How can we better protect ourselves from the recent npm supply chain attacks leaking secrets?

The recent wave of malware infecting hundreds of npm packages organization. sensitive secrets on platforms like GitHub has shaken the developer community. These supply chain attacks exploit malicious post-install noscripts and compromised maintainers, making it really challenging to trust the packages we depend on daily.

Many security best practices suggest disabling post-install noscripts, implementing strict package version cooldowns, validating package provenance, and minimizing dependency trees. Yet, even with these, the leakage of secrets remains a critical risk, especially when malicious code executes inside containers or developer environments.

Has anyone explored or implemented strategies that go beyond traditional methods to reduce the attack surface within containerised or runtime environments? Ideally, approaches that combine minimal trusted environments with strong compliance and visibility controls could offer better containment of such threats. Curious to hear what the community is trying or thinking about as more organizations wrestle with these issues.

https://redd.it/1p6z3ar
@r_systemadmin

I’m tired of playing “where did this update go?”

Every sprint review turns into a hunt for missing updates. Devs update GitHub, PMs update Trello, leads update Google Sheets, and nothing matches. Half our delays come from misalignment, not actual coding issues. Is there anything that pulls GitHub info directly into the project boards and makes reporting automatic? I'm done manually chasing pull requests like they're stray cats

https://redd.it/1p71tuw
@r_systemadmin

Chainguard alternative?

hey anyone got cheaper (or free) alternatives to chainguard images that actually get rebuilt weekly with patches? chainguard is killing our budget and my manager is about to have a stroke over the invoice 😂

i just need tiny base images that stay mostly cve-free without costing a kidney. what are y’all using?

https://redd.it/1p765wt
@r_systemadmin

Has anyone ever actually fixed anything by updating drivers in Device Manager?

I’ve been in IT for 5 years now, and not once has “Search automatically for updated driver software” in Device Manager ever found any missing drivers. I get that it only pulls generic stuff and not the proper manufacturer drivers, but why this crap is still widely recommended as a first troubleshooting step is beyond me.

Yet I still try it every now and then out of pure desperation… only to confirm what I already know: it is never a solution. Has this ever actually solved anything for anyone?

https://redd.it/1p73k01
@r_systemadmin

FreeRADIUS in production: 10 practices that eliminated random delays and weird spikes

I manage FreeRADIUS in one real project (no sensitive details, of course) where it handles a significant flow of authentication and accounting requests.
In the early days we saw everything: random delays, ODBC stalls, unexpected request spikes, duplicate storms, and periodic “mystery slowdowns.”

After months of tuning, log analysis, and observation, these practices made the system far more stable and predictable.
Sharing them here — maybe useful to someone.



# 1. Database latency watchdog (every 5 seconds)

A tiny query like SELECT 1 through ODBC.
If latency goes above a threshold → log immediately.
Helps distinguish “DB is slow” from “RADIUS is slow.”

# 2. Proper ODBC pool tuning

These values worked extremely well:

min pool = 8
max pool = 32
connection lifetime = 3600
query timeout = 5–8 seconds
login timeout = 2 seconds

Without a lifetime limit, stale connections accumulate and eventually collapse the entire chain.

# 3. Duplicate-request control

We added a small duplicate counter + a soft-limit.
When a device floods identical Access-Requests, FreeRADIUS can behave strangely.
This made such issues instantly visible.

# 4. Log handling: only rotated .gz files

Never touch active logs.
Use logrotate → compress to `.gz` → process archives only.
Touching “live” RADIUS logs is an easy way to corrupt them silently.

# 5. Weekly system-status snapshots

A single automated report containing:

RAM / SWAP usage
IO wait
Load average
SQL latency
ODBC pool state
log size growth
RADIUS response time

Week-to-week baselines make long-term patterns obvious.

# 6. RTT monitoring between nodes

Even if servers are in the same site or different regions.
If two nodes show identical RTT spikes → it’s a systemic event, not a local issue.

# 7. Docker maintenance (if containerized)

We run FreeRADIUS in Docker, so we use:

cleaning overlay2 layers older than 7 days
truncating large container logs
weekly `docker system prune`
healthchecks + auto-restart

This removed several unexpected IO stalls.

# 8. Reject-peak detector

If rejects per second go above a threshold → log it as a separate event.
Helps detect anomalies in real time (DB slowdown, traffic bursts, etc.).

# 9. Accounting/session logs: gzip → archive

Never read or write active accounting files.
Compress → move → remove local copies once verified.
Keeps live directories clean and safe.

# 10. Lightweight RCA notes for every incident

5–6 lines:

timestamp
what happened
root cause
impact
fix
current state

This saved hours of analysis when something similar happened again.

# Result

After implementing all of this, random slowdowns dropped dramatically, and incident resolution time became much shorter.

If anyone wants it, I can share:

the system-status noscript
ODBC configs
logrotate templates
duplicate-request checker
my reject-peak detector
or the safe directory layout we use

Just ask.

https://redd.it/1p77epw
@r_systemadmin

Has anyone found any AI use cases that work and deliver value yet? Other than smarter helpdesk support article suggestions...

I'm not talking about something where a user starts to enter a ticket about needing to reset their password, and the help desk system can find and suggest a support page about ... resetting passwords. That stuff has been around for a long time.

I'm talking current AI, or "AIOps" (which surprisingly really started ticking up in the past year). Even if the AI isn't automatically taking actions ... if it's able to quickly triage and bring all sorts of information together so by the time you get involved there's already an assessment waiting to be reviewed ... would be helpful.

It'd be interesting to know of any real-world examples where this is taking place. You don't have to name specific vendors (unless you want to) but I'd like to believe that somewhere out there, someone has stumbled on a few things that make their daily lives easier (personally, I'm playing around a lot with n8n on that front but that's not directly "AI" even though you can call AI engines into workflows with it).

https://redd.it/1p77zo3
@r_systemadmin

Are there any reasons to support TLS versions lower than 1.3 nowadays?

I am configuring a new host on Cloudflare, and I noticed that all versions of TLS, from 1.0 onwards, are enabled by default.

After a quick check, it seems that all modern browsers now support TLS 1.3. So is there any valid reason to keep TLS 1.0/1.1/1.2 enabled?

https://redd.it/1p78nnd
@r_systemadmin

Is Defender For Business any good?

Hi All, AV renewal time is coming up and have done my own research but wondered what the hive-mind here thinks about Defender for Business

On paper it seems like a no-brainer, we already have business premium licenses for some users, and per-endpoint it's cheaper than what we're using currently and since we're a MS environment it makes a lot of sense

However I'm getting that sinking feeling, if it's too good to be true then it probably is? Just wondered if there are any reasons we shouldn't go for it over our 'conventional' antivirus solution, or if anyone has run into any major issues with it

https://redd.it/1p74djj
@r_systemadmin

Windows 11 25H2 Long Path support

Has anyone used the long path regedit recently? I tried it on a few computers recently and it doesn't seem to work. Both notepad and Office applications are unable to open files when the combined length is longer than 260.

https://learn.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=registry

The documentation seems to support that it should only work with applications specifically designed to be compatible, but I remember it working with Office apps before. Anyone have any insight on this? Was there a recent change?

https://redd.it/1p7c153
@r_systemadmin

What is a special habit you have in your everyday sysadmin life?

I'll go first. Every time I press restart during server patching, I salute the VM or host in the hope that they will come back online quickly and I won't have to work any longer in the maintenance window.

https://redd.it/1p7gfi7
@r_systemadmin

What happened to the IT profession?

I have only been in IT for 10 years, but in those 10 years it has changed dramatically. You used to have tech nerds, who had to act corporate at certain times, leading the way in your IT department. These people grew up liking computers and technology, bringing them into the field. This is probably in the 80s - 2000s. You used to have to learn hands on and get dirty "Pay your dues" in the help desk department. It was almost as if you had to like IT/technology as a hobby to get into this field. You had to be curious and not willing to take no for an answer.

Now bosses are no longer tech nerds. Now no one wants to do help desk. No one wants to troubleshoot issues. Users want answers on anything and everything right at that moment by messaging you on Teams. If you don't write back within 15 minutes, you get a 2nd message asking if you saw it. Bosses who have never worked a day in IT think they know IT because their cousin is in IT.

What happened to a senior sysadmin helping a junior sysadmin learn something? This is how I learned so much, from my former bosses who took me under their wing. Now every tech thinks they have all the answers without doing any of the work, just ask ChatGPT and even if it's totally wrong, who cares, we gave the user something.

Don't get me wrong, I have been fortunate enough to have a career I like. IT has given me solid earnings throughout the years.

https://redd.it/1p7hmjn
@r_systemadmin

I remember when digicert didn't suck.

That is all.

https://redd.it/1p7jwzm
@r_systemadmin

How has Dell Command Update worked for you?

We recently did a slow release by installing Dell Command Update in new images (so not directly from Intune) and configuring it to update itself via the Intune ADMX. So right now, only about 5% of devices have Dell Command Update. We have it configured to update once per month.

How has it worked for you? Do you have any horror stories? Do you have any config recommendations?

https://redd.it/1p7m6dg
@r_systemadmin

Help desk tools for mid-size teams? (college project + real life need)

Doing a project on ITSM tools, and at the same time I’m helping a mid-size company part-time with internal IT ops.
Their current help desk setup is super outdated..

What tools do you guys recommend for 100–500 employees?

https://redd.it/1p7o9r0
@r_systemadmin

How do I get a sharepoint activity list (as shown when you go to "restore this library")?

Audit log reports and unified audit log are empty, looks like they weren't started before and I have now started them...

When I go to "restore this library", however, it gives me a chronological list of every change made to the sharepoint site and I can choose to restore to any given point/change.

Is there a way to export that list for the last 7 days, or to otherwise get that data?


Edit: If you go to the library and go to details -> activity you can see the history too... but I can't find any way to export it...

https://redd.it/1p7tfzr
@r_systemadmin

Our dev workflow feels like a group project gone wrong

Design uses Figma PMs use Sheets devs use Jira QA uses something called Testy dont ask. We spend more time syncing tools than shipping builds. There has to be a better way.



https://redd.it/1p7umve
@r_systemadmin

Thickheaded Thursday - November 27, 2025

Howdy, /r/sysadmin!

It's that time of the week, Thickheaded Thursday! This is a safe (mostly) judgement-free environment for all of your questions and stories, no matter how silly you think they are. Anybody can answer questions! My name is AutoModerator and I've taken over responsibility for posting these weekly threads so you don't have to worry about anything except your comments!

https://redd.it/1p7yid3
@r_systemadmin

Teams governance

Hi,

How is everyone else governing Teams these days? The general lifecycle management, self service, governance and overall experience of Teams from a sysadmin point of view seems really lackluster and annoying to deal with.

 


We have been scouting for a proper solution to govern our Teams and Sharepoint setup and allow for our end users to create Teams, with guard rails and governance such as a naming convention, forced ownership, automatic archiving and thing like that, but it is difficult to find the right solution, or perhaps i am just getting hit with this "FOMO" where if i pick a solution and find a better one the next day, i am dug in for at least a year.



 

So far we have looked at Teams Manager from Solutions2Share and gotten a quote on it. Seems a bit Pricey 17.000€ for a year for 1000-4000 users. We only have around 3000 users at the moment, which is why i hate the 1000-4000 tier, as you pay the same regardless of having 1000 users or 4000 users.



 

It seems like a good product though, and mayb it is the right choice. Maybe not, i was hoping for some recommendations for other products or some feedback from others using Teams Manager, pros, cons, what is annoying, what works well, what does not work well and so on.


 

Hopefully we are not the only organization using Teams and are tired of the manual workload of keeping it tidy heh.

https://redd.it/1p7zazg
@r_systemadmin

We need one view for everything. Is that too much to ask?

I need ONE platform that unifies everyone and lets us track dependencies in a way humans can actually understand. Design, product, marketing, and dev teams all contribute to our releases, but no one sees the same information. Marketing launches features before they’re done. Product teams write requirements no one reads. Devs don’t know what’s blocked until it's too late.

https://redd.it/1p7zmik
@r_systemadmin

How many of you have done AI related projects?

Interested if anyone has had any projects to implement AI in their environment.

Setting up a LLM (in cloud or on-prem), integrating AI into an app that you host, creating an AI tool for your m365 services, etc.

Not trying to make a point, just curious if anybody in the real world has had to do this.

https://redd.it/1p7y3fc
@r_systemadmin

Cleaning up a decade of user-level ACL chaos… I ended up building a tool to survive it

We had one of those “beautiful” environments where every department insisted on per-user NTFS permissions “for traceability”, inheritance disabled everywhere, and 500+ folders with unique ACLs.
You know the type... the kind where only the guy who left the company few years ago actually had Full Control on most of folders.

Auditing was a nightmare.
Figuring out “what does this user have access to?” was a nightmare.
Transitioning to groups was even worse because you first have to discover the full effective footprint of each user before you can rebuild anything cleanly.

I got tired of manually walking through Explorer, checking advanced security on every folder, and trying to piece together what actually exists. So over the last several months, I built a PowerShell-based GUI tool that lets me:

search any domain user or group and instantly see all explicit ACLs across shares
detect all unique ACL paths
compare two identities (“give me the same perms as that guy”)
and most importantly: use it to migrate from user-based ACLs → group-based structure much faster (find the user who represents the role, create a group, clone the ACEs onto the group, add the right members, remove the users)

I posted about it yesterday on r/PowerShell and the thread blew up... lots of debate, but also tons of admins saying they’re stuck in similar legacy environments and that visibility tools like this would have saved them days.

A few people asked if I could share the viewer part, so I published the read-only version, it’s just the ACL discovery / auditing engine with no write functions at all.
No credential storing, no privilege tricks, just reading explicit ACEs the user already has rights to read.

If anyone wants to take a look or give feedback, it’s linked on my profile (FSWorks Lab).

This whole thing came out of pure survival instinct, so if it helps someone else drag their file server out of permission hell, great.

Curious how many of you are still dealing with user-level ACL legacy… because based on yesterday’s reactions, it’s more common than I thought.

https://redd.it/1p82ll2
@r_systemadmin