What’s your best use case for AI in your company so far?

I’m looking to learn from examples - what have been so far your best implementation of AI in the org?

https://redd.it/1rasadb
@r_systemadmin

Migration Nightmare: How moving to a new server killed my email deliverability (and how I fixed it)

Just a quick heads-up for anyone planning a hosting migration. Everything seemed perfect until I noticed my user emails (activation keys, receipts) completely stopped arriving.

Turns out, the new server IP was 'cold' or had a poor reputation, landing everything straight in SPAM. Spent my day deep-diving into mail logs and DNS records. Had to double-check my SPF, DKIM, and DMARC settings to prove to Gmail/Outlook that I’m not a bot.

Lesson learned: Always warm up your IP and verify your records immediately after a move. Anyone else had this 'fun' experience?

https://redd.it/1raw91f
@r_systemadmin

OVH raises prices. My new offer is 55.1% higher starting April.

We, the consumers, are getting screwed big time right now. I'm starting to hate this AI thing that is causing us so much trouble.

https://redd.it/1rayqsw
@r_systemadmin

What are you using for large fileserver backups in 2026?

Hey all,


I am contemplating the best solution for security + cost.


We have the following

\-100TB of storage on one Windows Fileserver, \~30tb active data and \~70tb of archive

\-100TB of storage on a TrueNAS with about 50/50 of usable/archive data

\-Another \~10ish TB of data across a few processing servers, VMs, etc.


I have two spare fileservers with \~80TB of available storage on each that can be used as a new backup server.


I'd like to have a copy on site for one of them, then ideally have the other off-site and then replicated to the cloud. I'm looking for redundancy and immutability.


Are there any recommendations that could satisfy these requirements without absolutely breaking the bank?


Thanks!

https://redd.it/1rb0e6p
@r_systemadmin

Does anyone just know things without remembering exactly where you picked it up?

The noscript doesn't do a fantastic job of conveying what I mean.

I've been in the industry twelve years now. When I was starting out I learned everything about everything. I had this naive belief that I needed to know all of the underlying aspects of everything. But once you've done this long enough - you realize exactly where to make compromises and pick up tricks to get up to speed much faster. And you start to leverage tools and workflows in more creative ways that needing to know every underlying thing isn't needed.

A problem I see is junior people aren't curious or don't think big picture. There was a time I would pass on knowledge or advice more freely but people just don't care and it limits them.

Lately I've been wondering where I picked a lot of stuff up. So much has just become obvious or second nature. And it all ties back to the first paragraph about picking things up to make you more effectual / productive.

For example - we have a Stored Procedure that goes through a table in every customer database and compiles the data into a central database / table so we can pull reports from the data. This process was eating up a ton of CPU and taking hours to run. I looked at it, and it was using a merge over an insert into and it was also pulling the data directly from the customer tables.

Rather than waste time with changing the merge and possibly causing myself more work in rewriting - I just had the SP grab the data, and dump it into a temp table. That way, the merge would happen from that temp table. To me, that was the obvious cleanest fastest fix. After my change, the process ran in an average of 4 minutes and the CPU never climbed more than a couple percent. I'm not even a data analyst or DBA in specialty. I'm a systems engineer who was just curious enough to learn how things worked when I was younger. I realized being able to write SQL would make me mor effectual. But I will talk to devs of 20 years who complain their dev SQL server is slow but they have the memory limit set too high and after 20 years haven't learned to check that.

And I've just been thinking lately, when and where did I learn this crap and when did so much of what I do turn into pattern recognition and muscle memory.

I assume this is common to run into the longer you do this?

It feels like the further I get into my career, the industry expects so much more out of Systems people than anyone else. And maybe that's why I've grown so much... A lot of what we do is psychology and instilling confidence. I can't imagine admitting I don't know how to set the memory limit on a SQL server and the chain of command not losing all confidence in me and my abilities. Meanwhile, I have our CTO asking me, "Can you set basic setting x and y for the QA manager who owns the system. It's not their specialty and they don't know how."

https://redd.it/1rb22uz
@r_systemadmin

MAM IOS/Android error

Hello everyone,

I’ve been working on this for a few hours now and I’m trying to roll out MAM for some BYOD devices. I’ve followed several articles and watched a couple of deployment videos, but I’m still running into issues.

I created an Intune App Protection Policy and assigned it to two groups one security group and one Microsoft 365 group. I have a single test user with a Microsoft 365 Business Premium licence. When I check the user in the Intune Admin Centre, I can see they are Intune licensed, and it shows 37 check ins.

I’m using Microsoft Authenticator, and I’ve already re added the user account to the app. If I log in without a Conditional Access policy, everything behaves like a normal login and no policy seems to apply. However, when I enable the Conditional Access policy, I receive the following error:

"Access needed: Your organization requires that you have an Intune policy to access data for this account, but we couldn’t find one."

The Conditional Access policy is targeting all Microsoft apps, and I can see the included group contains the test user. The user’s country location is also correct.

Does anyone have any suggestions on what I might be missing? I am also looking for someone to help me ongoing with multiple Intune/Entra issues on a pay as you go basis please feel free to DM me.

Many thanks,

https://redd.it/1rb5mmf
@r_systemadmin

Helping someone gain access to sensitive data: Am I overthinking to refuse providing help?

Mods - Apologies if this is not the best subreddit to post this question. Please lmk if so, and I'll gladly remove my post.

An acquaintance approached me asking for help with recovering some important data from his GF's laptop as she had forgotten the password to it. During the text convo, he casually mentioned that it was his GF's ex's laptop, and she wanted to get access to potential evidence related to the molestation of their (GF and her ex) two daughters by her ex when they were young so she could bring charges against him.

The moment I read it, I refused to help him because I didn't have a good feeling about it. There's no way to know if she was legally the owner of the laptop or just stole it. Besides, if what this person claimed was true, then would I be exposing myself to potential charges by being in possession of explicit content involving children?

This person was very upset at my refusal to proceed with the data recovery effort.

Am I overthinking this?

https://redd.it/1rbdqpj
@r_systemadmin

Weekly 'I made a useful thing' Thread - February 27, 2026

There is a great deal of user-generated content out there, from noscripts and software to tutorials and videos, but we've generally tried to keep that off of the front page due to the volume and as a result of community feedback. There's also a great deal of content out there that violates our advertising/promotion rule, from noscripts and software to tutorials and videos.

We have received a number of requests for exemptions to the rule, and rather than allowing the front page to get consumed, we thought we'd try a weekly thread that allows for that kind of content. We don't have a catchy name for it yet, so please let us know if you have any ideas!

In this thread, feel free to show us your pet project, YouTube videos, blog posts, or whatever else you may have and share it with the community. Commercial advertisements, affiliate links, or links that appear to be monetization-grabs will still be removed.

https://redd.it/1rg2o3y
@r_systemadmin

Does your service desk tier 1 rep know how to change display scaling and how much are you paying them?

Serious question, not a joke. Can you tier 1 (entry/low) rep change display scaling on their window device? How much are you paying them?

Edit: for clarity, our tier 3 service desk is still a help desk rep but a senior level. Someone who can troubleshoot new issues. In traditional tiers this is probably tier 2 or 1.5?

Rant: I am about to cut ties with service desk completely after what was pulled recently. User submitted a ticket with a screenshot stating that they can not access certain web application. Screenshot shows an icon indicating that device must be rotated. It was not solved by tier 1 and escalated to tier 3. Tier 3 reached out to me directly asking for help. I responded with change windows scaling down to 100%. The reply that rep sent was telling end user to click on settings in web application and then change scaling to 100%

This is tier 3 rep, that does not know what changing scaling in windows is or how to do. Instead of trying it or asking for clarification a nonsense note was sent to end user which does not solve anything.

This position is paid 65k a year if I’m not mistaken. For tier 3.

I just lost my will to help…

https://redd.it/1rgs26b
@r_systemadmin

How do y’all handle coworkers that’s just not pulling their weight?

I can get behind competent people slacking since they know how to do the work when it counts but I have a guy that just doesn’t grasp it. Unless google literally spell out the solution or someone walk him through it he wouldn’t get how to begin troubleshooting it.

I wouldn’t mind it as much if I’m not dragged into his tickets so often. Just to figure they never bother research further than calling the vendor .

https://redd.it/1rgs89q
@r_systemadmin

I've made a massive mistake

​

I left a sysadmin role where I was comfortable and had spent five years, and I started a new sysadmin position this week. Almost immediately, I realised I’d made a mistake.

On my first day, I arrived to find an old Acer monitor with no stand, a broken desk phone, and no laptop. After a very brief introduction, I began reviewing the tenant and discovered it was several years old but essentially still in a “straight out of the box” state. There is no documentation, no asset register, and critical infrastructure including hardware and the firewall is end of life.

It quickly became clear that the IT Manager has no understanding of which vendors we use or what services they provide. I was told to start emailing various MSPs to figure out what they handle and was informed that I’d be responsible for managing this going forward.

I put together an eight-page document outlining serious security risks, only to then learn from the CEO that the company was hacked last year. On top of that, they never retrieve equipment from leavers and have no way to track company assets.

I feel like I’ve failed by leaving a great role for this situation, and I’m now facing the possibility of having to restart my job search. I’ve been completely honest with them about how misled I was during the interview process.

There’s also an expectation that I take on multiple, unrelated projects alongside day-to-day sysadmin responsibilities. I was told in the interview that this was a new role and a straightforward sysadmin position. What I later discovered is that another IT manager had previously been doing this job and was dismissed for gross misconduct. Another red flag is that the company doesn’t use job noscript everyone is expected to “wear multiple hats.”

At this point, I’m seriously considering walking out on Monday and looking for something else.


https://redd.it/1rh810e
@r_systemadmin

HELP PLEASE! Had my first real email compromise incident this week. Solo IT Admin. Here's what I did — what did I miss?

Long post, but hopefully useful to someone who ends up in the same situation. TLDR at the bottom.

So this week I dealt with my first legit email compromise at work. I'm the sole IT Admin at an SMB (\~250 mailboxes, \~82 internal users caught in the blast). No team to call on, no senior engineer to escalate to — just me, Google, and a lot of Microsoft docs.

A VP-level exec's M365 account got compromised and the attacker used it to blast malicious OneDrive/SharePoint sharing links to our internal employees and external customers(about 2000 emails sent in total). Because it came from a trusted internal account, a lot of people didn't think twice. It was a bad day.

Here's what I did, roughly in order:



# Containment

First thing — got the VP out of the attacker's hands. Reset the password, revoked all active sessions in Entra ID so they were signed out everywhere immediately. Then I pulled the malicious OneDrive file, killed all the sharing links tied to it, and went digging for inbox rules. Didn't find anything. Also checked to make sure the attacker hadn't registered their own MFA method on the account. Disabled users access to all platforms under my purview in our tech stack.

# Investigation

Pulled Entra ID sign-in logs to figure out where the breach started — looking for weird IPs, unusual locations, off-hours logins. Found some suspicious non employee logins from Miami and Arlington Va. Used Exchange Admin Center to run message traces and figure out how far the malicious emails actually went.

I also checked for OAuth app consents, new device registrations, and any delegated permissions that got added (found nothing).

# Remediation

I used Microsoft Purview Content Search to run a tenant-wide search for every email sent from the compromised account during the attack window. Found 164 malicious messages sitting in 82 mailboxes.

I used powershell to mass purge the emails from all internal users inbox.



# What I'm still trying to figure out / asking for help with

1. What did I miss in the investigation? Are there logs or artifacts I should've pulled that I didn't? I'm thinking about things like shadow inbox rules, deeper delegate access checks, hidden mail flow rules at the org level — anything that could've been left as persistence.

2. Customer notification — where's the line? The malicious links went to external customers too. At what point does this become a legal or compliance notification situation? Has anyone navigated this at an SMB level without a legal team on staff?

3. CA policy baselines? Anyone have a solid Conditional Access policy structure they'd recommend for an SMB M365 environment? Especially around admin accounts and high-risk sign-in handling.

5. Defender plan — what do I actually need? What's the minimum plan you'd want for real incident response tooling at this size? Is Defender for Business worth the jump?

6. How do you validate you actually got everything? Post-incident, how do you confirm there's no persistence left — hidden OAuth tokens, mail rules, rogue device enrollments? I feel like I got the obvious stuff but I'm not fully confident.

Anything else I should be looking out for or worried about? Anyway to tell how the attacker entered her accounts or gained access or track what they may have done while they had access to her credentials? This is giving me anxiety, some of our partners and customers are in a uproar.

TLDR: VP account got compromised, attacker sent malicious OneDrive links to \~82 internal mailboxes and external customers and partners. Reset/revoked the account, investigated logs, used PowerShell to purge 164 malicious emails across the org. Solo admin, first time doing this for real. What would you have done differently and what should I be doing next?

https://redd.it/1rhaevx
@r_systemadmin

OpenClaw is going viral as a self-hosted ChatGPT alternative and most people setting it up have no idea what's inside the image

Got OpenClaw running two weeks ago. Claude and GPT through my own Telegram, no third party routing, exactly what I wanted. Pulled the image, followed a guide, done.

Then I actually looked at what I pulled.

Official GHCR image has \~2k CVEs. 7 critical. Several with no patch available at all. The 1panel build is basically identical. Alpine/openclaw sounds like it should be minimal, it's not even Alpine, it's Debian 12 underneath with 1,156 vulnerabilities. Check yourself: docker run --rm alpine/openclaw cat /etc/os-release

Here's what makes this different from running any other bloated container. OpenClaw directly edits local files and executes system commands. It needs unrestricted machine access to function. ChatGPT runs sandboxed. This doesn't. So whatever image you pulled has your WhatsApp, your API keys, your filesystem, and 2,000 unpatched CVEs.

I'm not running it anymore until I find something cleaner. Has anyone found an image that's actually been stripped down, same functionality...?

https://redd.it/1rg2kc1
@r_systemadmin

1 month with Ubiquiti (so far)

We recently started testing with Ubiquiti to replace an existing Meraki deployment. After a very small test, we replaced about 30% of our APs with Ubiquiti APs. Then, we replaced two 48-port access switches with Ubiquiti switches. We have a small environment with only 2 physical sites, about 75 APs, 1 core switch, and about 15 48-port access switches. We are using self-hosted Unifi OS running on Rocky Linux 10 on Proxmox.

So far:

\--We noticed an issue with a single wireless client. It was a very old Android phone, and for whatever reason, it repeatedly connected and disconnected (once about every 2 seconds). The "solution" was to disable the 6 GHz radio for that one SSID; we honestly don't know why this "fixed" it. And it may not be a Ubiquiti-specific issue because this was the first 6 GHz radio we ever had in our environment. Eventually, we will turn on the radio again.

\--We had some weird intermittent client connection issues with the switches. We quickly reverted back to Meraki for these. We probably could have spent more time and energy on it and possibly fixed it, but it was just too much to deal with at the time. The issue did not occur in the lab testing, so I am not sure what it is. We may revisit it.

So our overall direction right now: use Ubiquiti for APs, not switches. This could change in either direction over time. I'll post again in a few months.

https://redd.it/1rhcdko
@r_systemadmin

Stale trust still showing up on other DCs after deletion

Good Morning!

Back many moons ago, my predecessor created a secondary domain to use for Exchange. He built the Exchange server AND DC as one server. This is the only server in this domain and it has been offline now for about three years. However I still see the Trust relationship in the Active Directory Domains and Trusts GUI. The Trust looks like this:

"Domains trusted by this domain (outgoing trusts)":

Domain Name "companyB.com"
Trust Type - Forest
Transitive - Yes

"Domains that trust this domain (incoming trusts)":

Domain Name - "CompanyB.com"
Trust Type - Forest
Transitive - Yes

I've deleted the trust via Active Directory Domains and Trusts GUI.

However, 30 minutes later, if I use the above tool to connect to my other DCs, It still appears, and when I click on the trust and properties I receive this error: ""A trusted domain object cannot be found for the trust to domain (olddomain). The trust may have been removed by another user." The remove button is greyed out.

I've forced replication using repadmin /syncall /APeD

If I open up adsiedit.msc, and connect to my current domain, I cannot find the old trust object under CN=-System to delete. Am I looking in the wrong place?



I still have access to the old DC for the no longer needed domain and trust. It's been powered off for several years. Should I simply turn it back on, recreate the trust on my current domain, then delete the trust while the old DC is active?





Thank you!

https://redd.it/1rh7cp0
@r_systemadmin

Network Solutions - Any success with DKIM for your companies Domain

I inherited a big mess with company email hosted at Network Solutions, but DNS hosted elsewhere. The split support isn't really a problem, just a pain.


I'm trying to implement DKIM aligned with our company domain. Emails have valid DKIM applied by Vade/OX, but of course that won't pass DMARC.


I won't bother relating the support horror story, I just would like to know if anybody has successfully setup DKIM for your own domain to use with Network Solutions Professional Mail.

https://redd.it/1rhbkwy
@r_systemadmin

Philips 34B1U5600CH – Does USB-C + HDMI work in 50/50 PBP with hardware KVM (no software)?

Does anyone own the Philips 34B1U5600CH and use USB-C (with power delivery) + HDMI simultaneously in 50/50 PBP mode?
Can you confirm the built-in KVM lets you switch keyboard/mouse between both inputs using only the monitor’s OSD, with no software installed?

https://redd.it/1rhgxxb
@r_systemadmin

CMMC L2

My org is starting to look at getting to CMMC L2 and there have been a lot of changes being made to make sure we achieve it by the end of the year.

Curious about other sysadmins who have been through this and what works and what doesn’t? I’m curious what pitfalls there are and how to avoid them.

https://redd.it/1rhimas
@r_systemadmin

Why do users insist on using work email for personal tasks?

It just makes no sense to me that I get people complaining about trying to send or receive emails when it has no work value at all. For example, one person was supposed to receive an email from their kid’s school about updated schedules but never did because it got caught in a spam filter that they could have checked themselves.

Why should I be dedicating resources to an issue that only affects their personal life, and why can’t they be bothered to have a personal email account?

https://redd.it/1rhk0xh
@r_systemadmin

Would a lightweight PAM / password rotation tool for <500 devices be useful?

Hi sysadmins,

I’m a developer working on a lightweight Privileged Access Management (PAM) / password rotation solution aimed at organizations managing fewer than 500 devices, and I’d really value some real-world feedback before going further.

The goal is to build something simpler and more affordable than enterprise PAM tools, but still reliable enough for production use.

Current ideas include:

• Reset and verify passwords for Active Directory and local machine accounts

• Automatic device discovery on the network

• Agent-based password rotation for devices that are not always connected (like laptops)

• Password vaulting and secure sharing

• Full audit logs for all password-related actions

• Verification that password changes actually succeeded on the device

• Managing credentials even when machines are off-network or remote

Target environment:

• Small and mid-size organizations

• Internal IT teams (not MSP-focused)

• Roughly 50–500 devices

I’m trying to understand:

• Would a tool like this be useful in your environment?

• What features would be must-have vs nice-to-have?

• What do current PAM tools do poorly?

• Would you consider buying a simple, reasonably priced solution in this space?

I’m especially interested in hearing from admins managing small-to-mid environments, where enterprise PAM tools feel too heavy or expensive.

Happy to chat 1-on-1 if anyone is interested — feel free to DM.

Thanks.

https://redd.it/1rhnirl
@r_systemadmin

My "I've made a massive mistake" moment

Reading another post on this sub reminded me of my own "I've made a massive mistake" moment - https://www.reddit.com/r/sysadmin/s/G7BjVaBkzy

I was a service desk analyst at a medium size organisation. The company overall was good to work for, and paid on the higher end for a service desk analyst in the area.

I had been with them for at least 3 years and I really wanted to get into a system administrator or network administrator role. Problem was all the people in these roles already were comfortable there and weren't going to be resigning anytime soon. The company also wasn't expanding, so there wouldn't really be any newly created roles. It would be potentially years before I would get into one of these roles at this company.

I start applying for other system administrator and network administrator roles, and eventually interview at another company as a system administrator.

Interview went well. We discussed why I wanted to leave my current role and I explain why, and discuss salary which was only slightly higher than my current salary, around 5% higher.

Although it would have been nicer to make a higher salary, it was at least getting into a role I wanted, and I didn't exactly have a huge amount of experience that wasn't service desk, so they offered and and I accepted the role.

I start my first day there. They tell me that everyone new in IT there starts out in service desk for 3 months. This was to get familiar with their systems, processes and business overall.

I was a little annoyed considering that I took this role to get out of service desk and that this wasn't mentioned in the interview, but fair enough. It was only for 3 months, so whatever, I'll just stick it out for 3 months.

Being new to the job, I do my job as good as I can. Every ticket is done well, has all the correct information, if it needs to be escalated has everything the team being escalated to needs including all troubleshooting, screenshots, etc.

My first pay came and I notice that it is quite a bit lower than what it should be. I check my payslip and it mentions my yearly salary at about 70% of what the salary in the interview was discussed.

The next day I raise this with my manager, politely mentioning there must have been an error when my pay was setup with HR or something.

He mentions that pay is what they pay their service desk analysts, so it is correct, but once I start as a system administrator it will become the wage discussed in the interview.

I was super annoyed at this, especially considering it's substantially less pay than the job I resigned from. I tell myself it's only for 3 months, just wait it out.

3 months comes up, then 3 and a half months comes up, and I'm still in service desk at this 70% of the agreed upon wage.

On the day of being there for 4 months I mention to my boss that it was discussed that everyone starts in service desk for 3 months, it's now been 4 months, and ask when I would be moving to my system administrator role.

He mentions funny I should bring that up, management were just discussing that. They had noticed that I have done really well in the service desk role. As such, they decided that they want to keep me there, and they would be moving another one of the service desk guys into the system administrator role.

To say I was livid at this would be an understatement, but I just put on a happy face. I knew at that moment I wanted nothing more to do with this company.

That night, I started applying at other companies and within a month, I had another offer as a system administrator elsewhere.

When I resigned, it was basically surprised Pikachu face with them. They couldn't understand why I was resigning after only 5 months in.

https://redd.it/1rhpifn
@r_systemadmin