“Every time I think of you, I feel shot right through with a bolt of blue…” – If your SQL Servers (and especially your SQL Server Availability Groups and Failover Cluster instances) could speak – that’s what they’d say about the times your security and sysadmin teams deploy a new AV tool without talking to the DBA team…
We’ve seen much heartbreak over this love triangle. We’ve also recently made a lot of money because of Sentinel One deployments done to AGs and FCIs with a next..next…next..finish style installation. *But the other tools are not off the hook – we’ve seen other tools break SQL servers*
Why Can’t We Be Ourselves Like We Were Yesterday?
“Mike,” you might say, “why the heck are you talking about antivirus tools in 2025?! I thought this was already solved 20 years ago.” You’re right – the “old days” where filter drives would cause random corruption here and there and everywhere in SQL server 6.5/7.0/2000 DBs got better. Filter drives got smarter. And many of the new classes of EDR/XDR/Enhanced/Next Gen AV/Endpoint protection tools (whatever you want to call the category!) don’t even use filter drivers like the antivirus products causing corruption in the days where “no exclusions” meant “yes corruption.”
Behold, a new era! The threats have changed. Sure, viruses still are the entry point for many threats – but the sophistication of the threat, the organized manner of ransomware, and the threat level meant a new generation and class of tools have been released. Many of these tools are doing traditional AV-like roles – but they are also now helping to protect against unauthorized system changes. That’s good. Except – sometimes the tools wind up “protecting” your servers from “your server.” When that happens – it can range from a little annoying to – your cluster database is corrupt, and you have to tear it down and rebuild it – for an AG, that’s a lot of work and annoyance – for a failover cluster instance – that’s a weekend of work reinstalling.
Not all the tools. But we’ve seen various tools cause issues on that spectrum. Our clients (or folks who found our number for an emergency call after this bit them) have seen it. They’ve used more hours than expected and had downtime. I’ll pick on one tool in particular below – but the MAIN point here is the point of the next section, first.
We Can Be Together Now
There’s no sense in telling me, “We’ve deployed this a dozen times, and it’s never caused an issue.” The wisdom of that foolish thought won’t set you free—not when your AG’s fallen hard at 2 AM two weeks after the new tool was deployed, and you were just doing routine maintenance and did a failover.
We need to get along and be on the same team. That’s all you have to do to avoid issues. You don’t have to stop an AV tool from coming in. They are coming in. They are necessary. But now it’s time to use your woo and make sure the process considers the databases. While I’ll explain some specific woes with ONE tool below, the steps here are what you need to do every single time.
Who cares if they say, “Why do you always get this way?” It’s your job as a DBA to advocate for your environment. But with the below steps, it’s fair to say that whenever you get this way, you WILL know just what to say (and do):
- Get Involved – ahead of time. Be part of the team that reviews these things. Tell your security and admin teams that the DBA team needs to be part of the release of new tools, major updates, and changes – even to AV tools. Make it known. Get a change process NOW. And make sure that the CISO and security teams don’t have some “get out of rules free” pass.
- Get In the habit of testing – BUILD a test environment. If you have an AG in prod – have an AG pre-prod. All changes go there. And then they stay there for testing. They stay there for a patch cycle, some failover. TEST HARD.
- Read the documentation and google it – Ask AI, ask Google, Look in the help forums.. “SQL Server problems and _____ AV” – what kind of things are people seeing? Understand what the risks are.
- EXCLUSIONS! Exclusions are key – if the AV tool is coming to your SQL Server. DO NOT LET THE VENDOR say, “well it’s not using filter drivers it works differently.” – ask if they support exclusions. PUT THEM in place. The microsot documentation is amazing for this. But don’t just say “follow this article.” Say “SHOW ME” – and don’t just look at the broad category – Look at the actual exclusions – see the section below…
Your goal as the DBA is to bridge the gap between the folks who depend on you protecting the databases and the teams who are well-meaning but tasked with something like security, not database availability and uptime. Forge relationships and sort this out.
Shot Right Through (With a Bold of Blue)
I am not picking on SentinelOne – it’s a fine EDR. That said – if we were sending gift baskets to the EDR vendor who has made us the most money over the past 5 years, I’d be sending a lot of gift baskets to Sentinel One. But remember – the lessons lead to the section above this. And the experiences we’ve seen necessitate those steps. But just look at a few things we’ve seen here.
Every time we see a cluster (and an FCI or AG on the cluster) fall, we get down on our knees and pray that it’s not the problem we find over and over—Sentinel One was deployed, and no one asked us or checked with the DBA team.
Alas, sometimes it is. And now our task will be tearing down a cluster and rebuilding it.
There are two specific things going on here:
- Exclusions! Use them. Most never have. And some just say “well we picked the default SQL Server exclusions so we are good” – but then they never looked to see that the file paths and wild cars are specific. And the Microsoft AV article above isn’t fully followed. So you need to customize these.
- MORE IMPORTANT – Sentinel One takes some VSS based snapshots (I don’t know why – They probably have a reason – but it will break things..) of your cluster DB. If this snapshot feature is not disabled – somehow those snapshots will cause the cluster DB eventually to get corrupt. Once that happens – I don’t care what you do – you are not getting the cluster happy again. Time to rebuild the cluster and add the instances back for an AG – or potentially rebuild it all and reinstall SQL for an FCI…
Make sure you disable those snapshots. Make sure you add the right exclusions. Or it gets expensive and ugly.
Don’t disable that snapshot – and you will see the error message that tells us you’re in trouble “The cluster database could not be loaded. The file may be missing or corrupt. Automatic repair might be attempted.” OOPS!
Why Does it Always Come Down to You?
Because you are the DBA!
Listen – more than half of the problems out there are communication problems or relationship problems. This is no exception. Yes, there are things that the EDR/XDR/Next Gen AV tools break in new and ugly ways. But if you follow the steps in the middle section. If you communicate with the other teams. If you build a culture of teamwork and test – these things are less likely to break.
That’s your job now. Take this. Look at your AV tools and follow the steps like you were about to deploy them:
- Check the exclusions – that Microsoft link is worth repeating. MAKE SURE THEY ARE THERE. SEE THEM YOURSELVES.
- Test
- Communicate
- And if you are using Sentinel One AND have a cluster! FIX THE SNAPSHOTS AND EXCLUSIONS NOW.
You can fix this without us—but you don’t have to go it alone. Whether it’s a cluster review, a health check, or a midnight disaster—we’re here: 1.888.SQL.DATA (888.775.3283) and option 1 will put you in touch with my cell. We’re here to help.
This is part of our “Regret” series—lessons from a SQL Server consultancy that’s seen it all.
