Psst. Hey Vendor – DBAs are secretly plotting against you! We hate what your products do to our environments. Sometimes we even work to get you replaced by someone else who makes a product in the same space but is “DBA-Approved”…
(Quick Edit March of 2021). In the time since writing this post, I’ve grown from a part-time SQL Server Consultant to the CEO of a thriving SQL Server Consulting practice. I’ve now officially lost track of how many clients we’ve helped with issues that just shouldn’t exist because of the worst-practice by software vendors. In fact, we just re-engaged with a client we helped out 2 years ago with some serious SQL Server performance issues. It’s their busy time of year, and they were being crushed by blocking and perf issues. Tara and I spent time digging in and looking and Tara gave the software vendor a really simple plan to resolve the issues. Sure enough, two years later our answer back to this client was, “I’m so sorry… The vendor code has not changed one iota…” Listen up, software vendors! You can do a better job. The world is full of people you can ask for help, posts you can read, consultants you can hire, DBAs on the market who can make your product shine. And sometimes – it really isn’t your fault! Clean up your house, and then let’s start getting your clients to fix their IO problems and performance woes brought on by understanding. We can help you, too.
What’s that? You want the DBA stamp of approval?
I write this blog to help folks, so I’ll pretend I’m not at work in a meeting railing on about how your product has no indexes or has way too permissive security or doesn’t think to recommend index maintenance. That’s right… I’ll help you. Why? Because I want you to succeed. I want to like working with you. I want you to be “DBA-Approved.”
A History
You don’t have to look too hard to find blog posts of DBAs annoyed with vendor gaffes. Some great vendors really understand their destination database environment. They go out of their way to make sure their client properly backs up and maintains the environment. They do performance tests and look at best practices. There are a lot who don’t.
I’ve worked as a consultant for a vendor who wanted to do a better job helping their customers handle the SQL Server environment they ship their app on. I’ve worked full-time for a vendor who didn’t. I can tell you that my experience with the latter was early on in my career as a tech support engineer, and I got a lot of painful calls from customers up a creek. Customers who maybe would have been better off with some prescriptive guidance…
But We Don’t Own Their Environment!
I know. But you didn’t tell the client that they really needed a DBA to look over their SQL Server environment, did you? You might have even marketed it as “so easy! A few clicks and our installation is complete!”. I’m not saying you should own their environment. I’m not saying you should coddle each customer and play the role of DBA. I am saying that you should…
Care Enough To Do It Right
That’s right. I’m appealing to your heart. Though, if you do a great job, it becomes marketing and word of mouth advertising. You’ll also be able to receive less “basic” support calls or deal with upset customers who got in trouble because of database issues. What about the lost business because of clients talking to other potential clients? Or IT staff commiserating with IT staff at user group meetings? Now I’m appealing to your bottom line, I hope?
Some Thoughts on Doing It Right
I’m not asking for much. This list is a good start. We might revisit this post in the future with more but starting with some basics; you’ll be on your way to being a “DBA-Approved” software vendor:
- Cheat – If you are looking to have your software sold and installed at a company that I work at, consult for, or have worked at. Be prepared to answer these questions. Maybe all don’t apply; maybe some look like gibberish. Learn about the terms and understand why a DBA would ask. Send me an e-mail or leave a comment if you are confused about why I ask a question.
- Learn and Know SQL Server – What kind of SQL Server expertise do you have on staff? You probably have some great developers but do you have someone with a DBA interest and skill set? Do you have someone who stays involved in the SQL Server community? Check out sites like SQLSkills, SQLBlog, SQL Server Central, Brent Ozar has a lot of great SQL Server training videos also. (Updated 2021 to remove PASS/SQLServerPedia and add the Brent links)
Speaking of PASS – PASS is the Professional Association for SQL Server users. It is a great organization with a lot of resources for use in the SQL Server space. They have a huge annual Summit with a lot of great technical content. Send someone to the conference and have them sit in the sessions. Have them talk to DBAs (we aren’t shy…)(There is no more PASS. . . Red Gate software bought their content and is starting to sharing it freely, though )- Documentation? Do you provide anything to your clients (especially those smaller shops that may not have a dedicated DBA, even if I think they should) around recommended best practices for database maintenance, backups, troubleshooting, etc.? You should.
- Best Practice Review – From some of the communities/blogs mentioned above, you can find many intelligent and experienced DBAs who can help you review your application from a database performance and best practices point of view. I would hazard to say that even just 12-24 hours of billable time can gain you a lot of customer goodwill. I know I’ve helped folks out with this in the past, and as a DBA on the receiving end, it makes a difference working with a vendor who has Database best practices in mind.
- Security! Please. Don’t. Ask. For. Sysadmin (SA). Rights!!! Please don’t even ask for DBO if you can avoid it. You should be using role-based security with least privileges. Please use Active Directory security and integrated with our AD, but I’ll even let you use SQL authenticated if you stop asking for SA rights! Let me review your deploys and see the scripts that get run. Do your deploys with the least privilege necessary, or heck, let me do it for you… I’m not a jerk; I’m trying to do my job as a DBA.
- Backup/Restore – It goes with the database maintenance documentation above. I wanted to call it out here separately. Work with your clients on best practices for backup and recovery. Learn about them yourselves first. Don’t just install someplace and expect it all to be fine. Ask your support teams if they’ve dealt with a client who lost more data than they were willing. Ask them how that call went.
- Don’t Go With Defaults – There have been many blog posts about this that you should read to get an idea of what I mean –My thoughts, Paul Randal, Aaron Bertrand, and plenty of others linked from those – Back? Alright, learn about how to install SQL Server and include that in the documentation. Talk about recovery models, so you don’t end up with huge transaction logs and bad advice being given to your customer’s IT support team from Google and forums.
Was That So Horrible?
I don’t think anything on there will make you freeze your code, hire a lot of developers or Project Managers. Sure, if you don’t have the staff’s expertise, you might engage a consultant and spend $10,000 or less reviewing your docs and plans, but how much will it cost you in a lost opportunity to not do that?
Thanks for listening, and I really hope you think about some of the points above or some of the points mentioned in my DBA Questions for Vendors list referenced earlier. I want your product to be a success, and I want to see us DBAs start writing positive vendor rants (well.. Asking a DBA, a pessimist by trade, to be positive is a bit much. Maybe we can at least stop with the negative rants if you take some steps with us).
Are you a software vendor? Who would you rather our team of SQL Server Consultants be working with? You, to proactively give your clients a great experience? Or your clients who are frustrated that they are spending money on us fixing your code?
Hey there Mike – Nice post. It’s true though – there are some really great ideas and products out there. They really know THEIR problem space well and they create a wonderful app around that space. However, when it comes to their data store – many don’t scale or do (as you’ve mentioned) quite horrible things!
I’m always excited when vendors DO ask the right questions and really care about the long term use and scalability of their apps. There are a lot of things that they can do (many of which are NOT difficult) to help their customers SIGNIFICANTLY!
Cheers,
Kimberly
Thanks, Kimberly!
Definitely true, it makes me excited to hear someone at a vendor trying to do it right. You guys at SQL Skills are definitely one such resource a software vendor could reach out to if they needed some consulting help to review their best practices. I know you’ve done it before and I am sure you have a lot of vendors in a great place because of your involvement.
No, don’t allow ’em to go with SQL Server-based logins. Windows logins! There are exceptions (What’s Up!, for instance), but those should be rare and well-reasoned. In What’s Up!’s case, it is used to monitor AD controllers. Obviously, if AD controllers are down, no Windows auth. And it needs to log that. But otherwise, no SQL Server-based logins!
Sounds fair to me, Brian. 🙂 I knew I could count on you for that comment, was actually half expecting it. I went back and forth while typing it.
There are some apps that just don’t natively integrate with AD and manage their own user tables/etc. so I didn’t want to single them out. But since I’m talking to vendors here, I guess I should say –> It isn’t that difficult to go windows authentication… DO IT!
Mike,
I must admit I am guilty of some but not all of the bad things you mention. At least the offenses were in the past during my “ignorant” days. Over the years I have learned and try my best to communicate during planning sessions on standards that we need to incorporate into our product.
Good points to consider. I’ll definitely use you “DBA Questions…” in the future.
Thanks
Sounds good, AJ. Please add any to the comments you like and I’ll make sure to try and keep the list updated. We all have a history… As long as you are growing, don’t worry about it – we all have to learn sometime.
Like SQLAJ, I was guilty of a few of these infractions in the past. At the end of the day, a solid DBA ends up saving me (a software vendor) time over the long term. I love it when my customer has a knowledgeable DBA.
Having said that, I may need to write a companion blog from the other perspective – tyrannical DBAs. I have one such customer who treats his vendors like ants he can fry under a magnifying glass.
Thanks for the comment, Jon… Glad to hear that a good DBA is a valuable asset with the clients of your software.
I will add though… I’m surprised you never noticed my gigantic Magnifying glass on my cube when we worked together… Why do you think I liked the window seat? mwooohahahahahahahh
Oh, is that what that was?! I thought it was the Eye of Sauron burning a hole in the back of my head.
Same thing, except for I use it for good.
Mike, nice post. Under backup/recovery vendors need to be sure that they include system database backups when they work with small shops without a DBA on site. I had an instance where the vendor WAS backing up their application databases, but NOT system databases. Well, msdb was corrupted in a power failure and guess what? Those scheduled application database backups stopped occurring. Then 3 months later the box failed and no recent backups.
Great point, Jack. Those system databases are often overlooked. By vendors and DBAs alike.
Good post Mike.
I am still amazed at how many times even the big vendors say they have to have sa rights. It instantly shows some laziness on their side that they didn’t take the time to identify the minimum required permissions needed.
Thanks for the comment, James. I agree with you about the appearance of laziness. The thing is, it wouldn’t take that long to address the security of an application and it would go a long ways to a better experience for users. My guess is that this is typically a “we’ve always done it this way” situation… An old CEO once gave a good illustration of this that he picked up someplace… About cutting the end off of a roast.. Snopes has the story in a few flavors – http://www.snopes.com/weddings/newlywed/secret.asp
Great post Mike!