The best EMTs and Medics are the ones who realize, “This isn’t my emergency,” early enough. You’ll be a better DBA or technologist the quicker you learn the freedom that comes from that line of thought. (Also, a warning – in this post, I talk about – in vague terms and no medical specificity about a couple of potential types of emergencies. If you’ve been through the stress of losing a loved one or having to call the ambulance on the worst day of your life, skip this one if you’d rather not risk being brought back to that moment in your memories.)
Yes. I’m mixing my wires and posting about observations gleaned from my time in the Fire and EMS world – it’s fresh in my mind since I’ve just recently started getting all of my previous EMS licenses back (Just passed my NREMT cognitive and psychomotor exams, license once the paperwork catches up.) But there are so many parallels that it’s worth bringing up the lessons that cross-apply here. Troubleshooting is big. So big that I won’t hire you at Straight Path if you can’t logically troubleshoot a problem. There are many steps to troubleshooting successfully. And I expect a 20 year SQL Server veteran to be able to more quickly rule in and rule out a lot of things in a matter of seconds. A 20 year Medic will do the same thing during their “form a general impression” phase. In fact, maybe I’ll post some parallels between the steps to assessing a patient and a problem in the coming weeks. For now, though, I want to talk about “distance.” You have to distance yourself from some elements of the problem to get close to other elements. I’ll explain…
A “Bad Call”
I’ve been on good calls in the ambulance. And I’ve been on bad calls. There are some common patterns on each… The difference makers are elements from the things you’d expect – preparation, teamwork, communication (yes, that means listening also), training, confidence (vs. pride/arrogance), etc.
But another thing is the ability to solve just who is having the emergency.
On a “serious call” with a seriously sick patient – things are dynamic. Let’s picture a “typical” witnessed Cardiac Arrest call. By the time we get to the scene – chaos is often the story. If there is no one with CPR training, there is someone on the phone with 911, dogs barking, folks screaming/crying/confused, people milling about holding hands shouting at the patient, someone invariably doing CPR as instructed by the Emergency Medical Dispatchers. Sometimes good CPR, sometimes bad CPR. This is a colleague, or a husband or wife, or kid, the stress can overwhelm “good high-quality CPR.” It’s disorganized. It’s amped up. It’s sad. It’s loud. Not to mention the condition of the patient. It’s a lot.
I’ve been on calls in a past life where the responders are sometimes “sucked in” to the emotions. The chaos. The noise. The frenetic pace. They become almost alarmed. Tension rises. They have “owned” the emergency – it’s their emergency. When it’s your emergency, your pulse quickens, your blood pressure rises (these things happen, either way, btw, adrenaline is real), and you start to get just a little frantic – barking orders, getting mad if the new person gave you the Pedi-mask when you clearly needed an adult mask for the BVM. Anger at the folks who last put the airway bag away, etc., etc. The stress rises, and as that happens – the quality of care goes down. It’s not enough to significantly change the outcome usually – but it’s enough to be noticed and to have that potential. The caregiver(s) who have “owned the emergency” is now in a high stakes game – not because they want to use their training and calmly (but swiftly) effect change – but they get almost so focused on the outcome and so into the emotions – that they can rattle some of that training out of them. They can become a bit useless. A provider in the emergency ends up so focused that they could lose training, lose muscle memory, skip steps, and not give their patient a fighting chance.
Maybe that’s extreme. But folks who are “in the emergency” are less effective than the folks who aren’t. The paradox here is – the more distance you have between you and the emergency – the better you’ll perform. The more you internalize the emergency and find yourself in it, the more “dumb things” you’ll do.
A Better Call
I won’t get into details – but there were so many more “better calls” than “bad calls” that first decade through my time on the ambulance. The calls were still far too often tragic situations, crises, and really sad and trying moments for the people whose emergencies we were being invited into.
But they were “better calls” because of many elements working together. Here, too, many things can account for a call going well (regardless of outcome) Again – preparation, teamwork, communication (yes, that means listening also), training, confidence (vs. pride/arrogance), etc. Again, here, though – this ability to create that necessary distance is key.
You can be on the most stressful call – a very sick pediatric trauma case. I’ve been on a couple. Those calls suck the life out of you (afterward). And they are high stakes, high stress. No one wants to lose a patient. But there’s something about a 2-year-old who had a multi-systems trauma and is decompensating and not going to end up right with his mom or dad pleading with you and coming along in the ambulance for the drive into the hospital that is different from an elderly person with a period of failing health experiencing respiratory or cardiac arrest at the end of a long life.
One such call is on my mind. It was just an all-around sad tragedy. Preventable, like most accidents, are, and senseless. And it ended up being the kind of call that had the “despite the best efforts of those on the scene, and at the hospital, I’m very sorry, but…” outcome. It was perhaps the worst of my years in. I still recall going home to hug my wife that night and sobbing while squeezing the life out of her after that one. I still recall that moment when “caregiver” mode was off, and “clean up the ambulance and put it back in service/document the call” mode started.
During that call, though, the team worked. All of the team – Fire/EMS/Paramedic Intercept – functioned as a team. And I didn’t witness any signs of folks making this “Their emergency.” We worked incredibly well as a team. The machine working. Because no one was “in the emergency,” – we were able to do all that could possibly be done. Everything was done to provide for a fighting chance. Times were limited on scene and en route so the best and most definitive care could be obtained at the hospital. In the hospital, the doctors and staff were able to triage, contemplate, decide, assess, discuss rapidly, and ultimately make a tough but essential call. I remember the emotions of that night for sure. But I also remember the machine working. The team working. Despite the massive adrenaline dumps we were all experiencing, and the patient’s age’s with added pressure, all the steps were done. And we were able to function. That’s perhaps the biggest sign that we weren’t “in the emergency” – we were actually able to function and try to make a difference.
Not all “Better calls” end up better afterward. Sometimes they still go the way they were going before we got there despite our efforts. And that was hard, especially in a small town. But that one call was also the moment that I knew I would be okay going on to the next level of license and continuing in the field.
Bring it back to DBAs, Mike!!
My premise – “Putting distance between you and the emergency is a key “skill” for an effective troubleshooter” – isn’t just for the EMT or Firefighter among us. It’s 100% true for consultants, DBAs, technologists, and anyone who is fixing others’ problems for a living.
While there is a progression of effective troubleshooting steps, the steps are useless if you are prone to skipping this principle. You can be the most capable technologist, with solid skills, vast knowledge, and even the best google searching skills out there and fail miserably if you ignore this rule.
A troubleshooter who is “in the problem” is stressed more than one who is out. Let me be clear – there will always be stress when dealing with a high priority problem! But the stress is very different when you tell yourself, “IF I DON’T FIX THIS ____ WILL HAPPEN! AND THEN _ _ ___ WILL HAPPEN! AND IT IS GETTING WORSE! AND OH, NO!! HOW WILL I ______ AND WHAT HAPPENS IF WE DON’T ___!?!? WHAT WILL THEY THINK!? WILL I EVER FIX THIS?!!? IF I DON’T FIX THIS!!!!” compared to saying, “Okay. This isn’t good. This stinks. But this isn’t life or death for me. We will do the best we can, and it is what it is.”
The first thing that happens when you put yourself in the problem is stress. The next thing that happens is tunnel vision. You can lose your periphery. You forget the big picture stuff, and you may miss some obvious stuff. Surely a problem that has you that worked up can’t be a simple problem – so you end up running right past the simple explanations and troubleshooting steps – looking for a rare needle in the haystack like problem with a correspondingly complex situation.
After that tunnel vision – panic. And I mean panic like a feeling of losing control after that comes blamestorming, missing the mark and running round and round, probably doing the same few things over and over.
Finally – you make the problem worse. Somehow. Someway. You likely are doing something now to make the problem worse. You’ve so bolted yourself to the outcome and the issue – that you don’t take a break to clear your mind and retrace the steps. You don’t ask for help because “this is my problem.” You end up making some silly mistakes, and at worst, you can cause further damage (I’ve been in way too many times after someone got stuck here and ended up doing odd things – like dropping a database without need and without verifying recent backups; detaching a totally corrupt – but recoverable database; taking a recoverable instance and doing who knows what to make it totally unrecoverable). At best – you increase the time to solution and increase the downtime.
Can We Stop It?
Yes. It’s a learned behavior. That first serious ambulance call where I was the lead tech? It was stressful – I lied to myself a bit and almost got sucked in. I’ve been there when troubleshooting a problem as a DBA (or worse, as a consultant) that I caused. I believe this is a skill that doesn’t just have to be taught – it more has to be learned, experienced and practiced intentionally.
A few steps to get us down that path:
- Breathe – Seriously – you’d be surprised how tight your shoulders are and how much you are holding your breath when heading into this path. Tell yourself to breathe. Relax your shoulders.
- Look Up – The “High Port” position Jocko Willink has described in some books and his podcast. Instead of looking down the scope of a rifle. Reset yourself and look around. Focus on all the info.
- Assess – and Reassess – Get back to the basics. “What happened first? What were the steps? What happened next?” gather details and facts.
- Name It – Tell yourself in your own head (or aloud if you must!) that “This is not my emergency.” – free yourself from the outcome. This situation is bad. It’s not great. It’s not heading towards an ideal spot – but it’s not “yours,” as in your life, your career, your health will go the way of this situation. You aren’t freeing yourself from the responsibility of doing your best – but you are stepping outside of the problem and reminding yourself that the outcome is less up to you than the steps and approach you take. Freeing yourself from the outcome, paradoxically, will better allow you to control and resolve the situation at hand.
- Communicate – “Hey. I’m getting stuck here. Help!” goes a long way. At Straight Path, I try and have two techs on serious P1 client issues. It’s not to rack up the bill (quite often when I’m the second tech, I’ll look the other way on some of my time, I work insane hours, so my hourly rate isn’t really that high 😉 ) – it’s because two sets of eyes working on an issue help make sure that someone can stay focused on bigger picture stuff (Communication with the client, looking at the big picture, watching the clock.)
- Timebox – Setup predefined “stop points” – in the fire service, it’s quite common for dispatch to radio to the incident commander on a scene and says, “You’re at your first 20-minute mark. Do you want us to continue calling those?” or even “First 20-minute mark – please do a PAR check” – this tells the incident commander to “pop the stack” and go to high port. Look at the big picture, check out the overall scene safety, look at the progression, find out where folks are, assess if you have the right resources, and have all the units conduct a Personal Accountability Report check – make sure that all the staff is where they should be, and no one has become missing. In tech – that’s setting up some targets for when you work on something, when you ask for help, when you go to the next level, etc. For example – if you spend 15 minutes and still have no clue what’s going on, try searching. If you spend 15 more and don’t – ask for help. Suppose you spend 30 together and don’t – escalate/etc. Setup windows where you’ll stop, look around and ask for help.
- Watch – Be there for the one who is the primary on an issue – be helpful outside marking time, asking if they are all set, trying to look at the big picture, etc.
There are probably many other things that can be done here. But the main point is – destress, formulate an attack plan, execute it calmly (but swiftly!) and be aware enough of yourself and your situation to know if you are starting to head down a bad path and get lost inside of the issue yourself.