Five Lessons Learned About Antimicrobial Stewardship Metrics

In this article an experienced antimicrobial stewardship pharmacist discusses lessons learned from navigating antimicrobial stewardship metrics.

Authored by: Timothy P. Gauthier, Pharm.D., BCPS, BCIDP

Article posted 13 December 2024

If you cannot measure it, you cannot determine if you have changed it.

Antimicrobial stewardship metrics are essential for identifying, acting on, and re-evaluating opportunities to enhance the safe and appropriate use of antibiotics. Without data all we have is opinion. To be successful and gain the trust of critical stakeholders, being data-driven is a must for antibiotic stewards.

Data in the realm of antimicrobial stewardship can come in many shapes and forms. It may be derived from work completed in the microbiology lab, units dispensed from the pharmacy, administrations performed by nurses, expenditures produced from drug purchases, documentation within any area of the electronic medical record, extraction from supplemental tools or data warehouses, and so much more.

Antimicrobial stewardship metrics are not simple. In my ~15 years working in the area of infectious diseases and antimicrobial stewardship I have learned a great deal and continue to learn more each year. With that in mind, I have composed this post in the hopes that it may help others who work in this space. Here are five lessons learned about antimicrobial stewardship metrics.

1. Every system is perfectly designed to get the results it gets

This is a quote from William Edwards Deming that says it well. Your data is only as good as the source you get it from! Understanding where the data comes from is critical to determining how valid it is. Note that it is not a yes/no on this, it is more on a spectrum and that may change over time as the inputs for the data evolve. All systems have limitations and potential confounders.

I have learned to never ever accept data someone sends you as truth. In fact, it is extremely common for data to be completely misleading. It is critical to recognize that the data source and system it is filtered through will determine the quality of the data.

Along this line of thought, I have also learned to be gentle with how I communicate my lack of confidence in data that is shared with me. It is first important to be grateful to receive information from your colleague. Once the person helping you knows that you appreciate and respect them, then you can start to dive into it with a critical eye. Often times the folks who provide data will be able to help us identify the methodological limitations used to develop it.

2. Communicating antimicrobial stewardship metrics gives us a chance to tell our story – and it needs to be done carefully because context matters

Anyone who has worked with CDC NHSN AUR SAAR data knows that is not very good. I feel it is accurate to describe the SAAR as a blunt tool for measuring antibiotic use. It does not account for indication or whether the antibiotic use was appropriate or not. It can at times be easily impacted by a few patients on appropriate therapy that require a prolonged course. When it comes to the SAAR, if it is at 1 then it’s expected antibiotic consumption, below one is less than expected, and above 1 is more than expected. So above 1 must be bad right? Well not exactly. Sometimes it’s fine – but beware your audience may see something above 1 and just jump to the conclusion that there is a problem.

When we are forced to used data with major flaws, it can be easy for folks we share it with to misinterpret it. When it comes to blunt tools for stewardship like the SAAR, I have learned it is better to use the data to tell a story than to try to get into the weeds on the specifics. Telling your story does not have to be complicated, but it does need to be fair and align with the needs of the organization or facility.

They say that he who controls the narrative controls the people. I think there is an element of truth here with stewardship metrics, but it is more about getting the team to understand the story so that you can all work more effectively together. It’s not about manipulating data to control people – that would be ill-advised.

3. Antimicrobial stewardship metrics are never perfect

Someone once told me that you have to stop when your work is “good enough” and not to chase perfection. Kind of like that saying – don’t let perfection be the enemy of progress.

Antimicrobial stewardship pharmacists have to balance clinical duties with programatic duties. I have learned that focusing on producing metrics that are the highest impact must be prioritized. You may need to evaluate a whole slew of metrics, but you do not have to prepare graphs for everything when it comes time to make the presentation.

The metrics we use are also commonly imperfect. Knowing when the data is reliable enough to use is another thing to beware.

Accepting that your metrics are never going to be perfect is important for developing realistic expectations for yourself and your antimicrobial stewardship program. Developing reasonable exceptions is critical for mitigating burnout.

4. Communication strategies on antimicrobial stewardship metrics should vary depending on the audience

In my experience pharmacists want all the details, physicians do not want to be told what to do, nurses are very oriented to bedside care, and leaders want the cliff notes of the cliff notes.

It is essential to adapt presentations of data to the audience. When preparing content keep your core goal in mind. Many people will not walk away with more than a couple take home points. You may find yourself adding slides and data that are more for you than for your audience. It has to be enough for them to understand it, but it does not have to be so much that it totally confuses them. I confess I am still working on this and it is really hard when our data and methods can be so complex. At the end of the day though, is it better to have a complicated presentation that nobody understands or a simplified presentation that people can follow? One strategy is to close your presentation by providing people with a few talking points based on the data. This is a strategy particularly helpful for when breaking down antibiogram data.

When developing presentations try to put yourself in the shoes of your audience. Let their interests guide what you create.

5. The pursuit for antimicrobial stewardship metrics is never-ending

Not only do stewards need to continuously seek out new data sources, we also need to endlessly seek to optimize the ones we already have. I like to shoot first for the most accessible data, even if it has a bunch of limitations, because something is better than nothing. More substantial data requests often take time to fulfill, so I think of those as longitudinal tasks. For making enhancements to existing data sources, those are often associated with one-off opportunities that arise, where something else is being updated and you can plug in a few add-ons.

A lesson on this topic is rather than try to develop new data sources, first explore what your colleagues are using. This means going to different departments, not just looking at pharmacy department reports. Infection prevention and control, quality assurance and performance improvement, and many other departments may have existing data sources/ reports that you can gain access to for stewardship purposes. Just like antimicrobial stewardship is a team activity, try to make stewardship data analysis a team item too. This also means stewardship programs should offer their data sources to other departments.

As you work with your data, beware use of the calendar year versus the fiscal year in your metrics. I have seen that trip people up and interestingly not every institution even uses the same months for their fiscal year.

As you get more data, take note that some of these semi-flawed metrics are better to measure over quarters rather than months, because the variability from one month to the next can be pretty substantial. Working with antimicrobial stewardship metrics is not a set it and forget it type of thing. It requires curiosity and innovation to do it well.

Closing comments

Working with antimicrobial stewardship metrics can be pretty challenging, but it can also be really fun to gain a greater understanding of the puzzle that is your healthcare organization. There is a lot of opportunity for improvement of antimicrobial stewardship metrics within various care settings. Perhaps one day you will help create a new and better one for us all!

Readings & Resources

Disclaimer

The views and opinions in this article are those of the author. They do not necessarily reflect the policy or position of any previous, current, or potential future employer.

RECOMMENDED TO YOU