Vivacity Consulting

Bringing energy and experience to your CME activity

Using Data to Improve Medical Education

By

A Case-Study: Using Data From Participant Evaluations

When the ACCME came out with the new Criteria (back in the 2000s!), many of us from  non-hospital-based organizations struggled to understand how to incorporate the concept of using data into our planning process. I remember whining a lot. “We’re a specialty society. We don’t have patient data!”  Not only that but, “we only see our attendees one time a year!”

Flash forward 7 years. The SAGES Continuing Medical Education Committee is pleased to announce the publication of the article entitled, “Evolution of practice gaps in gastrointestinal and endoscopic surgery: 2012 report from the Society of American Gastrointestinal and Endoscopic Surgeons (SAGES) Continuing Education Committee.” In this article, we describe the process by which we have collected data from our attendees through post-activity surveys, analyzed this data by “Learning Themes,” and then identified potential practice gaps which have or may be addressed at future educational activities.

If you have a subscription, you can access the article here:

http://link.springer.com/article/10.1007%2Fs00464-013-3263-2

I just read that some journal publishers are getting tougher about copyright violations, so message me if you would like me to send you a “draft” version of the paper. I’ll present a summary of this work during CMEPALOOZA, a web-conference being held March 20, 2014.

And congrats to my co-authors who did the bulk of the data work,

  • John T. Paige, MD
  • Timothy M. Farrell, MD
  • Simon Bergman, MD
  • Niazy Selim, MD
  • Alan E. Harzman, MD
  • Yumi Hori,
  • Jason Levine,
  • Daniel J. Scott, MD

Turns out specialty societies actually do have the ability to collect valuable data, and there are ways of using this data to, dare I say, improve the education delivered.

Wait a Minute: Rethinking the ACCME Accreditation Revisions

By

Rethinking the ACCME Accreditation Proposed Revisions

Today, I held a webmeeting with three of the smartest, most enthusiastic, ready-for-anything physician volunteers you can imagine.  We were conducting the overall mission assessment required by ACCME Accreditation Criterion 12 as the first step in a multi-month reaccreditation process. And it struck me.  If we eliminate C14 & 15, some of their motivation to get these changes accomplished might be reduced.  They are fired up to get these changes in place now so that we can measure the results.  Part of their enthusiasm stems from their core belief in the CME process, part of their enthusiasm is justifiably self-serving (they want to write a manuscript which may help them get promoted), but part of their enthusiasm, I really believe, comes from the fact that IT’S REQUIRED.  It’s human nature to respond to a deadline, and they know theirs is April 2014.

Maybe eliminating C14 & C15 might not be such a great idea.

ACCME reports that in the November 2012 cohort, 20-30% of providers were non-compliant with C15.  This could be a matter of timing (they didn’t start evaluating their program early enough to make changes and measure the effect.)  But it could also be because they missed the point.

C15 states, “The provider demonstrates that the impacts of program improvements, that are required to improve on the provider’s ability to meet the CME mission, are measured.” Without this, might we inadvertently stall the cycle of continuous improvement?

If we need to rethink this, what about the rest of the proposed changes?

I know that my job is easier when the rules are explicit – and Dr. Kopelow told us on the May 23, 2013 provider webinar that this is true for everyone. My smart, enthusiastic, BUSY physician volunteers would have been baffled if I tried to convince them that a mission statement should include anything more than our expected results … because “it doesn’t say so in the rules.” To me, it makes sense that we will continue to describe our purpose, content areas, target audience and type of activities in the mission statement, but to the person who doesn’t think about CME everyday, this might seem like a massive waste of time.

So, maybe ACCME Accreditation Criterion 1 should be left alone as well.

What do you think?  Make sure you submit your comments to the ACCME by July 2nd!

ACCME changes review

ACCME accreditation revisions – provider feedback due July 2nd!

 

From CME Police to CME Star

By

From CME Police to CME Star

How many of you feel like your committee of physician volunteers look at you and see this:

CME police

CME police

 

 

 

 

 

 

 

It’s not very comfortable to wear that hat day in and day out.

There are ways to re-frame your position, however, so that you can demonstrate the value of the CME planning and documentation process.

I recommend a basic training (1 hour webinar or in person) for any group of new or revamped CME stakeholders.  Sometimes, this training starts with the image of the Wall Street Journal, and the question “Does anyone wish to see our organization’s name in a headline here?”  After reminding (or enlightening) these stakeholders about why the rules exist, suddenly, we are much more clearly on the same page – because none of us want biased education to be presented in the name of our organization.

Second, I give an overview of the vernacular used in CME.

(By the way, how often do we inadvertently contribute to our own problems? Here’s an example of what not to write to your Course Director:

“Dear Dr. Scott, 
The ACCME requires that you review the attached disclosures for your planned enduring material. If you find COI, you will need to review the presentation to ensure we stay compliant with the SCS.”

The bold words are NOT self-explanatory. Nothing alienates a motivated, intelligent volunteer more than feeling like they cannot understand what in the world your email means!)

Finally, I outline the process our organization follows to ensure compliance with all of the rules we must follow, demonstrating what we need from the planners and what we, from the CME office, might provide.  Often, when I show examples of the treasure-trove of data that I can provide from post-activity evaluations, I see wonder and excitement in the eyes of previously less-than-enthusiastic committee members.

The results: more engaged and empowered stakeholders.  And maybe a new hat for you.

CME star

CME star

 

 

 

 

 

 

 

 

 

If you’d like help putting together a training like the one I describe, please contact me!