We believe in continuously evaluating the work of End Hep C SF to determine if we are making an impact and where we could do better.
End Hep C SF is a collective impact initiative, which means there are many partners at the table, all bringing different assets, but with the same goal – eliminating hepatitis C virus (HCV) as a public health threat in San Francisco. We use a systematic, ongoing process of evaluation to generate information that is fed right back into the initiative. It allows us to make our collective decisions based in evidence, and not just gut instincts or educated guesses.
We use Results Based Accountability (RBA) as our evaluation framework, because it holds us accountable for making a difference in the lives of people affected by HCV, while allowing us to learn, improve, and tell our story along the way.
RBA is a trademarked system for evaluation, developed by Mark Friedman and described in his book Trying Hard is Not Good Enough. It is especially suited to collective impact work because it looks at both individual programs or efforts (i.e., those of the individual End Hep C SF partners), as well as the collective impact of our combined efforts. The idea is that if we all work together to design and implement our programs to achieve the same goal, and if each partner’s individual program is strong, we will “turn the curve” on the things we care about.
Our Performance Measures assess the activities and programs that the various End Hep C SF partners are doing to contribute to our shared goal. These measures tell us: (1) How much did we do?, (2) How well did we do it?, and (3) Is anyone better off?
But we don’t just stop there. We want to see how End Hep C SF is making a difference at the population level. Are we actually impacting the health of the whole city when it comes to hepatitis C? Our RBA Indicators—HCV incidence (new cases), prevalence (total cases), morbidity (health problems due to HCV), and mortality (HCV-related death)—help us see if people who have or have had HCV are generally better off because of our work.
Our Year 4 RBA evaluation tells a compelling story about how the COVID-19 pandemic dramatically impacted our ability to turn the important curves in the right direction.
Year 4 was hugely affected by COVID-19 and San Francisco’s shelter-in-place order. We saw a big decrease in HCV-related services that could be provided in 2020, though by Q3 things were starting to rebound.
We invite you to engage with the entirety of our RBA indicators, performance measures, and stories about our progress. Part of holding ourselves accountable to making a difference is by being transparent about our successes and failures, how we respond to them, and whether we are making a difference for people affected by Hep C in SF.
End Hep C SF is tracking all of our Indicators and Performance Measures using a scorecard hosted by Clear Impact, a software program designed for use with RBA. The scorecard is below, and is updated once per quarter as we get new data.
When you initially look at it, you can see the list of Indicators and Performance Measures we’re tracking, and to the right you can see the current trend (up or down), and how much it’s changed since we started tracking it (“baseline”). If it’s red, it’s currently heading in the wrong direction. If it’s green, that’s a great sign!
To the left of each of the items, you can click the “+” sign, and it will expand that section into a graph (that’s how you can see the “curve”), and give you more information about the trend at each timepoint. Importantly, below each graph is a series of tabs:
- Why Is This Important? (Why are we tracking this; what does it tell us?),
- Story Behind the Curve (Why do we think we’re seeing the trends we are? Sometimes a curve is heading in the wrong direction, but that’s because something happened – like a global pandemic! – and we’re confident it will turn around again.)
- Technical Notes (What are the details about this dataset? This is for the data nerds out there.)