We believe in continuously evaluating the work of End Hep C SF to determine if we are making an impact and where we could do better.
End Hep C SF is a collective impact initiative, which means there are many partners at the table, all bringing different assets, but with the same goal – eliminating hepatitis C virus (HCV) as a public health threat in San Francisco. We use a systematic, ongoing process of evaluation to generate information that is fed right back into the initiative. It allows us to make our collective decisions based in evidence, and not just gut instincts or educated guesses.
We use Results Based Accountability (RBA) as our evaluation framework, because it holds us accountable for making a difference in the lives of people affected by HCV, while allowing us to learn, improve, and tell our story along the way.
RBA is a trademarked system for evaluation, developed by Mark Friedman and described in his book Trying Hard is Not Good Enough. It is especially suited to collective impact work because it looks at both individual programs or efforts (i.e., those of the individual End Hep C SF partners), as well as the collective impact of our combined efforts. The idea is that if we all work together to design and implement our programs to achieve the same goal, and if each partner’s individual program is strong, we will “turn the curve” on the things we care about.
Our Performance Measures assess the activities and programs that the various End Hep C SF partners are doing to contribute to our shared goal. These measures tell us: (1) How much did we do?, (2) How well did we do it?, and (3) Is anyone better off?
But we don’t just stop there. We want to see how End Hep C SF is making a difference at the population level. Are we actually impacting the health of the whole city when it comes to hepatitis C? Our RBA Indicators—HCV incidence (new cases), prevalence (total cases), morbidity (health problems due to HCV), and mortality (HCV-related death)—help us see if people who have or have had HCV are generally better off because of our work.
Our Year 4 RBA evaluation tells a compelling story about how the COVID-19 pandemic dramatically impacted our ability to turn the important curves in the right direction.
Year 4 was hugely affected by COVID-19 and San Francisco’s shelter-in-place order. We saw a big decrease in HCV-related services that could be provided in 2020, though by Q3 things were starting to rebound.
We invite you to engage with the entirety of our RBA indicators, performance measures, and stories about our progress. Part of holding ourselves accountable to making a difference is by being transparent about our successes and failures, how we respond to them, and whether we are making a difference for people affected by Hep C in SF.
End Hep C SF is tracking all of our Indicators and Performance Measures using a scorecard hosted by Clear Impact, a software program designed for use with RBA. The scorecard is below, and is updated once per quarter as we get new data.
When you initially look at it, you can see the list of Indicators and Performance Measures we’re tracking, and to the right you can see the current trend (up or down), and how much it’s changed since we started tracking it (“baseline”). If it’s red, it’s currently heading in the wrong direction. If it’s green, that’s a great sign!
To the left of each of the items, you can click the “+” sign, and it will expand that section into a graph (that’s how you can see the “curve”), and give you more information about the trend at each timepoint. Importantly, below each graph is a series of tabs:
- Why Is This Important? (Why are we tracking this; what does it tell us?),
- Story Behind the Curve (Why do we think we’re seeing the trends we are? Sometimes a curve is heading in the wrong direction, but that’s because something happened – like a global pandemic! – and we’re confident it will turn around again.)
- Technical Notes (What are the details about this dataset? This is for the data nerds out there.)
Program Performance Measures
The Program Performance Measures dashboard tracks the progress that End Hep C SF’s partner programs have made toward achieving the overall goal of eliminating HCV in San Francisco. The Performance Measures (PMs) listed below show key data points from the partner programs. These programs are funded by the San Francisco Department of Public Health (SFDPH) and are located in non-traditional testing sites at San Francisco-based community-based organizations. By providing HCV testing in non-clinical sites where folks access services for a variety of reasons, we can reduce barriers to HCV testing.
By clicking on each PM, you can see a graph of our progress over time, information and context about the data, and the strategies we are planning to “turn the curve” in the right direction. Several graphs include data on subpopulations that are more affected by HCV than the general population, including People Who Use Drugs (PWUD) and People Experiencing Homelessness (PEH). To see subpopulation testing data in relation to the overall testing metric, click on the box that says ‘comparison’ at the top left-hand side of the graph. Please note that all data is considered preliminary until three months after each quarter ends and is usually an underestimate of what the final number will be.
San Francisco’s Hepatitis C Data Dashboard
This data dashboard, called San Francisco’s Hepatitis C Data, shows San Francisco’s progress toward eliminating HCV and its harmful effects over time. By tracking Indicators (I) of our Desired Results (R), we can get an idea of how the city is doing in reducing new infections as well as HCV-related illness and death. By clicking on each indicator, you can see a graph of the city’s progress over time, information and context about the data, and the strategies we are planning to “turn the curve” in the right direction. While these population-wide indicators include both people who have received services from one of EHCSF’s programs as well as those who have not, our goal is for EHCSF to have the most significant possible influence on these indicators. Tracking this data can show us whether the city is headed in the right direction and lets us know if we should make program adjustments to help reverse undesired trends.