A hack for reviewing empirical literature for evidence-based social work practice in the field
Our social work Code of Ethics calls on us to be evidence-based in our practice, so that we can demonstrate our familiarity with a given practice area and its scholarly references, but there’s a lot of confusion out in the field about how to do this, based on what I see out there with my students and graduates. Not to mention the fact that I see a lot of resistance to the nature of the task of reviewing the literature in the quest of becoming evidence-based. This post is here to help, and recognizes that practitioners exist in a productivity-driven, fast-paced, time-starved and resource-short environment.
In order to have this conversation, we need to start with two definitions. We need to know what an empirical study is (because that’s the literature we want to focus on) and we need to know what a social work intervention is (because that’s the type of empirical study we want to find).
Empirical study: This is a study that is either based on primary or secondary data collected from direct observations in the field that is analyzed by the study authors. It will have a methods section that includes a sampling approach and a data analysis section. It is not an opinion article or essay that includes statistical data.
Social work intervention: According to Sundell and Olsson (2017) “In social work, interventions are intentionally implemented change strategies which aim to impede or eradicate risk factors, activate and/or mobilize protective factors, reduce or eradicate harm, or introduce betterment beyond harm eradication; thus social work intervention encompasses a range of psychotherapies, treatments, and programs.”
Many social work clinicians read materials that are not empirical intervention studies, which is a less than ideal prospect. It’s important to sift through all the commentary literature out there to find the empirical intervention study gems. And it is true that reading through the empirical literature on a given social work intervention for a particular diagnosis strikes fear into the hearts of many social workers who are phobic about reading statistics-heavy research and evaluation articles and reports. Schools of Social Work can support students and alumni in learning to demystify and translate the language of research and evaluation so that nascent clinicians can make good on their commitment to be evidence based, and that’s why you are here reading this today. We should be offering more CEU workshops on this topic to help our grads in this area!
In order to start to consider the empirical literature base on any given intervention or set of interventions that might be appropriate for a client, social workers should amass the relevant articles and reports on the topic. As a student, you will have access to your university library database. You can watch my short screencast with a hack on how to search quickly and effectively here. Having access to literature post graduation is a common complaint that many raise, although this is less of an issue now that we have freely-accessible information from the Cochrane Library, the Campbell Collaboration, the Society of Clinical Psychology’s list of empirically supported interventions, my own evidence based practice page and so on. Nowadays, many towns’ public libraries also have access to academic databases, so access to clinical journals is much more of an option for community-based clinicians than it was before as well.
As you begin reading through your materials, you can also begin the process of writing. Start to take notes that you can incorporate into the text of your literature review later on, if you plan to write something up to share with colleagues. The best way to do this is by creating informal annotations for each article. This can help you to remember what you read about and can save time later in the process of writing.
Part of approaching the task of literature review is knowing what to look for in the articles and reports that one finds. I argue that while a full academic literature review requires reading and understanding the *full* article as it relates to other articles in the stack, identifying themes, debates and gaps as they go, social work clinicians can take a slightly different approach for field-based empirical literature review.
In my opinion, a clinician with little time for reading, much less writing, should focus in on four things: (1) what intervention/s was/were tested, (2) who was sampled in the study, (3) what measures were used in the study (if a quantitative study was conducted) and (4) what the results of the study were). Each of these four areas should be thoroughly critiqued, as follows:
Understanding the intervention/s studied: One of the main reasons to dig into the literature is to get a sense of the range of interventions that are used to treat a given condition or presenting problem. You want to get enough of a sense of what these interventions are, what is actually done with clients clinically, that you could begin to replicate those actions. This also means that you need to get a sense of the research question that guided the study of this intervention.
Examining the sample: One of the most important things you can do in reviewing an empirical intervention study as a clinician is to examine the methods section in order to see who was included in the study. In other words, what were the inclusion and exclusion criteria for the study’s sample, and specifically for the treatment and/or control group, if any? Once you identify those things, the key thing you want to think about is, how does that study’s sample generalize to my own clients, or does it? This is a question that is not often posed when we think about evidence based practice. Too often, we just say “oh that is an evidence based practice because a study found it effective,” and we don’t think further about which population it was found effective on. We can do better than that!
Considering the measures utilized: Another reason we review the empirical literature as clinicians is to see how other evaluators and researchers have approached measuring process and outcome measures in their studies. We do this so that we can use those same approaches, if possible, in our own evaluations, for the purpose of comparability. It is not only ethically mandated for us to evaluate our practice, but it is a best practice to relate our own evaluations back to the existing literature, which is best done when using similar measurement instruments. Seeing which instruments are most commonly used can help you with this process.
Critiquing the study’s results: Perhaps the most important aspect of the literature review is seeing how effective any given intervention was on particular client populations. Effectiveness may differ based on whether treatment was inpatient or outpatient, individual vs. group treatment or on timeframe, for example. It is important to connect information about study results back to information about the study’s sample as well. Noting whether statistical significance between groups or timeframes is present is important, understanding that sometimes the lack of statistical significance is a good thing, such as when a new treatment is compared to an existing well performing treatment. And of course, considering not only statistical significance between groups or timeframes but also clinical significance or meaningfulness of the clinical scores (and effect size data, if available) is vital to a thorough literature review as well.
Synthesizing the results – or – telling the story of the literature:
After each of these four elements of all relevant articles are considered, the clinician should synthesize the results by comparing and contrasting the studies they have read in order to pull it all together by “telling the story” of what the articles, taken together, say about the intervention/s in question. Above all else, do not write a list of the articles. If nothing else, you will bore your professor to death (if not yourself).
To begin organizing what the literature says into a story, the writeup must demonstrate the connections and relationships between the articles and reports that have been considered. Some of the things to look for in your synthesis are: patterns or trends in study results, or samples used; themes in use of measurements or outcome found; debates about outcomes or gaps in study samples used. After identifying these items, a brief introduction should be written, before narrating the story, which can be done chronologically by publication date, thematically, methodologically or theoretically.
Telling the story of the literature chronologically is to follow the origins of the topic over time, but you want to do more than just present a list, which is really boring to read. Instead, think about presenting an analysis of patterns noted over time, perhaps commenting on turning points in the research, or key controversies or debates that have impacted the particular field of practice.
If you are taking a thematic approach to telling the story of the literature, you can organize your writing into sections that talk about different parts of the story. For example, if you are reviewing literature about interventions for improving mental health outcomes in immigrant communities, important themes might include mental health care policy interventions at the macro level or interventions that take into account cultural attitudes and beliefs at the micro level.
Using a methodological approach to organize data may also be an option to consider, comparing qualitative vs. quantitative research and evaluation literature for example. However, most of the empirical literature on clinical practice outcomes in social work will be quantitative.
To start telling your story, you might mix several of these approaches. Your overall structure might be thematic, for example, but each theme could be talked about in a chronological order, for example. After telling the story of your literature, pull it all together with a brief introduction and conclusion to share with your colleagues at your agency. This literature review grounds you in what the field has found helpful and not so helpful. It helps you to see who the literature has studied and not studied – and whether those populations are generalizable to your program’s population or not. It also helps you to determine which measurement instruments you might use to evaluate your own interventions, something else you are bound to do according to our Code of Ethics, but that’s a post for another day.
Good luck with reviewing the empirical intervention literature – and with sharing it with your colleagues to spread the evidence-based wealth and lift our profession to a better place!