Showing posts with label evaluation. Show all posts
Showing posts with label evaluation. Show all posts

Wednesday, March 28, 2012

"Hashtag Activism"- Is It Working For Public Health?

Last year on Facebook, my timeline was suddenly filled with status updates of just one word.  Red. Black. White.  "What is this?" I thought to myself.  It turned out that it was a breast cancer awareness campaign.  Facebook users were listing their bra colors to help prevent breast cancer.  And there are tons of these public health campaigns all over social networking sites.  "Like" our Facebook page to prevent cancer!  "Re tweet" this message to prevent heart disease!  For those of you familiar with my blog, you'll remember that I think "raising awareness" is the most counter-productive phrase used in our work.  It is not specific enough to measure for change and (on its own) it will not change health behaviors.  

So I thought of my frustration with these campaigns as I read a great article in the New York Times this week called, "Hashtag Activism, and Its Limits".  David Carr writes eloquently about the ease of supporting a variety of causes:

"If you “like” something, does that mean you care about it?  It’s an important distinction in an age when you can accumulate social currency on Facebook or Twitter just by hitting the “like” or “favorite” button.

The ongoing referendum on the Web often seems more like a kind of collective digital graffiti than a measure of engagement: I saw this thing, it spoke to me for at least one second, and here is my mark to prove it".

I like that David brings up the question of engagement here.  Many of these public health campaigns on social media just strive for "likes" on Facebook or "hits" on a website or "re tweets" on Twitter.  And not that they mean nothing, but those measures are just the tip of the iceberg in measuring audience engagement.  And audience engagement (beyond "raising awareness") is what could actually lead to public health activism, knowledge change, and ultimately behavior change. Leslie Lewis gives a great overview of Return on Engagement (ROE) on her blog "digital.good".  According to Leslie, ROE measures tend to be more qualitative and measure message reach and spread.  In addition to "likes", ROE also measures things like brand/campaign awareness, comments, shares, and returning visitors. 

I certainly do not think that all public health campaigns delivered via social media are ineffective.  On the contrary, I think that social media is an incredibly powerful tool for public health.  David Carr makes similar comments later in his article.  Challenging his initial skepticism of web activism, he lists several recent "campaigns" that have been quite effective (e.g., the reversal of Susan G. Komen de-funding Planned Parenthood).

However, to use social media effectively in public health, we must be strategic and we must evaluate.  

Some sample questions that I ask program planners:
  • What is the goal of the campaign?  (e.g., to drive traffic to your website; increase hotline calls; increase specific behaviors like breast self examination?).
  • How will the campaign activities (logically) lead to the desired goal/outcome?
  • Are your goals/outcomes measurable?
  • Have you thought about evaluation before launching the campaign?
  • Besides the ideas listed above, how are you measuring "audience engagement"?
 
 What other questions should we be asking?

Tuesday, August 23, 2011

The East Coast Earthquake, Real Time Twitter Chat, and Facebook Applications for Disasters

Well! Today was an interesting day at the office. Up and down the east coast, many of us felt the tremors resulting from an earthquake in Virginia. While I would like to report that we all stayed calm and participated in orderly, safe, and well rehearsed evacuations...that was not the case. It appeared that the shock of experiencing an earthquake (such a rare event on the east coast) caused a little chaos. On my way into my office to grab my bag before hitting the stairs, I experienced a "George Costanza" type moment as a fellow staff person almost knocked me down in her rush to get out. I heard similar stories from my husband who works 4 blocks away. Upon recognition of the earthquake, his co-workers made a beeline for the safest escape route...the elevator?!



After the shaking took us down 13 flights of stairs, I quickly turned to the only reliable source for real time information- Twitter. Since I was the only one in the area who either grabbed my phone or had twitter, I quickly read off what I knew: "It is a 5.9 earthquake in Virginia"; "My colleagues felt it in- Baltimore, DC, Boston, NYC, North Carolina"; "No damage except one broken window is reported in Philadelphia". After being given the go ahead to return to the building and settling back into our work, we received an official text/email from the University reiterating the information Twitter delivered an hour before. According to Twitter's official profile tonight, within one minute of the #earthquake, there were more than 40,000 earthquake-related tweets. They reached 5,500 tweets per second (TPS).



As I discussed in a related post back in March 2011, the question for public health professionals continues to be- "What is the role of social media in emergency preparedness and recovery?"



I believe we are making some strides in answering that question. Just yesterday, the Office of the Assistant Secretary for Preparedness and Response (ASPR)- located within the US Dept of Health and Human Services (HHS)- launched a contest: The ASPR Lifeline Facebook Application Challenge. The goal of the contest is to create applications that prepare individuals for disasters and build resilient communities. Those who opt into the application will be able to identify "lifelines" or Facebook friends that agree to be an individual's emergency contact and act on their behalf in case of an emergency. They will also be able to create a personal preparedness plan and share that plan and the application with others.



Even without the formal application, we have seen social network sites be used for checking in with friends/family and for getting information out quickly. For example, I follow the Philadelphia Office of Emergency Management on Twitter, so I got the message quickly that our 9-1-1 system was being inundated with calls since the earthquake and we should only use it with a real emergency...for infrastructure damage, call 3-1-1 instead.



While the Facebook application sounds like a great addition to emergency preparedness, it is important to also consider implementation issues which will impact its reach and effectiveness:

  • Is the application only available to Facebook members who download it ahead of time? Or will it be available to anyone via the mobile web?
  • Do these Facebook members typically update their profile via mobile devices in addition to stationary computers (which may not be available during an emergency)?
  • During the emergency, are there cell networks/wifi to support the communication? (e.g., many reported that cell networks were jammed immediately following the earthquake)
  • Do these "electronic" preparedness plans need to be rehearsed the same way as "in person" plans in order to increase effectiveness?
The HHS-ASPR contest runs August 22-November 4, 2011. I look forward to seeing the winning applications and hearing about how the dissemination will be conducted and evaluated.



What were your experiences today during the east coast earthquake? What did you hear/see from your colleagues? How did you get/send information to others? Please share in the comments section below.



Sunday, May 8, 2011

Will New Ads in Georgia "Stop Childhood Obesity" or Increase Stigma and Bullying?

On Friday's Today show, there was an interesting analysis of a new campaign from the Georgia Child Health Alliance (GCHA) aimed at reducing childhood obesity. According to the GCHA website, the Warning: Stop Childhood Obesity media campaign "is part of a large-scale public awareness campaign designed to educate Georgians on the childhood obesity epidemic facing our state. Backed by market research, the campaign’s warning messages about obesity are developed to reach parents and children using communication vehicles such as billboards, television, radio and more".

From the Today show segment (which featured the campaign's Director, a child actor featured in the ads, and a child psychologist) we learned that this media campaign is part one of a three part campaign. The three parts were briefly outlined:

1- Raise awareness about childhood obesity; letting kids voice their struggle in their own words.
2- "Activate"- focus on healthy eating and activity
3- Focus on real solutions

While the GCHA outlines their strategic mission for this campaign, they are hearing some major objections to their approach and it continues to grab national headlines. The major concerns voiced by objectors such as Rebecca Puhl (a weight discrimination expert from Yale University), are that the ads will increase stigma for overweight kids (which could increase their experience of bullying) and that the ads will be ineffective due to their fear-based approach. In my review of the ads, I have mixed (mostly negative) feelings about their development and implementation:
  • Strike One: The goal of this campaign is listed as "raising awareness". These may be my two least favorite terms in all of public health. "Raising awareness" is too vague and does not lend itself to being evaluated. In actuality, campaign developers usually want to "increase knowledge" or "change perceptions" or "change behavior" (e.g., calling the 800 number on the screen). These are all things which can actually be measured and should be stated more clearly.
  • Strike Two: When the Today show asked the Campaign director about the audience for these ads, he replied "parents, kids, and educators". Again, this is way too vague. Your message and call to action (i.e., what you want the viewer to do after watching the ad) would be completely different for each of those audiences. For example, you may want educators to reach out to the parents of overweight kids in their classes or you may want kids to tell an adult if they are being bullied about their weight. These messages need to be tailored to each audience.
  • Strike Three: These ads definitely fall into the "fear-based" category. As you watch them, the ads read "WARNING" in bold red letters and you hear a "boom" (kinda like on "Law & Order) as the statistics run across the screen. As I have mentioned in previous blog posts, fear-based approaches have been found to be ineffective in other areas of prevention (e.g., alcohol and other drugs).
  • In terms of redeeming factors, it does seem that the campaign was developed using formative research which included focus groups with overweight kids. The results of these focus groups were used to develop the dialogue read by child actors in the ads so that it would be "in their words". If kids are the audience for these ads, then the age appropriate priorities and dialogue (with the inclusion of child actors) is positive. From health behavior theory (e.g., Social Learning Theory), we know that kids will respond better if they relate to those in the ads.
Of course, it is unclear if they also focus group tested the ads and billboards after initial development, before they were rolled out. It is also unclear how they are being evaluated and what the ultimate goals are (beyond "increased awareness"). I'll be interested to see parts two and three rolled out and hope to include follow up thoughts here on Pop Health.