Publications – Social Media for Public Health

Links to Our Published Work:

Articles on Public Health Applications in Public Health & Hypothesis Testing

Assessing Causal Claims about Complex Engineered Systems with Quantitative Data: Internal, External, and Construct Validity Systems Enginnering, 2018. Engineers seek to design systems that will produce an intended change in the state of the world. How are we to know if a system will behave as intended? This article addresses ways that this question can be answered. Specifically, we focus on three types of research validity: (1) internal validity, or whether an observed association between two variables can be attributed to a causal link between them; (2) external validity, or whether a causal link generalizes across contexts; and (3) construct validity, or whether a specific set of metrics corresponds to what they are intended to measure.

The Emergence and Collapse of Knowledge Boundaries. IEEE Transactions on Engineering Management, 2017. This paper uses natural language processing tools to extract measures of semantic overlap between experts making risky decisions about medical device approval. Analysis supports, and extends, the predictions of leading theories of how information can be shared between representatives of different cultures to come to better shared decisions about risky medical decision making.

Effective vaccine communication during the Disneyland measles outbreak. Vaccine, 2016. This paper tests the predictions of Fuzzy Trace Theory — a leading theory of decision under risk — on the spread of information about the Disneyland Measles Outbreak of 2012-2015. Results support the theory’s predictions, leading us to propose a health communications framework.

Leveraging Big Data to Improve Health Awareness Campaigns: A Novel Evaluation of the Great American Smokeout. JMIR Public Health and Surveillance, 2016.

Awareness campaigns are ubiquitous, but little is known about their potential effectiveness because traditional evaluations are often unfeasible. For 40 years, the “Great American Smokeout” (GASO) has encouraged media coverage and popular engagement with smoking cessation on the third Thursday of November as the nation’s longest running awareness campaign. We proposed a novel evaluation framework for assessing awareness campaigns using the GASO as a case study by observing cessation-related news reports and Twitter postings, and cessation-related help seeking via Google, Wikipedia, and government-sponsored quitlines.

Zika vaccine misconceptions: A social media analysis. Vaccine, 2016.

News and Internet Searches About Human Immunodeficiency Virus After Charlie Sheen’s Disclosure.JAMA Internal Medicine, 2016.

Celebrity Charlie Sheen publicly disclosed his human immunodeficiency virus (HIV)–positive status on November 17, 2015. Could Sheen’s disclosure, like similar announcements from celebrities, generate renewed attention to HIV? We provide an early answer by examining news trends to reveal discussion of HIV in the mass media and Internet searches to reveal engagement with HIV-related topics around the time of Sheen’s disclosure

Understanding Vaccine Refusal: Why we need social media now. American Journal of Preventive Medicine, 2016.

Revisiting the Rise of Electronic Nicotine Delivery Systems Using Search Query Surveillance.American Journal of Preventive Medicine, 2016.

Introduction: Public perceptions of electronic nicotine delivery systems (ENDS) remain poorly understood because surveys are too costly to regularly implement and, when implemented, there are long delays between data collection and dissemination. Search query surveillance has bridged some of these gaps. Herein, ENDS’ popularity in the U.S. is reassessed using Google searches. Methods: ENDS searches originating in the U.S. from January 2009 through January 2015 were disaggregated by terms focused on e-cigarette (e.g., e-cig) versus vaping (e.g., vapers); their geolocation (e.g., state); the aggregate tobacco control measures corresponding to their geolocation (e.g., clean indoor air laws); and by terms that indicated the searcher’s potential interest (e.g., buy e-cigs likely indicates shopping)—all analyzed in 2015. Results: ENDS searches are rapidly increasing in the U.S., with 8,498,000 searches during 2014 alone. Increasingly, searches are shifting from e-cigarette- to vaping-focused terms, especially in coastal states and states where anti-smoking norms are stronger. For example, nationally, e-cigarette searches declined 9% (95% CI=1%, 16%) during 2014 compared with 2013, whereas vaping searches increased 136% (95% CI=97%, 186%), even surpassing e-cigarette searches. Additionally, the percentage of ENDS searches related to shopping (e.g., vape shop) nearly doubled in 2014, whereas searches related to health concerns (e.g., vaping risks) or cessation (e.g., quit smoking with e-cigs) were rare and declined in 2014. Conclusions: ENDS popularity is rapidly growing and evolving. These findings could inform survey questionnaire development for follow-up investigation and immediately guide policy debates about how the public perceives the health risks or cessation benefits of ENDS.

Germs Are Germs, and Why Not Take a Risk? Patients’ Expectations for Prescribing Antibiotics in an Inner-City Emergency Department. Medical Decision Making, 2015.

Background. Extensive use of unnecessary antibiotics has driven the emergence of resistant bacterial strains, posing a threat to public health. Physicians are more likely to prescribe antibiotics when they believe that patients expect them. Current attempts to change these expectations highlight the distinction between viruses and bacteria (“germs are germs”). Fuzzy-trace theory further predicts that patients expect antibiotics because they make decisions based on categorical gist, producing strategies that encourage risk taking when the status quo is bad (i.e., “why not take a risk?”). We investigate both hypotheses. Methods. We surveyed patients visiting the emergency department of a large urban hospital (72 [64%] were African American) using 17 Likert scale questions and 2 free-response questions regarding patient expectations for antibiotics. Results. After the clinical encounter, 113 patients completed the survey. Fifty-four (48%) patients agreed with items that assess the “germs are germs” hypothesis, whereas 86 (76%) agreed with items that assess the “why not take a risk?” hypothesis. “Why not take a risk?” captures significant unique variance in a factor analysis and is neither explained by “germs are germs” nor by patients’ lack of knowledge regarding side effects. Of the 81 patients who rejected the “germs are germs” hypothesis, 61 (75%) still indicated agreement with the “why not take a risk?” hypothesis. Several other misconceptions were also investigated. Conclusions. Our findings suggest that recent public health campaigns that have focused on educating patients about the differences between viruses and bacteria omit a key motivation for why patients expect antibiotics, supporting fuzzy-trace theory’s predictions about categorical gist. The implications for public health and emergency medicine are discussed.

Using Social Media to Perform Local Influenza Surveillance in an Inner-City Hospital: A Retrospective Observational Study. JMIR Public Health and Surveillance, 2015.

Background: Public health officials and policy makers in the United States expend significant resources at the national, state, county, and city levels to measure the rate of influenza infection. These individuals rely on influenza infection rate information to make important decisions during the course of an influenza season driving vaccination campaigns, clinical guidelines, and medical staffing. Web and social media data sources have emerged as attractive alternatives to supplement existing practices. While traditional surveillance methods take 1-2 weeks, and significant labor, to produce an infection estimate in each locale, web and social media data are available in near real-time for a broad range of locations. Objective: The objective of this study was to analyze the efficacy of flu surveillance from combining data from the websites Google Flu Trends and HealthTweets at the local level. We considered both emergency department influenza-like illness cases and laboratory-confirmed influenza cases for a single hospital in the City of Baltimore. Methods: This was a retrospective observational study comparing estimates of influenza activity of Google Flu Trends and Twitter to actual counts of individuals with laboratory-confirmed influenza, and counts of individuals presenting to the emergency department with influenza-like illness cases. Data were collected from November 20, 2011 through March 16, 2014. Each parameter was evaluated on the municipal, regional, and national scale. We examined the utility of social media data for tracking actual influenza infection at the municipal, state, and national levels. Specifically, we compared the efficacy of Twitter and Google Flu Trends data. Results: We found that municipal-level Twitter data was more effective than regional and national data when tracking actual influenza infection rates in a Baltimore inner-city hospital. When combined, national-level Twitter and Google Flu Trends data outperformed each data source individually. In addition, influenza-like illness data at all levels of geographic granularity were best predicted by national Google Flu Trends data. Conclusions: In order to overcome sensitivity to transient events, such as the news cycle, the best-fitting Google Flu Trends model relies on a 4-week moving average, suggesting that it may also be sacrificing sensitivity to transient fluctuations in influenza infection to achieve predictive power. Implications for influenza forecasting are discussed in this report.

Combining Search, Social Media, and Traditional Data Sources to Improve Influenza Surveillance. PLOS Computational Biology, 2015.

We present a machine learning-based methodology capable of providing real-time (“nowcast”) and forecast estimates of influenza activity in the US by leveraging data from multiple data sources including: Google searches, Twitter microblogs, nearly real-time hospital visit records, and data from a participatory surveillance system. Our main contribution consists of combining multiple influenza-like illnesses (ILI) activity estimates, generated independently with each data source, into a single prediction of ILI utilizing machine learning ensemble approaches. Our methodology exploits the information in each data source and produces accurate weekly ILI predictions for up to four weeks ahead of the release of CDC’s ILI reports. We evaluate the predictive ability of our ensemble approach during the 2013–2014 (retrospective) and 2014–2015 (live) flu seasons for each of the four weekly time horizons. Our ensemble approach demonstrates several advantages: (1) our ensemble method’s predictions outperform every prediction using each data source independently, (2) our methodology can produce predictions one week ahead of GFT’s real-time estimates with comparable accuracy, and (3) our two and three week forecast estimates have comparable accuracy to real-time predictions using an autoregressive model. Moreover, our results show that considerable insight is gained from incorporating disparate data streams, in the form of social media and crowd sourced data, into influenza predictions in all time horizons.

Discovering Health Topics in Social Media Using Topic Models. PLoS ONE, 2014.

By aggregating self-reported health statuses across millions of users, we seek to characterize the variety of health information discussed in Twitter. We describe a topic modeling framework for discovering health topics in Twitter, a social media website. This is an exploratory approach with the goal of understanding what health topics are commonly discussed in social media. This paper describes in detail a statistical topic model created for this purpose, the Ailment Topic Aspect Model (ATAM), as well as our system for filtering general Twitter data based on health keywords and supervised classification. We show how ATAM and other topic models can automatically infer health topics in 144 million Twitter messages from 2011 to 2013. ATAM discovered 13 coherent clusters of Twitter messages, some of which correlate with seasonal influenza (r = 0.689) and allergies (r = 0.810) temporal surveillance data, as well as exercise (r = .534) and obesity (r = −.631) related geographic survey data in the United States. These results demonstrate that it is possible to automatically discover topics that attain statistically significant correlations with ground truth data, despite using minimal human supervision and no historical data to train the model, in contrast to prior work. Additionally, these results demonstrate that a single general-purpose model can identify many different health topics in social media.

Twitter: Big data opportunities. Science, 2014.

Twitter Improves Influenza Forecasting. PLoS Currents, 2014.

Accurate disease forecasts are imperative when preparing for influenza epidemic outbreaks; nevertheless, these forecasts are often limited by the time required to collect new, accurate data. In this paper, we show that data from the microblogging community Twitter significantly improves influenza forecasting. Most prior influenza forecast models are tested against historical influenza-like illness (ILI) data from the U.S. Centers for Disease Control and Prevention (CDC). These data are released with a one-week lag and are often initially inaccurate until the CDC revises them weeks later. Since previous studies utilize the final, revised data in evaluation, their evaluations do not properly determine the effectiveness of forecasting. Our experiments using ILI data available at the time of the forecast show that models incorporating data derived from Twitter can reduce forecasting error by 17-30% over a baseline that only uses historical data. For a given level of accuracy, using Twitter data produces forecasts that are two to four weeks ahead of baseline models. Additionally, we find that models using Twitter data are, on average, better predictors of influenza prevalence than are models using data from Google Flu Trends, the leading web data source.

National and Local Influenza Surveillance through Twitter: An Analysis of the 2012-2013 Influenza Epidemic. PLoS ONE, 2013.

Social media have been proposed as a data source for influenza surveillance because they have the potential to offer real-time access to millions of short, geographically localized messages containing information regarding personal well-being. However, accuracy of social media surveillance systems declines with media attention because media attention increases “chatter” – messages that are about influenza but that do not pertain to an actual infection – masking signs of true influenza prevalence. This paper summarizes our recently developed influenza infection detection algorithm that automatically distinguishes relevant tweets from other chatter, and we describe our current influenza surveillance system which was actively deployed during the full 2012-2013 influenza season. Our objective was to analyze the performance of this system during the most recent 2012–2013 influenza season and to analyze the performance at multiple levels of geographic granularity, unlike past studies that focused on national or regional surveillance. Our system’s influenza prevalence estimates were strongly correlated with surveillance data from the Centers for Disease Control and Prevention for the United States (r = 0.93, p < 0.001) as well as surveillance data from the Department of Health and Mental Hygiene of New York City (r = 0.88, p < 0.001). Our system detected the weekly change in direction (increasing or decreasing) of influenza prevalence with 85% accuracy, a nearly twofold increase over a simpler model, demonstrating the utility of explicitly distinguishing infection tweets from other chatter.

How Social Media Will Change Public Health. Intelligent Systems, IEEE, 2012.

Recent work in machine learning and natural language processing has studied the health content of tweets and demonstrated the potential for extracting useful public health information from their aggregation. This article examines the types of health topics discussed on Twitter, and how tweets can both augment existing public health capabilities and enable new ones. The author also discusses key challenges that researchers must address to deliver high-quality tools to the public health community.

Articles using Survey Data

Breaking Down the Monolith: Understanding Flu Vaccine Uptake among African Americans Social Science & Medicine Population Health, 2018.

Black adults are significantly less likely to be immunized for seasonal influenza when compared to Whites. This persistent disparity contributes to increased influenza-related morbidity and mortality in the African American population. Most scholarship on vaccine disparities has compared Whites and Blacks. Employing Public Health Critical Race Praxis, this study seeks to shift the focus to explore differences within the Black population. Utilizing a nationally-representative 2015 survey of US Black adults (n = 806), we explore differences by gender, age, income, and education across vaccine-related measures (e.g., perceived risk, knowledge, attitudes) and racial factors (e.g. racial salience, racial fairness, perceived discrimination). We also explore differences by vaccine behavior in the past five years among those who vaccinate every year, most years but not all, once or twice, and never. Greater frequency of flu vaccine uptake was associated with better self-reported vaccine knowledge, more positive vaccine attitudes, more trust in the flu vaccine and the vaccine process, higher perceived disease risk, lower perceived risk of vaccine side effects, stronger subjective and moral norms, lower general vaccine hesitancy, higher confidence in the flu vaccine, and lower perceived barriers. Logistic regression results highlighted other significant differences among the groups, emphasizing areas to target for improved vaccination rates. We find great diversity within the Black community related to influenza immunization decisions, highlighting the need to “break down the monolith” in future research.

Determinants of Influenza Vaccination among High-Risk Black and White Adults Vaccine, 2017.

Adults with chronic conditions are at much greater risk of influenza-related morbidity and mortality, yet flu vaccine uptake remains suboptimal. Research focused on the high-risk population has been limited, particularly related to racial disparities in vaccination. We explore a broad range of demographic, racial, and psychosocial factors to identify predictors of vaccination among high-risk adults, with a focus on identify differences between Black and White adults.

The Influence of Social Norms on Flu Vaccination among African American and White Adults Health Education Research, 2017.

Adult influenza vaccination rates remain suboptimal, particularly among African Americans. Social norms may influence vaccination behavior, but little research has focused on influenza vaccine and almost no research has focused on racially-specific norms. This mixed methods investigation utilizes qualitative interviews and focus groups (n = 118) and national survey results (n = 1643) to assess both descriptive and subjective norms surrounding influenza vaccination. Qualitative results suggest a perceived descriptive norm that ‘about half’ of the population gets vaccinated. Participants describe differing norms by race and vaccine behavior. Quantitative results confirm a perceived descriptive norm that 40–60% of the population gets vaccinated. Both African Americans and Whites accurately identified race-specific vaccination rates relative to the general population. Individuals who report that a majority of people around them want them to be vaccinated were significantly more likely to be vaccinated, suggesting subjective norms are influential for both White and African American adults. While perceived descriptive norms are somewhat accurate (mirroring the actual influenza vaccination rate), emphasizing a suboptimal vaccination rate may not be beneficial. Health promotion efforts, particularly those targeting African Americans, may benefit from focusing on subjective norms and encouraging friends and family members to talk about the benefits of influenza vaccination.

Determinants of Trust in the Flu Vaccine for African Americans and Whites Social Science & Medicine, 2017.

Trust is thought to be a major factor in vaccine decisions, but few studies have empirically tested the role of trust in adult immunization. Utilizing a 2015 national survey of African American and White adults (n = 1630), we explore multiple dimensions of trust related to influenza immunization, including generalized trust, trust in the flu vaccine, and trust in the vaccine production process. We find African Americans report lower trust than Whites across all trust measures. When considering demographic, racial, and ideological predictors, generalized trust shows statistically significant effects on both trust in the flu vaccine and trust in the vaccine process. When controlling for demographic, racial, and ideological variables, higher generalized trust was significantly associated with higher trust in the flu vaccine and the vaccine process. When controlling for generalized trust, in addition to the baseline covariates, psychosocial predictors (i.e. risk perception, social norms, knowledge) are significant predictors of trust in flu vaccine and trust in the vaccine process, with significant differences by race. These findings suggest that trust in vaccination is complex, and that significant differences in trust between White and African American adults may be contributing to disparities in influenza immunization.

The Role of Risk Perception in Flu Vaccine Behavior among African-American and White Adults in the United States Risk Analysis, 2017.

Seasonal flu vaccination rates are low for U.S. adults, with significant disparities between African and white Americans. Risk perception is a significant predictor of vaccine behavior but the research on this construct has been flawed. This study addressed critical research questions to understand the differences between African and white Americans in the role of risk perception in flu vaccine behavior: (1) What is the dimensionality of risk perception and does it differ between the two races? (2) Were risk perceptions of white and African-American populations different and how were sociodemographic characteristics related to risk for each group? (3) What is the relation between risk perception and flu vaccine behaviors for African Americans and whites? The sample, drawn from GfK’s Knowledge Panel, consisted of 838 whites and 819 African Americans. The survey instrument was developed from qualitative research. Measures of risk perception included cognitive and emotional measures of disease risk and risk of side effects from the vaccine. The online survey was conducted in March 2015. Results showed the importance of risk perception in the vaccine decision-making process for both racial groups. As expected, those who got the vaccine reported higher disease risk than those who did not. Separate cognitive and emotional factors did not materialize in this study but strong evidence was found to support the importance of considering disease risk as well as risk of the vaccine. There were significant racial differences in the way risk perception predicted behavior.

Exploring Racial Influences on Flu Vaccine Attitudes and Behavior: Results of a National Survey of White and African American Adults. Vaccine, 2017.

Introduction: Racial disparities in adult flu vaccination rates persist with African Americans falling below Whites in vaccine acceptance. Although the literature has examined traditional variables including barriers, access, attitudes, among others, there has been virtually no examination of the extent to which racial factors including racial consciousness, fairness, and discrimination may affect vaccine attitudes and behaviors. Methods: We contracted with GfK to conduct an online, nationally representative survey with 819 African American and 838 White respondents. Measures included risk perception, trust, vaccine attitudes, hesitancy and confidence, novel measures on racial factors, and vaccine behavior. Results: There were significant racial differences in vaccine attitudes, risk perception, trust, hesitancy and confidence. For both groups, racial fairness had stronger direct effects on the vaccine-related variables with more positive coefficients associated with more positive vaccine attitudes. Racial consciousness in a health care setting emerged as a more powerful influence on attitudes and beliefs, particularly for African Americans, with higher scores on racial consciousness associated with lower trust in the vaccine and the vaccine process, higher perceived vaccine risk, less knowledge of flu vaccine, greater vaccine hesitancy, and less confidence in the flu vaccine. The effect of racial fairness on vaccine behavior was mediated by trust in the flu vaccine for African Americans only (i.e., higher racial fairness increased trust in the vaccine process and thus the probability of getting a flu vaccine). The effect of racial consciousness and discrimination for African Americans on vaccine uptake was mediated by perceived vaccine risk and flu vaccine knowledge. Conclusions: Racial factors can be a useful new tool for understanding and addressing attitudes toward the flu vaccine and actual vaccine behavior. These new concepts can facilitate more effective tailored and targeted vaccine communications.

Exploring the Continuum of Vaccine Hesitancy Between African American and White Adults: Results of a Qualitative Study. PLoS Currents Outbreaks, 2016.

Vaccine delay and refusal present very real threats to public health. Since even a slight reduction in vaccination rates could produce major consequences as herd immunity is eroded, it is imperative to understand the factors that contribute to decision-making about vaccines. Recent scholarship on the concept of “vaccine hesitancy” emphasizes that vaccine behaviors and beliefs tend to fall along a continuum from refusal to acceptance. Most research on hesitancy has focused on parental decision-making about childhood vaccines, but could be extended to explore decision-making related to adult immunization against seasonal influenza. In particular, vaccine hesitancy could be a useful approach to understand the persistence of racial/ethnic disparities between African American and White adults. This study relied on a thematic content analysis of qualitative data, including 12 semi-structured interviews, 9 focus groups (N=90), and 16 in-depth interviews, for a total sample of 118 (N=118) African American and White adults. All data were transcribed and analyzed with Atlas.ti. A coding scheme combining both inductive and deductive codes was utilized to identify themes related to vaccine hesitancy. The study found a continuum of vaccine behavior from never-takers, sometimes-takers, and always-takers, with significant differences between African Americans and Whites. We compared our findings to the Three Cs: Complacency, Convenience, and Confidence framework. Complacency contributed to low vaccine acceptance with both races. Among sometimes-takers and always-takers, convenience was often cited as a reason for their behavior, while never-takers of both races were more likely to describe other reasons for non-vaccination, with convenience only a secondary explanation. However, for African Americans, cost was a barrier. There were racial differences in trust and confidence that impacted the decision-making process. The framework, though not a natural fit for the data, does provide some insight into the differential sources of hesitancy between these two populations. Complacency and confidence clearly impact vaccine behavior, often more profoundly than convenience, which can contribute either negatively or positively to vaccine acceptance. The Three Cs framework is a useful, but limited tool to understanding racial disparities. Understanding the distinctions in those cultural factors that drive lower vaccine confidence and greater hesitancy among African Americans could lead to more effective communication strategies as well as changes in the delivery of vaccines to increase convenience and passive acceptance.

Public Acceptance of Peramivir During the 2009 H1N1 Influenza Pandemic: Implications for Other Drugs or Vaccines Under Emergency Use Authorizations. Disaster Medicine and Public Health Preparedness, 2015.

Objective: The Centers for Disease Control and Prevention estimated that up to 88 million H1N1 influenza cases, 398,000 hospitalizations, and up to 18,050 related deaths, including significant racial and ethnic disparities, occurred between April 2009 and March 13, 2010. The Food and Drug Administration (FDA) approved emergency use authorizations (EUAs), which allowed the distribution of unapproved drugs or the off-label use of approved drugs. In late 2009, peramivir was granted an EUA for patients with severe disease. This study examined factors associated with willingness to take peramivir. Methods: In 2010 we conducted a nationally representative survey with 2079 respondents randomly drawn from the Knowledge Networks research panel. Our completion rate was 56%. Respondents received information about peramivir from a fact sheet and then answered questions about their willingness to take the drug. Results: Overall, 48% of participants indicated that they would probably or definitely take peramivir. Seventy-nine percent definitely would take the drug if their doctor recommended it and there were no alternative treatments. There were significant racial differences in willingness. The term experimental to refer to the drug decreased willingness to accept peramivir among both whites and blacks. Conclusions: Trust in the FDA was important for peramivir acceptance. Particular care must be taken to ensure that patients and their families understand the complex nature of EUA drugs. Lessons learned can inform communication about future EUAs

Health Inequalities and Infectious Disease Epidemics: A Challenge for Global Health Security. Biosecurity and Bioterrorism: Biodefense Strategy, Practice, and Science, 2014.

In today’s global society, infectious disease outbreaks can spread quickly across the world, fueled by the rapidity with which we travel across borders and continents. Historical accounts of influenza pandemics and contemporary reports on infectious diseases clearly demonstrate that poverty, inequality, and social determinants of health create conditions for the transmission of infectious diseases, and existing health disparities or inequalities can further contribute to unequal burdens of morbidity and mortality. Yet, to date, studies of influenza pandemic plans across multiple countries find little to no recognition of health inequalities or attempts to engage disadvantaged populations to explicitly address the differential impact of a pandemic on them. To meet the goals and objectives of the Global Health Security Agenda, we argue that international partners, from WHO to individual countries, must grapple with the social determinants of health and existing health inequalities and extend their vision to include these factors so that disease that may start among socially disadvantaged subpopulations does not go unnoticed and spread across borders. These efforts will require rethinking surveillance systems to include sociodemographic data; training local teams of researchers and community health workers who are able to not only analyze data to recognize risk factors for disease, but also use simulation methods to assess the impact of alternative policies on reducing disease; integrating social science disciplines to understand local context; and proactively anticipating shortfalls in availability of adequate healthcare resources, including vaccines. Without explicit attention to existing health inequalities and underlying social determinants of health, the Global Health Security Agenda is unlikely to succeed in its goals and objectives.

Trust During the Early Stages of the 2009 H1N1 Pandemic. Journal of Health Communication, 2014.

Distrust of the government often stands in the way of cooperation with public health recommendations in a crisis. The purpose of this article is to describe the public’s trust in government recommendations during the early stages of the H1N1 pandemic and to identify factors that might account for these trust levels. The authors surveyed 1,543 respondents about their experiences and attitudes related to H1N1 influenza between June 3, 2009, and July 6, 2009, during the first wave of the pandemic using the Knowledge Networks online panel. This panel is representative of the U.S. population and uses a combination of random digit dialing and address-based probability sampling frames covering 99% of the U.S. household population to recruit participants. To ensure participation of low-income individuals and those without Internet access, Knowledge Networks provides hardware and access to the Internet if needed. Measures included standard demographics, a trust scale, trust ratings for individual spokespersons, involvement with H1N1, experience with H1N1, and past discrimination in health care. The authors found that trust of government was low (2.3 out of 4) and varied across demographic groups. Blacks and Hispanics reported higher trust in government than did Whites. Of the spokespersons included, personal health professionals received the highest trust ratings and religious leaders the lowest. Attitudinal and experience variables predicted trust better than demographic characteristics. Closely following the news about the flu virus, having some self-reported knowledge about H1N1, self-reporting of local cases, and previously experiencing discrimination were the significant attitudinal and experience predictors of trust. Using a second longitudinal survey, trust in the early stages of the pandemic predicted vaccine acceptance later but only for White, non-Hispanic individuals.

Determinants of parental acceptance of the H1N1 vaccine. Health Education & Behavior, 2013.

Although designated as a high-risk group during the 2009-2010 H1N1 pandemic, only about 40% of U.S. children received the vaccine, a relatively low percentage compared with high-risk groups in seasonal influenza, such as the elderly, whose vaccine rates typically top 70%. To better understand parental decision making and predictors of acceptance of the H1N1 vaccine, we examined data from a representative national sample of parents (n = 684), using the health belief model as a framework. The most important predictors of vaccine acceptance were “cues to action” at multiple levels, from intrapersonal to mass communication, including the influence of friends, family, the media, and modeling by the Obama family; costs and benefits and self-efficacy were also significant predictors of vaccine acceptance. Higher perceived levels of H1N1 risk were not associated with vaccine uptake. Results suggest that traditional measures of perceived risk may not account for the cost–benefit analysis inherent in vaccine decision making, and that messages designed to emphasize disease risk may be ineffective. The authors recommend emphasizing cues to action that support norming and modeling of vaccine acceptance.

Exploring Communication, Trust in Government, and Vaccination Intention Later in the 2009 H1N1 Pandemic: Results of a National Survey. Biosecurity and Bioterrorism: Biodefense Strategy, Practice, and Science, 2013.

With the growing recognition of the critical role that risk communication plays in a public health emergency, a number of articles have provided prescriptive best practices to enhance such communication. However, little empirical research has examined perceptions of the quality of communication, the impact of uncertainty on changing communication, use of information sources, and trust in specific government spokespersons. Similarly, although there is significant conceptual focus on trust and communication as important in vaccination intent and acceptance, little research has explored these relationships empirically. We conducted an online survey in late January 2010 with a nationally representative sample (N=2,079) that included Hispanic and African American oversamples. The completion rate was 56%. We found that public health officials were the most trusted spokespersons, with President Obama being the most highly trusted elected official. Demographic variables, including race, accounted for 21% of the variance in trust of the president. Perceptions of the quality of communication were high, including significant understanding of uncertainty and appreciation for officials’ openness about evolving information. Other factors that contributed to vaccination acceptance were quality of communication, closely following the news, and confidence in the vaccine because of a role model effect of the Obama daughters’ immunizations; these factors significantly increased trust in government actions. Because the challenges of communication often vary over the course of a pandemic, there is a consistent need to pay close attention to both communication content and delivery and prepare public health officials at all levels to be effective communicators.

US public support for vaccine donation to poorer countries in the 2009 H1N1 pandemic. PloS One, 2012.

Background: During the 2009 H1N1 pandemic, the global health community sought to make vaccine available “in developing nations in the same timeframe as developed nations.” However, richer nations placed advance orders with manufacturers, leaving poorer nations dependent on the quantity and timing of vaccine donations by manufacturers and rich nations. Knowledge of public support for timely donations could be important to policy makers during the next pandemic. We explored what the United States (US) public believes about vaccine donation by its country to poorer countries. Methods and Findings: We surveyed 2079 US adults between January 22^nd and February 1^st 2010 about their beliefs regarding vaccine donation to poorer countries. Income (p = 0.014), objective priority status (p = 0.005), nativity, party affiliation, and political ideology (p<0.001) were significantly related to views on the amount of vaccine to be donated. Though party affiliation and political ideology were related to willingness to donate vaccine (p<0.001), there was bipartisan support for timely donations of 10% of the US vaccine supply so that those “at risk in poorer countries can get the vaccine at the same time” as those at risk in the US. Conclusions: We suggest that the US and other developed nations would do well to bolster support with education and public discussion on this issue prior to an emerging pandemic when emotional reactions could potentially influence support for donation. We conclude that given our evidence for bipartisan support for timely donations, it may be necessary to design multiple arguments, from utilitarian to moral, to strengthen public and policy makers’ support for donations.

The social ecological model as a framework for determinants of 2009 H1N1 influenza vaccine uptake in the United States. Health Education & Behavior, 2012.

Research on influenza vaccine uptake has focused largely on intrapersonal determinants (perceived risk, past vaccine acceptance, perceived vaccine safety) and on physician recommendation. The authors used a social ecological framework to examine influenza vaccine uptake during the 2009 H1N1 pandemic. Surveying an adult population (n = 2,079) in January 2010 with significant oversamples of Blacks and Hispanics, this study found that 18.4% (95% confidence interval = 15.6-21.5) had gotten the 2009 H1N1 vaccine. Variables at each level of the social ecological model were significant predictors of uptake as well as of intent to get the vaccine. The intrapersonal level explained 53%, the interpersonal explained 47%, the institutional level explained 34%, and the policy and community levels each explained 8% of the variance associated with vaccine uptake. The levels together explained 65% of the variance, suggesting that interventions targeting multiple levels of the framework would be more effective than interventions aimed at a single level.

Racial Disparities in Exposure, Susceptibility, and Access to Health Care in the US H1N1 Influenza Pandemic. American Journal of Public Health, 2011.

Objectives: We conducted the first empirical examination of disparities in H1N1 exposure, susceptibility to H1N1 complications, and access to health care during the H1N1 influenza pandemic. Methods: We conducted a nationally representative survey among a sample drawn from more than 60 000 US households. We analyzed responses from 1479 adults, including significant numbers of Blacks and Hispanics. The survey asked respondents about their ability to impose social distance in response to public health recommendations, their chronic health conditions, and their access to health care. Results: Risk of exposure to H1N1 was significantly related to race and ethnicity. Spanish-speaking Hispanics were at greatest risk of exposure but were less susceptible to complications from H1N1. Disparities in access to health care remained significant for Spanish-speaking Hispanics after controlling for other demographic factors. We used measures based on prevalence of chronic conditions to determine that Blacks were the most susceptible to complications from H1N1. Conclusions: We found significant race/ethnicity-related disparities in potential risk from H1N1 flu. Disparities in the risks of exposure, susceptibility (particularly to severe disease), and access to health care may interact to exacerbate existing health inequalities and contribute to increased morbidity and mortality in these populations.

The Impact of Workplace Policies and Other Social Factors on Self-Reported Influenza-Like Illness Incidence During the 2009 H1N1 Pandemic. American Journal of Public Health, 2011.

Objectives: We assessed the impact of social determinants of potential exposure to H1N1—which are unequally distributed by race/ethnicity in the United States—on incidence of influenza-like illness (ILI) during the 2009 H1N1 pandemic. Methods: In January 2010 we surveyed a nationally representative sample (n = 2079) of US adults from the Knowledge Networks online research panel, with Hispanic and African American oversamples. The completion rate was 56%. Results: Path analysis examining ILI incidence, race, and social determinants of potential exposure to H1N1 demonstrated that higher ILI incidence was related to workplace policies, such as lack of access to sick leave, and structural factors, such as number of children in the household. Hispanic ethnicity was related to a greater risk of ILI attributable to these social determinants, even after we controlled for income and education. Conclusions: The absence of certain workplace policies, such as paid sick leave, confers a population-attributable risk of 5 million additional cases of ILI in the general population and 1.2 million cases among Hispanics. Federal mandates for sick leave could have significant health impacts by reducing morbidity from ILI, especially in Hispanics.

The Vagaries Of Public Support For Government Actions In Case Of A Pandemic. Health Affairs, 2010.

Government health measures in a pandemic are effective only with strong support and compliance from the public. A survey of 1,583 US adults early in the 2009 H1N1 (swine influenza) pandemic shows surprisingly mixed support for possible government efforts to control the spread of the disease, with strong support for more extreme measures such as closing borders and weak support for more basic, and potentially more effective, policies such as encouraging sick people to stay home from work. The results highlight challenges that public health officials and policy makers must address in formulating strategies to respond to a pandemic before a more severe outbreak occurs.

Public Willingness to Take a Vaccine or Drug Under Emergency Use Authorization during the 2009 H1N1 Pandemic. Biosecurity and Bioterrorism: Biodefense Strategy, Practice, and Science, 2009.

On April 26, 2009, the United States declared a public health emergency in response to a growing but uncertain threat from H1N1 influenza, or swine flu. In June, the World Health Organization declared a pandemic. In the U.S., hospitalizations due to swine flu numbered 6,506 on August 6, 2009, with 436 deaths; all 50 states have reported cases. The declaration of a public health emergency, followed by the approval of multiple Emergency Use Authorizations (EUAs) by the Food and Drug Administration, allowed the distribution of unapproved drugs or the off-label use of approved drugs to the public. Thus far, there are 2 antiviral medications available to the public as EUA drugs. It is possible that an H1N1 vaccine will be initially released as an EUA in the fall in the first large-scale use of the EUA mechanism. This study explores the public’s willingness to use a drug or vaccine under the conditions stipulated in the FDA’s nonbinding guidance regarding EUAs. Using Knowledge Networks’ panel, we conducted an internet survey with 1,543 adults from a representative sample of the U.S. population with 2 oversamples of African Americans and Spanish-speaking Hispanics. Our completion rate was 62%. We examined willingness to accept an EUA drug or an H1N1 vaccine, the extent of worry associated with taking either, the conditions under which respondents would accept an EUA drug or vaccine, and the impact of language from the EUA fact sheets on people’s willingness to accept a drug for themselves or their children. We also examined the association among these variables and race/ethnicity, education level, trust in government, previous vaccine acceptance, and perceived personal consequences from H1N1 influenza. These results provide critical insights into the challenges of communicating about EUA drugs and vaccine in our current pandemic.

Articles, Workshops, and Presentations on Social Media Tool Development

Examining Patterns of Influenza Vaccination in Social Media. Presented at the W3PHIAI 2017 Workshop (AAAI), 2017.

Traditional data on influenza vaccination has several limitations: high cost, limited coverage of underrepresented groups, and low sensitivity to emerging public health issues. Social media, such as Twitter, provide an alternative way to understand a population’s vaccination-related opinions and behaviors. In this study, we build and employ several natural language classifiers to examine and analyze behavioral patterns regarding influenza vaccination in Twitter across three dimensions: temporality (by week and month), geography (by US region), and demography (by gender). Our best results are highly correlated official government data, with a correlation over 0.90, providing validation of our approach. We then suggest a number of directions for future work

Towards Real-Time Measurement of Public Epidemic Awareness: Monitoring Influenza Awareness through Twitter. Presented at the AAAI Spring Symposium on Observational Studies through Social Media and Other Human-Generated Content, 2016.

This study analyzes temporal trends in Twitter data pertaining to both influenza awareness and influenza infection during the 2012–13 influenza season in the US. We make use of classifiers to distinguish tweets that express a personal infection (“sick with the flu”) versus a more general awareness (“worried about the flu”). While previous research has focused on estimating prevalence of influenza infection, little is known about trends in public awareness of the disease. Our analysis shows that infection and awareness have very different trends. In contrast to infection trends, awareness trends have little regional variation, and our experiments suggest that public awareness is primarily driven by news media.

Learning Multiview Embeddings of Twitter Users. Presented at the 54th Annual Meeting of the Association for Computational Linguistics, 2016

Low-dimensional vector representations are widely used as stand-ins for the text of words, sentences, and entire documents. These embeddings are used to identify similar words or make predictions about documents. In this work, we consider embeddings for social media users and demonstrate that these can be used to identify users who behave similarly or to predict attributes of users. In order to capture information from all aspects of a user’s online life, we take a multiview approach, applying a weighted variant of Generalized Canonical Correlation Analysis (GCCA) to a collection of over 100,000 Twitter users. We demonstrate the utility of these multiview embeddings on three downstream tasks: user engagement, friend selection, and demographic attribute prediction.

Collective Supervision of Topic Models for Predicting Surveys with Social Media. Presented at the Thirtieth AAAI Conference on Artificial Intelligence, 2016.

This paper considers survey prediction from social media. We use topic models to correlate social media messages with survey outcomes and to provide an interpretable representation of the data. Rather than rely on fully unsupervised topic models, we use existing aggregated survey data to inform the inferred topics, a class of topic model supervision referred to as collective supervision. We introduce and explore a variety of topic model variants and provide an empirical analysis, with conclusions of the most effective models for this task.

Demographer: Extremely Simple Name Demographics. Presented at the Natural Language Processing and Computational Social Science Workshops (NLP+ CSS), 2016.

The lack of demographic information available when conducting passive analysis of social media content can make it difficult to compare results to traditional survey results. We present DEMOGRAPHER, 1 a tool that predicts gender from names, using name lists and a classifier with simple character-level features. By relying only on a name, our tool can make predictions even without extensive user-authored content. We compare DEMOGRAPHER to other available tools and discuss differences in performance. In particular, we show that DEMOGRAPHER performs well on Twitter data, making it useful for simple and rapid social media demographic inference.

Collective Supervision of Topic Models for Predicting Surveys with Social Media. Presented at the Association for the Advancement of Artificial Intelligence (AAAI), 2016.

From ADHD to SAD: Analyzing the Language of Mental Health on Twitter through Self-Reported Diagnoses. Presented at the NAACL Workshop on Computational Linguistics and Clinical Psychology, 2015.

Many significant challenges exist for the mental health field, but one in particular is a lack of data available to guide research. Language provides a natural lens for studying mental health — much existing work and therapy have strong linguistic components, so the creation of a large, varied, language-centric dataset could provide significant grist for the field of mental health research. We examine a broad range of mental health conditions in Twitter data by identifying self-reported statements of diagnosis. We systematically explore language differences between ten conditions with respect to the general population, and to each other. Our aim is to provide guidance and a roadmap for where deeper exploration is likely to be fruitful.

Worldwide Influenza Surveillance through Twitter. In AAAI Workshop on the World Wide Web and Public Health Intelligence. Presented at the AAAI Workshop on the World Wide Web and Public Health Intelligence, 2015.

We evaluate the performance of Twitter-based influenza surveillance in ten English-speaking countries across four continents. We find that tweets are positively correlated with existing surveillance data provided by government agencies in these countries, with r values ranging from .37–.81. We show that incorporating Twitter data into a strong autoregressive baseline reduces mean squared error in 80 to 100 percent of locations depending on the lag, with larger improvements when reporting delays are longer.

HealthTweets. org: A Platform for Public Health Surveillance using Twitter. Presented at the AAAI Workshop on the World Wide Web and Public Health Intelligence, 2014.

We present HealthTweets.org, a new platform for sharing the latest research results on Twitter data with researchers and public officials. In this demo paper, we describe data collection, processing, and features of the site. The goal of this service is to transition results from research to practice.

Carmen: A Twitter Geolocation System with Applications to Public Health. Presented at the AAAI Workshop on Expanding the Boundaries of Health Informatics Using AI (HIAI), 2013.

Public health applications using social media often require accurate, broad-coverage location information. However, the standard information provided by social media APIs, such as Twitter, cover a limited number of messages. This paper presents Carmen, a geolocation system that can determine structured location information for messages provided by the Twitter API. Our system utilizes geocoding tools and a combination of automatic and manual alias resolution methods to infer location structures from GPS positions and user-provided profile data. We show that our system is accurate and covers many locations, and we demonstrate its utility for improving influenza surveillance.

Separating Fact from Fear: Tracking Flu Infections on Twitter. Presented at North American chapter of the Association for Computational Linguistics: Human Language Technologies (HLT-NAACL), 2013.

Twitter has been shown to be a fast and reliable method for disease surveillance of common illnesses like influenza. However, previous work has relied on simple content analysis, which conflates flu tweets that report infection with those that express concerned awareness of the flu. By discriminating these categories, as well as tweets about the authors versus about others, we demonstrate significant improvements on influenza surveillance using Twitter.

You are what you Tweet: Analyzing Twitter for public health. Presented at the International Conference on Web and Social Media (ICWSM), 2011.

Analyzing user messages in social media can measure different population characteristics, including public health measures. For example, recent work has correlated Twitter messages with influenza rates in the United States; but this has largely been the extent of mining Twitter for public health. In this work, we consider a broader range of public health applications for Twitter. We apply the recently introduced Ailment Topic Aspect Model to over one and a half million health related tweets and discover mentions of over a dozen ailments, including allergies, obesity and insomnia. We introduce extensions to incorporate prior knowledge into this model and apply it to several tasks: tracking illnesses over times (syndromic surveillance), measuring behavioral risk factors, localizing illnesses by geographic region, and analyzing symptoms and medication usage. We show quantitative correlations with public health data and qualitative evaluations of model output. Our results suggest that Twitter has broad applicability for public health research.

Book Chapters

Smith, M. C., & Broniatowski, D. A. (2016). Modeling Influenza by Modulating Flu Awareness. In K. Xu, D. Reitter, D. Lee, & N. Osgood (Eds.), SBP-BRIMS. Springer International Publishing.

It is important for public health officials to follow both the incidence of disease and the public’s perception of it, especially in the Internet-connected age. In the specific context of influenza, disease surveillance through social media has proven effective, but public awareness of influenza and its effects are not well understood. We build upon the existing Epstein model of coupled contagion with the aim of including modern media mechanisms for awareness transmission. Our agent-based model captures the unique effects of news media and social media on disease dynamics, and suggests potential areas for policy intervention to modulate the spread of the flu.

Hu, D., & Broniatowski, D. A. (2016). Designing a Crowdsourcing Tool to Measure Perceived Causal Relationships Between Narrative Events. In K. Xu, D. Reitter, D. Lee, & N. Osgood (Eds.), SBP-BRIMS. Springer International Publishing.

The computational study of narrative is important to multiple academic disciplines. However, prior research has been limited by the inability to quantify subjects’ comprehension of the causal structure within each narrative text. With the aid of big data technology and crowdsourcing tools, we aim to design a new approach to analysis the content of narratives in a data-driven manner, while also making these analyses scientifically replicable. The goal of this research is therefore to develop a tool that can be used to measure people’s understanding of the causal relationships within a piece of text.