Marketing Research

Authors:

Julia (Pengyue) Dou, MPH, Beth L. Hoffman, PhD, MPH, Piper Narendorf, BS, Tatiana Grinberg Limoncic, BA, Grace Carver, MYTH Youth Collaborative Member, Natalia Connor, MYTH Youth Collaborative Member, Christine Larkin, BS, Jaime E. Sidani, PhD, MPH, CHES

Introduction:

Our team of public health faculty, undergraduate, and high school (HS) students in the MYTH (Misinformation-identifying Youth in Tobacco and Health) Youth Collaborative is conducting a study on nicotine and tobacco products (NTPs) on social media. A crucial component of this research is the development of codebooks for human coding of social media content; however, we have encountered discrepancies in how team members of different ages define specific constructs. This vertically integrated project (VIP) therefore aimed to compare how coders of different ages defined and coded NTP content from YouTube to identify similarities and differences in historically challenging constructs.

Methods:

We manually extracted 300 YouTube shorts with #Zyn (a new NTP) on September 18, 2024. Teams of two HS, two undergraduates, and two faculty used an existing codebook to code 50 shorts per round across six rounds. Percent agreement was calculated after each round, followed by meetings to resolve discrepancies.

Results:

Preliminary results found similar overall average percent agreement within teams (HS: 91.2%, undergraduate: 92.5%, faculty: 92.0%) but varied by category. Coding neutral sentiment (HS: 81.0%, undergraduate: 89.0%, faculty: 78.3%) and comedic type (HS: 80.3%, undergraduate: 88.3%, faculty: 82.3%) were difficult for all teams. Only HS coders reached near 100% agreement on youth targeting.

Conclusion:

This in-progress study shows the feasibility and value of involving diverse age groups in coding processes. Preliminary findings suggest coder demographics interpret online content differently. Integrating youth perspectives can inform more accurate and contextually relevant strategies for studying emerging public health issues affecting youth.