What’s Political on TikTok? A User-Centered Approach

By Stephanie Wang, Jason Greenfield & Danaé Metaxa.

This post is part of a series based on research being presented at the workshop on News Futures at the CHI 2025 conference.

With over 1.5 billion users worldwide, TikTok is rapidly growing in popularity. While a third of U.S. adults now use the platform, we still know little about the nature of political content on TikTok, or how users interpret it. At the same time, U.S. lawmakers have voiced growing concerns that TikTok poses a national security risk due to its ownership by the Chinese company ByteDance. But without independent research, it’s hard to assess whether fears about political extremism or polarization are justified. We begin with a couple of fundamental questions: what do users actually see, and what do they consider political on TikTok?

Towards answering these, we introduce a user-centered approach to understanding political content on TikTok — one that foregrounds users’ own perceptions of what constitutes political content, rather than relying on top-down definitions. We surveyed 368 U.S.-based TikTok users about their experiences with political content on the platform, and collected annotations and video data from their personalized “For You” feeds using a custom browser extension. Based on the patterns in these survey responses, we created a taxonomy of different categories to describe the data. We applied this taxonomy to both user annotations and video transcripts. This data allows us to understand what kinds of content users considered political on TikTok, and what specific videos they labeled as political, and why.

Our study shows that users identify more issue-based content (like race and gender rights) as being political, rather than formal political topics in their feeds. Augmenting human experts with an LLM to analyze TikTok videos helped us efficiently map this landscape, offering a model for researchers and newsrooms who aim to understand and audit platform content. However, we discovered that LLMs also struggle to identify news content without prominent visual cues, which has implications for the development of content analysis pipelines for TikTok or other kinds of short-form video.

Studying Political Content on TikTok From the User Perspective

We recruited a politically balanced and demographically diverse sample of 368 TikTok users based in the United States. Participants first responded to an open-ended survey prompt: How would you describe the kinds of TikTok content you consider to be political, and what makes it political to you? Then, using a custom browser extension, participants logged into their personal TikTok accounts, scrolled through their “For You” feeds, and annotated 40 consecutive videos. For each video, they indicated whether or not they perceived it as political and briefly explained why. To validate this self-reported data, we randomly sampled one political and one non-political annotation per participant and manually assessed whether their description aligned with the video content.

We then developed a set of 31 categories based on patterns we found in survey responses. We used these categories to label data from the survey responses (N=368), user annotations of political videos (N=2,171), and the video transcripts themselves (N=1,115). For instance, we labelled whether responses pertained to topics such as “healthcare” or “housing”, or specific government institutions such as “SCOTUS”.

Scaling Data Analysis with LLMs

To scale our analysis, we combined expert human coding with a large language model (LLM). We used OpenAI’s GPT-4o model via the Chat Completions API, with temperature set to 0 to ensure consistent and deterministic outputs in our classification task. The LLM was prompted with the following, where “{codebook}” is a dictionary mapping the abbrevaited category name to its description, and “{text}” is the user description (replaced with “{transcript}” when we prompted the LLM to code the video transcripts):

Given the following predefined codes for political content: {codebook}, classify the following text into one or more of these categories: “{text}”. Respond with ONLY the category name (e.g., TRUMP, RACE, EDU, IMG).

We refined our set of categories by clarifying ambiguous definitions, reducing overlap among categories and reducing vagueness by adding examples. To illustrate, the category “Education” was amended from “Any mention of education”, to “Any mention of schools, education policy, or student loans, including references to Affirmative Action” to better capture the breadth of education-related political discourse, particularly in the context of contemporaneous events.

User Perceptions of Political Content

Our study found that TikTok users identified an average of 17.3% of videos in their feeds as political content. The content that participants considered political covered a broad range of topics, from formal political entities like politicians, legislation, and elections to issue-based themes like healthcare, abortion, and COVID-19.

A Disconnect Between Survey Responses and Real Feed Annotations

Analyzing survey responses, we found that participants most frequently mentioned formal entities such as Politicians, Elections, and Government, while specific policy issues (e.g., Economy, Abortion, Immigration) were less frequently mentioned. However, when users labelled the individual TikTok videos in their feeds, a different pattern emerged. The most common themes in these annotations centered on social issues such as Gender and Sexuality Rights (“LGBT person being discriminated against at work”), Race (“This is about systemic racism”), Police (“Police brutality”), and Economy (“This video speaks of the financial strain of inflation in America on young people.”).

This shift suggests a contrast between how people conceptualize political content in the abstract versus what they actually encounter in their daily TikTok experience — suggesting a form of saliency bias in how users recall political content.

Where is the news?

Given TikTok’s rising popularity as a news source for U.S. adults, we aimed to identify news content in our data but encountered challenges in reliably classifying it. Our LLM-based pipeline showed low agreement with human experts when identifying news content in video transcripts. For instance, a clip from The Rachel Maddow Show might not be labeled as news if the transcript lacked key visual or branding cues. The LLM-based pipeline used in the study performed poorly at labeling such content based on video transcripts. In future work we plan to incorporate visual inputs to the LLMs to better classify news content.

Turning instead to our manual review of videos, we observed few political videos posted by official news organizations. Instead, TikTok videos were likely to feature clips from news programs reposted by influencers or unaffiliated users. We hypothesize that traditional news media accounts have yet to fully breach TikTok, despite 52% of American TikTok users reporting they regularly get news from the platform and the News category appearing in 16% of user survey responses. As news content is commonly used as a proxy for political content in social media research, our finding suggests that studies that solely measure user exposure news to approximate exposure to political content may come up short and miss the bigger political landscape when studying TikTok.

Implications

This study contributes a new user-centered method for understanding political content on TikTok, one that centers real-world exposure and user perception over traditional approaches that rely on political keywords, public figures, or news content.

Our findings suggest several important implications for future research, as well as for practitioners looking to understand their audiences or use LLMs as a tool in this process.

First, because TikTok’s algorithm is highly personalized and user-level data is difficult to access, surveys are a common method for studying political content on the platform. Our findings suggest that survey-based measures can provide a useful high-level view of what TikTok users deem to be “political” content. However, we also observed a disconnect: in surveys, participants emphasized formal political content (Politicians, Elections), but when reviewing their actual feeds, they labeled more issue-based content (Race, Gender/Sexuality Rights, Police). This highlights the added value of in-situ feed data in offering a more precise view of political content on the platform.

Second, analyses focusing on traditional news may not be optimal for studying TikTok. In both user annotations and our manual review of videos, we saw few official news outlets. Instead, news content frequently appeared in repurposed formats, such as clips from TV segments shared by influencers, making detection based on standard news categories unreliable.

Finally, our exploratory work using a codebook + LLM approach suggests a promising direction for scalable analysis of political content. While strong human-LLM agreement was not achieved for all categories, performance was adequate for some codes. For other categories, like News, researchers could also try to leverage the full multimodal nature of TikTok videos, incorporating visual features into our audio analysis pipeline to improve detection.