Can Sophina Turn Me Into a TikTok Star?

Back in April, on the sidelines of the International Journalism Festival, I took part in some user testing that blew me away. Outside a cafe in Perugia, in Italy, TikTok star Sophia Smith Galer held out her phone for us to take down her QR code to download the prototype of her new app, Sophina, which will be released more broadly later this month.

Sophina uses generative AI to help you turn your text article into a script for a short video, for TikTok, Facebook Reels and the like, and then record it by acting as a teleprompter. Less than five minutes after downloading the app, out on the busy street, I — a camera-shy text journalist — had recorded a video based on my GAIN article about Apple’s inaccurate news summaries. I wasn’t quite set up to post it so, a month later, back in my kitchen, I decided to spend two hours with the app to see how many short videos I could post.

I wasn’t completely new to making TikTok videos. In lockdown I had shared some videos of writing Teeline shorthand. I hadn’t made TikTok video where I talked to the camera before as I was worried I was too stuffy, would stumble, stutter, and forget.

Thankfully, Sophina gave me what Lydia Chilton refers to as “activation energy”. The Assistant Professor in the Computer Science Department at Columbia University carried out research into student journalists using another AI tool, called ReelFramer, to help convert text articles into TikToks. Sophina, and ReelFramer, reduced the friction involved in moving from writing long text articles to making short videos.

I’m impressed with how Sophia identified a valuable point in the workflow to implement generative AI: the rewriting of a text article into the first draft of a video script. Importantly, Sophina doesn’t attempt to replace the human’s voice and face. This is something I had not managed to do when I was a product manager for a similar prototype last year. My prototype Podmorph took podcasts and, like Sophina, helped turn them into TikTok videos. However, Podmorph made the mistake of focusing on end-to-end automation. It was a technical feat. It meant minimal effort from users who only had to upload audio then select a quote and a picture. We managed to use AI to make a TikTok video by generating pictures based on an AI-driven analysis of AI-generated snippets of a podcast. We were very impressed with what was technically possible. But the output was not very engaging. In contrast, Sophina keeps the end part of the process — the telling of the story — human.

Editorial judgement

At the same time, I wasn’t impressed with the scripts Sophina generated. Sophia told me that Sophina uses OpenAI’s GPT-4o to rewrite the scripts. The tone seemed a bit too formal and a bit too similar to my articles. The script used “journalese”, the words we would never use when explaining something to a friend but time-pressed journalists use in written news articles. I changed “voiced concerns” to “complained” and “cease” to “scrap”. But I didn’t find a way to see the prompts, adapt them, or try out other models, such as Anthropic’s, which might have improved the scripts.

As one example of an error in the script-generation process, Sophina attempted to summarise a funny anecdote but took out the punchline. Since I first tested Sophina, it has been updated to add what tone you would like (I chose informal) and a description of your job (I chose journalist). Unfortunately these script problems didn’t go away. The tool killed a joke, again, and — as if to make it worse — subsequently added a clumsy segue “it’s not just a laughing matter”. It felt clichéd and unnatural.

I also didn’t always agree with the summarisation. Sophina had selected three examples out of six examples I had given. But it chose the least significant and least interesting of the examples. This didn’t surprise me, large language models, like OpenAI’s, predict the most likely next word based on probability, they don’t predict them based on truth, relevance, novelty, significance, or entertainment value, which are some of the reasons journalists might make a selection. Chilton observed the same limitation with ReelFramer: “It doesn’t quite get what’s the important part,” she told me. “We found that the journalists’ training was super valuable in deciding ‘these are the important points’, ‘this is what people need to know’.” She also observed that ReelFramer, whose prompt she said you could adjust, wasn’t very good at correctly judging the audience’s level of knowledge. “You know your audience as the journalist, you would know what needs explanation and how to explain that.” Likewise, I felt I had to add extra information to my story at the beginning of Sophina’s script about Apple’s summarisation feature just for it to make sense for my audience.

Automation bias

As I used the tool I started questioning my changes to the script — maybe the machine knew more about short video scripts than I did? Chilton observed the same phenomenon in her research participants: “There’s this battle in their heads, who’s the expert here? I’m an expert on journalism, but it’s [the AI tool] more of an expert on TikTok. And so people would actually forget some of their journalism training and sort of give up their agency to the AI.”

Ultimately though I ended up changing pretty much the whole script before moving on to filming the video.

Sophina the app is easy to use and the teleprompter feature helped me read the script and record it all in one shot. Although I think the script was a bit low on my screen and my eyes subsequently are looking down a bit rather than directly at the camera.

Producing my second video, I tried to save time by recording the video without checking the script. Only part way through I noticed a factual error and had to stop.

The script summarised in such a way that it accused someone of something I couldn’t prove they had done. This also didn’t surprise me. Defamation has been a common risk with generative AI output I found in early prototypes I worked on at the BBC. It doesn’t make a tool like this unusable, it just means that it’s probably wise that the person fact checking the script knows the facts of the story. As Sophina appears to be designed specifically for people who write long text articles to change their own articles into short videos, I think it’s reasonable that it’s the responsibility of the content maker, not the app, to ensure accuracy.

All errors corrected, I uploaded my video and left the two videos to the gods of the TikTok algorithm for a week. I came back to 270 views, no likes, and no comments.

So can Sophina turn me into a TikTok star? No. But it got me started. And, for me, that is enough to make it worthwhile.