Can Tumblr Build the TV Ratings Metric of the Future?
Header photo: Screenshot via Tumblr TV Features Tumblr
What do you suppose is the top show on television right now?
No, wait—before you answer that, answer this: What is the show on television right now that feels most like a creative god got a good, long look inside the most people’s souls, then used a big(ish) Hollywood budget to turn what they saw into small-screen magic?
That first question, whatever objective reality it may imply, is impossible to answer. Ask Nielsen, and they’ll tell you the show with the best viewership ratings in all age groups during the week ending August 5 (the most recent data available at the time of writing) was either America’s Got Talent (broadcast) or WWE Entertainment (cable); limit the results to scripted series and those options shift way down the full lists to The Big Bang Theory and Power. Ask Metacritic, they’ll tell you the show with the best average rating across the past 90 days was either The Staircase (critics, scoring out of 100), or Better Call Saul (users, scoring out of 10.0), while the most discussed this year has been Altered Carbon (critics, again scoring out of 100).
Rotten Tomatoes also has a scoring system, but ask them and they’ll tell you the most popular show on their own site at the moment is Castle Rock, even though it only has a score of 86% while four of the shows lower down the list—Cobra Kai at #3, The Sinner at #6, Vida at #7, and The 100 at #10—are all sitting at 100%, as is the top show on their “Certified Fresh” list, Better Call Saul, which doesn’t appear on the “Most Popular” list at all. CherryPicks, in the meantime, will tell you that Rotten Tomatoes is overwhelmingly reflective of the male critical perspective, and just wait until fall for their competing female-critics-only ratings aggregator to join the fray. Twitter’s pretty catholic in its tastes, but pinning the trending topics there down long enough to gather hard data is not a job I’ll do for free (Nanette, though; I think most recently the Twittersphere loved Nanette). Over at IMDB, Orange is the New Black is up three spots this week to take over the #1 spot on that site’s “Most Popular (as determined by IMDb users)” list, while here at Paste, that same time period saw Succession presiding over our weekly Power Rankings like the aging lion of a family media conglomerate.
Viewership. Critical ratings. Popularity. Certified freshness. Whatever strange alchemy steers Paste’s small (but mighty!) Power Rankings ship. There’s data of some kind propping up every one of these ranking systems (me, I’m part of that data), but there’s no consistency from one algorithm to the next. Worse yet, there’s no real sense of the quality of people’s emotional, social or creative engagement with any of these shows, regardless of the algorithm being used. I quite evidently find value in criticism (and tolerate the industry necessity of numerical scoring), but I simultaneously believe that above all else, television is a form of art, and art is meant not just to be criticized, but to be engaged with—emotionally, socially, and creatively.
So, look: I don’t doubt that the eight critics (not ours) who gave The Staircase an average Metascore of 92 were moved to… something by the “epic” scope of its true crime reportage, but what emotional engagement from the audience at large does that score reflect? Where are the “THIS is the Bad Place!”-type GIFs from The Staircase, the gleaming artistic moments that television-loving humanity in 2018 might grab ahold of to whip out in daily life whenever the moment calls for it? Where’s the fan art of, I don’t know, “Jean-Xavier de Lestrade’s Razor” (the ambiguity-laden corollary to Occam’s, naturally)?
There aren’t any. Because The Staircase, critically well-received as it has apparently been, is just not that kind of art. Nor is America’s Got Talent. Nor is Better Call Saul. Nor is Sharp Objects, which at least is consistent in appearing somewhere within all those ratings-driven lists rounded up above. Nor are most of the mostly prestige-y shows that make most of these mostly prestige-y (or adult-demo-skewed) lists. Castle Rock might eventually be, but that’s getting ahead of myself.
First, I need to introduce Tumblr’s Senior Content Insights Manager, Amanda Brennan—taxonomy-loving librarian, self-professed Can’t Hardly Wait trash, and the genius behind Tumblr’s in-house data crunching blog, Fandometrics (recently rebranded as Fandom dot Tumblr dot com), which this Pretty Little Liars trash has been using for teen television data insights since 2017’s end-of-year teen television review. (Teen television, for reasons I regularly get into in depth, is rarely included in critical round-ups/”Best Of” conversations. Tumblr, for reasons we’re about to get into, changes that.)
Screengrab of Ships, TV Shows, Anime & Manga, and Video Games Fandometrics lists for the week ending August 6th, 2018.
“Tumblr has always been the place to go to celebrate the things that you love,” Brennan tells Paste, “so we kind of used [Fandometrics] as a way to put some metrics behind it. Putting data to the fans is something I’ve always been really passionate about.”
Passionate is an understatement. When Brennan joined the company in November 2013, the first project she was given to work on was the 2013 Year in Review. “I was basically handed a hundred thousand-line spreadsheet,” Brennan laughs, “and they were like, ‘Can you categorize all of the tags?’ And I was like, ‘Heck yeah, I can!’ And then I did it by hand. At the time it was this new thing that they hadn’t really done before, so all of this, the “most reblogs” list, it was the first time we ever categorized these things people were talking about. It was just a way to really highlight these things in our community.”
That Year in Review list grew into Fandometrics, which in addition to its continued Year in Review lists now encompasses a nine-category “Week in Review” series—covering TV Shows, Movies, Music, K-Pop, Video Games, Ships and Anime & Manga—as well as one-off “Fandometrics In-Depth” lists like “The Avengers,” “Stranger Things 2,” and “Halloween,” all of which work off a taxonomy that Brennan, whose academic training is in information science, has been building since she started at Tumblr five years ago. Shaping that taxonomy are all the original posts, searches, and likes/reblogs (“notes,” in Tumblr-speak) for some five-digit number of tags, which are then weighted according to a secret formula to account for things like visibility and likely expanded user interaction, before being aggregated into each weekly list. Or, as the blog’s About page puts it, “To make a long story short: We weight and normalize the number of actions to create a more accurate picture of each fandom’s influence across Tumblr.”
For a social media platform with the youthful/artsy/fannish/GIF-loving reputation that Tumblr has, this dedication to deep data analysis may, at least to outsiders, be surprising. For Brennan—and for the fans whose passion drives the Fandometrics engines—it’s completely natural. If Rotten Tomatoes, Metacritic, IMDb and Twitter are where people go to declare the artistic value of some new show, movie, or musician, Tumblr is where they go to obsess over the art they already love. And what is more creatively generative than obsession—especially when there is so much visual content available for fans to riff (or GIF) off of?
“Those websites are where people are going to say, I give this [thing] an 8 because of X, Y, Z. They’re very about giving a [thing] a ranking,” Brennan says of ratings-aggregating sites. “On Tumblr, it’s like, I love this thing because of X, Y, Z. It’s not like this is my number ten favorite movie ever, it’s more about the passion you feel when watching it. It doesn’t matter if the content is, like, bad—it’s more about the creative experience you get if you’re making fan art or making GIFs, and just about really diving into your emotions and your emotional connection to the content. Like, I love the movie Can’t Hardly Wait, and it is a bad move. It does not hold up. I know I am trash for [loving] this movie, and I know this movie is trash. I would rate that a 4 or 5 if we’re talking about the actual quality of the film, but on Tumblr, whenever I see a GIF set of it, I will proudly reblog it. It’s a different kind of approach to the content—it’s more about passion and the positive feelings around the things.”
This, without a doubt, accounts for that wild discovery I made at the end of 2017 when eulogizing the three teen television titans that bowed throughout the year (The Vampire Diaries, Pretty Little Liars and Teen Wolf) and found Fandometrics telling me that rather than any one of those three topping the live-action TV list for the year, it was some random Norwegian public television series called SKAM that wasn’t even available legally outside of Norway and therefore had to have not just torrents, but transcripts and translations, crowdsourced by obsessive fans. (Rotten Tomatoes, meanwhile, listed Alias Grace as 2017’s top show, while Metacritic called it for The Leftovers.)
Collage of 2017’s Top 10 lists from Rotten Tomatoes, Metacritic and Fandometrics.
Brennan was equally surprised to see SKAM appear in her spreadsheets. “At first I was like, what is this? Then I looked at the tag and it was full of GIF sets. Even though it was an international show and wasn’t in English, GIF sets convey emotion that you don’t need language to share.”
This is a theme that keeps coming back around in our conversation, that the more GIF-able, fanart-able content a piece of media has, the more likely it is to catch hold of Tumblr users’ interests. With all of SKAM available uniquely via streaming Internet video, a SKAM-shaped GIF tsunami was all but inevitable.
That’s not the only aspect of SKAM that primed it for Tumblr obsession, though. As Brennan explains, “That show was really interesting because it’s a set of teens [who] all come from different backgrounds, so there’s something everybody can identify [with]. So much of Tumblr is, you come here because you love something and you end up learning about yourself. You can be in any part of Tumblr and just really dive into the things that make you you, and SKAM hits all of those notes. I think it’s one of the coolest stories in general of stuff that has happened on Tumblr. This show that may not have had this wide audience, but via the translation into GIFs and English and other languages, it [conveyed] stories that teens are thirsty for. Young people in general just want to see themselves represented and have their feelings seen on a screen.”
That Tumblr is for young people—and femme people, and queer people, and fiercely progressive people—is taken as a given, despite user information being scrubbed from the data Brennan and the rest of the Fandom/Fandometrics team uses. Tumblr culture is nothing if not inclusive, but the clearest voices seem to resound from accounts that list their owners as some combination of young, femme, queer, and a bit radical.
“People really love the gritty teenage show,” Brennan says, simultaneously predicting that Netflix’s The Chilling Adventures of Sabrina is going to hit it big with the Tumblr crowd. “People love shows with canonically queer characters. I saw a huge surge [with] Brooklyn Nine-Nine when Rosa came out as bi. A lot of people were like, ‘Oh, I want to watch this show now!’ Wynonna Earp is another one, that’s like the perfect formula for Tumblr: Well-written, starring women, there’s a queer canonical relationship, and she speaks like a Tumblr user, Wynonna, she’s got that kind of sassy vibe, she doesn’t care what people think, she’s just her truest self. Anything where a character can be the truest form of themselves is very much Tumblr. Steven Universe is a great example. Steven Universe is often one of our top shows because it is so genuine, about not being afraid to be who you are and just loving who you want to love. It’s just such a positive message. A big theme we’ve seen across Tumblr as a whole is just positivity and self-care and how to take care of yourself mentally as well as physically, just positive mental health vibes. Even shows that might not hit anywhere on the Nielsen ranking, like The Bold Type, people really identify with that. Kind of going back to Wynonna Earp, it shows very strong women, in leading roles, and queer characters.”
Brooklyn Nine-Nine, #WayHaught, Steven Universe, The Bold Type—so far, so predictable, at least for this teen television critic. But not everything reflected by Fandometrics’ data is so obvious at first glance. Like, for example, Big Brother 20, which is currently holding steady at #3 in TV Shows. (Fandometrics also includes data on titles’ movements week to week.)
“Big Brother has been trending massively this season,” Brennan confirms. “It always comes back, and then [in its off-season] Big Brother Canada trends. I think [it’s] that kind of raw emotion of the reality show, where people can dive into that, plus it’s constantly on. What I love about our TV list, and partly why I wanted to start the weekly rankings in general, is that it doesn’t look like Nielsen. I think that it’s kind of a way to say like, hey, this is what teens are into, this is what the younger demographic is into, and this is what they’re passionate about. Like, you can watch a TV show and be a fan of it, but to participate in the fandom is the next level of dedication and passion, and people come to Tumblr to do it because this is where the other people are who are celebrating the thing that they love, and whether it’s because they love a character, or a storyline, or whatever it is about the show that clicks to them, it’s clicking to a lot of people.”
That said, no matter how much clicking new shows do within a fandom, Fandometrics shouldn’t be seen as a replacement for Nielsen (or IMDb, or Metacritic, or Rotten Tomatoes/CherryPicks). Truly, it couldn’t even become one if it tried—part of that user info stripped from the data Brennan and her team uses? Audience size. “The topics on the Fandometrics lists represent the top 10-25% of all the conversations on Tumblr,” she explains. “They’re ranked by a trending score, which measures conversation volume through searches, original posts, reblogs and likes. [It’s] the volume of conversation [that] is a key metric for Tumblr to best understand its users.”
Volume is a compelling metric, sure—especially for critics like me who want to keep a finger on the pulse of the Youth—but while it may supply fascinating insight into a young-but-mighty corner of the wider television audience, Nielsen and its cohorts are saved by the fact that Fandometrics’ proprietary measuring stick is purpose-built to reflect the Tumblr community, and the Tumblr community only. And while that community is where the most passionate fandoms find fertile ground, it is by no stretch of the imagination where the fandoms of shows with the most viewers end up. It may not even be where the most fans of the most passionate fandoms end up, given that passion—as the still-active Tumblr users on Twitter are underscoring as they recoil in horror from the “grass is always greener” piece published in The Outline earlier this week—can often leave a fandom utterly inhospitable.
“[U]h, I’m there everyday, it’s a hellscape, would not recommend,” tweets @elizabethminkel, a literal academic Tumblr researcher.
“The idea of all of Twitter, mass-migrating to Tumblr in 2018 expecting to find a giant meme party circa 2012-2014,” Vox’s Aja Romano adds (and you can almost hear the cackling scoff), “and finding instead the Purity wars, the Discourse, and the great Voltron ship war.”
“This is the worst thing I have ever seen please no,” tweets @lizbelsky at the top of a thread that, not for nothing, later includes a gif (very Tumblr) of Brooklyn Nine-Nine’s Gina (extremely Tumblr) dancing obliviously by herself while the precinct goes to hell behind her, captioned “you on tumblr in 2018” (extra, super Tumblr).
So Tumblr, like everywhere else where humans gather on the Internet, can be Bad. And that Badness, in turn, might skew both what trends on Brennan’s lists, and to what degree. Or it might not, as vitriol fills a show’s tag-bucket just as well as adoration. While we spoke long before The Outline’s piece appeared, for her part, Brennan was upfront about the fact that volume doesn’t necessitate love. “We have instances where people trend because they did something terrible, like Logan Paul,” she notes from a slightly different angle, “[but] luckily, Tumblr users are smart enough to understand that [that was] trending because it’s a negative story.” Still, she concludes, “most of it is just gushing about things you love the most.”
In any case, both the Paul example and the response to The Outline’s piece are case in point as to why Fandometrics won’t—and shouldn’t—be replacing Nielsen et al. anytime soon. All eyeballs (and obsessions) may be good eyeballs (and obsessions) where ad revenue is concerned, but Tumblr’s never going to be able to provide to advertisers the raw data from studios and networks that Nielsen does. Meanwhile, as self-selected or critically blinkered as Metacritic, IMDb and Rotten Tomatoes can be, their putative commitment to sifting artistic and/or storytelling quality from pure entertainment value is something that Fandometrics’ focus on passion can’t match.
Still, that focus on passion is, in turn, the one thing that none of those other ranking sites even get near illuminating, and thus is the very reason that Fandometrics needs to be given a permanent place in pop culture’s critical metrics tool belt. The television landscape is large; it contains multitudes. And only with the full constellation of critical tools will we be able to measure what shows, if any, are striking the most chords.
To that end: Over the course of writing this piece, Castle Rock still didn’t climb into Fandometrics’ Top 20, but what with Stephen King’s creepy “multiverse” having had decades to build up the kind of fandom that might fall into Tumblr’s brand of obsession, the show itself proving to be supremely GIF-able, and Halloween lurking around the corner, I won’t be surprised when it eventually does.
Until then, I may not yet be convinced to give Big Brother a try, but I will trust the Tumblr machine enough to catch up on Andi Mack, which jumped three spots this week to #5 following its Season Two finale. I know well enough about that show, at least, to know its tag volume will be filled with love.
Alexis Gunderson is a TV critic whose writing has appeared on Forever Young Adult, Screener, and Birth.Movies.Death. She’ll go ten rounds fighting for teens and intelligently executed genre fare to be taken seriously by pop culture. She can be found @AlexisKG.