TV seems to know what you want to see; algorithms at work

Your TV is inside your mind.

It knows what you watch. More than that, it knows how you watch. When you pause a program, your TV is taking notes. When you rewind or fast-forward, the machine jots that down too.

But here’s maybe the scariest part of all: Your TV knows what you want, maybe even before you do.

This is where technology has led us. The algorithms that spit out online recommendations for television series, movies and more are taking artificial intelligence to a new level. Top providers such as Netflix, Hulu and Amazon — which tens of millions of Americans get either through set-top boxes such as Roku or via personal computers — employ large engineering teams dedicated to cracking the code of what users want and guiding them to it.

Nothing less than the future of the entertainment business is at stake, as the industry continues its landmark shift from broadcasting to time-shifting and niche programming.

“You can use algorithms and user interfaces to get the right content to the right person … as opposed to just throwing something to the airwaves with a hope and a prayer,” said Todd Yellin, vice president of product innovation at Netflix, which has at least 60 staff members devoted to recommendations. Already, he added, such research has proved that many of the categories that have guided the TV industry for decades — such as statistics about viewers’ age and gender — are meaningless in terms of determining what people actually watch.

Most recommendation systems are based on what’s known as “collaborative filtering,” a mathematical process that can sort large data sets into groups that share certain affinities or characteristics. That information can then be used to offer viewing advice — based not on what a friend or a critic might think, as in the past, but rather on what many other users with similar taste have chosen over many visits to a site.

For example, if you watch “Downton Abbey” on Amazon, the recommendation service might suggest you try “Cranford,” a series starring Judi Dench about life in a small British village during the Victorian era. That is because the algorithm has determined that many viewers who match your taste and viewing habits and have watched “Downton” often go on to watch and like “Cranford.”

It might sound simple. But computerized recommendations are an imperfect and still-developing area of research with a lot of flaws (more about those in a minute). And yet companies keep at it because of the potential payoff.

“We are always looking at ways to improve and evolve our algorithms and recommendation engine,” said Tian Lim, Hulu’s chief technology officer.

Why the devotion? Netflix, which has more than 50 million subscribers worldwide, says about half of all its viewing comes from recommendations. At Hulu — which has more than 6 million subscribers for its paid Hulu Plus service — the figure is closer to 75%. Netflix takes the mission to improve recommendations so seriously that several years ago it held a contest with a $1-million prize to whomever could best improve its system (a team of AT&T researchers won).

Michael Ekstrand, a recommendations expert and computer-sciences professor at Texas State University, said that such systems may be far from perfect but that they are good enough to affect business already and getting better all the time. “They can result in a substantial lift in peoples’ movie-watching [and] purchasing,” he said.

The ultimate goal is to get so good at predicting what customers want that, in the not-too-distant future, a TV show or movie calibrated precisely to that person’s taste and habits will start playing whenever someone logs on to a site.

Businesses have been trying to harness the power of predictive statistics for decades. Long before computers became commonplace, marketers were comparing hard copies of sales charts and statistical tables in a bid to forecast buying trends, Ekstrand said.

Modern online recommendations began in 1992, when two American computer scientists, John Reidl and Paul Resnick, developed a tool that collected user ratings to suggest articles on Usenet, an Internet discussion system.

During the first dot-com wave in the late 1990s, tech startups seized on collaborative filtering as a way to stand out from competitors. Amazon began offering an algorithm-based recommendation service in 1998, just a few years after the company launched (an Amazon spokeswoman did not respond to requests for comment).

Netflix has been recommending movies and TV shows to subscribers since the days when it was best known for shipping red envelopes with DVDs through the mail.

Now it’s a giant of streamed entertainment, not to mention a producer of acclaimed series such as “House of Cards” and “Orange Is the New Black.”

“We daily process billions and billions of [data] entries to come up with the recommendations for every single session, and the algorithms are quite sophisticated,” said Carlos Gomez Uribe, an MIT-trained statistician who serves as vice president of innovation at Netflix.

It’s all part of the company’s push to expand its global subscriber count, which this year sailed past 50 million.

And keeping those customers happy as well as luring new ones are key to its business: In October, Netflix stock was hit hard after the company missed its growth targets.

Since recommendations are often the first thing a subscriber sees when he or she logs on to Netflix, “putting that right foot forward is important to us, and that’s why we invest so much to get it right,” Yellin said.

Of course, if algorithms could reliably decipher and predict human preferences all the time, every TV show would be a smash hit. But collaborative filtering has plenty of blind spots and glitches.

One problem is what is known as the “gray sheep.” That is a nonconformist whose taste is hard to compare with that of other people. Sometimes she likes what other “Downton Abbey” fans like. But sometimes she doesn’t. She’s impossible to pigeonhole. Algorithms have a difficult time accounting for such people.

Another unsolved problem involves new users, about whom the computer simply doesn’t have enough data about behavior to make any useful recommendations.

Of course, the user can always answer a list of questions about what sort of TV shows or movies he likes. But that leads to another problem, what might be called the self-reporting bias. People tend to report an idealized view of their own behavior, while what they actually do is another matter.

“They may say, ‘Yeah, I like these serious dramas,’ but they go watch a bunch of comedies,” Ekstrand said. “That stated preference might be aspirational. The user wants to be the kind of person who enjoys watching Scorsese movies.”

Netflix learned early on about the problem of self-reporting. But during the glory days of DVDs, the company had little data to go on besides a list of the titles the customers ordered and — assuming they actually bothered to offer their input — the star ratings they gave to movies or TV shows. The company didn’t even know whether customers had actually watched the DVDs they returned.

That’s all changed.

“With streaming, obviously, we have a ton more data,” Uribe said. “We know when you fast-forward, when you pause, whether you re-watch it, how much of it you watched and so on.”

That means, he said, more information for the algorithms, which are getting ever more precise at tailoring to each user.

Of course, algorithms aren’t always the way to go. HBO offers its original series and a small selection of theatrical movies on its popular HBO Go streaming service. Sometimes the network will hand-pick certain titles for a certain seasonal theme, such as a recent roundup of programming for Veterans Day.

But it leaves algorithms out of it — a choice made easier because HBO has a much smaller library than Netflix and is collecting programs that have already run on the cable network.

“Instead of an algorithmic approach to recommendations, we do more active curation of programming on HBO Go, which enables us to choose featured content based on what we believe users will be likely to enjoy and what’s important right now,” an HBO spokeswoman wrote in an email.

But the companies that rely on algorithms believe that such recommendation systems are only in their infancy. In the future, users may indeed feel that the machine is reading their mind and knows exactly what they want to see.

Netflix’s Yellin points out that the machine may never know if a user is, for instance, in a bad mood because his boss bawled him out that afternoon. And even if the machine did know, would it be best to recommend a pointed comedy — “Horrible Bosses,” anyone? — or maybe some escapism, a la “Pulp Fiction”?

Such problems will keep the engineers busy for years.

“We should go very far with analyzing behavior and trying to help our users find something great to watch,” Yellin said. “It’s never going to be that utopian, but we’ll keep striving for it.”

Twitter: @scottcollinsLAT