Virtual Video Store Clerks Take On The Geeks For Netflix Prize
from the wisdom-of-the-movie-geeks dept
We’ve been fascinated with the Netflix Challenge for a while now. That’s Netflix’s offer of $1 million to whomever can improve on their system for recommending movies by 10%. While there were a lot of early success stories in making improvements, all of the attempts seemed to bog down, making much more gradual improvements, but not getting close enough to hit that 10% mark. Earlier this year, we wrote about the surprising run up the leaderboard by a (previously) anonymous individual who approached the problem from a very different perspective, that of a psychologist, rather than a coder, which apparently was quite helpful in getting good results through a very different method.
Now some other folks are trying something completely different, relying on more of a “crowdsourcing” system, combined with a gaming element. They’ve set up a virtual video store, called Video Store Clerk, and set it up as a game for movie buffs. The game players act as a video store clerk, and can see how particular users rated three movies, and are then asked to predict how they would rate a fourth movie, with points given to correct answers. The idea is that they’ll be able to use these crowdsourced predictions to create an even better model than the purely algorithmic model being worked on by various teams.
This reminds me of the research work by Luis von Ahn to do things like tag images via the “ESP Game.” von Ahn has had numerous successes in creating fun casual online games, where the output data is actually very useful for taking on some sort of problem that is quite difficult to do algorithmically (such as identifying what’s really in an image). The real question, though, is if movie recommendations really work that way as well. Perhaps I need to be a bigger movie buff, but so far, I’m not particularly good at figuring out how others would rank a movie. And, unlike the ESP game, frankly, the Video Store Clerk game isn’t all that fun as currently designed. After playing it once, I had no desire to try again. Still, I’m intrigued by the different approach, and wonder if a more advanced (and more fun) version might be much more effective.
Filed Under: competition, games, netflix challenge, online games, video store, virtual store clerk
Companies: netflix
Comments on “Virtual Video Store Clerks Take On The Geeks For Netflix Prize”
$1 million will buy a lot of hours of OLPC children’s time. Just sayin’.
@AC: Dumb reasoning. Just sayin’
Seriously though, that’s like saying that the $100m it takes to make the new blockbuster would be better spent on tackling child poverty in the 3rd world. Yes it would, but the point is that it’s not *spent*, it’s invested in an attempt to make a profit.
Netflix isn’t simply giving away $1m. It’s using that as an incentive to get people to improve their service for them, and therefore increase their profits.
@PaulT
You missed his point. AC’s saying to wire up all those OLPC-using kids to do the crowdsourced ratings/predictions. He’s not making a point about solving poverty. It was actually pretty funny.
And ironically, your post shows why it’s so hard to predict what movie someone will like. People like different movies for different reasons, often quirky ones. What makes one bad movie a cult classic and another simply a forgotten throwaway? Some people “get” the point of a flick, and others don’t. How about mood and setting? Ever watch a movie one ime and think it was great but the second time you saw it, you couldn’t see what was so good about it?
It’s a very challenging problem to solve.
Re: @PaulT
I get the joke but in reality, how many kids in 3rd world countries are movie buffs?…have even seen a movie? Sorry for the depressing thought.
If someone succeeds in this challenge, I am sure that the $1m will be better spent than if Netflix had brought in Accenture or KPMG and paid them a $1m to “consult” on building a better system.
Kudos to Netflix management on thinking outside the box.
Another amazing comment
Netflix is cool, I love the service, and Blockbuster hasn’t quit spamming me since I went with Netflix instead of them… The “Movies You’ll Love” section is cool, I rate movies I’ve seen all the time, and often pick movies for my cue from there. Would be amazing if they make it better (with a game?) it’s already pretty cool…
librarians...
Funny, this is what librarians do most of their day- Read-alikes for movies and books. I do it everyday and if my results are on target or not… forget programmers coding.
Netflix and patents
I have two nagging doubts about the Netflix prize:
1. The leaders seem to all be achieving their results by tuning a combination of methods which may mean that there is going to be little in the way of generally useful algorithm or insight coming out of the process.
2. There are a lot of patents already issued on collaborative filtering and there is a real chance that the winner will find himself being harrassed by a “patent licensing firm”.
Re: Netflix and patents
Probably a good thing you can’t actually Patent Code, only the software that compiles it..
what if someone went and patented from being used in HTML without paying that person..?
It’s the same with algorithms, I don’t pay anyone for writting 2+2 = 4
And that could technically be the result of an algorithm like:
dim a as int
dim c as int
c = a & a
output=a & ” + ” & a & ” = ” & c;
^–Now i know thats overkill to make a point but you can tell, this can’t be patented, it’s not possible to restrict a programming language in that way.
The game would be a lot more fun if you got points for a near miss, especially as movie ratings can be so subjective. 10 points for a correct selection, 5 for being one star off, and nothing for more than that. Then maybe they could get folks to play more than once.
I don’t know enough about the specs of the OLPC to say this affirmatively, but they could potentially be using them to watch movies and become film buffs.
Some suggestions for the game maker:
Don’t end the game. Let people move up when they make enough points. That way you get more ratings. You can still look at the data to see if they’re really bad predictors, and just throw the data away, but keep the game engaging.
Let people skip a rating. If they’re not familiar with any or most of the movies, you just get bad data by forcing people to pick a rating to get to the next one.
result?
Has Netflix gotten close to the desired end result via this crowdsourcing experiment?