I was actually surprised at how funny he is - many times I find the personality type that supports being so good at that type of thing tends to be lacking in social skills (broad generalization of course) but he comes off as humble, personable and funny. His answer to the Now do you know how people felt when they were competing against you? question was especially interesting to me.
Ken is super-great. He was a fun and friendly guy when I met him at a book signing. I've become a regular reader of his blog, where he shares random trivia and his love of cinema, hipster bands and the 1980's era of Games Magazine.
And if you noticed he's a computer programmer and you're wondering, the answer is emacs.
Not being from the US (and not getting Jeopardy in my country), i've never heard of this guy. He's incredibly entertaining in this interview, definitely caught me off guard considering how serious and straight-faced he seemed throughout the Watson matches (Thanks, YouTube).
This interview is the funniest thing i've read today (3 hour work commute, I read LOTS.)
Oh, and did anyone else keep wanting to upvote his answers?
Ken Jennings got famous as being the longest-lasting Jeopardy contestant ever. He remained undefeated in Jeopardy for over 70 games, and won over 2 million dollars. He was very likeable on the show as well, especially given that we were able to enjoy so much of the show with him on it, which is unusual for contestants.
He's somewhat notorious in the IT industry because his day job (not sure if he still holds it) was a programmer, which at least endeared him to me.
Ah, OK. Given the negative connotation (at least for its more common usage) I was wondering if there was some Ken Jennings backlash out there that I'd missed.
I watched that match back in high school (his streak corresponded with my own academic bowl team's season), and "What is a hoe?" was my response, too.
I'd never heard that semantic sense of "rake" before, and felt somewhat scandalized that "hoe" would be a response on national television (while in high school, I felt great affinity toward the puritans).
Does anyone else think "hoe" is actually a valid answer there, though? It's hard to tell from Ken's facial expression whether he "purposefully answered incorrectly" or just found himself amused at the situation.
That's odd, I've always thought that given its college-bowl atmosphere, Jeopardy! is uniquely poised to attract people who are both erudite and of quick wit.
The guy who hacked Press Your Luck fits more your stereotype; given its flashing lights and general brouhaha, PYL seems more of a magnet for that sort than Jeopardy!. There are also quite a lot of that sort in The Price is Right fandom (yes, it has a fandom!).
I love how he handles the, almost guaranteed, defeat with great grace and humor. For comparison, Kasparov was in tears. Not the same scenario of course, but I truly admire Kennings' attitude and acknowledgment of IBM's (and humanity's) accomplishments exemplified by Watson.
Kasparov was pissed off about the human intervention to Deep Blue during the game. The scene where you see him jump out is where his frustration at knowing and realizing that humans had altered Deep Blue get the better of him.
I don't know about tears, as such, but the NOVA documentary about Watson included a video clip from the end of the famous Kasparov-Deep Blue game. It showed him angrily storming away from the table after being beaten; the way the video was cut, in combination with the narration, gave the impression that he was something of a sore loser. Note, however, that it was a very short clip, and there was probably a lot of context that didn't come through.
Maybe he was angry because he felt like he wasn't performing at the top of his game, and that if he did perform at the top of his game, he might have won. I'd feel pretty angry if that were the case.
"""I AM PLAYING A PRIME-TIME GAME SHOW AGAINST A SUPER-ADVANCED ROBOT! This is the coolest thing I will every do in my life by a factor of a million. The future is here."""
My kids think I'm a little weird because I think this battle is so cool (I think the more you understand about the implementation, the cooler this is), but this statement pretty much sums it up.
IBM: "There's a lot of you in Watson"
Ken: "If it goes amuck and kills humanity and stuff so sorry lolz my bad!"
Chat participant: "I read your first in practically a day, I loved it."
Ken: "I'd like to thank my mom for taking part in the chat!"
Q: "There was an article in the Post the other day about trivia now being trivial because of the ability to look up everything in an instant..."
A: "...in a not-too-distant future where nobody knows their state capitals anymore, maybe trivia geeks will be revered for their even more fantastical-seeming abilities! We will be like gods to you, carried on litters to your feasts."
According to cnet[1]: "Correction at 12:04 p.m.: After this story was published, we heard from PBS producer Michael Bicks that it was not, in fact, Watson that crashed during the show's taping. He would like to make clear the following: "I missblogged last night--It was not Watson, but the system that was the interface between Watson and the Jeopardy computer, completely separate from Watson, that crashed during the taping."
I had no idea. Out of everything I've heard, including the buzzer response time advantage, this seems to be the one thing that isn't really fair. If Ken was a narcoleptic and fell asleep every couple of minutes, he wouldn't have been allowed to be a competitor.
I still value what they're doing and of course it's fun to see the crazy thing play and win at Jeopardy, but I think the fact that it crashes continuously should at least be made more visible to the viewing audience. "Yes, folks, it can beat Ken at Jeopardy, but we still have to press the reset button every once in a while just like you do at home." Skynet may be getting smarter, but it still has fundamental problems.
... no, I'm sort of shocked that's been so covered up. Several of the team members have been at my university giving presentations and talks on Watson (including one of the software engineering practices behind it) and I'm somewhere between shocked and appalled it wasn't mentioned.
It's unfair that Watson is allowed to basically use it's robotic reaction to time ring in instantly when the light turns on, but all audio and video daily doubles are avoided because IBM didn't bother to add audio and video inputs.
If the show is catering to Watson's shortcomings, why couldn't they cater to human reaction time shortcomings?
Watson can't do this: it only buzzes once it has an answer in mind and a sufficiently high confidence interval.
Well, they could have programmed it any way they wanted to. It would have been smarter just buzz in right away given that they knew it would be right most of the time.
Regardless, Watson dominated because it was confident about an answer most of the time before the light turned on. If a human also wants to buzz in right when the light turns on, he or she will always be beaten by Watson. So, the game again boils down to who buzzes in first, regardless of confidence.
Q: I have already read plenty of doomsday reports for the blue-collar workforce that this technology could replace customer service representatives, in-patient counseling, bank tellers, cashiers, etc. Do you think Watson could replace Alex Trebek?
I'm starting to wonder what Watson could do with the database of Stack Overflow.
It looks like you're writing an algorithm that is O(N^2)! Would you like me to optimize it? Let me take a screenshot so Jon Skeet can laugh at you. click
PROGRAMMER frantically presses CLIPPY's close button
"They give Brad and me bottled water during every commercial break. I think you can see where this is going if the machine starts to build a big lead."
"But I wouldn't call this unfair...precise timing just happens to be one thing computers are better at than we humans. It's not like I think Watson should try buzzing in more erratically just to give homo sapiens a chance."
Wouldn't it be better to build a distribution of champion level response times and have Watson draw randomly from that distribution? That would seem more fair.
Here's a crazy idea... what if we changed the rules to Jeopardy so that if the buzzer presses are within a 50ms window then randomly select a contestant.
I think this would be a good change for even all human games.
I think that's a great idea, allow for different pools of time after the buzzer, 0-50ms, 50-100ms, etc and if you end up in the same pool as another player there is a random draw for who gets to answer first.
Seems to me they should let people click their buzzers as soon as they want to, and then just take whoever clicked first. Having to wait for the question to finish being asked seems like it makes it a lot more random rather than skill-based.
According to Wikipedia:
"Before Trebek's second season, contestants were able to ring in at any time after the clue had been revealed, and a buzzer would sound whenever someone rang in. According to Trebek, the buzzer sound was "distracting to the viewers" and sometimes presented problems, as contestants would inadvertently ring in too soon, or ring in so quickly that by the time he finished reading the clue, the contestant's five-second limit had expired. He also said that, by not allowing anyone to ring in until the clue was finished, home viewers could play along more easily, and faster contestants would be less likely to dominate the game."
The first rule of competitive game playing is that anything in the rules is... game.
Illustrating that the computer has the advantage over humans is the entire point. Competitive game play is not about 'fair,' in fact, the best players specifically try to make the game as unfair as possible, within the rules of the game.
But the rules have already been bent to help Watson in areas where it's weak:
• receives question (and answers) as coded text, rather than having to sight-read or listen to them
• no audio/video clues
• no categories that require an extra Trebek-explanation
It may even be receiving a direct signal when it's OK to ring in, rather than assessing Trebek's cadence and the same light-indicator as the human contestants rely upon.
So other 'balancing' changes shouldn't be out of the question.
I particularly like the idea that the penalty for a moment-too-early buzzing could be eliminated, and/or treating all buzzes in a small window around the traditional earliest-allowed moment as simultaneous. That still rewards quickness, without heavily advantaging the computer's ability to perfectly synchronize with the buzzing-period.
Once two buzzes are considered simultaneous, the contestant chosen to answer could rotate in round-robin fashion, or (for maximum drama) be the one furthest behind.
AFAIK, Watson was just using the light signal to decide when to start buzzing in. Just the light, ignoring when Trebek was going to finish saying something.
Presumably a human could consistently beat Watson to the buzzer by predicting when the light would go on.
Not sure about 'consistently', though trying to predict the light seems a reasonable strategy to attempt.
There's a lock-out that applies if you buzz too early, and the variance of the 'beats' between Trebek's finish and the light, if that's a human-mediated process, may be far larger than Watson's light-fired reaction time.
The easiest and fairest fix would be to hook Watson's buzzer up to a dataset of previous Tournament of Champion question-winning buzzer times. Or just use Ken and Brad's previous buzzer times for even more control. Pick a new human-achievable reaction time for every question.
We already know that computers have an advantage over human reaction time. We don't need Watson to tell us that, my laptop can press a button faster than I can. This is like requiring each player to run a mile before being able to answer the question... and then letting one guy drive a Corvette instead.
Of course, the impressive thing is that Watson can answer the questions at all.
The point being that at the highest levels of play, all of the contestants know most of the answers. This is a very important thing to remember. It means that even in all-human games, it's not "do you know the answer" but "can you ring in fast enough."
So it still is really quite impressive that reaction time matters at all to Watson, because it implies that it can compete at the highest levels.
There's a whole book on the concept of game balance, focused on arcade fighting games (Street Fighter), written by David Sirlin, who balanced Super Puzzle Fighter II Turbo HD Remix and Super Street Fighter II Turbo HD Remix.
Particularly relevant is the section on 'scrubs', the people who arbitrarily ban certain techniques for being 'cheap' instead of coming up with ways to counter them:
As he works explicitly in the world of game balance, he acknowledges that it must always be reasonably possible to counter all usable techniques, or else the game is flawed. He even advocates banning techniques that honestly can't be countered, as a way of turning bad (unbalanced) games into good (balanced) ones.
So I suppose the upshot is that Jeopardy!, as it is now, allows any reaction time physically possible given the limitations of the buzzer and the constraints of the enforced time limit (which penalizes you if you attempt to buzz in before Alex finishes, or for approx. 0.2 seconds after he finishes). You could imagine a Jeopardy! variant with different rules in that respect, but it would not be the same game.
Interestingly, Ken Jennings says about Watson what most people say about ordinary players:
Watson sometimes takes some time to get acclimated to a category, so starting at the bottom gives me a chance to rack up some money before it gets confident. In theory!
Supposedly that was Mr. Jennings' reasoning for jumping around during his streak, that the other players didn't have quite as long a time as him to consider the category.
Not sure how tongue in cheek that remark was, though.
I guess the difference is, Watson can easily retain the learning that is built up through a category, even if he hops around the board the entire round. For humans, the context switch is a lot more costly and a category change can catch you by surprise.
A lot of this advantage is from correctly hitting the buzzer in time. When my college roommate was on the show, he said very often the trick was not buzzing in early and getting locked out. It was apparent that most contestants knew the correct answer for most questions. I imagine Watson would not have buzzer jitters.
To provide a better challenge for both the humans and the machine, you should be able to buzz in as soon as the question is visible. Watson no doubt is doing most of the work in the time it takes the humans to load up the question into their mind.
My understanding is that the shows are taped and you obviously know the outcome. You made the recent comment on MSNBC that in order to win you had to play and bet recklessly. Do humans have an advantage in terms of betting (game theory) Your comment seems to be a tell that a human won, any guidance appreciated.
– February 15, 2011 10:59 AM Permalink
A.
KEN JENNINGS :
I would say that Watson has the wagering edge--like you say, it's all game theory and math, and even a cheap PC is pretty good at doing math at high speeds. That said, a human player might be more willing to take risks that Watson is too smart to try. In the practice games I saw, betting big on Daily Doubles and Final Jeopardy seemed like the only way to cancel out Watson's big buzzer advantage.
* * * *
HE DIDN'T ANSWER THE QUESTION! Is this confirmation that Ken Jennings wins?
