This is what's known as a "superstimulus" - something that pinpoints our brain's reward and pleasure centers orders of magnitude more than anything that existed in the evolutionary environment. Heroin and candy bars fall into the same category.
Of course, heroin and candy bars aren't exactly civilization-level risks - they're regulated to various degrees, and the people that overconsume simply self-select out of existence. I'm skeptical that an entire society would fall under the spell of a game (even a really good one) before society develops protections against it (through regulation and cultural norms).
Having tried to implement a certain form of this, I can say that forcing yourself into a certain mode of thinking, top down, is EXTREMELY mentally taxing. It's very difficult to keep up the mental focus necessary for an appreciable amount of time, and I would not be surprised if this actually impeded mental performance on an individual level because of the effort involved, though it's possible overall group performance could still increase.
It seems likely that most people would see things differently were they to have her experience. I like the talk, but I understand why some people do not. But I would not say she was "so easily" drawn into mysterious or magical thinking. She was drawn into it by a massive, life-changing medical event that attacked and damaged the organ which creates her mind. I think she's using the only words, metaphors, and ideas she has available to describe what is probably indescribable at base.
A philosophical zombie is not a decision tree incapable of learning. A philosophical zombie is something that is exactly like a human in every way except that it lacks consciousness. The concept has more than a few problems - see http://lesswrong.com/lw/p7/zombies_zombies/
If a decision tree (or a clever algorithm or whatever) REALLY did perfectly mimic a thinking human (including our ability to learn), do you still think it wouldn't REALLY be intelligent? That there's some sort of important, qualitative difference between being shifted through a decision tree and sensory inputs and feedback loops building up neuron action potentials? That may be true, but the evidence is heavily stacked against such a worldview.
That said, I agree that this is a bad approach to AI. It's not like people haven't been trying to stitch together various subproblems of intelligence for the better part of 50 years.
>Complexity research doesn't need its own institute, because complexity research is a pseudoscience that means whatever you want it to mean
This. As far as I can tell, "complexity science" is really just a dumping ground for things we haven't figured out yet. Some pattern seemingly ‘emerges’ out of nowhere? Complexity! Feedback loops causing unpredictable behavior? Complexity! Non-linear or ‘chaotic’ behavior that's difficult to interpret? Complexity! Don’t know where something starts or stops? Complexity! Self-reference or deep, recursive hierarchies doing something weird? Complexity! You can’t talk about any of them without using the phrase “don’t know”. It’s a field defined by our ignorance.
Hrm, this submission is over 300 karma points as of this comment, which means that hacker monthly #2 will have to include an entire copy of hacker monthly #1...
Of course, at some point letting problems work themselves out just becomes laziness.
"Mr Coolidge's genius for inactivity is developed to a very high point. It is not an indolent inactivity. It is a grim, determined, alert inactivity, which keeps Mr Coolidge occupied constantly" - Columnist Walter Lippmann, 1926
