There is some interesting stuff here, but if you want a one-sentence explanation of why Tcl has declined I'd suggest: It doesn't have a hook.
What does Tcl offer? Half this article is about Tk, but Tk is (a) a generic GUI toolkit, and therefore competing directly against everything from Java Swing to Flash to (the big winner) HTML/JS/CSS. And (b) not Tcl-specific, right? Didn't I see Perl bindings for Tk go by?
As an embeddable language Tcl somehow got eclipsed in the branding battle by Lua (don't ask me why; not my field, all I can do is repeat glib rumors) and in web-app land it got crushed by PHP just like Perl did. If you want to toy with extensible syntax in your language you can use Ruby DSLs; if you want the real thing you go to a Lisp. Tcl's a lousy language for mathematics and I can't imagine replacing a shell script with it (I think I vaguely remember a Tcl shell, but maybe I dreamed that, because it has no mindshare). It doesn't run in the browser. So what is Tcl for? Just tell me in a sentence. And, no, "being a nice average general-purpose language" is not a brand worth anything.
I think you should get an afternoon and learn Tcl, because it is clear from your post that you don't know it enough. A few remarks:
1) Ruby DSL, really? After you understand Tcl you'll discover how Ruby is not really a language suited for DSLs compared to TCL, even if Ruby is quite suited for this use case indeed. Tcl is not just much more DSL-ish as a syntax and structure, but also has some kind of "run time lisp macros" (uplevel) that let you do everything in your DSL.
2) Tcl built-in event loop provided a way to write very fast evented I/O apps easily. Something node.js is providing now was, in a different form, available since 10 years, and nobody noticed. However it is still very interesting, now with support for coroutines Tcl is very suited to take the state between firing of events.
3) Tcl as alternative of shell script is great, for a number of reasons: from the [glob] command to an [exec] command that is able to mimic a lot of what a shell can do. See the Redis unit test for an example of scripting stuff with Tcl.
4) Tcl vs Lua: Lua IMHO wins in the implementation, Tcl has not an ANSI target, and is a much larger implementation. But actually it is possible to write a large subset of Tcl in a very small space. I tried it in the past with http://jim.tcl.tk, and indeed now this implementation is in use in the embedded world.
Tcl lost at this point, but programmers willing to understand it will do a great thing, it is like to be exposed to a different new world. It's like learning Scheme or FORTH: mind changing.
Actually, I was technically a professional Tcl programmer in 1999, and I remember "uplevel" very well, because I got to watch as the MIT Scheme wizards put it through all its paces. Oh, the things those folks could do with Tcl. I really can't overstate how much fun it was to read their new libraries as they came out.
I admit I've forgotten most of my Tcl, though. Because I haven't had a compelling need to use it in ten years, so I haven't. See above.
Tcl is indeed "a different new world, like learning Scheme" (though Scheme has SICP, and Tcl has... what?) but this essay is not about "why Tcl makes a great academic paper" or "why Tcl is a great way to spend a Saturday night" but rather "why Tcl is not popular as a mainstream programming language". Being "mind-blowing" is not only not a path to mainstream popularity, but it's also a field with a lot of stiff competition: Scheme, Forth, Factor, Erlang, Io, Common Lisp, Haskell... several of these are on my personal todo list ahead of revisiting Tcl.
It's not that nobody noticed that TCL had async i/o, but that it didn't have closures, which make doing async i/o (or even user interface programming) much more difficult in TCL.
In terms of this point, the lack of closures forced one to NOT write that messy garbage of code that people complain that the newbs are doing in node. I.e. it forces one to separate out the behavior from the construction and so systems (rather than off-the-cuff demos) are cleaner and easier to read and maintain.
Are you joking? It requires you to use a bunch of global variables and other auxiliary keys and data structures to keep track of the state that you would have been able to capture in closures.
Without closures, it's even harder to write reusable "components" that you can make more than one instance of. Look over that TCL code and try to figure out how it handles making more than one instance of each kind of window. It supports multi-player mode by making multiple X11 server connections, so it can have more than one instance of each kind of window, one for each X11 server connection, and some windows can have multiple instances on each server (like the edit and map windows).
I had to write a lot of boilerplate infrastructure for creating/destroying windows and handling events, to store all the state for each window in global variables and structures, keyed by the window id, linked together symbolically, so windows on different screens don't clash, and even fix some bugs in Tk itself relating to having multiple windows on different servers: http://code.google.com/p/micropolis/source/browse/trunk/micr...
To create each window it would reload a file like this each time that set up all the globals and window links and bindings like "LinkWindow $head.map $win" and "bind $win <Visibility> {[WindowLink %W.view] Visible [string compare %s FullyObscured]}". Ugly shit man! Give me closures any day:
http://code.google.com/p/micropolis/source/browse/trunk/micr...
Multi-head X11 support was not well tested in 1993, it got confused if the X11 servers had different visuals, and the TCL menu handling code had to be rewritten to keep the state in per-window structures instead of globals that made the assumption that only one menu could be popped up across all screens at once. Notice the "set screen [winfo screen $w]" and "set tk_priv(posted@$screen) $w" stuff:
http://code.google.com/p/micropolis/source/browse/trunk/micr...
> 3) Tcl as alternative of shell script is great, for a number of reasons: from the [glob] command to an [exec] command that is able to mimic a lot of what a shell can do. See the Redis unit test for an example of scripting stuff with Tcl.
The thing TCL doesn't have are pipes. Not even Unix pipes, just some way of chaining little blocks together, so that complexity extends rightward, as you type.
Yes, tcl's `exec` command implements a special syntax for using UNIX pipes, but what I'm referring to is not UNIX pipes, but a language feature - an interactive language needs to implement some kind of pipe/chain/cascade syntax, so you can use the results of a command without having to go back and surround it with parentheses or brackets.
Tcl is at least as good at DSLs as Japanese Tcl. It has the same spare syntax and the facility (via uplevel) to define new control structures.
I don't think it's the hook that's missing (Tk was a major hook; if the hook was all that mattered, Tk would have kept up with the times).
I think it's just that Tcl is a crappy language. It's always love with me and Tcl, but like this article says: everything in Tcl is a string. The "language" is just a stream of commands --- it parses comments! For normal programming tasks, it has all the syntactical downsides of Lisp notation, but none of the upside. It's slow.
There was a time when Tcl was killer because it was the easiest language to (a) embed and (b) extend in C. But lots of languages do that well now, and Lua specializes in it. Embeddability was the last hook for Tcl. It's hard to see why anyone would choose it for a system today.
As a TCL programmer, you are permitted to imagine "everything is a string". In most TCL implementations, though, your "strings" are stored as objects underneath, and converted when necessary. So if you say:
set foo [+ 1 2]
incr foo 3
puts $foo
the variable `$foo` will not be converted into a string until the last line.
> I can't imagine replacing a shell script with it
I can and have. Whenever a shell script starts getting at all complex, I grab a scripting language. These days it's Ruby, but in the past it was Tcl, and it did a fine job: it has a bunch of built in commands that are convenient for system tasks, and plays very nicely with the environment it lives in. So, quite the contrary: due to the OO (although this is being fixed) and 'GC/reference' problems, I'd be more concerned about big systems in Tcl, and not at all with little scripts, where it still is an excellent choice.
You're quite right about "the web" being the big winner in terms of "cross-platform toolkits" and as a bonus deployment is also way easier. Tcl can do the web though. See: http://flightaware.com/ . But of course in that field, things didn't play out like they did with Tk, where it was probably the most popular option.
Also, while I use Ruby these days quite happily, Tcl has it beat in terms of DSL's, IMO, due to the extreme simplicity/plasticity of the syntax.
Why Lua (probably deservedly) started beating Tcl in the 'embedding' space is covered in the article.
Python, by the way, does pretty well with being a "nice average general-purpose language", although I do sense some envy on their part over the popularity of things like Ruby on Rails. So I do and I don't agree with your conclusion. If Tcl were where Python is today, it would be a much healthier language, even if not the biggest thing out there. Also, by having a 'reservoir' of people using a language, you're more likely to get someone doing something that turns into a big hit.
The frustrating thing about "the web" as a cross-platform GUI is the cutting-edge demos these days are of things we last saw on the PCs of the 1980s. Its like we stepped back 30 years and are celebrating ourselves.
This is because the new wave of developers doesn't have any connection to the old scene to see how absurd this state is. What would be trite and unimpressive in the context of what was done decades ago on far less powerful hardware with far more low-level languages is instead very impressive to those that are used to incredibly complex runtimes (e.g. browsers and modern operating systems) and have performance issues as a result. I think computing needs a low-level renaissance. "The web" is no panacea; to the contrary, it's going to keep expanding in unavoidable complexity to negate all improvements in algorithms and hardware over time.
If I ever made a tool with a command interface (like gdb or ftp), I would probably use Tcl as the command language. The only alternative seems to be a homebrew design. None of the other scripting languages I can think of have a sufficiently convenient syntax for interactive use. But that's a very small niche these days.
Interesting you mention TCL's focus on command-line usability. I think it was a great step forward there, but there's a fundamental thing it's missing, which is complexity extending to the right. What I mean by that is, say you have a command like
tcl> foo { bar baz zot }
The only way to use the result of the command is to surround it with square brackets, which requires me to jump all over the line:
tcl> quux [foo { bar baz zot }]
This makes it pretty difficult on a readline-style interface to cobble together small commands Unix-style.
And while interactive languages may be a small niche today, I think part of the reason is that nobody's solved the usability problems as elegantly as the Unix shell has.
What does Tcl offer? Half this article is about Tk, but Tk is (a) a generic GUI toolkit, and therefore competing directly against everything from Java Swing to Flash to (the big winner) HTML/JS/CSS. And (b) not Tcl-specific, right? Didn't I see Perl bindings for Tk go by?
As an embeddable language Tcl somehow got eclipsed in the branding battle by Lua (don't ask me why; not my field, all I can do is repeat glib rumors) and in web-app land it got crushed by PHP just like Perl did. If you want to toy with extensible syntax in your language you can use Ruby DSLs; if you want the real thing you go to a Lisp. Tcl's a lousy language for mathematics and I can't imagine replacing a shell script with it (I think I vaguely remember a Tcl shell, but maybe I dreamed that, because it has no mindshare). It doesn't run in the browser. So what is Tcl for? Just tell me in a sentence. And, no, "being a nice average general-purpose language" is not a brand worth anything.