This is what's frustrating about being a Tcl fan: Tcl had this something like 15 years ago, because it was necessary for Tk. "Tcl don't get no respect" :-/
That said, though, what separates the "men from the Erlangs" is that in Tcl, Ruby, Python, Javascript, and company, "never block the reactor" is the name of the game, you can't do a while { 1 } { something } in your event handler or it will wedge the whole thing.
In Erlang, thanks to its built in scheduler, you can.
What always bugged me about Twisted, or similar design is that you have to really flex your brain to get proper error handling (or maybe I'm just not experienced enough with them? :) please correct me if I'm wrong) It seems that you need a lot of shared state to kill some other process that is connected to your action in some way... You need to set a flag that will stop the other code path in some place. Then you need to know how to resolve the conflict (is one process providing resources for the other one? how do you synchronise closing down?)
In Erlang (or a similar multiprocess / actor-based system) you just kill the computation you want to stop. It's up to the killed side (or their supervisor) to know how to handle it - but it doesn't happen during the next computation loop, or when it's convenient - it handles the condition now.
I learned Tcl back in college (say 15 years ago) and have not used it since. However, while investigating things like EM and Twisted, I for some reason thought of Tcl. I (and others [1]) asked on the Tcl mailing list about creating a "C10K" server in Tcl but unfortunately Tcl does not have support for epoll, kqueue, etc. at this time.
Ruby's OO is nicer than Tcl's in some ways. Incr Tcl is sort of an ugly attempt to make Tcl look a bit like C++. Indeed, the Tcl core team came up with their own OO thing recently, that actually looks a bit more like Tcl:
No. Things that are not Erlang turn you into a human compiler, forced to manually chop your code into pieces for the pleasure of the computer. Erlang allows you to actually write things without doing the chopping up the scheduler should do.
Event-based libraries are great hacks on top of weak languages/runtimes that allow those weak languages to hobble forward into the present, but they shouldn't be confused for the right approach in the future. I do hope they don't become trendy.
Of course, look for them to become trendy, then for one or more of them to rediscover the pleasure of actually writing code in a straightforward manner, so they will provide you with a preprocessor that tries to do the chopping up for you, but still won't solve the problem of doing too much work in one timeslice. Then they'll rediscover that cooperative multitasking was abandoned years ago for good reason.
You know, there's nothing new under the sun, but sometimes you just wonder... this much hype around cooperative multitasking? Seriously?
I don't want anything else to decide what order my code runs. I want it to run in the order I decide.
Once you move away from the stupid "I want these 100 things to run at the same time", and realize that's impossible unless you have 100 CPUs, then you can start to move forwards and learn a better way.
Event based / async programming is simple once you get used to it, it's hugely liberating, and _extremely_ efficient. It reduces code complexity, and means you _know_ what order your code will execute in. Something that I kinda like knowing.
That's flat out ridiculous, to the point I have a hard time not just calling it stupid. Nothing stops you from using thread pools in Erlang, if that's what you want to do. It's just that you aren't required to do it, either. You compare "an idiot writing Erlang" with "a smart person writing in an event-based system", but who cares about that comparison?
Event-based async programming is a hack around language weaknesses and is a strict subset of Erlang's capabilities, in that anything you could write in those frameworks you can write in Erlang. Yes, including "knowing when things execute". In fact I'd say you get more control because you have more choices; you don't have to lock things up if you want to actually run a computation of some sort.
I suggest you spend some quality time with http://www.erlang.org/doc/design_principles/users_guide.html . You can stop at the "included applications" chapter. Erlang is a friend to event-based stuff, after all, it just handles it naturally, without you having to do the scheduling unless you want to.
You don't really know a priori the order everything is executing in, unless you control the remote ends of all the connections and you have no timers. I've lost days of my life to timer cancellation bugs.
Sure, you don't know the exact order your code executes if it has external inputs :) but you know that your code will execute 'linearly' and then adhere to any program flow constructs you put in, and nothing else.
You can be 100% sure that it won't suddenly stop executing here, and jump over to some totally disconnected bit of code for a while, then jump back.
You don't have to worry about a whole class of bugs which take up weeks/months of peoples time.
Not sure what you mean @ timers, I was really talking about async/event driven in general. Timers seem like an implementation specific detail on top of that. I haven't really seen any issues myself there :/
"You don't have to worry about a whole class of bugs which take up weeks/months of peoples time."
I reiterate, you need to spend some quality time with Erlang. This time I recommend http://erlang.org/download/erlang-book-part1.pdf ; Erlang has some powerful answers to those "whole class of bugs".
(Actually, not being aware of that does rather make sense of the rest of your views. I have first-hand experience that Erlang really does make those problems go away; I like to say that Erlang takes concurrent programming from an exponential problem to a polynomial one. It still isn't "trivial" and I don't know that it ever can be, but it makes it so a human can do it.)
I don't get what you're saying to be honest... When you have 2 actors that are executing "independently", you're just doing (very-pseudo-code) this: (assume recv == wait_for_N_bytes_or_timeout_after_M_seconds)
case recv(10, 5) in
timeout -> handle_this_and_throw
data -> do_something(data)
in do_something you've already made a progress to another state (unless you want to handle the FSM explicitly, then nothing is preventing it) and can throw to cleanup.
In an event based scenario:
def got_some(data):
buffer += data
switch(current_processing_state):
case state1:
buffer = try_to_handle(buffer)
case state2:
....
Now you might have enough data in the buffer to handle it, or you might not. You cannot check it before, because each state might need different amount of data. Try to insert timeouts into this code in a way that also closes the in-state resources now... yeah... forget about throwing exceptions really. Any internal exception thrown by bad code is very likely to hit your code in a bad way unless you handle all possible exceptions in every state separately.
I see more code and more complexity in event style. Care to provide some counter-example? (and I'm not saying it's not possible - just haven't really seen one yet)
That's strange. I always get a headache when I have to think about order of execution (e.g. in C or Ocaml). Haskell (which is not parallel by default, so it's slightly off-topic here), manages the order for me, and I feel I have an easier time writing code, less code.
Why should you need to write the code differently from that? It's nice that the language/environment handles things for you, rather than manually dividing things up into little chunks.
Why should you need to write the code to do each individual part of the calculator?
I disagree, dividing up the work yourself into chunks means you retain complete control of when your code executes. That actually decreases the complexity of your code, since you know when everything will execute.
As luck would have it, I'm sitting here writing a fuzzer for a trading protocol in EventMachine as we speak.
Ruby and async network programming go together like gin and vermouth. It's one of the reasons I'm so happy with Ruby's block/lambda support, and so irritated with Python's: many of the basic programming idioms for evented network code have direct support in the Ruby language.
I've done a fair bit of Twisted Python programming in the past, and I find EventMachine to be far more intuitive and flexible. On the other hand, EventMachine will crash on me once in a blue moon, and it swallows exceptions, which is extremely frustrating. They're both great packages.
The big problem that still needs solving for EventMachine is evented adapters for an ORM (any ORM). You've always been able to talk to MySQL over an evented socket, but that's clumsy compared to DataMapper.
I am still totally not sold that Ruby threads are tenable, so a lot of the "advanced" stuff in EventMachine does nothing for me. It's great even without it.
I am still totally not sold that Ruby threads are tenable, so a lot of the "advanced" stuff in EventMachine does nothing for me.
FWIW, threads based on an event backend are a solved problem in Perl. The Coro library is what EM is trying to be; they should really take a look. ("Oh but it's PERL!!!!" Yeah, and it actually works.)
I'll admit to being a major Perl hater. However, this Coro library looks great! Perl threads are horrible pieces of trash, and the Coro library author not only admits this but does something about it by implementing threads the way nature intended them to be. If I'm ever forced to code in Perl again and need or want threads, I will turn to this library first.
The way the Coro author implements Perl threads, by the way, sounds similar to how MacRuby and Rubinius are attempting to remove the GIL in ruby by copying interpreter state so that multiple OS threads can run simultaneously.
edit: after reading further, it turns out these threads are cooperative in that only one can run at a time. Also, it looks like they act more like Ruby fibers in that one must give up control explicitly or make a blocking call that implicitly gives up control. I'd rather the core scheduler be able to preempt threads directly. What if one of the threads takes a long time or ends up in an infinite loop?
"This module collection manages continuations in general, most often in the form of cooperative threads (also called coros, or simply "coro" in the documentation). They are similar to kernel threads but don't (in general) run in parallel at the same time even on SMP machines"
Usually people dismiss Coro because they want to take advantage of multiple cores without running multiple processes. But the reality is that you'll see a maximum 8x speedup if you run 8 OS threads, but if you rewrite your critical section in Haskell, you'll see a 50x speedup. So really , Coros are exactly the right abstraction for "scripting languages". IO performance is great, and expressiveness is great. Speed can be gained in other ways.
I agree entirely about not needing threads in a scripting language like perl for CPU performance reasons, and being able to configure the threads to be scheduled automatically is a big win in my book. Thanks for letting me know about the library!
How does the Perl Coro library compare to coroutines in Lua or Scheme? (It's been almost a decade (!) since I used Perl really actively, and it seems to have changed quite a bit.)
Behind the scenes, it's basically the same thing. It's cooperative so your code does not get preempted where you don't want it to be; the locking is implicit.
But you can switch to preemption and explicit locking if you desire. At that point, IMO, it's time to ditch Perl / Python / Ruby and use a language with a sane STM implementation, because locks suck.
It works best in Thin (EventMachine) or Rainbows! (Fibers).
If you are doing I/O you should use NeverBlock (http://www.espace.com.eg/neverblock/) they've even implemented a MySQL driver, the best part is they're 'drop-in' so you don't have to learn a new API, they just hide the async calls from you.
Fibers I think are closer to Erlang style than the event loop, and much more straight forward to program.
> Honest question: is there a purely event-driven I/O framework (no threads anywhere)?
You might look into a continuation-based framework. With continuations, you can do purely event-driven IO written in a direct style: the framework can grab the "callback" itself by snagging the continuation of its slow IO functions.
You still run into the same problems with blocking (non-IO, or non-framework) code taking up more than its fair share of time, however.
I see. My (wrong) understanding was that EM used a thread pool to do file I/O like Node.js. It appears to add the file descriptor to the event loop's watch list.
I recently had a chance to play around with EventMachine, as I wanted to set up an event-based scheduler within my application.
I was not impressed.
If you try and mix 'EventMachine' and 'other threadsafe code', it seems to fall down pretty hard. In my case, it would run anything executed via next_tick, but would refuse to run any timers... unless I defined a periodic timer in the main event loop, after which timers started working... but only every time the periodic timer executed.
I dug through the code to see if I could fix the problem, and gave up after trying to figure out why they seem to implement timers in two different ways, and it doesn't seem very threadsafe (next_tick is, but none of the timer code looks to be).
EM also expects that your entire application will run inside the EventMachine loop; there's no way to ask EM for a Reactor that you can then hand events.
If you're not running threaded code, then EM will probably work just fine; the Thin webserver looks to be a really nice Rack deployment option if you're on MRI.
If you need to mix EM up with anything else running on the same VM, look elsewhere.
Take a look at Juggernaut. It's an event server based on EventMachine that provides nice integration with Rails. Streams events to the client via Flash.
That said, though, what separates the "men from the Erlangs" is that in Tcl, Ruby, Python, Javascript, and company, "never block the reactor" is the name of the game, you can't do a while { 1 } { something } in your event handler or it will wedge the whole thing.
In Erlang, thanks to its built in scheduler, you can.