Erlang (is great but) is still much closer to the static-location (Actors) paradigm than what I’m aspiring for. For example, if you have stateful calculations, they are typically implemented as isolated (static-location) loops that aren’t textually co-located with the message senders.
As a programmer, I hope that your answer continues to abstract away the problems of concurrency from me, the way that CPU designers have managed, so that I can still think sequentially except when I need to. (And as a senior engineer, you need to — developing reliable concurrent systems is like pilots landing planes in bad weather, part of the job.)
I was doing some Java code recently after spending a decade in async code and boy that first few minutes was like jumping into a cold pool. Took me a moment to switch gears back to everything is blocking and that function just takes 500ms sometimes, waiting for IO.
Flo lead-author here! This is spot on :) Flo aims to be a bit less opinionated than Timely in how the runtime should behave, so in particular we don't support the type of "time-traveling" computation that Timely needs when you have iterative computations on datasets with retractions.
This is also one of the core differences of Timely compared to DBSP, which uses a flat representation (z-sets) to store retractions rather than using versioned elements. This allows retractions to be propagated as just negative item counts which fits into the Flo model (and therefore Hydro).
Currently, Hydro is focused on networked applications, where most parallelism is across machines rather than within them. So there is some extra overhead if you want single-machine parallelism. It's something we definitely want to address in the future, via shared memory as you mentioned.
At POPL 2025 (last week!), an undergraduate working on Hydro presented a compiler that automatically compiles blocks of async-await code into Hydro dataflow. You can check out that (WIP, undocumented) compiler here: https://github.com/hydro-project/HydraulicLift
These code examples aren't fully documented yet (which is why we've not linked them in the documentation), but you can take a look at a (more-real) implementation of Paxos here: https://github.com/hydro-project/hydro/blob/main/hydro_test/.... We're also working on building more complex applications like a key-value store.
Hi, I'm one of the PhD students leading the work on Hydro!
DFIR is more of a middle-layer DSL that allows us (the high-level language developers) to re-structure your Rust code to make it more amenable to low-level optimizations like vectorization. Because DFIR operators (like map, filter, etc.) take in Rust closures, we can pass those through all the way from the high-level language to the final Rust binaries. So as a user, you never interact with DFIR.
PyO3 is the library that enables Python <-> Rust bindings, Maturin is a build tool for packaging PyO3 Rust libraries (which export Python APIs) as Python packages!
Not yet, but this is something I've been investigating. The general plan is to have safe Rust bindings to the underlying Tree Sitter APIs used by custom scanners, and then have the Rust Sitter proc macro expose a Rust scanner as an `extern` function that the Tree Sitter runtime can call back to.
