I found this a couple months ago and it was an amazing resource (thanks dmitrygr)!
I ended up using atc1441's base station firmware[0] because it's a bit of a pain to actually pop open the case and get the programming pins on the Chroma 74
It's been a long while since I worked there, but IIRC one of the big issues at the time was that the symlink "support" couldn't actually work properly:
- some users want symlinks to be synced as symlinks (and they might not know it, old Apple file formats have internal symlinks), so their files got corrupted if they were accessed on another computer
- some users want the sync client to follow the symlink and copy all the files at the destination, which requires the client to listen to filesystem events for potentially the entire filesystem, which is no longer permitted in many cases
so, not exactly a vulnerability, but a mix of security and product issues.
another former contributor to pb-jelly, though no longer at Dropbox.
protoc plugins have an interesting bootstrapping problem as well: the protoc-gen-$LANG interface requires the ability to ser/de protobuf messages that describe the proto file's AST. If your build system builds almost everything from scratch, including the protoc plugin, this means that you need to have a variant of your protoc plugin linked to a working proto implementation...
That's not to say this is impossible or even difficult, but at the time that I last looked at it (more than a year ago at this point), it made it fairly unpalatable to move the codegen from Python to Rust.
I’ve been playing with Rust on the ESP32 using these tools on and off over the last couple of months or so, relying more in linking to the existing FreeRTOS/C HAL for accessing the peripherals.
One thing I haven’t quite figured out: a substantial number of onboard features are configurable at image build time, which then change the effective C libraries via preprocessor definitions. So, it doesn’t quite work to naively run bindgen on the header file once, and then embed that in a -sys crate and put it on crates.io
I think you can hand craft a -sys crate with all the appropriate config options as cargo features, but that seems both very time-consuming and likely to drift from the newest esp-idf versions. The alternative of running bindgen periodically (e.g. as part of the build process) works fine for one project, but then how do you publish reusable libraries online?
The ESP32 API surface is huge, and the libraries have many configuration settings. Rust-ifying this is a huge endeavour. One way could be to just interface with the vendor HAL using Rust wrapper, but doing that in a safe way that respects the lifetime rules can't be easy? Also AFAIK some lower level radio APIs are locked and hidden behind a binary blob, so the ESP32 can't be misused as a jammer.
Its very tempting to try this due to the huge amount of resources the hardware has, but it is a huge amount of work. Just being able to access the registers isn't very valuable, the ESP is about Wifi, BT and I2S. If there are no ergonomic Rust APIs for those, why even bother?
It's true and a bit surprising how much of the ESP32's API's are configurable or change based on config options. Espressif has built an impressive amount of functionality on top of FreeRTOS. It'd be nice to see Rust more widely adopted, but it seems to require an entirely rewritten SDK as so much of the API are C macros and defines.
Given that I decided to go a middle ground and use Nim with its new ARC (GC reference counting and move semantics). In theory it's less efficient than compile time Rust memory management but it works well in practice. Wrapping the C API is trivial and handles all of the C defines as well as writing in C, since it compiles down to it. And you can use any Nim or C libraries. It's surprisingly effective!
Except you still get data races from the C API's which Rust would likely help prevent. So it's cool to see someone toying with an xtensa llvm backend. I wish LLVM still fully supported a C backend so you could compile Rust to C and build that. Hopefully in the long run, more native rust libraries will exist for more full functionality embedded.
Was looking at Zig for the same reason, manual memory management but less error prone due to "defer", and a design ethos that is very reminiscent of C. Can even compile C directly! I really, really like the aesthetics of the language, but there is no xtensa backend yet.
Rust might just be a leap too big and can only work in blank-slate scenarios.
Zig seems promising too! But it’s still LLVM based (?), which seems to be a limitation in the embedded world. Nim’s templates/macros are nice for fiddly stuff. Not sure how Zig handles places where compile time macros can make things nice. Yah I figure in ~5 years Rust might be better positioned in embedded work as a goto solution.
The compiler toolchain can cross compile to esp8266, but I think the pure Rust HAL is less developed there. If you use Rust to write components/code that link against the ESP-IDF, I’d imagine that would work?
I have the toolchain in a Docker container if you want to try it out.
I'm not Sujay, but I worked on this system once upon a time too! (not anymore)
1. It does redo the minimization phase, but the actual execution is extremely fast, so this cost is minimal. Storing test outputs gets pretty expensive when you are running millions of tests, and since there are very few failures, recomputing this is worthwhile
2. Yes and no! the article talks about this a little, but the "heirloom" system does essentially this, and the "native" filesystem variant of Trinity runs the same Trinity tester code against a real filesystem. The "no" is due to the issues with randomized testing -- since any operation that you do can affect the RNG, the exact operation that is run for a particular seed can change if you swap any part of the system. For regression tests, the operation sequence can be put into a separate, non-randomized test.
3. both
4. Testing in Nucleus is a sliding scale from "unit-test-like" to "integration-test-like" -- Trinity is mocking plenty of functionality; CanopyCheck is simultaneously testing many different components. It would probably be more accurate to say that CanopyCheck is testing a smaller subset of components with much greater fidelity, and Trinity is testing as much of the sync engine is practical.
If you have time, I have another round of questions:
1. Did you try formal methods like TLA+ on the client? I think that the logic covered by CanopyCheck may be a nice target.
2. Do you have some tests with several clients running at the same time on a shared directory? In particular, I think of the termination invariant where the clients are fighting because several users have reorganized the directory by moving a lot of stuff, and each client is trying to converge in a different direction (ie they are making operations that cancelled the ones made by other clients).
3. The article says "In the Nucleus data model, nodes are represented by a unique identifier". Does it happen that a node has to change its identifier? For example, in a scenario like this one:
1. We thought about this at one point, IIRC, and various parts of Dropbox sync have been formally modeled. But CanopyCheck and Trinity are useful in part because they test the real, production code -- the hash-indexing bug that Sujay mentioned elsewhere on this page is an implementation error, not a design error.
2. Kind of. There are tests for interactions between instances (in particular, between different Dropbox folders on the same machine), but running Dropbox twice on the same folder is explicitly not supported.
3. Yes, this happens. There's logic to handle these changes, since some applications expect to use in-place edits and others swap in a new file -- you don't actually need grace to have an interesting situation. You can experiment with this yourself in a Dropbox folder :)
I can't speak for Slack, but it's not unreasonable to believe that a single machine's available output bandwidth (~10-40Gbps) can be saturated during a deploy of ~GB to hundreds of machines. Pushing the package to S3 and fetching it back down lets the bandwidth get spread over more machines and over different network paths (e.g. in other data centers)
We do it similarly except we push an image to a docker registry (backed by multi-region S3), then you can use e.g. ansible to pull it to 5, 10, 25, 100% of your machines. It "feels" like push though, except that you're staging the artifact somewhere. But when booting a new host it'll fetch it from the same place.
As I recall, this was investigated at one point on Sync Engine Classic and is definitely a thing that folks were thinking about for Nucleus, for pretty much exactly this reason.
The biggest challenges with this kind of work (not specific to the USN journal) are in the lack of reliable cross-platform support for features which can be shimmed to look similar. The more unique the code path, the harder it is to test en masse.
It's also the case that the execution environment on Windows machines tends to be very diverse due to a long history of backwards compatibility and the ability to set complex domain policies -- Dropbox strives for a good user experience, and "go talk to IT to have them change this setting" is rarely one of those.
disclaimer: worked on sync at Dropbox; don't work on it anymore; don't have current context
disclaimer: was on the team, but didn't write the article / don't work on this anymore
Rust was adopted at Dropbox for some serving infrastructure use cases more than a year before the sync rewrite was started, which was about four years ago. I'd say we solidly predated the "rewrite it in Rust" meme.
I believe that this rewrite was only successful because of Rust's ability to both interact safely/efficiently with underlying OS APIs (they're pretty much all C-like) and to encode complex concepts into the type system and the compiler. Rust isn't the only language with these properties, but it is one of the few -- and it's one that we really enjoyed using.
It depends on what you mean by checking all the boxes:
Go is tricky if you run on diverse platforms and want to do a lot of FFI, since cgo overhead is significant. The type system in Go is also not very powerful, which is both good and bad.
Haskell tends to hit performance walls that are very difficult to debug, and has a pretty similar learning curve to Rust (most people you hire onto the team won't know the language already).
The predominant competitor in this space is probably a high-level dynamic language combined with C/C++ library code. With good tooling and good practices to mitigate footguns, the extensive library support in C++ has a lot to offer.
Of course, I think Rust makes a better trade-off there, but early in the project it was not at all obvious that the good parts outweighed the fact that we would probably have been the biggest user of any library we depended on. We had some fun adventures in stress-testing HTTP/2 support here :)
If you're interested in graphics, animation, simulation, machine learning, or other topics, knowing the concepts in linear algebra will be worthwhile.
Most higher-level math will be easier for you if you have familiarity with vector calculus and linear algebra, so if you're the kind of person who benefits from a structured class when learning new/weird ideas, it's worth studying in school. (I'd also generally recommend a course in probability and statistics)
I ended up using atc1441's base station firmware[0] because it's a bit of a pain to actually pop open the case and get the programming pins on the Chroma 74
[0]https://github.com/atc1441/E-Paper_Pricetags