It may be due to AI proliferation, or the culturural bias I have, but I increasingly find em-dashes jarring.
As you point out, authors use them for the "natural and chaotic shifts of thought we all have" and when there are lots of these shifts it feels like I have to keep track of multiple conversations at once.
For example, in the article we have:
If your goal is to have other people read—and hopefully enjoy—your writing, you should make an effort to edit your thoughts.
When I read this I instinctively pause the 'main' thought/voice, read the aside, then re-establish my train of thought. In my opinion the sentence reads just as well without the aside:
If your goal is to have other people read and enjoy your writing you should make an effort to edit your thoughts.
[edit - putting comma back in to break up the long sentence]
If your goal is to have other people read and enjoy your writing, you should make an effort to edit your thoughts.
I think this is the only aside formatted like this in the article. The other em-dashes take the place of pauses in sentences, places I would normally use a comma or semicolon, or are used to introduce a list where I would typically use a colon.
Again this is probably a cultural thing, maybe a reaction to AI as well, but I find the em-dash a lot more though-interrupting than the other punctuation choices and I wonder if it's something I'll get used to or not.
I think I took out an extra comma too, which hurts readability.
Personally I write with too many asides, normally done with commas and parentheses. It's a comforting habit to fall into, and makes getting your thoughts out so much easier, at the expense of interrupting the reader's train of thought.
I don't normally notice when I'm writing with asides so the jarring em-dashes were a good reminder to try and edit them out where I can.
There are subjects which naturally lend themselves to “shifts” inter-sentence. Not really technical matters if the technical matter is just a list of specifications: here is how this works by default and if this and that then this happens. More like subjects relating to social issues and philosophy.
A person may have no preconceived notions about what the frobricator does. So you just list it out. But they may have plenty of preconceived notions about some more abstract-but-relatable subject. The writer anticipates that. And they have to navigate this subject at many levels at the same time, either inter-sentence or inter-paragraph; now I am talking about X, but not the X you think [the writer anticipates] but the X in itself. And not the X that group A considers, nor the one that B considers...
These subjects are more common in the “humanities”.
Certainly some authors overdo it and just seem to produce sentences with multiple semicolons and em-dashes because it’s their style/they are showing off. They are not writing clearly.
(Here I am using """""smart quotes""""".[1] Why is no one arresting me for that?)
I think we have seen a rise in the use of more intricate prose among technologists concurrent with the rise of AI penmanship. There’s more “flavor” now. Less of, either, straightforward prose or just boring and stodgy prose. More supposed personality.
Whatever the cause, this could be an emergent property of authors competing for readership by writing on the same subject but in a more supposedly engaging and personal way. And if an author who doesn’t like writing prose but wants to promote something regardless could get help from a program which happens to be literate in English as well? Well. Now it is easier to ramp up the word count.
> > If your goal is to have other people read and enjoy your writing you should make an effort to edit your thoughts.
Now the sentence says something different. The original said: If your goal is to get people to read ... And hopefully also enjoy.
Just getting people to read has the primacy in the original.
> > If your goal is to have other people read and enjoy your writing, you should make an effort to edit your thoughts.
This is certainly how I expect a programmer to write.
> Again this is probably a cultural thing, maybe a reaction to AI as well, but I find the em-dash a lot more though-interrupting than the other punctuation choices and I wonder if it's something I'll get used to or not.
Some groups of people ponder the great questions of life.[2] Programmers ponder if there is really a categorical difference, in principle, between their own consciousness and that of their smart fridge. And whether em-dash users are bots.
I've found having yt-dlp available on my iPhone useful, and used Pythonista to achieve that, but haven't figured out how to get the new requirements to work yet. Would love any ideas people have!
> (I forget how git format-patch handles the case where there are two parents.)
As per [0] merge commits are dropped:
Note that format-patch will omit merge commits from the output, even if they are part of the requested range. A simple "patch" does not include enough information for the receiving end to reproduce the same merge commit.
I originally thought it would use --first-parent (so just diff vs the first parent, which is what I would want) but apparently no! It is possible to get this behaviour using git log as detailed in this great write-up [1].
Out of interest, what do you think it would look like if communicating was algorithmic?
I know that it doesn't feel like I am doing anything particularly algorithmic when I communicate but I am not the hommunculus inside me shuffling papers around so how would I know?
This goes far to explain a lot of Chinese room situations. We have an intuition for the way something is. That intuition is an unshakeable belief, because it is something that we feel directly. We know what it feels like to understand Chinese (or French, or English, or whatever), and that little homunculus shuffling papers around doesn't feel like it.
Hopefully we have all experienced what genuine inspiration feels like, and we all know that experience. It sure as hell doesn't feel like a massively parallel search algorithm. If anything it probably feels like a bolt of lightning, out of the blue. But here's the thing. If the conscious loop inside your brain is something like the prefrontal cortex, which integrates and controls deeper processing systems outside of conscious reach, then that is exactly what we should expect a search algorithm to feel like. You -- that strange conscious loop I am talking to -- are doing the mapping (framing the problem) and the reducing (recognizing the solution), but not the actual function application and lower level analysis that generated candidate solutions. It feels like something out of the blue, hardly sought for, which fits all the search requirements. Genuine inspiration.
But that's just what it feels like from the inside, to be that recognizing agent that is merely responding to data being fed up to it from the mess of neural connections we call the brain.
You can take this insight a step further, and recognize that many of the things that seem intuitively "obvious" are actually artifacts of how our thinking brains are constructed. The Chinese room and the above comment about inspiration are only examples.
I cannot emphasize enough how much I dislike linking to LessWrong, and to Yudkowsky in particular, but I first picked up on this from an article there, and credit should be given where credit is due: https://www.lesswrong.com/posts/yA4gF5KrboK2m2Xu7/how-an-alg...
By the way, the far more impactful application of this principle is as a solution (imho) to the problem of free will.
Most people intuitively hold that free will is incompatible with determinism, because making a choice feels unconstrained. Taken in the extreme, this leads to Penrose and others looking for quantum randomness to save their models of the mind from the Newtonian clockwork universe.
But we should have some unease with this, because choices being a random roll of the dice doesn’t sit right either. When we make decisions, we do so for reasons. We justify the choices we make. This is because so-called “free will” is just what a deterministic decision making process feels like from the inside.
Philosophically this is called the “compatibilist” position, but I object to that term. It’s not that free will is merely compatible with determinism—it requires it! In a totally random universe you wouldn’t be able to experience the qualia of making a free choice.
To experience a “free choice” you need to be able to be presented with alternatives, weight the pro and con factors of each, and then make a decision based on that info. From the outside this is a fully deterministic process. From the inside though, some of the decision making criteria are outside of conscious review, so it doesn’t feel like a deterministic decision. Weighing all the options and then going with your gut in picking a winner feels like unconstrained choice. But why did your gut make you choose the way you did? Cause your “gut” here is an unconscious but nevertheless deterministic neural net evaluation of the options against your core principles and preferences.
“Free will” is just what a deterministic application of decision theory feels like from the inside.
I anticipate this will make it much easier for me to get my little team of actuaries formatting their code.
The impact of uv on our onboarding and utilisation of python has already been huge, and any way I can easily improve their code hygiene is a good one. Yes, I could get them to use ruff standalone, or set up pre-commit hooks etc for them, but the simple mental model that comes from `uv <do something>` is really beneficial.
Will have a play around with this and see how it goes. Would love to be able to hook into other formatters too, but not sure if that's easy or even possible already. For example, I would love it if `uv format` also formatted our SQL/dbt models.
At that point I think you start wanting Makefiles, or preferably IMO justfiles. Then you can run `just format` and have it format your combined Python/SQL/Bash/TypeScript project.
Someone mentioned recently that the slowness of rustc is in large part due to llvm. I know that is probably orthogonal to the work here, but I do like the idea of building the compiler with different toolchains, and that there may be follow on effects down the line.
Depends on the workload, but yes, codegen is a huge part of the total compilation time.
That said, that doesn't mean LLVM is always where the fixes need to be. For instance, one reason rustc spends a lot of time in LLVM is that rustc feeds more code to LLVM than it should, and relies on the LLVM optimizer to improve it. Over time, we're getting better about how much code we throw at LLVM, and that's providing performance improvements.
I'm completely ignorant so forgive me if this is obvious: in the effort of the parent article - to compile rustc with gcc - will rustc still be feeding lots of code to LLVM, or would that code now be fed to gcc?
Which codegen backend the building compiler uses is independent of which codegen backend(s) the built compiler uses.
Similarly, you can build Clang using itself or using GCC. The resulting compiler should behave the same and produce the same machine code, even if its own machine code is somewhat different.
The produced binaries could still have artifacts from the original compiler in them, e.g. if "compiler built-in" libraries or standard libraries were compiled with the original compiler.
Both GCC and rustc use a multi-stage build process where the new compiler builds itself again, so you reach an idempotent state where no artifacts from the original compiler are left.
There's an experimental compiler backend [1] using cranelift [1] that's supposed to improve debug build times. I never see it mentioned often in threads about Rust's long compilation time so I'm not sure if I'm missing something.
It's slow because the borrow checker is NP complete. LLVM may or may not generate slower code than GCC would for rustc, but I doubt it's anywhere close to the primary cause of the lack of snappy.
You're wrong it's been debunked that the borrow checker is any appreciable part of the compile time - Steve Klabnik actually verified it on here somewhere.
I say “usually” because of course sometimes bugs happen and of course you can conduct degenerate stress tests. But outside of those edge cases, it’s not an issue. If it were, blog posts that talk about lowering compile times would be discussing avoiding the borrow checker to get better times, but they never do. It’s always other things.
Is there any tool for Rust that does profiling that detects what part of compilation time is caused by what? Like, a tool that reports:
- Parsing: x ms
- Type checking: y ms
- LLVM IR generation: z ms
And have there been any statistics done on that across open-source projects, like mean, median, percentiles and so on?
I am asking because it should depend a lot on each project what is costly in compile time, making it more difficult to analyse. And I am also curious about how many projects are covered by "edge cases", if it is 1%, 0.1%, 0.01%, and so on.
The post my original comment is on discusses doing this at length.
> And have there been any statistics done on that across open-source projects, like mean, median, percentiles and so on?
I am not aware of any. But in all the posts on this topic over the years, codegen always ends up being half the time. It’s why cargo check is built the way it is, and why it’s always faster than a full build. If non-codegen factors were significant with any regularity, you’d be seeing reports of check being super slow compared to build.
I have actually seen a few posts here and there of 'cargo check' being slow. I have also heard of complaints of rust-analyzer being slow, though rust-analyzer may be doing more than just 'cargo check'.
>Triage notes (AFAIUI): #132625 is merged, but the compile time is not fully clawed back as #132625 is a compromise between (full) soundness and performance in favor of a full revert (full revert would bring back more soundness problems AFAICT)
Update: They fixed it, but another issue surfaced, possibly related, as I read the comments.
I do not know if the borrow checker and Rust's type system have been formalized. There are stacked borrows and tree borrows, and other languages experimenting with features similar to borrow checking, but without formal algorithms like Algorithm W or J for Hindley-Milner, or formalizations like of Hindley-Milner and problems like typability for them, I am not sure how one can prove the complexity class of the borrow checking problem nor a specific algorithm, like https://link.springer.com/chapter/10.1007/3-540-52590-4_50 does for ML.
I also am not sure how to figure out if the complexity class of some kind of borrow checking has something to do with the exponential compile times of some practical Rust projects after they upgraded compiler version, for instance in https://github.com/rust-lang/rust/issues/75992 .
It would be good if there was a formal description of at least one borrow checking algorithm as well as the borrow checking "problem", and maybe also analysis of the complexity class of the problem.
There isn't a formal definition of how the borrow checking algorithm works, but if anyone is interested, [0] is a fairly detailed if not mathematically rigorous description of how the current non-lexical lifetime algorithm works.
The upcoming Polonius borrow checking algorithm was prototyped using Datalog, which is a logical programming language. So the source code of the prototype [1] effectively is a formal definition. However, I don't think that the version which is in the compiler now exactly matches this early prototype.
EDIT: to be clear, there is a polonius implementation in the rust compiler, but you need to use '-Zpolonius=next' flag on a nightly rust compiler to access it.
Interesting. The change to sets of loans is interesting. Datalog, related to Prolog, is not a language family I have a lot of experience with, only a smidgen. They use some kind of solving as I recall, and are great at certain types of problems and explorative programming. Analyzing the performance of them is not always easy, but they are also often used for problems that already are exponential.
>I recommend watching the video @nerditation linked. I believe Amanda mentioned somewhere that Polonius is 5000x slower than the existing borrow-checker; IIRC the plan isn't to use Polonius instead of NLL, but rather use NLL and kick off Polonius for certain failure cases.
That slowdown might be temporary, as it is optimized over time, if I had to guess, since otherwise there might then be two solvers in compilers for Rust. It would be line with some other languages if the worst-case complexity class is something exponential.
> IRC the plan isn't to use Polonius instead of NLL, but rather use NLL and kick off Polonius for certain failure cases.
Indeed. Based on the last comment on the tracking issue [0], it looks like they have not figured out whether they will be able to optimize Polonius enough before stabilization, or if they will try non-lexical lifetimes first.
Recent moonlanders have been having trouble landing on the moon. Some are just crashing, but tipping over after landing is a real problem too. Hence the joke above :)
Mars landers have also had a chequered history. I remember one NASA jobbie that had a US to metric units conversion issue and poor old Beagle 2 that got there, landed safely and then failed to deploy properly.
As you point out, authors use them for the "natural and chaotic shifts of thought we all have" and when there are lots of these shifts it feels like I have to keep track of multiple conversations at once.
For example, in the article we have:
If your goal is to have other people read—and hopefully enjoy—your writing, you should make an effort to edit your thoughts.
When I read this I instinctively pause the 'main' thought/voice, read the aside, then re-establish my train of thought. In my opinion the sentence reads just as well without the aside:
[edit - putting comma back in to break up the long sentence] I think this is the only aside formatted like this in the article. The other em-dashes take the place of pauses in sentences, places I would normally use a comma or semicolon, or are used to introduce a list where I would typically use a colon.Again this is probably a cultural thing, maybe a reaction to AI as well, but I find the em-dash a lot more though-interrupting than the other punctuation choices and I wonder if it's something I'll get used to or not.