Thanks! HN was part of the origin story of the book in question.
In 2018 or 2019 I saw a comment here that said that most people don't appreciate the distinction between domains with low irreducible error that benefit from fancy models with complex decision boundaries (like computer vision) and domains with high irreducible error where such models don't add much value over something simple like logistic regression.
It's an obvious-in-retrospect observation, but it made me realize that this is the source of a lot of confusion and hype about AI (such as the idea that we can use it to predict crime accurately). I gave a talk elaborating on this point, which went viral, and then led to the book with my coauthor Sayash Kapoor. More surprisingly, despite being seemingly obvious it led to a productive research agenda.
While writing the book I spent a lot of time searching for that comment so that I could credit/thank the author, but never found it.
No you can just book a ticket like with all the others. Always buy a ticket online. Otherwise you'll be stuck in a queue. The Sagrada stopped doing on the spot visits because the queue was getting too long and Battló did as well. Only at la pedrera (or casa Milà as it's really called) can you still buy a ticket on site. But I wouldn't. You're just wasting your time waiting while all the prebooks go ahead of you.
Battló is the best one for me by far. I love the organic shapes and the light well and the soft wood etc. Wow
Here's a high level overview for a programmer audience (I'm listed as an author but my contributions were fairly minor):
[See specifics of the pipeline in Table 3 of the linked paper]
* There are 181 million ish essentially different Turing Machines with 5 states, first these were enumerated exhaustively
* Then, each machine was run for about 100 million steps. Of the 181 million, about 25% of them halt within this memany step, including the Champion, which ran for 47,176,870 steps before halting.
* This leaves 140 million machines which run for a long time.
So the question is: do those TMs run FOREVER, or have we just not run them long enough?
The goal of the BB challenge project was to answer this question. There is no universal algorithm that works on all TMs, so instead a series of (semi-)deciders were built. Each decider takes a TM, and (based on some proven heuristic) classifies it as either "definitely runs forever" or "maybe halts".
Four deciders ended up being used:
* Loops: run the TM for a while, and if it re-enters a previously-seen configuration, it definitely has to loop forever. Around 90% of machines do this or halt, so covers most.
6.01 million TMs remain.
* NGram CPS: abstractly simulates each TM, tracking a set of binary "n-grams" that are allowed to appear on each side. Computes an over-approximation of reachable states. If none of those abstract states enter the halting transition, then the original machine cannot halt either.
Covers 6.005 million TMs. Around 7000 TMs remain.
* RepWL: Attempts to derive counting rules that describe TM configurations. The NGram model can't "count", so this catches many machines whose patterns depend on parity. Covers 6557 TMs.
There are only a few hundred TMs left.
* FAR: Attempt to describe each TM's state as a regex/FSM.
* WFAR: like FAR but adds weighted edges, which allows some non-regular languages (like matching parentheses) to be described
* Sporadics: around 13 machines had complicated behavior that none of the previous deciders worked for. So hand-written proofs (later translated into Rocq) were written for these machines.
All of the deciders were eventually written in Rocq, which means that they are coupled with a formally-verified proof that they actually work as intended ("if this function returns True, then the corresponding mathematical TM must actually halt").
Hence, all 5-states TMs have been formally classified as halting or non-halting. The longest running halter is therefore the champion- it was already suspected to be the champion, but this proves that there wasn't any longer-running 5-state TM.
Having worked on non-linear simulators/ODE solvers off and on for a decade, I agree and disagree with what you're saying.
I agree with you that that is 100% the point: you don't already know what's going to happen and you're doing modelling and simulation because it's cheaper/safer to do simulation than it is to build and test the real physical system. Finding failure modes, unexpected instability and oscillations, code bugs, etc. is an absolutely fantastic output from simulations.
Where I disagree: you don't already know what's going to happen, but you DO know generally what is going to happen. If you don't have, at a minimum, an intuition for what's going to happen, you are going to have a very hard time distinguishing between "numerical instability with the simulation approach taken", "a bug in the simulation engine", "a model that isn't accurately capturing the physical phenomena", and "an unexpected instability in an otherwise reasonably-accurate model".
For the first really challenging simulation engine that I worked on early on in my career I was fortunate: the simulation itself needed to be accurate to 8-9 sig figs with nanosecond resolution, but I also had access to incredibly precise state snapshots from the real system (which was already built and on orbit) every 15 minutes. As I was developing the simulator, I was getting "reasonable" values out, but when I started comparing the results against the ground-truth snapshots I could quickly see "oh, it's out by 10 meters after 30 minutes of timesteps... there's got to be either a problem with the model or a numerical stability problem". Without that ground truth data, even just identifying that there were missing terms in the model would have been exceptionally challenging. In the end, the final term that needed to get added to the model was Solar Radiation Pressure; I wasn't taking into account the momentum transfer from the photons striking the SV and that was causing just enough error in the simulation that the results weren't quite correct.
Other simulations I've worked on were more focused on closed-loop control. There was a dynamics model and a control loop. Those can be deceptive to work on in a different way: the open-loop model can be surprisingly incorrect and a tuned closed-loop control system around the incorrect model can produce seemingly correct results. Those kinds of simulations can be quite difficult to debug as well, but if you have a decent intuition of the kinds of control forces that you aught to expect to come from the controller, you can generally figure out if it's a bad numerical simulation, bad model, or good model of a bad system... but without those kinds of gut feelings and maybe envelope math it's going to be challenging and it's going to be easy to trick yourself into thinking it's a good simulation.
I got a walkie talkie set as a Christmas present when I was 8. Which was kind of an evil thing to do given I had no siblings or friends to play with. One day I turned one set on and listened for a while and I thought I heard someone talking behind all the static noise. So I said something and was shocked to hear the voice talking back to me. Fast forward a few decades, next week is my wedding and that voice on the other side of the radio is my best man.
As a former and long smalltalker who learned MVC from the ParcPlace crowd…
I used to say things like this. M and V were always pretty unambiguous, but “controller” was kind of like “Christianity”, everyone talks like it’s a unifying thing, but then ends up having their very own thoughts about what exactly it is, and they’re wildly divergent.
One of the early ParcPlace engineers lamented that while MVC was cool, you always needed this thing at the top, where it all “came together” and the rules/distinctions got squishy. He called it the GluePuppy object. Every Ux kit I’ve played with over the years regardless of the currently in vogue lets-use-the-call-tree-to-mirror-the-view-tree, yesteryears MVVM, MVC, MVP, etc, always ends up with GluePuppy entities in them.
While on the subject, it would be remiss to not hat tip James Depseys MVC song at WWDC
“Mad props to the Smalltalk Crew” at 4:18, even though he’d just sung about a controller layer in cocoa that was what the dependency/events layers did in various smalltalks.
Way back when, I did a masters in physics. I learned a lot of math: vectors, a ton of linear algebra, thermodynamics (aka entropy), multi-variable and then tensor calculus.
This all turned out to be mostly irrelevant in my subsequent programming career.
Then LLMs came along and I wanted to learn how they work. Suddenly the physics training is directly useful again! Backprop is one big tensor calculus calculation, minimizing… entropy! Everything is matrix multiplications. Things are actually differentiable, unlike most of the rest of computer science.
It’s fun using this stuff again. All but the tensor calculus on curved spacetime, I haven’t had to reach for that yet.
Weird! Sorry to hear that commenting (including on HN) didn't make this person any friends. It has made me a bunch of friends, including some very close in-person ones. I don't think I'm an oddball in that regard!
Of particular note: comment culture is how I managed to engage with local politics here in Chicagoland, through which I met a ton of my neighbors and got actively involved with campaigns and the local government itself. Those are all in-person relationships that were (and remain) heavily mediated by comments.
Back in the 90s, I implemented precompiled headers for my C++ compiler (Symantec C++). They were very much like modules. There were two modes of operation:
1. all the .h files were compiled, and emitted as a binary that could be rolled in all at once
2. each .h file created its own precompiled header. Sounds like modules, right?
Anyhow, I learned a lot, mostly that without semantic improvements to C++, while it made compilation much faster, it was too sensitive to breakage.
This experience was rolled into the design of D modules, which work like a champ. They were everything I wanted modules to be. In particular,
The semantic meaning of the module is completely independent of wherever it is imported from.
Anyhow, C++ is welcome to adopt the D design of modules. C++ would get modules that have 25 years of use, and are very satisfying.
Yes, I do understand that the C preprocessor macros are a problem. My recommendation is, find language solutions to replace the preprocessor. C++ is most of the way there, just finish the job and relegate the preprocessor to the dustbin.
I may be the only person who ever understood every detail of C++, starting with the preprocessor. I can make that claim because I'm the only person who ever implemented all of it. (You cannot really know a language until you've implemented it.) I gave up on that in the 2000's. Modern C++ is simply terrifying in its complexity.
(I'm not including the C++ Standard Library, as I didn't implement it.)
Your first prompt is testing Claude as an encyclopedia: has it somehow baked into its model weights the exactly correct skeleton for a "Zephyr project skeleton, for Pi Pico with st7789 spi display drivers configured"?
Frequent LLM users will not be surprised to see it fail that.
The way to solve this particular problem is to make a correct example available to it. Don't expect it to just know extremely specific facts like that - instead, treat it as a tool that can act on facts presented to it.
For your second example: treat interactions with LLMs as an ongoing conversation, don't expect them to give you exactly what you want first time. Here the thing to do next is a follow-up prompt where you say "number eight looked like zero, fix that".
This isn't a failure of PowerPoint. I work for NASA and we still use it all the time, and I'll assure anyone that the communication errors are rife regardless of what medium we're working in. The issue is differences in the way that in-the-weeds engineers and managers interpret technical information, which is alluded to in the article but the author still focuses on the bullets and the PowerPoint, as if rewriting similar facts in a technical paper would change everything.
My own colleagues fall victim to this all the time (luckily I do not work in any capacity where someone's life is directly on the line as a result of my work.) Recently, a colleague won an award for helping managers make a decision about a mission parameter, but he was confused because they chose a parameter value he didn't like. His problem is that, like many engineers, he thought that providing the technical context he discovered that led him to his conclusion was as effective as presenting his conclusion. It never is; if you want to be heard by managers, and really understood even by your colleagues, you have to say things up front that come across as overly simple, controversial, and poorly-founded, and then you can reveal your analyses as people question you.
I've seen this over and over again, and I'm starting to think it's a personality trait. Engineers are gossiping among themselves, saying "X will never work". They get to the meeting with the managers and present "30 different analyses showing X is marginally less effective than Y and Z" instead of just throwing up a slide that says "X IS STUPID AND WE SHOULDN'T DO IT." Luckily for me, I'm not a very good engineer, so when I'm along for the ride I generally translate well into Managerese.
My dad headed up the redesign effort on the Lockheed Martin side to remove the foam PAL ramps (where the chunk of foam that broke off and hit the orbiter came from) from the external tank, as part of return-to-flight after the Columbia disaster. At the time he was the last one left at the company from when they had previously investigated removing those ramps from the design. He told me how he went from basically working on this project off in a corner on his own, to suddenly having millions of dollars in funding and flying all over for wind tunnel tests when it became clear to NASA that return-to-flight couldn't happen without removing the ramps.
I don't think his name has ever come up in all the histories of this—some Lockheed policy about not letting their employees be publicly credited in papers—but he's got an array of internal awards from this time around his desk at home (he's now retired). I've always been proud of him for this.
Funny to see this pop up again (I'm the author). The year is now 2025 and I still use Chase as a personal bank and I'm now discovering new funny banking behaviors. I'll use this as a chance to share. :)
My company had an exit, I did well financially. This is not a secret. I'm extremely privileged and thankful for it. But as a result of this, I've used a private bank (or mix) for a number of years to store the vast majority of my financial assets (over 99.99% of all assets, I just did the math). An unfortunate property of private banks is they make it hard to do retail-like banking behaviors: depositing a quick check, pulling cash from an ATM, but ironically most importantly Zelle.
As such, I've kept my Chase personal accounts and use them as my retail bank: there are Chase branches everywhere, its easy to get to an ATM, and they give me easy access to Zelle! I didn't choose Chase specifically, I've just always used Chase for personal banking since I was in high school so I just kept using them for this.
Anyways, I tend to use my Chase account to pay a bunch of bills, just because it's more convenient (Zelle!). I have 3 active home construction projects, plus pay my CC, plus pretty much all other typical expenses (utilities, car payments, insurance, etc.). But I float the money in/out of the account as necessary to cover these. We do accounting of all these expenses at the private bank side, so its all tracked, but it settles within the last 24-48 hours via Chase.
Otherwise, I keep my Chase balance no more than a few thousand dollars.
This really wigs out automated systems at Chase. I get phone calls all the time (like, literally multiple times per week) saying "we noticed a large transfer into your account, we can help!" And I cheekily respond "refresh, it's back to zero!" And they're just confused. To be fair, I've explained the situation in detail to multiple people multiple times but it isn't clicking, so they keep calling me.
I now ignore the phone calls. Hope I don't regret that later lol.
Presenting information theory as a series of independent equations like this does a disservice to the learning process. Cross-entropy and KL-divergence are directly derived from information entropy, where InformationEntropy(P) represents the baseline number of bits needed to encode events from the true distribution P, CrossEntropy(P, Q) represents the (average) number of bits needed for encoding P with a suboptimal distribution Q, and KL-Divergence (better referred to as relative entropy) is the difference between these two values (how many more bits are needed to encode P with Q, i.e. quantifying the inefficiency):
Information theory is some of the most accessible and approachable math for ML practitioners, and it shows up everywhere. In my experience, it's worthwhile to dig into the foundations as opposed to just memorizing the formulas.
The Kodak Research Labs (like Bell Labs) let their researchers play. In the 1960's my father (who later devised the Bayer filter for digital cameras) coded this algorithm for "Jotto" the 5 letter word version of Mastermind.
Computers were so slow that one couldn't consider every word in the dictionary as a potential guess. He decided empirically on a sample size that played well enough.
I became a mathematician. From this childhood exposure, entropy was the first mathematical "concept" beyond arithmetic that I understood.
Oh hey, I wrote this. Been a long time. I had the lucky of break of working in machine translation / parsing when the most important invention of the century happened in my niche field.
I'm pretty interested in the intersection of code / ML. If that's your thing here are some other writing you might be interested in.
I believe it absolutely should be, and it can even be applied to rare disease diagnosis.
My child was just saved by AI. He suffered from persistent seizures, and after visiting three hospitals, none were able to provide an accurate diagnosis. Only when I uploaded all of his medical records to an AI system did it immediately suggest a high suspicion of MOGAD-FLAMES — a condition with an epidemiology of roughly one in ten million.
Subsequent testing confirmed the diagnosis, and with the right treatment, my child recovered rapidly.
For rare diseases, it is impossible to expect every physician to master all the details. But AI excels at this. I believe this may even be the first domain where both doctors and AI can jointly agree that deployment is ready to begin.
> "When you consider that classical engineers are responsible for the correctness of their work"
Woah hang on, I think this betrays a severe misunderstanding of what engineers do.
FWIW I was trained as a classical engineer (mechanical), but pretty much just write code these days. But I did have a past life as a not-SWE.
Most classical engineering fields deal with probabilistic system components all of the time. In fact I'd go as far as to say that inability to deal with probabilistic components is disqualifying from many engineering endeavors.
Process engineers for example have to account for human error rates. On a given production line with humans in a loop, the operators will sometimes screw up. Designing systems to detect these errors (which are highly probabilistic!), mitigate them, and reduce the occurrence rates of such errors is a huge part of the job.
Likewise even for regular mechanical engineers, there are probabilistic variances in manufacturing tolerances. Your specs are always given with confidence intervals (this metal sheet is 1mm thick +- 0.05mm) because of this. All of the designs you work on specifically account for this (hence safety margins!). The ways in which these probabilities combine and interact is a serious field of study.
Software engineering is unlike traditional engineering disciplines in that for most of its lifetime it's had the luxury of purely deterministic expectations. This is not true in nearly every other type of engineering.
If anything the advent of ML has introduced this element to software, and the ability to actually work with probabilistic outcomes is what separates those who are serious about this stuff vs. demoware hot air blowers.
A few years ago, on my birthday, I quickly checked the visitor stats for a little side project I had started (r-exercises.com). Instead of the usual 2 or 3 live visitors, there were hundreds. It looked odd to me—more like a glitch—so I quickly returned to the party, serving food and drinks to my guests.
Later, while cleaning up after the party, I remembered the unusual spike in visitors and decided to check again. To my surprise, there were still hundreds of live visitors. The total visitor count for the day was around 10,000. After tracking down the source, I discovered that a really kind person had shared the directory/landing page I had created just a few days earlier—right here on Hacker News. It had made it to the front page, with around 200 upvotes and 40 comments: https://news.ycombinator.com/item?id=12153811
For me, the value of hitting the HN front page was twofold. First, it felt like validation for my little side project, and it encouraged me to take it more seriously (despite having a busy daily schedule as a freelance data scientist). But perhaps more importantly, it broadened my horizons and introduced me to a whole new world of information, ideas, and discussions here on HN.
I met Woz briefly at a Coffee Shop in NYC, many years ago. I was pitching investors for funding and had no idea what I was doing.
He gave me some quick feedback that made a world of difference in SF (years later, when I tried raising angel money): get closer to people first. You don’t have to be their friends - just show up to where they show up (bars, meetups, library - anything), then tell them what you’re doing, and they’ll write you checks without you asking.
This is still the key reason why raising money in the Bay Area is so much easier than anywhere else.
I didn’t get any check from him that night, but he came across as such a human person - like someone you know for years - even to a complete stranger trying to go straight for his wallet.
Tech would be amazing if we had 1000x more Wozes around.
You reminded me of an awesome Google engineer I met at BSidesSF last year who tirelessly answered my questions, and when I clicked on the video, it was you! That was a really inspiring moment for me, thank you.
> My peers in tech who are reluctant to have children often express fear that it will interrupt the arc of the careers they've worked so hard to build.
> That, I think, is the primary tension: not between the family and the state, as Boyle argues, but between individual and collective ambitions. Both the state and the family ask us to make sacrifices for something bigger than ourselves — and this, perhaps, is why they have historically fought each other for mindshare. What tech offers is the opposite: a chance to realize a vision that is entirely one's own. Tech worships individual talent, and it's a unique thrill to live and work among peers who don't shy away from greatness. But it also means that tech has to work harder than other industries to demonstrate that starting a family doesn't require giving up these ambitions.
I'm the breadwinner in my family, and my husband is a SAHD. I have a 2yo and I'm 6 months pregnant with our second. Stereotypically, having a kid made me care less about professional ambitions — but I don't care zero. And as the breadwinner, earning money, ideally more money every couple years or so, is a high priority. God, the pressure to keep up. It's hard to balance with being a present mom.
I live in the SF Bay Area and being able to attend events and network in person has been a huge boon to my career. Being "in the scene" pays off. I can't really do that anymore, not without losing time with my kid, and I'm just not willing to make the sacrifice. Traveling to conferences, etc., is even more off the table. Don't even talk to me about commuting. But I know these lifestyle changes will have repercussions next time I need to find a job.
To secure jobs with the kind of flexibility I require as a mom, I need to be a high performer, an impressive candidate with plenty of connections. Being a mom makes it harder — more expensive, let's say — to be that kind of exceptional worker bee. Oy vey!
How can you tell that any Windows or Mac clone UI is a re-implementation? Easy: try to move your mouse diagonally into the Send To menu after letting it pop up. If the send-to menu closes as you mouse over the item into the submenu, it's a clone. If the menu stays up even if you brush over another menu item, it's either real or a Good Clone. :)
For the fun history, @DonHopkins had a thread a few years back:
Mine was entirely mechanical (driven by punch cards and a hand-crank), and changed all of the pixels in parallel, but a lot of the mechanism development looked extremely familiar to me.
Let me share a personal story. Back in 2014 when I was working at Cloudflare on DDoS mitigation I collaborated a lot with a collage - James (Jog). I asked him loads of questions, from "how to login to a server", via "what is anycast" to "tell me how you mitigated this one, give me precise instructions you've run".
I quickly realised that these conversations had value outside the two of us - pretty much everyone else onboarded had similar questions. Some subjects were about pure onboarding friction, some were about workflows most folks didn't know existed, some were about theoretical concepts.
So I moved the questions to a public (within company) channel, and called it "Marek's Bitching" - because this is what it was. Pretty much me complaining and moaning and asking annoying questions. I invited more London folks (Zygis), and before I knew half of the company joined it.
It had tremendous value. It captured all the things that didn't have real place in the other places in the company, from technical novelties, through discussions that were escaping structure - we suspected intel firmware bugs, but that was outside of any specific team at the time.
Then the channel was renamed to something more palatable - "Marek's technical corner" and it had a clear place in the technical company culture for more than a decade.
So yes, it's important to have a place to ramble, and it's important to have "your own channel" where folks have less friction and stigma to ask stupid questions and complain. Personal channels might be overkill, but a per-team or per-location "rambling/bitching" channel is a good idea.
I'm reminded of the famous story of (I think) the central beam in a building at Oxford. The story goes something like:
The central beam was beginning to fail and the Oxford administration knew they needed to replace it. When they went around for quotes, no one could replace the beam because it was 100 ft in length and sourced from an old growth tree. Such logs were simply unavailable to buy. To solve the issue, the staff begin to look at major renovations to the building's architecture.
Until the Oxford groundskeeper heard about the problem. "We have a replacement beam," he said.
The groundskeeper took the curious admins to the edge of the grounds. There stood two old growth trees, over 150 feet tall.
"But these must be over 200 years old! When were they planted?" the admins asked.
In 2018 or 2019 I saw a comment here that said that most people don't appreciate the distinction between domains with low irreducible error that benefit from fancy models with complex decision boundaries (like computer vision) and domains with high irreducible error where such models don't add much value over something simple like logistic regression.
It's an obvious-in-retrospect observation, but it made me realize that this is the source of a lot of confusion and hype about AI (such as the idea that we can use it to predict crime accurately). I gave a talk elaborating on this point, which went viral, and then led to the book with my coauthor Sayash Kapoor. More surprisingly, despite being seemingly obvious it led to a productive research agenda.
While writing the book I spent a lot of time searching for that comment so that I could credit/thank the author, but never found it.