> However, contemporary applications rarely run on a single machine. They increasingly use remote procedure calls (RPC), HTTP and REST APIs, distributed key-value stores, and databases,
I'm seeing an increasing trend of pushback against this norm. An early example was David Crawshaw's one-process programming notes [1]. Running the database in the same process as the application server, using SQLite, is getting more popular with the rise of Litestream [2]. Earlier this year, I found the post "One machine can go pretty far if you build things properly" [3] quite refreshing.
Most of us can ignore FAANG-scale problems and keep right on using POSIX on a handful of machines.
Cranshaw mentions "tens of hours of downtime" as a drawback. Given that downtime sometime corresponds with times of high load aka, the really important times, that's going to be a deal killer for most.
But his architecture does seem to be consistent with a "minutes of downtime" model. He's using AWS, and has his database on a separate EBS volume with a sane backup strategy. So he's not manually fixing servers, and has reasonable migration routes for most disaster scenarios.
Except for PBKAC, which is what really kills most servers. And HA servers are more vulnerable to that, since they're more complicated.
> Cranshaw mentions "tens of hours of downtime" as a drawback. Given that downtime sometime corresponds with times of high load aka, the really important times, that's going to be a deal killer for most.
I posit that this kind of "deal killer" is most often a wish list item and not a true need. I think most teams without a working product think these kinds of theoretical reliability issues are "deal killers" as a form of premature optimization.
I worked at a FANG doing a product where we thought availability issues caused by sessions being "owned" by a single server design was a deal killer. I.e. that one machine could crash at any time and people would notice, we thought. We spent a lot of time designing a fancy fully distributed system where sessions could migrate seamlessly, etc. Spent the good part of a year designing and implementing it.
Then, before we finished, a PM orchestrated purchase of a startup that had a launched product with similar functionality. Its design held per-user session state on a single server and was thus much simpler. It was almost laughably simple compared to what we were attempting. The kind of design you'd write on a napkin over a burrito lunch as minimally viable, and quickly code up -- just what you'd do in a startup.
After the acquisition we had big arguments between our team and those at the startup about which core technology the FANG should go forward with. We'd point at math and theory about availability and failure rates. They'd point at happy users and a working product. It ended with a VP pointing at the startup's launched product saying "we're going with what is working now." Within months the product was working within the FANG's production infrastructure, and it has run almost unchanged architecturally for over a decade. Is the system theoretically less reliable than our fancier would-be system? Yes. Does anybody actually notice or care? No.
Maybe. In that example, if the service has run for over a decade, it seems plausible that whatever contractual penalties they would have had to pay out for occasional downtimes would be far less than the initial and ongoing development time required to implement a far more complex solution, not to mention the additional hardware/cloud costs.
I would consider it dishonest to promise your customers a certain uptime, knowing you likely won't meet it. And some customers, particular more lucrative ones, want to see historical uptime and/or evidence that you have a resilient architecture.
That is not at all to say that it is a deal breaker for everyone, but it certainly will be for some companies.
> Running the database in the same process as the application server, using SQLite, is getting more popular with the rise of Litestream.
As someone who uses SQLite a lot, I'm suspicious of this claim. Litestream is strictly a backup tool, or, as its author puts it, disaster recovery tool. It gives you a bit more peace of mind than good old periodic snapshots, but it does not give you actual usable replication,* so I doubt it meaningfully increased SQLite adoption in the RDBMS space (compared to the application data format space where it has always done well).
For folks' context, the new tool that's being discussed in the thread mentioned by the parent here is litefs [0], as well as which you can also look at rqlite [1] and dqlite [2], which all provide different trade-offs (e.g. rqlite is 'more strongly consistent' than litefs).
A lot of these are FAANG scale problems primarily when it comes to server/backend infrastructure.
For a lot of "embedded" use cases like robotics, where you are trying to squeeze out maximum performance from a single machine (the hardware you have on the robot), POSIX is and will remain a hindrance.
I just don't see it. If it's an internal tool that has no scale at all, fine. No SLA, no problem. Good enough is good enough in a lot of cases.
But what about global customers? Most of the planet just eats the latency? What about single node failure? You usually need to scale past n=1 for a public facing service. It's not just about Google scale.
If you have an application server then you still have RPCs coming from your user interface, even if you run the whole DB in process. And indeed POSIX has nothing to say about this. Instead people tend to abuse HTTP as a pseudo-RPC mechanism because that's what the browser understands, it tends to be unblocked by firewalls etc.
One trend in OS research (what little exists) is the idea of the database OS. Taking that as an inspiration I think there's a better way to structure things to get that same simplicity and in fact even more, but without many of the downsides. I'm planning to write about it more at some point on my company blog (https://hydraulic.software/blog.html) but here's a quick summary. See what you think.
---
In a traditional 3-tier CRUD web app you have the RDBMS, then stateless web servers, then JavaScript and HTML in the browser running a pseudo-stateless app. Because browsers don't understand load balancing you probably also have an LB in there so you can scale and upgrade the web server layer without user-visible downtime. The JS/HTML speaks an app specific ad-hoc RPC protocol that represents RPCs as document fetches, and your web server (mostly) translates back and forth between this protocol and whatever protocol your RDBMS speaks layering access control on top (because the RDBMS doesn't know who is logged in).
This approach is standard and lets people use web browsers which have some advantages, but creates numerous problems. It's complex, expensive, limiting for the end user, every app requires large amounts of boilerplate glue code, and it's extremely error prone. XSS, XSRF and SQL injection are all bugs that are created by this choice of architecture.
These problems can be fixed by using "two tier architecture". In two tier architecture you have your RDBMS cluster directly exposed to end users, and users log in directly to their RDBMS account using an app. The app ships the full database driver and uses it to obtain RPC services. Ordinary CRUD/ACL logic can be done with common SQL features like views, stored procedures and row level security [1][2][3]. Any server-side code that isn't neatly expressible with SQL is implemented as RDBMS server plugins.
At a stroke this architecture solves the following problems:
1. SQL injection bugs disappear by design because the RDBMS enforces security, not a highly privileged web app. By implication you can happily give power users like business analysts direct SQL query access to do obscure/one-off things that might otherwise turn into abandoned backlog items.
2. XSS, XSRF and all the other escaping bugs go away, because you're not writing a web app anymore - data is pulled straight from the database's binary protocol into your UI toolkit's data structures. Buffer lengths are signalled OOB across the entire stack.
3. You don't need a hardware/DNS load balancer anymore because good DB drivers can do client-side load balancing.
4. You don't need to design ad-hoc JSON/REST protocols that e.g. frequently suck at pagination, because you can just invoke server-side procedures directly. The DB takes care of serialization, result streaming, type safety, access control, error reporting and more.
5. The protocol gives you batching for free, so if you have some server logic written in e.g. JavaScript, Python, Kotlin, Java etc then it can easily use query results as input or output and you can control latency costs. With some databases like PostgreSQL you get server push/notifications.
6. You can use whatever libraries and programming languages you want.
This architecture lacks popularity today because to make it viable you need a few things that weren't available until very recently (and a few useful things still aren't yet). At minimum:
1. You need a way to distribute and update GUI desktop apps that isn't incredibly painful, ideally one that works well with JVM apps because JDBC drivers tend to have lots of features. Enter my new company, stage left (yes! that's right! this whole comment is a giant ad for our product). Hydraulic Conveyor was launched in July and makes distributing and updating desktop apps as easy as with a web app [4].
2. You're more dependent on having a good RDBMS. PostgreSQL only got RLS recently and needs extra software to scale client connections well. MS SQL Server is better but some devs would feel "weird" buying a database (it's not that expensive though). Hosted DBs usually don't let you install arbitrary extensions.
3. You need solid UI toolkits with modern themes. JetBrains has ported the new Android UI toolkit to the desktop [5] allowing lots of code sharing. It's reactive and thus has a Kotlin language dependency. JavaFX is a more traditional OOP toolkit with CSS support, good business widgets and is accessible from more languages for those who prefer that; it also now has a modern GitHub-inspired SASS based style pack that looks great [6] (grab the sampler app here [7]). For Lispers there's a reactive layer over the top [8].
4. There's some smaller tools that would be useful e.g. for letting you log into your DB with OAuth, for ensuring DB traffic can get through proxies.
Downsides?
1. Migrating between DB vendors is maybe harder. Though, the moment you have >1 web server you have the problem of doing a 'live' migration anyway, so the issues aren't fundamentally different, it'd just take longer.
2. Users have install your app. That's not hard and in a managed IT environment the apps can be pushed out centrally. Developers often get hung up on this point but the success of the installed app model on mobile, popularity of Electron and the whole video game industry shows users don't actually care much, as long as they plan to use the app regularly.
3. To do mobile/tablet you'd want to ship the DB driver as part of your app. There might be oddities involved, though in theory JDBC drivers could run on Android and be compiled to native for iOS using GraalVM.
4. Skills, hiring, etc. You'd want more senior devs to trailblaze this first before asking juniors to learn it.
First, simply, I don't know anyone that puts their DB connections "on the internet". That live, raw, database "SQL" socket to be poked, prodded, hammered, and cajoled by complete strangers (or their automatronic minions).
Second, is DB latency. Sending "coarse" grained service requests across the planet is one thing compared to the potential volume of random SQL commands that folks do.
Mind, much of that can be mitigated if you build a stored procedure service layer. But classic "2 tier" "client/server" work didn't do that exclusively, just just threw out SQL willy nilly as the need dictated.
As old school as I am, even I tend to shy away from the "monolithic DB". You think your app was a monolith before, wait until it's all baked into Oracle. I've always found the DB to be a "bad citizen" when it comes to things like versioning, source code control, etc.
Even if I were doing a desktop app, I still think I would prefer a middle tier ("application server") managing service endpoints than cram it all into the DB, especially today.
What specifically are you concerned about? An HTTP connection to NGINX is just a TCP connection anybody can open, a database connection to postgres is the same. Both applications need to be extremely careful with those connections to protect against attacks, but database security is already a huge priority, so they both have invested heavily in that.
One concern I can think of is NGINX might have better DoS protection. What else do you have in mind?
Shodan knew at least 600,000 PostgreSQLs listening on the open internet when I last looked. Presumably quite a few are mistakes, of course. But people do it and the sky doesn't fall. Same for SSH or many other types of server. Of course the web ecosystem has 30 years of accumulated work so yes, you'd be missing stuff like Cloudflare, reCAPTCHA etc. Better for more controlled contexts than something like HN.
Latency is easy to screw up whether you do web apps or direct SQL connections. You have to be conscious of what a request costs, and you can easily batch SQL queries. Yes, you have to watch out for frameworks that spam the DB but those are bad news anyway, and of course there are lots of web frameworks that generate inefficient code. Not sure it's so different.
Your app will have to deal with DB versioning whether it's a web app or not. Tools like Flyway help a lot with linking your DB to version control and CI.
Nonetheless, I totally understand where you're coming from. Thanks for the thoughts.
We don't know either way, but a standard Postgres install doesn't let remote connections do much. You still have to authenticate before anything is allowed. It's not much different to sshd in this regard. A typical web server is far more promiscuous, with a massive surface area exposed to unauthenticated connections. There have been way more disasters from buggy web frameworks/apps that get systematically popped by crawlers, than from people running RDBMS.
Thanks for this detailed comment. I would like to add that I have seen solutions which facilitate this architecture, but still have your application be on the web. One example is postgREST which generates a REST api from your database. Using a separate schema and views you can tightly control what gets exposed and how, but all security and logic still only happens in the database. Do you have any opinions on similar solutions?
Since I've seen a similar thing in the 90s, I have a practical point to make.
If a two-tier app sends out emails, PLSQL/dbplugin does it. Now every ops task for sending emails involves the DB and by extension, your data is at stake. To launch a new parallel process, or to roll a new version, or to spread to a different location, or to kill a frozen process, or to measure much RAM a new feature has eaten, these are all DB tasks despite the fact that the task was just for a send-email feature.
Anything happening server-side (i.e. not on a user's device) needs to pass DBA middlepersons.
To put it back on feet, the architecture might be: the DB is one of the services. A frontend can talk to a database, and the two can work out the protocol, the authn/authz, the load balancing. They don't need any CRUD "backend" that is not really a "back" "end" but just a glorified boilerplate SQL-to-JSON converter.
The tradeoff is that you lose a lot of implicit trust. An email service cannot trust the frontend with the business rules. If user is allowed to only send to a set of recipients - it's an email service that needs to query that set from the DB.
Yes, you can go for a mixed approach. As you observe, it might not change that much because most of the issues aren't dependent on how many tiers you have. If you have middlemen between you and prod they're probably there anyway, regardless of architecture. And something will have to query the DB to find out who the user can email. Whether that's a DB plugin written in Python, a web server or whether it's an email microservice that connects to the DB over the network, it's going to boil down to how much you care about service isolation vs distributed systems complexity.
If you wanted isolation of an email service in this design, you'd use the DB as a task queue. The app triggers a procedure (written in SQL, Python, Java or whatever) which verifies the business logic and then does an insert to a tasks table. The email microservice wakes up and processes the queued tasks. That's a pretty common design already.
Ah, so you are saying to just bundle the DB with business logic and let it call other components (if any exist).
I thought about the whole idea over the weekend a bit and I'd say it is worth a try.
If you say that you have the distribution problem figured out, that makes it viable, it was the biggest obstacle in the 90s. What I'd expect it to mean is that to roll out a significant DB change, the frontend can self-update and not lose a hour-worth of users' unsaved work.
Also I think when selling this, you don't need to avoid the Delphi nostalgia that much. Everyone old who sees "remove the middle tier" will instantly go into mental mode of "uh-oh, those who do not learn from history are bound to repeat it". You are seeing a lot of it around this subthread - if you acknowledge upfront that you know you build on that past exp, it adds credibility.
Yes, exactly. Glad to hear that! Thanks for the words of advice, it's helpful.
People always have different thresholds for what "solved" means. Today Conveyor gives you a Chrome-style update experience on Windows where the app updates in the background even if it's being used. The user isn't disrupted. Ditto on macOS if the user states they want silent background updates at first update (we'll probably tweak this so the user isn't asked and must opt-out explicitly). The user won't lose any unsaved work or be surprised by sudden changes.
So to make a DB change, you need to do it in a backwards compatible way until the clients have had a chance to fully update. Probably that means being compatible for a few days, if you have users who aren't always online. This is probably solved enough for the general case.
The developer experience in that mode is exactly the same as compiling Markdown to HTML using Jekyll or Hugo.
The next step is to go further, so code changes can take effect much faster than a background update cycle. It requires the client to have some notion of a page, screen, activity etc - some point between interactions where you can make changes without disrupting the user. And it requires the client to be informed by the server if code has changed, even whilst the user is running the app. This takes more work, but it's on the roadmap. Mostly we think the async model is OK. You have to change your schemas in backwards compatible ways even in the web case to avoid site outages or different web servers getting confused during a rolling upgrade.
Yeah. Never used Oracle Forms but did use Delphi a lot. Borland never tried to solve distribution any better than Microsoft did. These firms were born in the era when "shipping" was meant literally and showed no interest in the problem of how to change software more than once every few years. Then people realized you could iterate a web app every day if you wanted to, the way they worked gave you total insight into what users were actually doing, you could use scripting languages better than VB and more. Businesses wanted the agility, devs wanted to use UNIX and Perl instead of VB/Delphi and the web was off to the races.
There were other issues of course, it wasn't just about distribution. Too bad so many downsides came along with the upsides. The real goal for OS research should be IMHO to find ways to combine what people like about web dev with what people like about desktop/mobile dev. All the action is above the POSIX layer.
> devs wanted to use UNIX and Perl instead of VB/Delphi
What do you think drove this? Presumably plenty of people in the dark mass of 9-to-5 devs were happy with VB/Delphi. Jonathan Edwards has written [1] that VB came from "a more civilized age. Before the dark times… before the web." Did nerdy devs like me, with our adolescent anti-Microsoft attitude (speaking for myself anyway; I was born in 1980), ruin everything?
The language and libraries situation on Windows wasn't great during this time.
Delphi was good but a compiled language with manual memory management. It was very easy to write code that crashed, which would nuke the user's state leaving little evidence of what happened. It also had a lot of legacy quirks like not allowing circular dependencies between compilation units, networking support was poor (iirc TCP, HTTP and other classes required you to buy a third party library). The VCL was a wrapper around Win32, which had some great strengths but also really frustrating weaknesses e.g. extremely poor/non-existent layout management, poor support for typography and no support for styling or branding. There were many positive aspects of course.
Microsoft gave you VB or C++, both with Win32 again. The C++ developer experience was horrid. VB was at least a scripting language with garbage collection, but, it was also constrained by the fact that B stood for "Beginners" so Microsoft were very reluctant to fix any of its legacy or add more powerful features.
Compared to that situation, scripting languages and especially Perl had some massive advantages:
1. Ran on UNIX/Big Iron which is where all the best hardware and databases could be found. Lots of devs liked UNIX because it was CLI and developer oriented.
2. Unashamedly designed for experts with tons of powerful features, quality of life stuff like integrated regex, garbage collection, proper stack traces, error logs you could view via telnet within seconds etc.
2. CPAN provided an ever growing repository of open source libraries, instantly accessible, for free! On Windows there were very few libraries, they were mostly quite expensive and closed source, no easy way to discover them (pre-Google) and only C++/Delphi devs could write them. VB was sort of a consumer-only language. Open source culture started with RMS at the MIT AI lab and so was very UNIX centric for a long time. Arguably it still is.
Really, it's hard to overstate how revolutionary proper garbage collection + CPAN was. GC is a force multiplier and CPAN is the granddaddy of all the open source library repositories we take for granted today. Imagine how unproductive you'd be without them.
The big downside was that Perl had no UI libraries and didn't really run on Windows. So how do you use it to write apps for normal people? Then Netscape started adding interactivity features to the web and it was all totally text based. Text was Perl's forte! Add the <form> tag, CGI, HTTP and now you're cooking with gas. Bye bye hateful 0xC00005 Access Violation errors and useless bug reports like "I clicked a button and the app disappeared".
The web was a huge step back for users, who went from having pretty sophisticated GUIs with fast table views, menus, shortcut keys, context menus, Office integration, working copy/paste, instant UI response etc to ... well, the web. But often users will suffer through that if it makes their devs more productive because all they really care about are features and competitive advantage. The web was crude but it let people escape the Windows ecosystem to one with open source + GC + pro languages + big iron UNIX.
"a more civilized age. Before the dark times… before the web."
There is revisionist history, and then there is that statement. That statement ... is almost offensive.
... Why did the "golden age" end? Because Microsoft sucked in so so so so so so so many ways. That statement is a bald faced lie, a whitewashing attempt to forgive Microsoft from inflicting millions of man years in damage to the IT industry over three decades of anticompetitive practices to keep their substandard software in a position of market dominance.
But for anyone beyond the MCSE factory programmer (aka NOT the thought leaders of the industry) aside from those profiting handsomely from the Microsoft evil empire, did not like Microsoft.
In the days of DOS, you had DOS or UNIX. Which was better?
In the days of Windows in the 3.1 and even pretty much in the windows 95, you didn't have preemptive multitasking (something that non-windows had for 20 years at that point). It crashed CONSTANTLY, had no security, required restarts for practically anything that was installed.
Meanwhile the UNIX people would brag about machines not being rebooted for years. This was before the constant patch cycle was a thing.
Microsoft's apis were slapdash and disorganized, and frequently went out of favor.
During this time Microsoft would constantly crush and screw over competitors and engage in constant anticompetitive behavior. If you didn't suck on the teat of Microsoft, your platform was under unrelenting assault, not by pure technical achievement, but by the full smorgasborg of corporate dirty tricks, FUD, bribery/kickbacks, lobbying, lies, secret apis, golf schmoozing with nontechnical CIOs to close deals, no or undermined standards, etc.
The graveyard is long: Sybase, Lotus 123, Netscape, Novell.
The Microsoft times were a time of being stuck with one option: and OS that crashed constantly or is utterly porous to attackers. A browser that has incompatible web apis and a disbanded developer team (IE 6) that gets corporate mandated and is a thorn in your side in the entire IT stack for two decades. Databases stolen from companies (Sybase signed SUCH a bad deal, it astonishes me to this day) running on platforms that can't stay up. Office software with inaccessible file formats and byzantine and closed programmatic apis for accessing it. A substandard desktop UI.
If you used Microsoft software with an ounce of historical knowledge or awareness, you could see the prison. You had no practical choices. All the executives in your company were bought and paid for. Microsoft had sales forces that tracked non-Microsoft systems and targeted them within companies by any means necessary. Every new piece of software written in MIcrosoft had to pay the "sociopath management tax" and go through extensive reviews on how it could be used to further or maintain Microsoft's empire and control.
Their software was deliberately dumped in beta form onto the market to crowd out the competitors.
None of this is an "adolescent" attitude. I'll repeat myself: MILLIONS OF MAN HOURS OF DAMAGE. You know, probably billions. Decades x tens of millions of workers.
This isn't just IT programmer frustration. This is bad applications forced on non-programmer users. This is better companies, better software, better IT industry denied proper funding and profits. Instead, Microsoft took trillions of dollars in revenues from them. This is undermining a free market, free ideas, and freedom for Microsoft's profit.
I agree with you about the massive damage caused by Microsoft, but ... it was there. In the DOS times, there was no choice between DOS and Unix on PCs -- there was effectively just DOS. Aside from niche things like Coherent (IIRC), Unix was only available on workstations and up, too expensive for small businesses and consumers.
Also, VMS and other OSes were the ones that ran for years without rebooting. Unix at the time was not so stable. Before Tcl, John Ousterhout wrote a log-structured file system for Unix because their Unix systems crashed often enough and took so long to boot that a possible loss of data but fast boot-up was deemed better than the lengthier downtimes with the existing file system.
So the PC market went with Microsoft and its encompassing environment, much to everyone's detriment. Fortunately, we've all moved on to the web and JavaScript and everything is now sunshine and roses. :-)
The only thing the anti Microsoft speech always misses, like the anti-FANG nowadays, it that the competition also has themselves to blame.
Bad management, bad products, not willing to reduce prices.
You see it nowadays on the Linux Desktop, instead of uniting, everyone goes do their own little thing.
No wonder it doesn't work out.
Linux won on the server room, thanks to being a cheap UNIX clone, and now with cloud computing and managed language runtimes, it hardly matters if it is there, or they are running on top of a type-1 hypervisor.
Both stances are too extreme. Yes, the web didn't bring "dark times", it was adopted for sound reasons. But your view of Windows vs UNIX is equally far out. Until the web got good Windows vs UNIX was no contest for anything meant for ordinary users. To the extent MS was anti-competitive it was hurting Netscape and Apple, not UNIX:
• Until Linux, UNIX didn't even run on PCs at all, only expensive proprietary hardware. Windows ran on cheap machines made by a competitive market. Businesses wanted PCs for lots of reasons (e.g. Office).
• UNIX wasn't a single OS, it was many forks that were not compatible. You couldn't just write an app for UNIX, put it on CDs and sell it.
• Anything GUI related was terrible compared to Win32. Motif was the standard but cave man tech compared to what you could do on the client with Windows.
• UNIX was wildly more expensive.
• PCs had a thriving gaming culture. It meant people used the same tech at home as at work, creating a self-training workforce. UNIX vendors didn't care about games, they were too 'serious' for that. Their loss.
• Windows 95 did do pre-emptive multi-tasking, by the way. Only Win16 apps that hadn't been ported were cooperatively multi-tasked. Those apps didn't stick around very long because porting to Win32 was easy and Windows 95 was an absolute monster success. People queued up at midnight around the blocks of PC stores to buy it, it was so popular.
• Windows apps crashed a lot compared to UNIX apps because the average Windows machine ran way more apps than the average UNIX machine, ran on way lower quality hardware that was often flaky, had way more diversity of peripherals and configurations, and apps were deployed in places where it was hard to get crash logs (no networks).
• Windows machines didn't have uptimes of years because nobody cared. They were workstations. You turned them off at the end of the day when you went home because otherwise they'd burn their CRTs in and their lifetime was reduced. The culture of leaving non-server machines on all the time and never rebooting them only began once ACPI and other power management tech started to become common (something non-Apple UNIX struggles with to this day). And once it was useful to do so, Microsoft had a version of Windows that could have uptimes of months or years, no big deal.
Yes! I ran Coherent 4.0 on a 386SX laptop when I was in high school (before moving to Linux.) Coherent had incredible documentation, something that is very rare today. I still remember that book with the shell on it, and learned a ton about systems administration and POSIX programming from it.
You're right, I'd forgotten about Xenix. Never heard of Coherent. SCO I thought was big iron but I'll take your word for it that I'm wrong about that. There sure were a lot of UNIX vendors back then!
That is the price for the software + the hardware to actually run it at an acceptable speed.
And all things being equal you could still get OS/2 as low as $49,
> The suggested introductory price of OS/2 2.0 is $139. However, the cost falls to $99 for users upgrading from DOS, which includes just about anyone, and to $49 for users who are willing to turn in their Microsoft version of Windows.
True, OS/2 was much cheaper. Coherent was relatively cheap for a Unix clone, which is basically what I was getting at. SCO Xenix / Unix was in the $500+ range. A C compiler wasn't even included, if I recall.
Most RDBMS have auditing features these days. Unless there's a lot of complex logic on the client that's sufficient. Otherwise yes, there are plenty of libraries and services for collecting and analyzing logs these days. I've looked at streaming Java Flight Recorder logs too, but it's probably overkill.
Desktop apps that query/update the db directly were a thing back in the '90s. They were an example of what we called "client/server". They were swiftly superseded by the web, which sort of hijacked the client/server architecture. As you noted, the basic reason is desktop app distribution and updating is hard. If your company can beat this problem, then great, because removing the intermediate web layer makes a lot of sense in certain cases (eg, enterprise deployments).
I have built a proprietary Swing desktop app that we use inside the tugboat company I run that does this. The wrinkle is that instances of this app are only intermittently connected to the Internet and our central instance of PostgreSQL. The Swing app uses a local SQLite instance and synchronizes it in the background when a connection is available. The users never experience any latency.
Careful schema design to support synchronization without collisions is the only real difference between this kind of app and CRUD apps that expect to always be able to reach the Internet.
I’d love to hear more about how you solved the synchronization process, if you’d be willing to share. How do you handle conflicting changes, like local and remote both changing the same field, or one deleting a resource while another modified it?
I’m trying to understand more real world examples of syncing offline with online.
I regret to report that I am not doing anything especially clever (e.g., CRDT). In some cases, I am doing things that are expensive in terms of storage, bandwidth, or local computation to facilitate synchronization.
Basically, my schema design prohibits use of UPDATE and requires that every row have a timestamp. The clients maintain a synchronization log to ensure they have fetched every available row. The keep track of which rows have not yet been sent up to the server.
This means that finding the current state of things that can change means doing a "SELECT column ORDER by timestamp DESC LIMIT 1" in order to see the latest state and always doing INSERT instead of UPDATE to update state.
In some cases, I am storing a delta in a row instead of a complete state representation. This means that some views have to replay the changes to show the current state. I cache the result of these.
I do some general high level caching on the client side to make all of this as fast as possible. I have watchdogs set on the local GUI to warn me when latency of the GUI event loop is over 200 milliseconds. I use these warnings to focus effort on caching and other optimizations.
Yep. OS vendors dropped the ball on distribution, browsers picked up the slack. But browsers also dropped the ball in a lot of ways. E.g. browsers do a lot of useful stuff but none of that is available for servers or CLI apps. There's lots of scope to do better. Also, on the desktop you have POSIX or POSIX-ish features at least :)
BTW, Conveyor is free for open source projects, and currently free for commercial too. The current versions focus on solving the basics really well. Deployment/update is like for a website statically generated from Markdown. You can build fully signed-or-self-signed and self-updating packages from cross-platform artifacts on whatever OS you happen to use with one command. So, you can package up Electron and JVM apps given just inputs like JS or JAR files, from your dev laptop or Linux CI box, and you get packages for every OS. You also get a simple download HTML page that detects the user's OS and CPU.
To do a new release you just re-build and re-upload the site. Clients will start updating immediately. On macOS it uses Sparkle, on Windows the OS will do the updates in the background and Linux users get packages.
It does native apps too but then of course you need to compile the binaries for each OS yourself.
One possibility we're researching is to page code in from the database on demand. That way you only have to push updates to the client occasionally, like when refreshing the core runtimes. For changing business logic the client would use SQL queries to speculatively load code based on what other clients have been requesting. If it works it means you can get rid of minification, bundling, all the other hacks web devs do to reduce requests and round-tripping, whilst keeping the "instant" deployment browsers give you.
Web applications can come close to directly accessing the database by using GraphQL with something like Hasura or PostGraphile. PostGraphile even uses Postgres's row-level security. A colleague and I once did a project using Hasura with a SPA-style JavaScript front-end and a separate back-end service driven by Hasura webhooks for doing the actual computation, and we ended up being unhappy with that approach. Some of our problems were related to the SPA architecture, but some were related to our use of GraphQL and Hasura.
We ended up starting over with a typical server-rendered web application, where the server itself accesses the database and communicates with the computation-heavy back-end service over gRPC, using a minimum of client-side JavaScript. I remain happy with that architecture, though I continue to explore different ways of integrating modest amounts of client-side JavaScript for interactivity and real-time updates. And to bring it back to the topic of my previous comment, if we assume that there has to be a server rendering HTML anyway, then I think it often makes sense to reduce complexity by bringing the database into that server process. I haven't yet tried that in production, though.
I think my preference is to use HTTP primarily as it was originally intended, for fetching HTML, as well as for form submissions. For finer-grained interactivity, I think it's better to use WebSockets as opposed to REST-ish requests and responses. I'm not dogmatic on that, though.
On web apps versus packaged desktop apps, I'm still inclined to go with web whenever feasible, and only develop a packaged desktop app if a web app just won't work. Being able to use an application without an installation step is really powerful for pre-sales demos or trials, for onboarding new users, and for applications that may only be used occasionally by any given user. Even for an application that you use all the time, a web app can be fine, as the popularity of Google Docs demonstrates. For example, if you just want to get the browser chrome out of the way, desktop browsers support "installing" a web app as an OS-level app with no browser chrome. IMO, Hydraulic's Eton demo app could just as well be a web app.
I look forward to your blog post, though even your preliminary HN comment offers a lot to think about.
The Eton Notes app is just a way to show what the download/install UX looks like when combined with a Compose app. It's mostly just a mockup with no real functionality.
Yes, for transient apps that are only used occasionally or where the user isn't committed e.g. social networks, the combination of sandboxing + no integration/unintegration steps, automatic deletion from the cache etc is really useful. Of course there's no rule that says only web browsers can supply these features. Other kinds of browser-like thing can do so too.
It's also worth thinking a bit outside the box. Although we claim web apps don't have installation steps, that's not really true most of the time. The lack of any explicit integration step ("install") means the browser doesn't know if the user values anything they do on the site. So you have to provide data persistence, and that in turn means you need a signup / account creation flow. It also means you're on the hook to store, replicate and back up any data any user ever creates, even if they only used it once and then never return.
Well, a lot of users really hate creating yet another account, especially if they aren't committed yet. It's tedious, they think you'll spam them with marketing emails and they're probably right, plus they don't want to make another password. Or they make one, abandon for a year, then can't remember how to log in.
You might think that's so fundamental it can't be any other way, but it's really just a side effect of how browsers have evolved. Think about how you might do demos outside the browser. You could just have a trial mode where the app spins up a local in-process RDBMS like H2 that writes the data into the app's private data area (on Windows) or home directory on macOS/Linux. No accounts necessary - just one or two clicks to download+trigger app install, and you're done. If/when the user decides to graduate, then they create an account and the app uploads the data from the local DB to the real remote DB. If they abandon it and don't care, it's not your problem, it costs you nothing. If they run low on disk space the OS will suggest they delete old unused apps at that point and you'll get tossed along with the rest.
Mostly though, this is about making developers more productive. If the primary determinant of your success is feature throughput and not shaving a few seconds off your onboarding, e.g. you're making specialised apps, internal apps, enterprise software, then optimizing for better dev environments can make sense. Installation just isn't that bad.
what about the ux side of things ? I love everything you mention about perf but I never ran into a single 90s era app that had interesting ergonomics. Web can be shitty or worse but there are some valuable UX aspects now.
I mean smooth async, fuzzy matching and reactive sub trees. I'm not a modern fanatic, I actually enjoy the good old as400 or win311 model a lot, and use old emacs daily. But there was a big chunk of the 90s where GUIs were really really subpar.
Could you elaborate? What does "smooth async" and "reactive subtrees" mean in the context of UX, that sounds more like developer experience than user experience.
Generally if you can do it on mobile you can do it elsewhere, right? If you want something like ReactJS and coroutines/async/await, look at Jetpack Compose. It's inspired by ReactJS but for Android/Desktop: https://developer.android.com/jetpack/compose
You don't need any particular UI toolkit though. Many years ago I did a tutorial on "functional programming in Kotlin":
It uses JavaFX with a library called ReactFX that adds functional utilities on top of the UI framework. It shows how to do async fuzzy matching of user input against a large set of ngrams. I guess that's in the region of what you mean too.
smooth async is basically mastered ajax with lean client/server events and nice loading widgets
nothing I said was web only, but web has focused on it and made it basic lingo; I admit not being aware of the javafx world evolution when I used it it was crude (circa 2010).
reactive subtrees ~= two way data binding over dom like trees, i don't recall running into such ui rendering models before
I think if you compare like with like it probably wasn't so bad. Web stuff was a lot cruder in 2010 too. JavaFX has two way data binding into the scene graph (a.k.a. DOM) and did from the start:
> Unix was the first operating system written by programmers for programmers
This is of course far from true; CTSS and especially ITS (which was so programmer friendly that rather than what is called today a “shell”, it’s user interface was a debugger) long predated it and both Thompson and Ritchie were exposed to them while at MIT to work on Multics. The supreme programming environments for programmers, the MIT Lispms and the D-machines at PARC are the ultimate in programmer-friendly OSes that are, for various important reasons, unmatched today.
Also, in their early papers about Unix they talked about how they didn’t need all that fancy stuff that mainframe systems like Multics needed, like VM and IO modules (Unix just does everything through the kernel in a simple fashion). Well, it was for a small computer system. There were good reasons that mainframes were structured the way they were and by the mid 80s that model started migrating to workstations (e.g. Britton-Lee accelerator and graphics subsystems) and by the 90s that model was already quite common in microcomputers too (for example with mpus in disk controllers and network interfaces).
But the Unix cultists ignored that revolution. They came late to the party with regards to networking; even when developing the sockets interface they relied heavily on the PDP-10 approach, but seemingly didn’t learn anything else from that.
Glad they are finally noticing that the world has changed.
I deeply believe UNIX only surviving and grew into the cultists that you mention, because AT&T was forbidden to sell their Bell Labs research, and so it widespread as free beer everywhere.
Had it been a commercial product from the get go, and history would have gone quite differently.
Posix used to feel like a nice compromise that worked for high level applications, one-off tasks, and low-level system software. I remember the first time I created an ISO file using "cat /dev/cdrom > some_file.iso", and I thought it was amazing that you didn't need a separate application for that (like you do on windows).
Now the posix APIs feels like the worst of both worlds.
Most applications shouldn't really store data on the filesystem, and probably not even configuration. Most applications use a database for a zillion reasons (e.g. better backup, restore, consistency, replication, transactions, avoiding random writes, ...). Yet the filesystem is also not a great interface if you're building a database: you want more control over flushing and buffering and atomic file operations. The filesystem is mainly good at storing compiled code.
The "everything is a text file" interface is not great anymore, either. Simple manipulation of the output of basic commands (like text 'ps') requires error-prone parsing and quite some skill with things like grep, sed, awk, sort, etc. That's OK for experienced sysadmins working on individual machines, but doesn't integrate very well in larger systems where you have lots of automated access to that information. I have some experience doing sysadmin-like work, but I still run into problems with simple things like spaces in filenames and getting the quoting and escaping right in a regex. One option is for it to be more database-like, but that's not the only option. Erlang provides nice programmatic access to processes and other system information while still feeling very free and ad-hoc.
Of course, Linux is still great in many ways. But it feels like some of the core posix APIs and interfaces just aren't a great fit, and a lot of the interesting stuff is happening in non-posix APIs. And applications/services are being built on higher-level abstractions that aren't tied to single machines.
> Most applications use a database for a zillion reasons
Is it time to look at database-first operating systems again? There have been a few. Tandem's Guardian was very well regarded in its day. Unfortunately, Tandem was acquired by DEC/Compaq, which tried to move it to the Alpha around 1997. Then, after HP acquired what was left of Compaq, HP tried to move it to Itanium. (There was a MIPS version in there somewhere.) After all those bad decisions, in 2014 it was finally ported to x86, by which time few cared.
There's still an HP offering.[1]
In Guardian, the database system owned the raw disks. There were no "files". If you wanted something that looked like a file, it was just a blob in a database entry.
Everything was transaction-oriented, like SQL. That was the whole point. Everything was a redundant, recoverable transaction.
The OS was cluster oriented. Adding or replacing servers was normally done without shutdown. Databases were replicated, and you could add new disks, wait for them to come into sync, and remove the old ones.
All this in the 1970s. They were way ahead of their time.
If you take an advanced database like Oracle it's quite close to an operating system already and only really relies on the OS for boring stuff like boot-up, storing the db's code files etc:
- "Everything is a $FOO" philosophy, check. Where POSIX has files, RDBMS has tables.
- CLI/shell, check.
- Installable apps, check. (you can create stored procs by INSERTing java/javascript/etc into system tables).
Probably more parallels that don't occur right now. Lots of companies use the DB as their primary OS with the underlying kernel relegated to starting up the DB and configuring the hardware.
> Now the posix APIs feels like the worst of both worlds.
I think, like anything, it depends on what you're doing, and how you're doing it.
> The "everything is a text file" interface is not great anymore, either.
Text doesn't seem markedly different from a JSON API? Perhaps "worse is better" more often when you are composing several programs/apps to make one system? Even complex distributed systems.
> But it feels like some of the core posix APIs and interfaces just aren't a great fit
And I think these APIs can thrive on top of (the good parts of) POSIX. I'm not so certain we need to fundamentally rethink this layer, because this layer seems to work pretty well at what it does well. It should be pruned occasionally, but I'm not sure we need a whole new abstraction.
FWIW, I think this what the article is saying. Let's create some new interfaces where the old interfaces don't model the hardware well[0].
"Text doesn't seem markedly different from a JSON API?"
Json would be terrible as a primitive.
I was imagining something more like erlang, which is very OS-like. The types are very permissive/fleixble, and there's lots of ways to transform and manipulate results or feed it into something else without feeling code-heavy. It still feels comfortable to do ad-hoc stuff. The advantage is that in erlang you usually don't worry about basic stuff like record and field separators.
"Perhaps "worse is better" more often when you are composing several programs/apps to make one system?"
That's probably true as you zoom out. But even basic utilities in posix within the same system require a bunch of parsing. Imagine if a function call in a programming language required an intermediate text format. That's how unix feels sometimes.
There is the alternative of an object based interface like Windows offers.
I thought it was an interesting idea until I actually got around to experimenting with Powershell recently. The ergonomics are more akin to Python than any of the GNU tools. I can’t help but wonder why not just use a language like Python rather than a heavyweight scripting language if it’s a similar LOE (speaking in general, I’m aware of CLR)? The benefit of the text based interface to me is the expediency of performing one-off CLI actions. I inclined think an object interface actually impedes this goal.
> POSIX provides abstractions for writing applications in a portable manner across Unix-like operating system variants and machine architectures. However, contemporary applications rarely run on a single machine. They increasingly use remote procedure calls (RPC), HTTP and REST APIs, distributed key-value stores, and databases, all implemented with a high-level language such as JavaScript or Python, running on managed runtimes. These managed runtimes and frameworks expose interfaces that hide the details of their underlying POSIX abstractions and interfaces. Furthermore, they also allow applications to be written in a programming language other than C, the language of Unix and POSIX. Consequently, for many developers of contemporary systems and services, POSIX is largely obsolete because its abstractions are low-level and tied to a single machine.
> Nevertheless, the cloud and serverless platforms are now facing a problem that operating systems had before POSIX: their APIs are fragmented and platform-specific, making it hard to write portable applications. Furthermore, these APIs are still largely CPU-centric, which makes it hard to efficiently utilize special-purpose accelerators and disaggregated hardware without resorting to custom solutions. For example, JavaScript is arguably in a similar position today as POSIX was in the past: it decouples the application logic from the underlying operating system and machine architecture. However, the JavaScript runtime is still CPU-centric, which makes it hard to offload parts of a JavaScript application to run on accelerators on the NIC or storage devices. Specifically, we need a language to express application logic that enables compilers and language run times to efficiently exploit the capabilities of the plethora of hardware resources emerging across different parts of the hardware stack. At the same time, it would be an an interesting thought experiment to ponder how different would the hardware design of these devices be without the CPU-centrism in POSIX.
It sounds like the author discovered the motivation behind the last time the POSIX standard was obsoleted, and some of the reasons why Plan9 was developed.
This falls right in line with the recent "hardware offload for all" article posted on HN, and the talks around Hubris that Bryan Cantrill has given. It's an exciting time for computing, I think.
I've often lamented that I was born too late to watch Unix/POSIX be born and grow up. That's always felt like a golden time in computing. I also wasn't in the industry yet for the early 2000s and the rise of web-scale, though I was at least aware. It's exciting to think about what systems will do over the next twenty years. I'm looking forward to it.
My answer to the title is "no" for the same reason we're still using email protocols designed in 1985. I guess the counterpoint would be systemd.
TFA says "Therefore, we argue that the POSIX era is over, and future designs need to transcend POSIX." So they're actually not answering their own question, they're saying "the answer should be x" which is subtly different. There are a lot of should-be's in computing.
Since I moved into managed languages I never cared about POSIX any longer, as long as the runtime supports the underlying hardware, OS or hypervisor, I am good.
Ironically, while I am an heavy email user, at work it is mostly used for internal newsletters, all the communication happens over Slack.
There was a great talk I can't find right now about how the share of computer hardware that it's directly controlled by the OS kernel keeps on shrinking.
In a modern smartphone, the OS directly controls just one of a dozen chips. The rest are running their own OSes which do their own thing independently of the main OS.
I'm guessing it's Tim Roscoe's keynote on how most fundamental questions about what the hardware is actually doing are invisible to traditional OS abstractions:
Richard Stallman POSIX now will have to do one of the hardest stuff in the software realm: resist planned obsolescence while replacing some interfaces with "in-the-end better" interfaces, that on the long run. This is extremely though: often, replacing "in-the-end better" interfaces is actually planned obsolescence.
For instance event based programming, "epoll" should replace "select". Synchronous programming, signalfd, timerfd, etc... but then "signals" should be more accurately classified, for instance in monothreaded applications segfault won't be delivered from a "signalfd" in a "epoll" syscall... and why not keep only the "realtime signal" behavior?
If POSIX only "add" new "options", in the end, you'll get gigantic spec just being the mirror image of the very few implementations since its size will make it unreasonable for a new implementation from scratch.
The "leaner POSIX" route seems the right way here, but it is really easier to say than to do.
Because fork() was very simple and conceptually "easy" to do when it first was introduced, and is now massively complex and has huge implications on every part of the system. It's not compositional, isn't thread safe, insecure by default (inherits env/fds), and it's also slow with all the state it must copy. And at a conceptual level it doesn't work in environments where the nature of a "process" and "address space" aren't synonymous. For instance if your application uses a hardware accelerator (NIC, GPU, whatever) fork() isn't ever viable or sensible, since the hardware resources can't be duplicated safely. And it'll never work in a design like WebAssembly (just an example of this idea, but WASM isn't the only one), again "process" and "virtual memory address space" are not the same. Consider that posix_spawn can make reasonable sense in WebAssembly at a first guess ("launch this wasm module"), but fork() in contrast is much more difficult when it implies COW semantics.
The reality is fork() is pretty much exclusively used to launch new processes these days, outside a few specific cases. Today, it's a poor fit for that problem. And the answer is what Windows has been doing (and POSIX has now had) for a long time: explicitly launching processes by giving a handle/pathname to an executable like posix_spawn. That's the first solution, anyway; a better one would be more capability-oriented design where you have to supply a new address space with all its resources yourself.
True but it's not meant to be. I'm not sure there are many if any valid use cases for forking and not immediately exec-ing and using threads together in the same application.
> insecure by default (inherits env/fds)
Inherits env and open file descriptors by design. It's pretty much always been understood that if you fork in most scenarios you immediately exec. You can set file to close on exec and set a new env if desired, and not do that if it's not.
> and it's also slow with all the state it must copy.
I thought it was mostly COW?
> And at a conceptual level it doesn't work in environments where the nature of a "process" and "address space" aren't synonymous.
Yeah valid argument. posix_spawn man page says:
> "The posix_spawn() and posix_spawnp() functions are used to create a new child process that executes a specified file. These functions were specified by POSIX to provide a standardized method of creating new processes on machines that lack the capability to support the fork(2) system call. These machines are generally small, embedded systems lacking MMU support.".
posix_spawn is POSIX and has existed along side fork since POSIX.2001. So your saying you want every application ever written to be automatically portable to systems the can't support fork, therefore get rid of fork entirely? I guess.
1. Call a function that creates an empty child process in a suspended state.
2. Call functions that: map/write memory into the child process’s address space; add file descriptors to the child process; set the child process’s initial register values; and so on.
3. Call a function to unsuspend and run the child process.
fork() is terrible, but embryonic processes also have a lot of performance issues and prevent a number of valuable security mitigations. In general a spawn() style mechanism seems like a better approach (despite the deficiencies of specific examples like posix_spawn()).
I think it goes without saying that since I stated fork() is terrible that I am not advocating for spawning new processes via any of the variants of exec() since they all depend on fork(). I directly stated posix_spawn() was the right technique (despite its deficiencies and a somewhat terrible interface).
My point was that embryonic processes aren't really the right solution since they require exposing a whole bunch to powerful primitives like read/write of remote address spaces in order to spawn processes. That ends up being slow (because each one of those calls is necessarily a syscall and requires manipulating page tables to mess with the other process). It also means to prevent abuse you need to be very carefully control when to revoke those privileges.
The obvious solution is to take all the operations you would have done to the remote process, encode them in some form, and pass them off to a secure agent (in this case the kernel) that can do them in bulk. That solves both perf issues (1 syscall, and no repeated round tripping through multiple address spaces), and the security issue (you no longer need to expose primitives to manipulate the remote process to every process that is allowed to spawn a new one).
so does posix_spawn, since you can leak file descriptors into parallel spawning threads/processes before the execve (where they are closed if having O_CLEXEC).
Yeah, that is a problem, and it is totally fixable. Checkout `POSIX_SPAWN_CLOEXEC_DEFAULT` on macOS for an example. Again, just because there are API deficiencies doesn't mean that the idea is wrong.
> These services cannot expect to run faster from year to year with increasing CPU clock frequencies because the end of Dennard scaling circa 2004 implies that CPU clock frequencies are no longer increasing at the rate that was prevalent during the commoditization of Unix.
Definition:
> Dennard scaling, also known as MOSFET scaling, is a scaling law which states roughly that, as transistors get smaller, their power density stays constant, so that the power use stays in proportion with area; both voltage and current scale (downward) with length.[1][2] The law, originally formulated for MOSFETs, is based on a 1974 paper co-authored by Robert H. Dennard, after whom it is named.[3]
It's interesting that this is almost entirely about the filesystem API, but barely touches on all the other things that the object-store proponents and other "POSIX is dead" advocates usually focus on. Hierarchical directory structure, byte addressability, write in place, permissions model, persistence and consistency requirements, and so on. If everybody switched to a new API today, POSIX would still be very much alive. I'm not going to argue about whether the rest of POSIX should die or not, having played an active role in such discussions for about twenty years, but for that to happen it has to be a lot more than just changing how requests and responses are communicated.
They can be running bare metal on top of a type 1 hypervisor, on top of a UNIX, Windows, mainframe, micro, or whatever.
Android and ChromeOS, which although based on the Linux kernel, don't expose a POSIX userspace to regular app developers as public API, only OEMs get to play with it, or placing the devices into developer mode like Crostini.
It doesn't matter at all on the XBox and Nintendo, but it does on PlayStation.
Its relevancy depends pretty much how deep one still is into classical UNIX like workflows, but like COBOL and Fortran it isn't going away anytime soon.
I agree. We need a standard specification for distributed computing. Categorize all the different aspects of a distributed system, then write standard APIs for them all.
The whole point behind POSIX was so people didn't need to constantly reinvent the wheel and glue things together to do simple things on different platforms. Today if you need to do anything in the cloud, you must "integrate" the pieces, which means to take out scissors and glue and toothpicks and a text editor and literally arts-and-crafts the pieces together. Want to add something to your system? More arts and crafts! Rather than waste all that time, if everything just used a common interface, everything would just work together, the way it (mostly) does with POSIX.
This is bad for the companies that make all their money off proprietary/incompatible interfaces - lock-in devices that are advertised as a plus because they can be "integrated" with enough time and money. But it's better for our productivity and the advancement of technology if we didn't have to integrate at all. Just run the same thing everywhere.
> The primary use case for Unix was to multiplex storage (the filesystem) and provide an interactive environment for humans (the shell) [2], [3]. In contrast, the primary use case of many contemporary POSIX-based systems is a service running on a machine in a data center, which have orders of magnitude lower latency requirements.
And yet the most pleasant development environment for producing the latter is going to be the former.
It was a nice read. Thanks for sharing the article on HN. However, I feel the fellowing claim in the article is not backed by any detailed facts/analysis.
> Virtual memory is considered to be a fundamental operating system abstraction, but current hardware and application trends are challenging its core assumptions.
What are specifically challenging the vm assumptions?
The overhead of the TLB and page walks have been increasing over time as memory gets larger - 4kb pages made sense in some contexts but not others so page sizes are now increasing, and this is complicating APIs and kernels. The complexities of huge pages on Linux being a case in point.
I suspect he may also be thinking of the prevalence of one virtual machine:one app in the cloud, where you can argue (and the Seastar guys do) that the process abstraction just slows you down because the unit of isolation is the VM anyway.
Why would POSIX need to be replaced? Sure, POSIX is a bit clunky to use, but who uses it directly? And there aren't any performance benefits you can get from replacing POSIX. This is just one of those questions that CS academics ask themselves because they ran out of algorithms to discover.
I've come around to the opinion that (POSIX-style) filesystems are no longer a great idea. Instead, I think something more like Git (plumbing, not porcelain) would be preferable as the underlying storage abstraction.
What exactly are you thinking of? Git manages files after all.
If you mean the underlying data structures, that's basically what modern filesystems are. XFS, APFS, BTRFS etc are all copy-on-write file systems that use git-like structures underneath. In the same way that git branches are "instant" because no data is actually copied, so too can these file systems clone files or whole trees instantly without copying any real data. You can easily "branch" a directory tree, rsync the results to a remote machine, etc.
For one thing, it would be nice if every directory had a hash that covered all its contents (recursively), like Git tree objects. That way, all sorts of tools that need to check which files in a directory tree have changed – including `git diff`, `make`, file sync tools, and indexed search tools – could immediately skip directories with no changes, without needing a separate fsmonitor tool. The cost would be higher overhead recalculating hashes when files are constantly being updated.
It would also be nice to support filesystem transactions, somewhat analogous to Git commits. POSIX file APIs make it difficult to avoid race conditions in the best circumstances, and extremely difficult if there’s a security boundary involved. You can never check something about a path (e.g. “is this a symlink?”) and then rely on that thing being true, because it could have been concurrently modified between the check and whatever you do next. So you have to rely on the limited atomic semantics available - for example, instead of explicitly checking for a symlink, you can use O_NOFOLLOW - but those are not always sufficient, depending on what you’re trying to do. It shouldn’t be this way. I should be able to take a frozen snapshot of the filesystem, inspect it to my heart’s content, make the changes I want, and finally atomically commit or abort.
Regarding copy-on-write clones, can any of those filesystems actually clone arbitrary directories? In APFS’s case, you can make copy-on-write clones of files, and you can make copy-on-write snapshots of entire volumes, but you can’t just take some random directory and make a copy-on-write clone of it (without individually cloning all the files under the directory). I believe the same limitation exists for some or all of the modern Linux filesystems.
I'm giving a (slightly updated) version of my talk at the Storage Network Industry Association Storage Developer's Conference (2022) in Freemont, CA next thursday:
Filesystems do have timestamps, but they aren't propagated to the root for performance reasons - normally you want file IO to be fast more than you want to be able to quickly test if a large directory tree changed and propagation would make all writes contend on a single lock (for the root dir hash/ts).
Agreed about fs transactions. Not sure about cloning directory trees. I thought btrfs could do it but I never used that feature. You might well be right.
The thought is somewhat inchoate still. I'm working with a pure functional language (Joy https://joypy.osdn.io/ ) and when it came time to add filesystem support I balked. Instead, I'm trying out immutable 3-tuples of (hash, offset, length) to identify sequences of bytes (for now the "backing store" is just a git repo.) Like I said, it's early days but so far it's very interesting and useful.
I get what you're saying about modern filesystems, and I agree. I guess from that POV I'm saying we could stand to remove some of the layers of abstraction?
2. The DAT or IPFS protocols, which are based on the idea of immutable logs storing file data, identified by hashes, with public keys and signatures to handle mutability.
I'm seeing an increasing trend of pushback against this norm. An early example was David Crawshaw's one-process programming notes [1]. Running the database in the same process as the application server, using SQLite, is getting more popular with the rise of Litestream [2]. Earlier this year, I found the post "One machine can go pretty far if you build things properly" [3] quite refreshing.
Most of us can ignore FAANG-scale problems and keep right on using POSIX on a handful of machines.
[1]: https://crawshaw.io/blog/one-process-programming-notes
[2]: https://litestream.io/
[3]; https://rachelbythebay.com/w/2022/01/27/scale/