> The thing is, there is a lot of existing software in the world written in C and C++.
This is true, and I think that there is a type of language that I don't think have been explored yet: Cross language ABI language.
That is, the language is fully designed for being the glue between multiple languages to allow for migration in the future. I think it would probably end up being C.
> There are several languages that have managed a transition away from a base language into a more capable, flexible successor language, though: TypeScript is an evolution of JavaScript, Swift is an evolution of Objective-C, and C++ itself is an evolution of C.
And yet none of these have fully escaped the chain of their predecessors. And that is almost the point, people actually wanted to do what JavaScript could do, what C could do, what Objective-C could do. Then we put cover on it, but it is still a wooden table at the end of the day.
Have Hyrum's Law teach us nothing? Unless you consciously make the effort to REMOVE features, it will always be there. The fact that cross language boundary is complicated is a sign that the new language is actually shedding off technical debts. If I wrote a massively complicated function in C that leaks file handles, the answer isn't to read /proc/fd before and after, then manually unlink the new FDs. The answer is to actually dig through the code or rewrite it.
> That is, the language is fully designed for being the glue between multiple languages to allow for migration in the future. I think it would probably end up being C.
Unfortunately, C is woefully insufficient for such a language. At a minimum, you'd want to have a language that can distinguish between nullable and nonnullable pointers, pointer-with-array-len (so C++'s std::span or Rust's slice), and read-only versus read-write pointers (which const only kind of does). Expressing no-alias relationships and very basic lifetime relationships is also pretty critical for several languages. Multiple return values is something I'd love to see, in addition a success-object-or-error-object (something like a Result type). Having a dedicated not-necessarily-null-terminated UTF-8 string type is also useful.
While raw C is not super helpful for generating nice bindings, I think you could get most of the way there if there were annotations for everything, e.g. like how the __counted_by__ attribute was added.
The problem with almost all of those is that most languages won't support it; that's a FFI specific to Rust.
I mean, what do you do with those languages that don't have the concept of lifetime relationships, like Python? Or those in which read/write pointers make no sense (Tcl)? How about languages with linear datatypes? Or languages that don't allow self-referential data structures (like Rust)?
The only practical approach is to have an interface that is the lowest common denominator of most languages, which is what we already have with C.
I did work in CORBA in the 90s (and someone else mentions COM down below, which I also worked a little with), and the way I'd go about it[1] is similar to swig but in a CORBA/COM type fashion - an IDL that will generate bindings for specific languages. This requires quite a lot of prescient thinking - after all, lifetime ownership is a fairly new thing, so who knows what new requirement will come in the future?
In some future language, you may want FFI to encompass closures, in which case you'd have extra syntax for capture by value vs capture by reference, and you may not necessarily want ownership change in the function call (for example on a single-threaded callback function, the value captured by reference is safe for either party to modify).
Nail down the data interface and you're 80% there.
======================
[1] The best way to go about it would be using something like s-expressions: allow some basic execution to encompass every type of object a language might ever need.
However on Windows, given its prevalence, especially since Vista, one would expect that by now Windows team would have bothered to make it less hairy, but elas it has never been a priority, when when they actually did something (WinRT) is was woefully mismanaged and now no one cares.
Indeed, example all the languages that target JVM, CLR and WebAssembly, and the missing bits between what the platform can do, what the "systems" language exposes, and what everyone else is able to expose or consume.
This is true, and I think that there is a type of language that I don't think have been explored yet: Cross language ABI language.
That is, the language is fully designed for being the glue between multiple languages to allow for migration in the future. I think it would probably end up being C.
> There are several languages that have managed a transition away from a base language into a more capable, flexible successor language, though: TypeScript is an evolution of JavaScript, Swift is an evolution of Objective-C, and C++ itself is an evolution of C.
And yet none of these have fully escaped the chain of their predecessors. And that is almost the point, people actually wanted to do what JavaScript could do, what C could do, what Objective-C could do. Then we put cover on it, but it is still a wooden table at the end of the day.
Have Hyrum's Law teach us nothing? Unless you consciously make the effort to REMOVE features, it will always be there. The fact that cross language boundary is complicated is a sign that the new language is actually shedding off technical debts. If I wrote a massively complicated function in C that leaks file handles, the answer isn't to read /proc/fd before and after, then manually unlink the new FDs. The answer is to actually dig through the code or rewrite it.