I'd say there are converging standards like Parquet for longterm on disk, Arrow for in memory cross language, and increasingly duckdb for just standard SQL on that in memory or on disk representation. If I had to guess most of the data table things vanish long term because everyone can just use SQL now for all the stuff they did with quirky hacked up APIs and patchy performance because of those hacked up APIs.
This worked well for me for some things I've recently been learning/working on. One improvement I'd add is the citations of where information have come from aren't hyperlinks it would be very useful if they were!
A bigger problem in this respect with Wikipedia is it often cites secondary sources hidden behind an academic fire/paywall. It very often cites review articles and some of these aren't necessary entirely accurate.
Well also years of Wikipedia proving to be more accurate than anything in print and rarely and not for very long misrepresenting source materials. For LLMs to get that same respect they would have to pull off all of the same reassuring qualities.
It lingers in fat tissue and once at a low enough level your liver doesn't really clear it. But that kind of level isn't necessarily linked to increased risks of diabetes or heart disease.
Hi there, leading DuckPGQ developer here :) Thanks for the shoutout! I've been busy working on an internship at DuckDB labs so DuckPGQ has gotten less attention, but I'll get back to it soon (December most likely) and will update the extension to support DuckDB v1.4.0 and v1.4.1 this week hopefully.
PGQ requires you to write using SQL and read using a graph query language. GQL is a standalone language that supports reads/writes. But much of the community is still using cypher.
As far as I can tell, this has nothing to do with CAP theorem or distributed systems. It's just being used as an analogy.
> [CAP theorem] states that any distributed storage system can provide only two of these three guarantees: Consistency, Availability and Partition safety.
> In the realm of graph databases, we observe a similar “two out three” situation. You can either have scalable systems that are not fully open source or you can have open source systems designed for small graphs. Details below.
(the article follows)
> This is one solution to the CAP theorem for graphs. We can store a billion scale graph using this method in parquet files and use a free, cheap and open source solution to traverse them, perform joins without storage costs that are prohibitively high.
The real question being raised in the blog post is - should the next generation graph databases pursue a local-only embedded strategy or build on top of object storage like many non-graph and vector embedded databases are doing.
Specifically, DuckLake (using system catalog for metadata instead of JSON/YAML) is interesting. I became aware of Apache GraphAr (incubating) after writing the blog post. But it seems to be designed for data interchange between graph databases instead of designing for primary storage.
I only mentioned it because I clicked it wondering if someone had found a way to "cheat" CAP for graph databases. When I saw that it was being used as an analogy and not literally, I figured I'd comment.
I still don't quite get the analogy. What are the 2 out of 3 that you can have? The second paragraph I quoted gives a classic 1 out of 2 dilemma - either scalable _or_ open-source.
DuckDB is scalable (can handle TPC-H 1TB) and open source, but doesn't support graphs natively. It supports some graph queries on a SQL native columnar storage.
With the proposed solution, you'll be able to query larger graphs on an open source graph native engine. Thus beating the "CAP theorem for graphs".
DuckPGQ is an interesting option, but unfortunately, that project hasn't been touched in a few months and does not currently work with the latest version of DuckDB.
I haven't touched Perl in about ten years but your example is kind of insane. This is the one place Python is more annoying than Perl???
my $a = ["foo", 123, {"bar" => [[1,2,3], {"qux" => "quux"}]}];
$a->[2]{"bar"}[1]{"spam"} = "eggs";
use JSON;
print(encode_json($a))
That's the same as your example (minus the tuple type), but where Perl shines over Python (a lot) is you could have done the following:
use JSON;
my $a = [];
$a->[2]{"bar"}[1]{"spam"} = "eggs";
print(encode_json($a))
which would yield the json: [null,null,{"bar":[null,{"spam":"eggs"}]}]
To do this in Python is truly grim:
import json
from collections import defaultdict
# recursive defaultdict using lambda
datastructure = lambda: defaultdict(datastructure)
a = [None] * 3
a[2] = datastructure()
a[2]["bar"][1]["spam"] = "eggs"
print(json.dumps(a))
and thats doing it the unpythonic way, if you were to do this like the typical Python dev would accept in an MR for a large corp you would have written:
import json
from collections import defaultdict
def datastructure():
return defaultdict(datastructure)
a = []
# expand list to at least length 3
while len(a) < 3:
a.append(None)
a[2] = datastructure()
a[2]["bar"][1]["spam"] = "eggs"
print(json.dumps(a))
They would still hate you for defaultdict(datastructure) though. Because absolutely no one in Python realises its got that level of expressionism one of the few places it does.
Why is there an arrow between $a and [2], but not between [2] and {“bar”}, in this?
$a->[2]{"bar"}
The highest rated SO answer here explains a little, and it’s the kind of situational nitpickiness that let to me dropping Perl like a hot potato when something better matching my mental model came across my radar: https://stackoverflow.com/questions/1817394/whats-the-differ...
I suppose it's because very soon people got tired of writing
$a->[2]->{"bar"}
which is equivalent and also works, but suffers from the explosion of punctuation that Perl often gets criticised for. There's an element of Do What I Mean where the first arrow says this is a reference and work the rest out for me.
A problem very solved in tooling for both languages, other than plenty of novice library maintainers existing in the ecosystem. Which is hardly a fault of the language and more choosing those vendors for your projects.
