Hacker Newsnew | past | comments | ask | show | jobs | submit | oulipo2's commentslogin

Assuming I'd like to replicate my production database for either staging, or to test migrations, etc,

and that most of my data is either:

- business entities (users, projects, etc)

- and "event data" (sent by devices, etc)

where most of the database size is in the latter category, and that I'm fine with "subsetting" those (eg getting only the last month's "event data")

what would be the best strategy to create a kind of "staging clone"? ideally I'd like to tell the database (logically, without locking it expressly): do as though my next operations only apply to items created/updated BEFORE "currentTimestamp", and then:

- copy all my business tables (any update to those after currentTimestamp would be ignored magically even if they happen during the copy) - copy a subset of my event data (same constraint)

what's the best way to do this?


You can use "psql" to dump subsets of data from tables and then later import them.

Something like:

  psql <db_url> -c "\copy (SELECT * FROM event_data ORDER BY created_at DESC LIMIT 100) TO 'event-data-sample.csv' WITH CSV HEADER"
https://www.postgresql.org/docs/current/sql-copy.html

It'd be really nice if pg_dump had a "data sample"/"data subset" option but unfortunately nothing like that is built in that I know of.


pg_dump has a few annoyances when it comes to doing stuff like this — tricky to select exactly the data/columns you want, and also the dumped format is not always stable. My migration tool pgmigrate has an experimental `pgmigrate dump` subcommand for doing things like this, might be useful to you or OP maybe even just as a reference. The docs are incomplete since this feature is still experimental, file an issue if you have any questions or trouble

https://github.com/peterldowns/pgmigrate


Indeed, but is there a way to do it as a "point in time", eg do a "virtual checkpoint" at a timestamp, and do all the copy operations from that timestamp, so they are coherent?

That's not how science works. Fundamental science works on much longer timescales (10/20/50/100 years), that are not accessible to companies

You don't understand how science works... you wouldn't have nuclear reactors or GPS etc if it wasn't for government run projects

We WARMLY welcome all researchers here in Europe! Please come, we love science (and arts) and want to build an inclusive, open-minded society together!

You might welcome them all, but you don't have jobs for most of them.

Still better than the US where they have no jobs AND they are persecuted...

> and want to build an inclusive, open-minded society together!

Which will be guaranteed by strict monitoring of your private chats!


Well, we're still fighting against this. Still better than the US situation though

Trust me, every scientist in America has been clawing for every eurpoean research grant opportunity there is. Competition is stiff

I've tried HyperDX and SigNoz, they seem easy to self-host and decent enough

So linear type + mutable value would be quite close to Rust, right?

Rust has affine types, not linear. It also doesn't have mutable value semantics, it uses references, lifetimes, and borrowing.

I've never seen any significant difference in linear vs affine types.

To me it just seems like Rust has Linear types, and the compiler just inserts some code to destroy your values for you if you don't do it yourself.

I guess the only difference is that linear types can _force_ you to manually consume a value (not necessarily via drop)? Is that what you are going for?


Affine types are "may use" and linear types are "must use," yeah. That is, linear types are stronger.

See https://faultlore.com/blah/linear-rust/ for a (now pretty old but still pretty relevant, I think) exploration into what linear types would mean for Rust.


Interesting, for me the "between Rust and Go" would be a nice fit for Swift or Zig. I've always quite liked the language design of Swift, it's bad that it didn't really take off that much

One thing working on this project has already done is give me more appreciation for a lot of Zig's design.

Zig really aims to be great at things I don't imagine Rue being useful for, though. But there's lots of good stuff there.

And lots of respect to Swift as well, it and Hylo are also major inspiration for me here.


I think with Swift 6 Apple really took it in a wrong direction. Even coding agents can’t wrap their mind around some of the “safety” features (not to mention the now bloated syntax). If anything, Swift would go down as a “good example why language design shouldn’t happen by committee in yearly iterations”.

Checkout Borgo: https://github.com/borgo-lang/borgo

I also find that D is good between language. You can do high level or low level whenever you need it.

You can also do some inbetween systems programming in C# if you don’t care about a VM or msft.


> You can also do some inbetween systems programming in C# if you don’t care about a VM or msft.

C# Native AOT gets rid of the JIT and gives you a pretty good perf+memory profile compared to the past.

It's mostly the stigma of .NET Framework legacy systems that put people off, but modern C# projects are a breeze.


AFAIK there’s still holes like reflection and you have some work, but if that’s changed that’s really good. I suspect it’ll be hard for C# to escape the stench of “enterprise” though.

I’m looking forward to seeing how it shapes out over the next few years. Especially once they release union types.


FWIW JIT is rarely an issue, and enables strong optimizations not available in AOT (it has its own, but JIT is overall much better for throughput). RyuJIT can do the same speculative optimizations OpenJDK Hotspot does except the language has fewer abstractions which are cheaper and access to low-level programming which allows it to have much different performance profile.

NativeAOT's primary goal is reducing memory footprint, binary size, making "run many methods once or rarely" much faster (CLI and GUI applications, serverless functions) and also shipping to targets where JIT is not allowed or undesirable. It can also be used to ship native dynamically or statically (the latter is tricky) linked libraries.


They know AI is inefficient and mostly just a glorified "template filler" at this point...

Using uv you can do

uvx --with psycopg2-binary --from sqlit-tui sqlit


What would be the main differences with Harlequin?

I asked the same thing on Discord where he also showcased it. He answered the question with this.

> I wanted to make something that makes intuitive sense to me, and I wanted to make a sql version of what lazygit does, namely you can just jump in and there's no need for external documentation. To navigate the harlequin with keyboard only you have to learn the keybindings. Also I though it was quite hard to connect to databases via the cli and install adapters (again, documentation necessary). I wanted to have a tool that you can just run and its intuitive. There's definitely more features in harlequin and I went with more lightweight, though sqlit has SSH tunnels, which harlequin lacks.

When I linked lazysql[0], he replied with this

> That was my original inspiration. Same problem with harlequin, I didn't feel like it was intutive and to follow the "lazy" mindset of lazydocker and lazygit, it should just be easy and not require looking up keybindings. Lazysql ended up with too many features for my taste to the point it was difficult to use

[0]: https://github.com/jorgerojas26/lazysql


I've recently been trying to find a good SQL TUI client, and one of the ones I tried was Harlequin. The main disadvantage, which every other SQL TUI I tried also suffers from, is that they don't comprehensively show every type of database object in the browser pane (e.g. tables, views, indexes, procedures). For example Harlequin only shows tables. This means that when you want to view one of the missing ones, you have to run a query. This means that you have to already know that the object you are looking for exists: there's no way to get a good overview of what you have in the database just by glancing at the browser. For me this sort of defeats the point of having a TUI in the first place.

It seems like the OP's offering has the same problem, although it does offer more objects than Harlequin. I'll be bookmarking it to see how it develops.


Appreciate that take!

I'm looking into indexes/triggers/sequences. That will be added in a near-future release, I hope you can benefit from it once I release it :)


Great. If you added those this would probably become my go-to SQL client.

Hi,

I've added this now: https://github.com/Maxteabag/sqlit/releases/tag/v1.1

Thank you for the suggestion.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: