I am surprised to see that there isn't a lot of information about performance regression testing.
Correctness testing is important but the way SQLLite is used, potential performance drops in specific code paths or specific type of queries could be really bad for apps that use it in critical paths.
While I’ve worked in HFT and understand the sentiment, I can’t recall any open-source project I’ve used coming out with performance guarantees. Most use license language setting no guarantee or warranty. Are there notable projects that do include this consideration as their core mission?
I believe every sensible open-source developer strives to keep their software performant. To me, a performance regression is a bug like any other and I got and fix it. Sure, there's no warranty guaranteed in the license, yet no-one who takes their project even a little seriously takes it as "I can break this any way I want".
Most important property of blockchains is verifiable transparency. That is a huge deal and yes, most financial and government systems would be much better for people if they were built with that type of verifiable transparency.
Decentralization is important but isn't as important. There are many successful chains that aren't decentralized at all and are quite useful.
Most top/real projects in blockchains aren't immutable. Pretty much everything is upgradable/changeable, including blockchains themselves.
Trustless - even this part isn' true for most blockchain systems. They all contain various levels of trusting different entities.
It is the transparency and verifiability that is the key idea and most important improvement that blockchains bring.
>It is the transparency and verifiability that is the key idea and most important improvement that blockchains bring.
You can have transparency and verifiability without blockchain, so long as you trust the parties publishing the ledger. So once again, trustless is really the key part to blockchain.
I have been advocating for the longest time that "repeatable read" is just a bad idea. Even if implementations were perfect. Even when it works correctly in the Database, it is still very tricky to reason about when dealing with complex queries.
I think two isolation levels that make sense are either:
* read committed
* serializable
You either go all the way to have a serializable setup, where there are no surprises. OR, you go in read committed direction where it is obvious that if you want have a consistent view of the data within a transaction, you have to lock the rows before you start reading them.
Read committed is very similar to just regular multi-threaded code and its memory management, so most engineers can get a decent intuitive sense for it.
Serializable is so strict that it is pretty hard to make very unexpected mistakes.
Anything in-between is a no man's land. And anything less consistent than Read Committed is no longer really a database.
I honestly don't think I've seen people reason about read committed well, especially as an application grows it becomes very difficult to understand all the cases in which locks are grabbed/data is accessed. (I feel that way about multi-threaded code and locks too, but that's for another time).
So I really only see serializable to be the only sane isolation model (for r/w transactions), and snapshot isolation is a good model for readonly transactions (basically you get a frozen in time snapshot of the database to work with). This also happens to be the only modes in which Spanner gives you: https://cloud.google.com/spanner/docs/transactions
Yeah there are many use cases where you aren't really using transactions, or where hard accuracy isn't a requirement (you're taking a percentage out of millions of rows) but you need fast write performance. It's very nice and can help you put off transitioning to a "real" data warehouse or a cache for a while.
Here is my issue with this. We are assuming you need to read a "consistent snapshot" in some type of real time application. Because if it isn't real time, you can always have those snapshot type of querying on "replicas", since that is a lot easier to implement correctly without sacrificing performance.
So assuming you are looking at reading "consistent snapshot" in the context of a real time transaction. If the data that you want to read as a "consistent snapshot" is small, locking + reading is good enough in most cases.
If the data to read is too large (i.e. query takes long time to execute, and pulls a lot of data), you are going to have ton of scaling issues if you are depending on something like "repeatable read". Long running transactions, long running queries, etc are bane of all the database scaling and performance.
So you really want to avoid that anyways, you would almost always be much better of changing your application logic to make sure you can have much shorter, time bounded transactions and queries and setup better application level consistency scheme. Otherwise you will at some point hit scaling/performance problems and they will be an absolute nightmare to fix.
Under MVCC this does not require locks which is a non trivial overhead savings for use cases that can tolerate slightly stale data, but want that data to be internally consistent, ie no read skew (and write skew is irrelevant in read only queries). This combination of strict serializable + read only snapshots is quite common with recently developed databases.
That is sort of my point. I think "repeatable read" is a fools gold. You think you wont need to do locking, but it is too easy to make incorrect assumptions about what guarantees "repeatable read" provides and you can make very subtle mistakes which leads to rare, extremely hard to diagnose correctness issues.
Repeatable read type of setups also make it much easier to accidentally create much longer running transactions, and long running transactions/too many concurrent open transactions/etc can create really unexpected, very hard to resolve performance issues in the long run for any database.
I agree that "read committed" is clearer in knowing what you're getting than "repeatable read". The latter can be convenient and workable if you accept that you will need to do locking to avoid write skew, etc.
I've used both "read committed" and "repeatable read" with MySQL and learned to deal with each in their own way.
The problem I've seen is with large/long-lived transactions that impact performance, where the solution is to divide writes into smaller transactions in the design--"read committed" does tend to encourage smaller transactions.
If you have never seen a 10x engineer you either have never worked with truly talented people or you haven’t worked on actual hard problems.
There aren’t just 10 or 100x engineers , there are even infinityX engineers when you work on actual hard problems because some of those can only be solved by handful of people.
The thesis is that the "10x engineers" don't output a linear multiple of the same kind of work their peers do. Instead, they implement solutions that their peers wouldn't or couldn't over any length of time.
It’s interesting how obvious this is for other domains. For example, imagine you are hiring for an orchestra, and are told just to increase the violin section by five to make up for the lack of good violinists.
The 10x engineer identifies common subproblems across different problems, and then creates a layer of reusable code which becomes a library. Then uses this library to write code 10x faster.
The 10x engineer tries to standardize problems and solve them en-masse.
The 1x engineer sees every problem as a different problem that requires a different solution.
The 10x engineer talks about abstractions, the 1x engineer talks about concrete implementations.
The 10x engineer believes in investing in productivity. The 1x engineer wants a visible solution right now.
The 1x engineer never has time because they're too busy making themselves less productive by owning more and more code to maintain. The 10x engineer is always looking for code to remove and refactor, and making the solution as small as possible.
The 10x engineer will talk about programming language features and libraries that promote reusability, the 1x engineer only talks about tangible stuff.
The 1x engineer talks about their car and weekend, the 10x engineer talks about technology.
I check all of the 10x engineer boxes from above, or at least I think I do, and that speaks as if I am a 10x engineer but I genuinely don't think I am. I think 10x engineer is something different. 10x engineer usually doesn't care about the things from above.
I think it is more about the ability to grasp vast amounts of complexity in relatively short amount of time and be able to sustain it. Business and market wise this is what is going to make a difference. Some engineers aren't even able to tackle such complexity because of so much layers of it which is why at some moment it just becomes ... too much. Others that can do it, it would take them considerably larger amount of time to do it.
> The 1x engineer talks about their car and weekend, the 10x engineer talks about technology.
Call me a proud member of the 1x engineer club. When I close my laptop at the end of the day or even when I’m having lunch with my coworkers, few people would know that I’m an “engineer” at all. The list of things that I care about more than technology in my non work life is a mile long. Technology is a means to an end - to support my addiction to food and shelter.
This is fine for a certain class of work, and disastrous when applied to another class of work.
To use an analogy a physician's assistant[1] doing brain surgery could be a big problem. if you need the latter. Also, you're probably overpaying if you hire the latter to administer Tylenol ...
Spoiler alert: most of the 2.7 million developers in the US aren’t “solving hard problems”. I interact with the people who are writing the services for the largest cloud provider. When I meet them for lunch, they are seldomly talking about work.
For awhile, I was working with the service teams involved with the various AI services trying to publish an open source demo around their services. When we left for lunch we were talking about travel, cars, our families, places to eat, etc.
I bet the well adjusted ones don’t talk about plumbing in every sentence.
There is so much more to life than technology - this is coming from someone who did hobby programming in assembly language on four different processors by the time I graduated - in 1996.
I sincerely doubt the 10x brain surgeon talks only about brain surgery in their downtime. I would assert that a true 10x person is smart enough to maintain good work-life balance.
There is an infinite pool work you can do. Smart, real 10x’ers understand what is a priority and are smart enough to pack their shit at the end of the day, go home and do nothing at all with their work.
I've been a 10x engineer, I've been a 1x engineer, and I've been a 0.5x engineer. There isn't some consistent set of things, it's a ton of circumstance.
Now, as a leader type, I would prefer to hire 1x engineers to 10x engineers, for a whole litany of reasons. 1x engineers are largely predictable, where as 10x can be 10x in any wild direction, because they're more or less deciding for themselves a bunch of shit they really shouldn't be.
10x engineers like to decide for themselves things like how to go to market or which features to build in what order, without doing any of the requisite research or information gathering necessary. It's infuriating because 10x engineers think passion or some arbitrary definition of "Correctness" gives them power, when in reality they're just operating with a heavily limited set of information (gee, I wonder how they had all that time to build stuff, maybe because they weren't attending meetings???).
Sorry, this turned into a not-very-coherent rant, but the point is 10x engineers aren't worth hiring as anything other than literally employee #1, and even then it's a huge dice roll.
Well if you honestly find the pointless complicated code you write for whatever stupid pointless startup more interesting than cars I guess maybe I just think you’re a moron? Wasn’t that the point of my original post???
I bike and walk more than you do by the way. Actually come to think of it my van has appreciated! LOL!
Because they are smart enough to realize they could make a shitload more money in their current occupation and treat their cycling as a hobby. Often times once you make your hobby turn into an job it no longer is fun. …maybe… never actually tested that theory personally so I don’t actually know if it is true.
High availability + uptime is now necessary for any business no matter how small. If you need to run 24/7 available high uptime services you can’t avoid distributed systems one way or another. (Distributed systems vary by complexity of course but anything that has some failover type of mechanism is going to be a distributed system )
Also, industry is now transitioning where even small business might need geo distribution , not just for compute but also for data. That is going to make distributed systems even more complex even for the small guys.
Cloud services have made a lot of this stuff easier for average Joe but you can’t really avoid dealing with at least some fundamental concepts of distributed systems
Not sure I agree. My Hetzner vps has less downtime than some AWS regions. To be honest, I have yet to experience any downtime from them over multiple years. There is a good chance you introduce more potential downtime by blowing up the complexity with a distributed system.
I have been wondering for a while now, how does Forbes still maintain its brand?
It has now been at least two decades that it has had close to zero credibility, but somehow in general population it is maintaining its brand as a legit publication
People always have been misled by branding. For example, AT&T today actually has (almost?) nothing to do with the original AT&T; the name was effectively purchased.
These days, I find very few people will even engage in evaluating the credibility of what they read - they seem to scorn the issue. Look at all the links to Wikipedia here on HN, for example - something I didn't see until maybe a year or two ago: Wikipedia was always seen as a bit dubious and uncertain. I haven't seen anyone argue otherwise, but again they seem to scorn or resent even raising the question.
> For example, AT&T today actually has (almost?) nothing to do with the original AT&T
Of all valid examples available, you picked AT&T, which is definitely out of set? Seriously?
Okay, they do not control the Northeast (that's Verizon), but other than that it'll be the equivalent of asking if Russia can trace back through the pre-1919 Russian Empire or the current Germany from Prussia.
P.S. Please look at this merger chart: https://si.wsj.net/public/resources/images/P1-BZ033_LIONDO_9.... Although it's unlikely that AT&T and Verizon will merge again (unless the US FTC lost all anti-trust enforcement), your statement can at least be proven wrong in this context. Maybe use Polaroid as an example next time?
> Wikipedia was always seen as a bit dubious and uncertain
Base knowledge has to come from somewhere, so where it would have been an encyclopedia fifteen years ago it is wikipedia now. That's just the times moving on - if you have material concerns with the linked articles you are free to edit them (although it must be said you will have a hard time doing so with rich folks - they have firms to whitewash their pages).
> Base knowledge has to come from somewhere, so where it would have been an encyclopedia fifteen years ago it is wikipedia now.
That agrees with the parent and GP comments: The assertion was that people rarely evaluate credibility, and the common citation of Wikipedia is given as an example. So we all agree that Wikipedia is commonly used.
If you mean that Wikipedia is inevitably used, that seems hard to suppot. There are an incredible number of sources available, and there's no reason Wikipedia had to be popular now. If people cared more about accuracy, a source with more focus on it would be more popular. I never use Wikipedia beyond checking the schedule for sporting events.
> if you have material concerns with the linked articles you are free to edit them
The problem is that I don't know what is accurate and what isn't on Wikipedia, and I don't have time to research everything. Also, editing Wikipedia is often very difficult and frustrating.
> although it must be said you will have a hard time doing so with rich folks - they have firms to whitewash their pages
And a hard time with anything controversial, or a page where an editor or editors who feel ownership, or one with an astroturfing campaign, or many others. Wikipedia is the original disinformation source, discovered long ago by some groups.
Ignorance most likely? I've noticed things getting worse, but everything in this article is new information to me. I also didn't know Huffington Post was written by contributors.
I think it’s all about placement and reach. They are good at SEO and article promotion on social media. Since people see it all the time they just assume it can’t be that bad.
Nah, most of USDT is on TRC20 because gas fees are way cheaper there and most of the exchanges support withdrawing USDT using TRC20.
So all arbitrageurs, market makers, etc, transfer USDT between different entities through TRC20 network. USDT on TRC20 started to increase only after Ethereum gas fees got ridiculously high and it became too expensive to use even for pretty high volume market makers.
Absolutely not. Large Tron issuances started well before Ethereum gas fees soared. We are talking a couple of years ago.
Also, market makers moving Tether around do not give two shits about $60 in gas fees or whatever. Additionally, a huge amount of Tether has gone into DeFi on Ethereum and L2s (or bridged to newer L1s, which typically don't have a Tron bridge).
People who have no idea how crypto markets work always get riled up by these tweets. Contrary to popular belief, there is generally huge demand spike for USDT when crypto market has flash crashes, not other way around.
More USDT got issued because there was opportunity to make 0.1-0.2% on each new USDT since demand for it was that high (i.e. people were paying real USD to buy USDT at a premium, imagine that).
USDT (and USDC) demand spikes during crypto crashes because people actually cash out to these instruments because they are so widely used and trusted by actual market players. People aren't cashing out of crypto ecosystem during most crashes, money actually stays in crypto networks.
> Contrary to popular belief, there is generally huge demand spike for USDT when crypto market has flash crashes, not other way around.
This is not contrary to popular belief at all. Tether almost always prints a few billion around large pullbacks, and the cynical view is that this is to stabilize markets by buying the dip.
> More USDT got issued because there was opportunity to make 0.1-0.2% on each new USDT since demand for it was that high (i.e. people were paying real USD to buy USDT at a premium, imagine that).
Uh yeah this is not at all how stablecoin premia get arbed. If you'd like a real answer, I'd be happy to explain. But I get the feeling that you're pretty invested in crypto and just want answers which align with your beliefs.
While I am no huge volume arbitrageour/market maker, I actually make a living running arbitrage systems and market making systems specifically for stablecoins.
I do consider to know what I am talking about, considering my systems make hefty profits and have no real exposure to volatile crypto assets (or even long term exposure to crypto in general, my profits are in real world dollars). This is why I know that USDT has actual real utility and it is currently best instrument in its category. USDT is indeed somewhat shady but it’s premise and utility are all sensible and it is hugely profitable for its founders. They have zero reasons or incentives to do suspect things that could risk its peg.
If something goes horribly wrong for USDT it is going to be due to incompetence and not because of some long term planned con.
I'm not really sure what you're getting at here. If you're generating income but arbing stablecoins, then you know exactly what I'm talking about. I can say with near certainty that you have never, and never will, directly mint or redeem Tether...which proves my point.
> This is why I know that USDT has actual real utility and it is currently best instrument in its category.
I don't disagree at all.
> If something goes horribly wrong for USDT it is going to be due to incompetence and not because of some long term planned con.
Market making on crypto exchanges is difficult because of high fees, small tick sizes, and minuscule spreads. Of course you can get volume discounts, but still.
On which exchanges do you trade and do you rely on volume discounts to make profit?
What does any of that have to do with the discussion? That doesn't in any way address the issue with Tether's lack of transparency. The issue is whether Tether actually has the capital to back up it's stablecoin.
I sometimes wonder how many of the commenters actually read the article itself. All the discussion in comments is about residential properties when in the article it clearly says:
"
Another concern the report outlines is that the U.S. anti-money laundering regime is focused on residential purchases, when a significant portion of the cases GFI reviewed involve commercial real estate transactions.
"
Or another:
“When you talk about residential real estate, the heart of it is identifying who is the beneficial owner, [because] if you find out who the beneficial owner is, it also tells you who the criminal is,” Kumar told ICIJ. “In a commercial real estate investment, you don’t have to own the majority stake to be a criminal. You can own 2% of a $500 million property, and you are [still] laundering millions through it.”
Correctness testing is important but the way SQLLite is used, potential performance drops in specific code paths or specific type of queries could be really bad for apps that use it in critical paths.