It's not supposed to be some red line absolute max price, but rather "how much is this item worth to you?" You set that as your max bid price. If you get it at auction for less than that, you got a good deal, but if you buy it for more, you got a bad deal. If someone outbids you, then maybe it was worth it to them, but you (supposedly) would not have wanted to buy the item for that much, and would rather use your money for something else.
For tricky-to-price items like unique art pieces, the idea that you can pin this down might be a fantasy, but for commodity items it's pretty reasonable. If you can buy the same thing at costco dot com for $500, then it's probably not worth more than $500 to you, and if at auction you get outbid and it sells for $500.01 then you'll shrug and go order the same thing for a cent less, having wasted only a few minutes of your time. If the item you're bidding on is discontinued (e.g. it's last year's model) but you can buy a slightly better one for $550, and you can spare that extra $50, then again you won't be too sad about getting outbid. Online auctions are more popular for used items, but again in that case you usually still have an idea of what a used item is worth to you.
The logic of "you shouldn't ever need to snipe, just bid your max price" only works if we assume that the max price is a red line though. If I "value something" at $5000 (as in I want to buy it at $5000), and I bid $5000, and someone bids $5000.01, I would probably be happy if I sniped them and got the item for $5000.02.
I'm not defending "you shouldn't ever need to snipe, just bid your max price" as a hard principle, just trying to explain where the idea comes from. Sniping can be strategic for lots of reasons: you don't have to commit to a bid until the last second (in case you find a similar item for cheaper elsewhere), you deny other people information, you might avoid anxiety from wondering whether your bid will win, etc.
That said, the max price is supposed to be a price where you are not especially happy to get the item at that price, but not really sad either, a price where you would say "well, I hoped for better but I guess that's a fair deal". That's not realistically pinned down to the cent. But if you set a max price at $5000 and would be happy to get the item at $5000.02 (for some reason other than satisfaction from sniping), then you set your max price wrong, or at least differently from how economists expect you to set it.
> But if you set a max price at $5000 and would be happy to get the item at $5000.02 (for some reason other than satisfaction from sniping), then you set your max price wrong, or at least differently from how economists expect you to set it.
I think this is the problem. When most sciences observe reality diverge from the model, they see that as a flaw in the model. When economists (at least you HN "economists") observe reality diverge from the model, they seem to see that as a flaw in reality.
My phrase "how economists expect you to set it" is probably wrong here, since I'm not an economist, I've just read the most basic theory about how to use this tool, and also used it myself (on eBay, you know, years ago when the site was mostly auctions). So I don't really know what "economists expect", but rather the basic guidelines for using this tool. You got me there.
> I think this is the problem. When most sciences observe reality diverge from the model, they see that as a flaw in the model. When economists (at least you HN "economists") observe reality diverge from the model, they seem to see that as a flaw in reality.
But like, to double-check here: "reality" means your imagined use of a tool that you do not in fact use, right? Like you say you "don't do auctions" and I'm trying to explain what that option is for, and you're countering that the basic "how to use this tool" explanation is a wrong model of reality?
See also the paper Ribbon filter: practically smaller than Bloom and Xor: https://arxiv.org/abs/2103.02515, which is a similar idea though not by the same authors.
IIRC, binary fuse filters are faster to construct than ribbon filters, but typically not quite as space-efficient. There are also frayed ribbon filters (by me) which are slower and more complex to construct but more space-efficient. There's no paper for those, just a Rust implementation.
Ribbon filters are deployed in Mozilla's Clubcard for distributing compressed certificate revocation lists: https://github.com/mozilla/clubcard and https://jmschanck.info/papers/20250327-clubcard.pdf. CRLs are an almost ideal application of this sort of compressed set tech, since the aggregator runs batch jobs and needs to distribute the set to very many clients. It's not perfectly ideal because CRLs require frequent updates and none of these methods support delta updates. There is a straightforward but inelegant workaround, which is to send a compressed set that represents the delta, and query both on the client.
A second special case of this theorem is Pascal's theorem, which says (roughly) that a variant of the elliptic curve group law also works on the union of a conic C and a line L (this union, like an elliptic curve, is cubic), where the group elements are on the conic. One point O on the conic is marked as the identity. To add points A+B, you draw a line AB between them, intersect that with the fixed line L in a point C, draw a second line CO back through the marked identity point, and intersect again with the conic in D:=A+B. This procedure obviously commutes and satisfies the identity law, and according to Pascal's theorem it associates.
Under a projective transformation, if the conic and line don't intersect, you can send the line to infinity and the conic to the units in (IIRC) a quadratic extension of F (e.g. the complex unit circle, if -1 isn't square in F). Since the group structure is defined by intersections of lines and conics, projective transformations don't change it. So the group is isomorphic to the group of units in an extension of F. If they do intersect ... not sure, but I would guess it instead becomes the multiplicative group in F itself.
The multiplicative group of F can be used for cryptography (this is classic Diffie-Hellman), as can the group of units in an extension field (this is LUCDIF, or in the 6th-degree case it's called XTR). These methods are slightly simpler than elliptic curves, but there are subexponential "index calculus" attacks against them, just like the ones against the original Diffie-Hellman. The attack on extension fields got a lot stronger with Joux's 2013 improvements. Since no such attack is known against properly chosen elliptic curves, those are used instead.
Annoyingly, while that d = e^-1 usually isn't used in practice (except in cases where you care about side-channel / fault resistance more than the 4x speedup), the Carmichael totient itself still is used in practice. At least if you want to conform to FIPS 186-5 / SP800-56B, which says that the private key includes d = e^-1 mod the Carmichael totient LCM(p-1,q-1), even if you're going to use the CRT. And that means you have to compute LCM(p-1,q-1), which also has side-channel considerations.
Do the standards require strong primes for RSA? I think FIPS doesn't ... it gives you that option, either for the legacy reasons or to get a proof with Pocklington's theorem that (p,q) really are prime, but just choosing a random (p,q) and running enough rounds of Miller-Rabin on them is considered acceptable IIRC.
Internally, most signature algorithms use hash functions. RSA-PSS, EdDSA and ML-DSA use them to provide something like randomness, and the security analysis of those signature schemes includes arguments assuming (in some very particular, technical ways) that the hash function outputs "look random".
Classical DSA and ECDSA do not use hash functions this way, but in my opinion they aren't stronger for it: they're basically assuming instead that some other mathematical function "looks random", which seems riskier than assuming that about a hash function. I've heard that the reason for this is to get around Schnorr's patent on doing it with hash functions, which has since expired.
The SHA3 and SHAKE hash functions (underlying e.g. ML-DSA) are explicitly designed to "look random" as well.
There are some signature schemes that try not to make such strong assumptions: in particular SLH-DSA targets properties more like first- and second-preimage resistance, target-collision-resistance, and so on.
All the algorithms you mention are PKI. RSA uses two large prime numbers. I don't see what hash sequences have to do with this at all.
PKI isn't even really about randomness. RSA does use a kind of randomness to generate its large primes, but that is beneficial and not required. The primary consideration is the math to reverse guess a factor of two primes or the square root of a large number, or something else computers currently find cheap to compute in one way but extremely expensive to reverse.
The intro textbook descriptions of cryptographic systems omit a lot of very important details.
When using RSA to sign a message m, in practice you don't send m^d mod N. That would generally be insecure, depending on what kinds of messages your system sends and/or accepts. In practical systems, instead you hash m, and then adjust the hash through a (possibly randomized) process called "padding" to be a value in [0,N). There are different standards for padding, and the better designs use additional hashing.
The security of the system depends in part on the hashed-then-padded message "looking random", i.e. not having structure that can be exploited by an attacker. It turns out to be tricky to formalize what exact randomness property you need, so cryptosystems are often analyzed in the "random oracle model" (ROM) in which the hash function has impossibly strong randomness properties.
It seems that usually, if you use a strong hash function, a scheme that's proved secure in the ROM is secure in real life (or at least it's not the ROM part that breaks); the counterexamples are usually really contrived. This article is about a somewhat-less-contrived, but still not quite realistic, example where something that's secure in the ROM would break due to the ROM being an unrealistic model.
As I understand the paper, the point is that Fiat-Shamir does *not* give a correct proof of the program's output.
They gave a (maliciously constructed) program whose outputs are pairs (a,b) where certainly a != b (instead the program is constructed such that a = b+1 always). But you can get the corresponding Fiat-Shamir protocol to accept the statement "I know a secret x such that Program(x) = (0,0)", which is clearly a false statement.
Adding to some other comments in the thread: finding missing or extra numbers is closely related to error-correcting codes, especially binary linear codes. In an error-correcting code, you have a string of bits or symbols, with symbol x_i appearing at position i. You choose the code so that valid sequences have a certain mathematical property, and then if one or a few symbols are corrupted, then you can use that property to correct the errors. The property is typically that a certain linear function called the "syndrome" is zero, meaning that sum(x_i * G_i) = 0 where each G_i is some strategically chosen vector, particular to the code. The math for how to correct is particular to the chosen G_i, and it's a really interesting field of study.
In a typical error-correcting code usage, you have an encoder which takes your message, and adds some extra symbols at the end which are calculated so that the syndrome is zero. Then when receiving your message, the receiver calculates the syndrome and if it's not zero, they know that at least one error has occurred. By using the code's decoding algorithm, they can figure out the fewest (and thus hopefully most likely) number of changes which would result in that error syndrome, and use this information to (hopefully) correct the transmission error.
For the missing numbers problem, you can set x_i to "how many times does the number i appear?". Then since the syndrome is sum(x_i * G_i), you can compute the syndrome on an unordered list of the i's. You are expecting the syndrome to be the same as the syndrome of full set 1...n, so when it is not, you can figure out which few x_i's are wrong that would lead to the syndrome you observed. You have an advantage because you know how many numbers are missing, but it's only a slight one.
The author's solution is called the Hamming code: you set F(i) = i, and you do the additions by xoring. Using error-correcting codes generalize to more missing numbers as well, including using xor, but the math becomes more complicated: you would want to use a fancier code such as a BCH or Goppa code. These also use xor, but in more complicated ways.
If you imagine a polynomial L(z) that's zero at all the missing numbers, you can expand the coefficients out. For example, with 2 missing numbers (x,y), you have:
L(z) = z^2 - (x+y)z + xy.
You already have x+y, but what's xy? You can compute it as ((x+y)^2 - (x^2 + y^2))/2. This technique generalizes to higher powers, though I forget the exact details: basically you can generate the coefficients of L from the sums of powers with a recurrence.
Then you solve for the roots of L, either using your finite field's variant of the quadratic formula, or e.g. just by trying everything in the field.
* But wait, this doesn't actually work! *
Over fields of small characteristic, such as F_2^m, you need to modify the approach and use different powers. For example, in the equations above, I divided by 2. But over F_2^m in the example shown above, you cannot divide by 2, since 2=0. In fact, you cannot solve for (x,y) at all with only x+y and x^2 + y^2, because
So having that second polynomial gives you no new information. So you need to use other powers such as cubes (a BCH code), or some other technique (e.g. a Goppa code). My sibling comment to yours describes the BCH case.
This will depend on the field, and for F_2^m you want odd powers: sum(x), sum(x^3), sum(x^5) etc. Using sum(x^2) won't help because squaring over F_2^m is a field homomorphism, meaning that sum(x^2) = sum(x)^2.
This is also how BCH error-correction codes work (see https://en.wikipedia.org/wiki/BCH_code): a valid BCH codeword has sum(x^i where bit x is set in the codeword) = 0 for t odd powers i=1,3,5, ... Then if some bits get flipped, you will get a "syndrome" s_i := sum(x^i where bit x was flipped) for those odd powers. Solving from the syndrome to get the indices of the flipped bits is the same problem as here.
The general decoding algorithm is a bit involved, as you can see in the Wikipedia article, but it's not horribly difficult:
• First, extend the syndrome: it gives sum(x^i) for odd i, but you can compute the even powers s_2i = s_i^2.
• The syndrome is a sequence of field values s_i, but we can imagine it as a "syndrome polynomial" S(z) := sum(s_i z^i). This is only a conceptual step, not a computational one.
• We will find a polynomial L(z) which is zero at all errors z=x and nowhere else. This L is called a "locator" polynomial. It turns out (can be checked with some algebra) that L(z) satisfies a "key equation" where certain terms of L(z) * S(z) are zero. The key equation is (almost) linear: solve it with linear algebra (takes cubic time in the number of errors), or solve it faster with the Berlekamp-Massey algorithm (quadratic time instead, maybe subquadratic if you're fancy).
• Find the roots of L(z). There are tricks for this if its degree is low. If the degree is high then you usually just iterate over the field. This takes O(#errors * size of domain) time. It can be sped up by a constant factor using Chien's search algorithm, or by a logarithmic factor using an FFT or AFFT.
You can of course use a different error-correcting code if you prefer (e.g. binary Goppa codes).
Edit: bullets are hard.
Further edit just to note: the "^" in the above text refers to powers over the finite field, not the xor operator.
Yesterday I linked to an implementation (with complexity quadratic in the number of errors) I helped to create in another comment in this thread.
> constant factor using Chien's search algorithm
Chien's search is only really reasonable for small field sizes... which I think doesn't really make sense in this application, where the list is long and the missing elements are relatively few.
Fortunately in characteristic 2 it's quite straight forward and fast to just factor the polynomial using the berlekamp trace algorithm.
For tricky-to-price items like unique art pieces, the idea that you can pin this down might be a fantasy, but for commodity items it's pretty reasonable. If you can buy the same thing at costco dot com for $500, then it's probably not worth more than $500 to you, and if at auction you get outbid and it sells for $500.01 then you'll shrug and go order the same thing for a cent less, having wasted only a few minutes of your time. If the item you're bidding on is discontinued (e.g. it's last year's model) but you can buy a slightly better one for $550, and you can spare that extra $50, then again you won't be too sad about getting outbid. Online auctions are more popular for used items, but again in that case you usually still have an idea of what a used item is worth to you.
reply