I'd be very interested to hear any thoughts you might have about Jung Chang's book "Wild Swans".
I read this book a year or two ago and learned a lot from it, but I also learned that many people who grew up in China take issue with the author's account. I'd be grateful for any remarks you may be able to share.
You’ve touched on a very sensitive and important point.
It’s true that many people who grew up in China have a complicated relationship with narratives that focus on negative historical periods. There is often a defensive reaction, a feeling that such stories are 'smearing' the country's image.
However, as a writer, I believe that truth is always more important than a curated image. Authentic memories are often scarce, precisely because they are difficult to tell. My goal with the '404' series is to provide a piece of that missing truth—not to judge, but to document a reality that actually existed. In the long run, I believe a society is better served by facing its complex past than by forgetting it.
Hello from Germany, and thanks for the blog post. Fascinating read. I liked how you intertwined the personal point of view with the bigger picture.
"Facing a complex past" is a big theme in Germany, too, of course, and I think it's the only proper way to deal with it. Direct witness accounts and retelling are important and add something that a dry history book can't provide. Keep up the good work!
Wild Swans has received criticism for ignoring statistics and human demographic evidence on the scale of the famine, and therefore lacks an element of verifiability and number inflation. However the author wrote in a spirit of truth telling from familial experience as I understand it: she was finishing her PhD when I was studying at the same UK university (York)
What are you looking for exactly? And what issues did you hear from others who grew up in China? Most of the historical / political events (Great Leap Forward, Cultural Revolution) are fairly accurate, while personal / family experiences are necessarily subjective. China is a huge, diverse country with a vast range of experiences from people growing up in different regions and eras (just like the US, or Europe), so it's hard to dispute any personal / family experience.
The Mathlib Initiative is a new programme of Renaissance Philanthropy. We exist to support Lean's open-source library of formal mathematics known as Mathlib https://leanprover-community.github.io/
I think it is worth comparing this problem with the question of the behaviour of a particle placed at the apex of a cone. I claim it is clear that in this case, the problem is clearly not well-posed because the apex is a singular point: the slope at the apex is undefined. The singular nature of the slope (first derivative) is the issue.
This "dome" is essentially the same issue just with the singularity buried one level deeper: you need to take second derivatives to see it. Indeed a planar cross section containing the vertical axis through its center is a graph of the equation $y^2 = |x|^3$ (up to constants) and this is not twice differentiable at $x = y = 0$. Newtonian mechanics is governed by a second order differential equation, so we need a C^2 regularity assumption to get uniqueness.
So for me there is not really any more philosophically interesting than the question about a particle balancing at the apex of a cone.
I am so glad to read this comment. Wodehouse is sublime in all what he wrote - if you end up reading Jeeves series, I would recommend Right Ho, Jeeves and Code of Woosters
thats all theory
thing is that I mess with large two volt(nominal)
storage cells,the largest are over 250lbs and sit
like dumb beasts,waiting to oblige anyones low voltage requests,hundreds of amps on tap
be nothing to bolt ,some nice shiny copper handles to the terminals and mist them down with some warm salt water
I also mess around with microscopes,and compared to bugs,humans are very poorly made,so many tiny
things are flawless living perfection,and some like wolf(jumping) spiders are smart,smart enough
that they see us seeing them,and are ok with that
one thing that I have observed that plays into the
static electricity thing,is that many of the tiny
critters that I watch,are impecably clean,no dust
or dirt on them at all,perfectly clean,unlike a human finger,which is one zillabutt uggly thing,under magnification
The computer did find the answers itself. I.e., it found "even integers" for P1, "{1,1}" for P2, and "2" for P6. It then also provided provided a Lean proof in each case.
It would make a lot of sense for the lean-code-formalisation of the problems done by the researchers fed to the AI to be provided. Not assuming bad intent in not providing them, but it would help understand better the results.
formal definition of first theorem already contain answer of the problem
"{α : ℝ | ∃ k : ℤ, Even k ∧ α = k}" (which mean set of even real numbers).if they say that they have translated first problem into formal definition then it is very interesting how they initially formalized problem without including answer in it
That wasn't very nice. Are you curious about anything? Happy to help. I'd proactively do it, but I don't want to guess at whats in your mind. My initial guess is you think I think that engaging with the public is an infinite loop. I don't!
I think maybe parent comment is referring to it essentially just employing a zerg rush but with the speed and reaction time of an AI?
Not 100% sure...
Unrelated, iirc the starcraft functionality was an early example of generalizing a pretrained NN, alphaGO, and showing that it could adapt to learn and defeat games across strategic domains, especially after it learned so much strategy from the most difficult, widely played, and most strategically-varied physical game available.
Exactly, a problem and its answer are just different ways of describing the same object. Every step of a proof is a transformation/translation of the same object. It would be disingenuous to say that some heavy lifting isn't done in formalizing a problem but it seems that step is also performed by a machine:
"We established a bridge between these two complementary spheres by fine-tuning a Gemini model to automatically translate natural language problem statements into formal statements, creating a large library of formal problems of varying difficulty."
I'm confused, is the formalization by Gemini or "manually"? Which is it?
"AlphaProof is a system that trains itself to prove mathematical statements in the formal language Lean. It couples a pre-trained language model with the AlphaZero reinforcement learning algorithm, which previously taught itself how to master the games of chess, shogi and Go."
Huh, so MCTS to find the ‘best’ token using a (relatively) small, quick language model? Sounds like an interesting approach to small model text generation too…
Yeah I am not clear the degree to which this system and LLMs are related. Are they related? Or is AlphaProof a complete tangent to CHatGPT and its ilk?
It's a math Language Model. Not even sure it's a Large Language Model. (Maybe shares a foundational model with an English LLM; I don't know)
It learns mathematical statements, and generates new mathematical statements, then uses search techniques to continue. Similar to Alpha Go's neural network, what makes it new and interesting is how the NN/LLM part makes smart guesses that drastically prune the search tree, before the brute-force search part.
(This is also what humans do to solve math probrems. But humans are really, really slow at brute-force search, so we really almost entirely on the NN pattern-matching analogy-making part.)
These kind of LLMs are also very interesting for software engineering. It's just a matter of replacing Lean with something that is more oriented towards proving software properties.
For example, write a formal specification of a function in Dafny on Liquid Haskell and get the LLM to produce code that is formally guaranteed to be correct. Logic-based + probability-based ML.
My reading of it is that it uses the same architecture as one of the Gemini models but does not share any weights with it. (i.e it's not just a finetune)
This is really interesting. I would have expected the understanding to be that humans make a guess, test it, and learn from what did or did not work. The lessons learned from the prior tests would impact future guesses.
Do you know if a system like the OP is learning from failed tests to guide future tests, or is it a truly a brute force search as if it were trying to mine bitcoin?
This quote from the article sounds like it learns from failed tests:
>We trained AlphaProof for the IMO by proving or disproving millions of problems, covering a wide range of difficulties and mathematical topic areas over a period of weeks leading up to the competition. The training loop was also applied during the contest, reinforcing proofs of self-generated variations of the contest problems until a full solution could be found.
Reading between the lines a bit, that does answer the question I had though don't think I I clarified very well.
I read that to say the model's token weights are adjusted as it goes, so in an LLM sense it is kind of learning. It isn't reasoning through an answer in the way a human does though. Meaning, the model is still just statistically predicting what an answer may be and checking if it worked.
I wouldn't chalk that up to learning at all. An AI solving complex math doesn't even seem too impressive to me with the predictive loop approach. Computers are well adept at math, throwing enough compute hardware at it to brute force an answer isn't suprising. I'd be really impressed if it could reliably get there with a similar number of failed attempts as a human, that could indicate that it really learned and reasoned rather than rammed through a mountain of failed guesses.
>with a similar number of failed attempts as a human
I'd be hard to know how many failed attempts the human made. Humans are constantly thinking of ideas and eliminating them quickly. Possibly to fast to count.
Ive never competed in math competitions at this level, but I would have expected it to be pretty clear to the human when they tested a different solution. As complex as the proofs are, is it really feasible that they are testing out a full proof in their head without realizing it?
Hmm, I think it comes down to what the definition of "testing" and "attempt". A human will generate many ideas, and eliminate them without creating full proofs, by just seeing that the idea is going in the wrong direction.
It sounds like AlphaProof will doggedly create full proofs for each idea.
yeah but it doesn't understand the exact syntax on an absolute level, does it...? I understood this to be the same as any language model applied to programming languages (aka Formal Languages). Is that mistaken?
As far as I understand, and I may be wrong here, the system is composed of two networks: Gemini and AlphaZero. Gemini, being an ordinary LLM with some fine-tunes, only does translation from natural to formal language. Then, AlphaZero solves the problem. AlphaZero, unburdened with natural language and only dealing with "playing a game in the proof space" (where the "moves" are commands to the Lean theorem prover), does not hallucinate in the same way an LLM does because it is nothing like an LLM.
Yes, but the problem space means that invalid outputs can be quickly identified - whereas general programming isn’t necessarily amenable to rapid checks.
I mean, aren’t you just describing formal language syntax? Seems like a fundamentally similar situation —- the computer can automatically flag any syntax errors in a millisecond by checking it against the generating grammar for that language. Thats what makes a formal language in the first place, I think!
I do think this language is considerably more robust than the typical programming language, which means a sound program is more likely to end up being valid/“correct”. But still, that’s a difference of degree, not kind, IMO
I don’t mean syntax errors - I mean the difficulty of validating code that contains side effects (like http requests, database access etc).
Validating a math proof either terminates in a reasonable time (in which case it’s useful for training), or does not (in which case the AI should be discouraged from using that approach).
If you are curious, I encourage you to look up The Sphere Eversion Project [1].
It was a project in which we formalised a version of Gromov's (open, ample) h-principle for first order differential relations. Part of the reason it was carried out was to demonstrate what is involved formalising something in differential topology.
I'd be very interested to hear any thoughts you might have about Jung Chang's book "Wild Swans".
I read this book a year or two ago and learned a lot from it, but I also learned that many people who grew up in China take issue with the author's account. I'd be grateful for any remarks you may be able to share.
reply