Thanks for the link. This illustrates my concern pretty well. Instead of having one highly relevant index we now have a multitude of smaller services that hold information about the podcast universe. And afaik they dont confederate in any way. So one would need to query all of them...
What's your basis for claiming that Tinygrad can't compute 2nd order partial derivatives (i.e. Hessians) needed for LBFGS? Tinygrad like PyTorch uses automatic differentiation which has no problem supporting nth order derivatives.
OP does not (seemingly) claim that tinygrad can't compute hessians, only that a first-order optimization method was the only thing tried.
Also, as a quasi-newton method, L-BFGS does not require explicit (pre-)computation of the hessian (it implicitly iteratively estimates its inverse in an online manner).
Commercial pilot here. Instead of climate change, we should be talking about continuous descent profiles (CDPs) that have become more common in the past years 5-10 years. These profiles with idle engines allow for a smoother, more fuel-efficient descent by reducing the need for level-off segments. However, CDPs can increase the perception of turbulence during descent. This is because aircraft remain at higher altitudes for longer periods, where atmospheric instability and wind shear are more pronounced. This increased turbulence is not due to climate change but rather the result of these optimized descent procedures aimed at reducing fuel consumption and minimizing environmental impact.
Interesting! I didn’t know that descent profiles had changed this decade. Now that you mention it, I seem to recall far less leveling off than when I was younger, at the very least you’ve incepted the idea into my head now.
Also worth noting that to a passenger, CAT is the worst feeling you’ll have on most flights — the “oh shit we’re not flying anymore” vibe is real bad, and usually when you hit proper air again, the sudden jerk feels bad as well.
As someone with like 8 flight hours to my name, I’ll say to a learning pilot, stalling feels much worse than CAT would, it’s a different sort of not flying, it’s like “oh shit the plane forgot how to fly, what now”.
In a broader economic context, they take profitable but unpredictable companies and make them boring. Google is the more recent example. Apple (pre-return of Job) is another. Here's a good article: https://www.inc.com/justin-bariso/apple-googlemckinsey-how-a...
I mean, that was practically already the case with the Google Sycamore processor. IBM claimed that they could simulate the 53-qubit circuit in something like 24 hours on a supercomputer (with a bunch of bespoke optimizations), but a 54-qubit version would have been completely classically intractable. We didn’t need to get to 100+ qubits, and those came out before now.
Truth. Not to mention the stunning asymmetry in energy usage. At a minimum, if we can solve an equivalent computational problem using a quantum computer with orders of magnitude less energy than a classical HPC, QC merits consideration. The carbon footprint of data centers is far from negligible.
Computational advantages aren't the only types advantages we should care about.
We can barely simulate a rough approximation of the brain of a worm. The world is filled with many things more impressive than early examples quantum computing.
The big difference is that these early quantum devices (non-scalable noisy quantum computers) are *programmable* and *universal*. It is the difference between an analog computer that can simulate one thing of fundamentally bounded size and digital computers that can simulate "anything" with *in principle* unbounded size.
You are off by 9 orders of magnitude at least. The classical super computer in these comparisons costs 0.3 Billion dollars and this does not count the many Trillions that took to develop the tech.
Even on this measure, the (useless for now) quantum tech wins.
I was assuming it was transistor count to qubit count, didn’t count the 0’s but either way not 9 orders of magnitude off so not sure what exactly you are saying, but either way all I am saying is that it isn’t impressive to a lot of people precisely because it is useless right now. So you are off by 100% in my opinion, way more than 9 orders of magnitude ;)
The people that have created these devices never claimed that they can be used for any useful computation, neither did the people you are talking to here. However, these technology demonstrators do show a programmable computation (sampling from a particular probability distribution) that is infeasible on anything but a supercomputer and becomes just impossible once you add a couple more qubits.
Sure, we do believe these devices, when made more reliable, will also do "useful" computations that are infeasible on supercomputers, but we are aware that we need to build the devices first in order to convince you.
9 orders is the difference between a few dollars and a billion dollars.
Quantum supremacy just means that a quantum computer is better than a classical computer at solving some (toy) task, not that one country's quantum computer is better than another country's.
"Mine is bigger!" does not define quantum supremacy. In particular, IBM does not claim to have performed any computations with theirs. By what they do report, that is because it cannot do any, yet.
ex-IBMer here. I rode this from 2001 until 2016 when it became clear that breaking up the company (as they recently did) would become the only sensible path forward.
While many poor decisions were made inside of the company, I ultimately blame Wall Street for IBM's downfall. Remember that back in 2011, Palmisano finished strong with IBM's Watson winning Jeopardy, IBM Software delivering consistent >80% profit margins on $10Bs in revenue, and a strong services backlog.
Many don't know that Palmisano's departure was preceded by a Wall Street mediated competition for the successor. IBM Software Group SVP, Steve Mills, was the obvious choice. The guy was a lifetime IBMer, intellectually superb, allegedly with photographic memory, effective public speaker, and with a proven history of leading (at the time) the 3rd largest software business in the world.
Unfortunately, Wall Street didn't like Mills because he did not come across well on CNBC. The guy is chubby and doesn't look like a conventional CEO. So Ginny Rometty, with a claim to fame based on building IBM Global Business Services from on the PwC acquistion became the leading candidate. Ginny is "media friendly" and the diversity factor didn't hurt.
Once Ginny came on board, leadership style changed from long-term to fickle and neurotic. Instead of committing to the hard work of building complex technology (e.g. cloud), any signs of technological challenges became reasons for business strategy changes at the top level. What started as a build decision (IBM SmartCloud) turned into a buy decision (Softlayer), followed by a build decision (IBM Bluemix), and so on.
However, Ginny's biggest failure was her inability to raise capital on Wall Street. IBM's engineers weren't failing at building cloud technology because the engineers were terrible (some were, normal distribution rules still apply) but because cost cutting policies starved engineering teams though attrition and lack of hiring. Staffing a team meant bringing in internal hires w/o the right skill set or taking a gamble on offshore (global) resource. At the same time Google was hiring left and right with comparative ease (as an aside, now they are dealing with the consequences).
Bottom line, Ginny couldn't get the capital to fund internal engineering efforts for the scale of the transformation needed to sustain IBM's success with machine learning (Watson) and cloud.
I place the blame on Wall Street since they made the bet on Ginny and then left her out to dry.
Wall Street? Couldn't raise capital? Maybe IBM could have used some of the $110 billion dollars they spent on buying back shares to, I don't know, build a real business? IBM could potentially have built what is now AWS but what did they do instead? Share buybacks. What do they have to show for that?
IBM has been essentially a financial engineering machine for many years, with their ridiculous 5-year EPS targets and all the rest of it. While I was there they even removed plants, copiers, coffee machines, televisions, pads, pencils, pens and everything else in sight to "save" money.
The "leadership" of IBM drove the company in to the ground, period.
>Maybe IBM could have used some of the $110 billion dollars they spent on buying back shares to, I don't know, build a real business?
You just answered your own question. As I pointed out, leadership was "picked" by Wall Street investors. Share buybacks and financial engineering exist to keep the investors happy.
If they didn't buy back shares and invested in R&D the stock would have gone straight down instead of sideways. As I said the leadership "couldn't get the capital to fund internal engineering efforts for the _scale_ of the transformation needed to sustain IBM's success"
> Google was hiring left and right with comparative ease (as an aside, now they are dealing with the consequences).
In the five years since you left IBM in 2016, GOOG has tripled in value, while IBM has lost 8 percent. I think I know which set of consequences I'd prefer to deal with as an executive, shareholder, or employee.
IBM generates $15b of free cash flow per year. They did not need to raise anything from Wall Street. And the notion that Wall Street would somehow interfere with an internal succession process because they thought a brilliant tech executive was not very polished is laughable.
I was an intern at IBM from 2010-2015, and then an full time engineer for a short while before moving on.
I agree completely that the change in leadership marked an inability to really bring WATSON to market, and some other products too.
While I was an intern, I sat next to ExtremeBlue interns and team members working on Watson. The technology was awesome and they were working like mad. Sleeping under the desk. By 2015 the push had stopped and everyone knew it was a failure to launch.
"I agree completely that the change in leadership marked an inability to really bring WATSON to market, and some other products too."
Watson for all it's academic glory, was kind of vapourware as a product.
It just never really did anything materially useful. They did a lot of hospital trials that fizzled. Way, way over-promised.
A little bit along the lines of ElementAI - it's hard to commercialized AI directly.
Google et. al. do it right by incrementally improving things like translation, voice recognition ... it's making waves within academia for it's quality, but it's something consumers are hardly aware of.
Since IBM is not a consumer company, there's only so much sugar and hype that Watson can provide (winning Jeopardy was a brilliant thing) before the rubber has to hit the rode.
I'm not so sure if it's fair to blame a new leader for that one ... because I'm not sure what a realistic business application for Watson would be. (Maybe there are hard and real products, but they're probably going to be very context specific etc..)
Building great tech is hard enough, then building great products on top of that - just as hard. Doing it within a massive multinational that makes it's money from grifting governments and megacorps into paying for really expensive services probably just is not going to work.
MS, Google, FB ... they have the operational fidelity and product line to leverage AI internally, IBM, not so much.
I don't know what the future of IBM looks like aside from just being another Accenture.
Interesting perspective. They were advertising WATSON everywhere and it did seem like they wanted it to be big. I worked for rational doing... less savoury stuff but it did pay the bills!
I worked in the Watson group during the time you mention - the problem that they made was to attempt to productize the original Jeopardy system. The Jeopardy system was built with the single-minded goal of winning jeopardy. It was much too heavy weight for real-world applications - it required unbelievable computing resources and simply could not be deployed in the cloud as it could only run on Power machines.
Ultimately much of the system had to be rewritten and this took years by which time, it was just another competitor in a crowded market
Sam's EPS target was the singular focus during that time. Executive management squeezed everything for the last penny during that period, IMO setting the path to failure.
Exactly! Palmisano was squeezing every last cent and starving the company of resources for developing the future. Even by 2008, it was clear hat cloud computing was the next big wave and yet IBM built out its own half-baked internal cloud offering before acquiring Softlayer
IBM was the entire IT industry of 1950s, 1960s, and most of 1970s. It is hard to explain IBM's dominance in today's terms. IBM used to be so dominant that the entire financial industry could not operate without their IBM mainframes and today major banks still run on IBM.
The downside (to IBM) was that Wall Street decided "never again" and fought hard to prevent another company of IBM's scale from reappearing.
>The injection is intramuscular, not into the bloodstream.
Recent research (per No Agenda shownotes) showed that unlike traditional vaccines, Moderna mRNA spread through the bloodstream producing and distributing spike protein in the entire body.
What do you mean by “spread through the bloodstream”?
Of course even in an IM injection, some will end up in the blood stream. The better question is the overall amount that stays within the deltoid vs that that travels elsewhere, and how the compares to natural infection. How does the virus deposit itself around the body vs. how does the spike protein via vaccination deposit itself around the body. As well, with the way the vaccine works, once it hits a cell, that cell will express the spike protein to the surface and that’s it, instead of contributing to further viral spread in the same region.
