Agents, besides tool use, also have memory, can plan work towards a goal, and can, through an iterative process (Reflect - Act), validate if they are on the right track.
If an agent takes a Topic A and goes down a rabbit hole all the way to Topic Z, you'll see that it won't be able to incorporate or backtrack back to Topic A without losing a lot of detail from the trek down to Topic Z. It's a serious limitation right now from the application development side of things, but I'm just reiterating what the article pointed out, which is that you need to work with fewer step workflows that isn't as ambitious as covering all things from A-Z.
It's not unprecedented, but I question if it is the right move while the industry is experiencing unprecedented growth.
"The Global Data Center Chip Market size is expected to be worth around USD 57.9 Billion by 2033, from USD 14.3 Billion in 2023, growing at a CAGR of 15.0% during the forecast period from 2024 to 2033."
Thinking about the Rule of 72, that CAGR is astonishing. Where will the power come from to run these data centers? Are we going to see an explosion of solar and wind farms with the express purpose of powering data centers? I say solar and wind because they are easier to get approved and built compared to nuclear or gas/coal.
Once you are working with local LLMs you quickly run into CUDA Out of Memory error. Managing input context input sizes in prompts is really critical. Also keeps cost down.
You can use a lower-end GPU (like the RTX 3060), which also uses less energy.
But you are right, you won't be encountering model API costs when running it locally.
Yes, because somehow humans can't drive without LIDAR in the day or night or rain.
I'm not saying Tesla FSD is any good but the idea that a robot could never drive without just cameras seems to be false given 1.6 billion human drivers that drive with only eyes.
Your eyes and your visual system are much higher performance than any camera and computer currently available for the task of driving. Tesla also has far from the best available hardware, both in cameras and compute.
Still, even with this high performance human sensor suite, people commonly get into accidents in bad weather.
The Waymo approach of using other sensing modalities in order to compensate for the ways in which the cameras/processing aren't as good as a human makes a lot of sense, and in addition, it gives them the ability to exceed human performance using lidar and radar when cameras are having a hard time.
Once we have mass produced lidars and radars, the cost will come down, and not many people are going to care about an extra $1000-$2000 worth of sensors on a car if it significantly improves the safety.
There's no camera that can match the dynamic range of a decent pair of eyes. Some of this is due to active control of the iris at fairly high speed, so you can argue about whether it's the eye or the brain, but the overall result is higher dynamic range than you can get even with normal HDR techniques.
And there's a huge increase (apparently around 36%) in road fatalities during bad weather.
People have much higher safety standards for self-driving cars than they do for human drivers. Just look at how one fatality led to the total abandonment of both the Cruise and the Uber self-driving program.
Did they ever have LIDAR? I completely agree that the removal of radar was a cost-cut too far and a decision made with uneducated hubris (his deeply-flawed "humans only need vision so cars only need that too" thought process). If they ever want to actually compete in this space, I expect we'll see one of these (or both) quietly re-added.
They used it on test mules to create labeled training data for their older monocular depth / Occupancy Network models (which they still use as part of the supervisory policy enforcement and active safety layer, alongside the end-to-end model).
They’ve never had LiDAR in their cars and it would not ever have been practical for them to have done so. Nobody has any mid or long range LiDAR in vehicles as the scale that Tesla sells.
