This is the weird part, I saw the same comments in other threads. People keep saying how everyone yearns for local LLMs… but other than hardcore enthusiasts it just sounds like a bad investment? Like it’s a smaller market than gaming GPUs. And by the time anyone runs them locally, you’ll have bigger/better models and GPUs coming out, so you won’t even be able to make use of them. Maybe the whole “indoctrinate users to be a part of Intel ecosystem, so when they go work for big companies they would vouch for it” would have merit… if others weren’t innovating and making their products better (like NVIDIA).
Intel sold their GPUs at negative margin which is part of why the stock fell off a cliff. If they could double the vram they could raise the price into the green even selling thousands, likely closer to 100k, would be far better than what they're doing now. The problem is Intel is run by incompetent people who guard their market segments as tribal fiefs instead of solving for the customer.
that's a dumb management "cart before the horse" problem. I understand a few bugs in the driver but they really should have gotten the driver working decently well before production. Would have even given them more time tweaking the GPU. This is exactly why Intel is failing and will continue to fail with that type of management
Intel management is just brain dead. They could have sold the cards for mining when there was a massive GPU shortage and called it the developer edition but no. It's hard to develop a driver for games when you have no silicon.
I think you're massively underestimating the development cost, of the number of people who would actually purchase a higher vram card at a higher price.
You'd need hundreds of thousands of units to really make much of a difference.
Well, IIUC it's a bit more "having more than 12GB of RAM and raising the price will let it run bigger LLMs on consumer hardware and that'll drive premium-ness / market share / revenue, without subsidizing the price"
I don't know where this idea is coming from, although it's all over these threads.
For context, I write a local LLM inference engine and have 0 idea why this would shift anyone's purchase intent. The models big enough to need more than 12GB VRAM are also slow enough on consumer GPUs that they'd be absurd to run. Like less than 2 tkns/s. And I have 64 GB of M2 Max VRAM and a 24 GB 3090ti.
This makes sense in some ways technologically, but just having a "centralized compute box" seems like a lot more complexity than many/most would want in their homes.
I mean, everything could have been already working that way for a lot of years right? One big shared compute box in your house and everything else is a dumb screen? But few people roll that way, even nerds, so I don't see that becoming a thing for offloaded AI compute.
I also think that the future of consumer AI is going to be models trained/refined on your own data and habits, not just a box in your basement running stock ollama models. So I have some latency/bandwidth/storage/privacy questions when it comes to wirelessly and transparently offloading it to a magic AI box that sits next to my wireless router or w/e, versus running those same tasks on-device. To say nothing of consumer appetite for AI stuff that only works (or only works best) when you're on your home network.
It most likely won't be a separate device. It'll get integrated into something like Apple TV or a HomePod that has an actual function and will be plugged in and networked all the time anyway. The LLM stuff would be just a bonus.
Both are currently used as the hub for HomeKit devices. Making the ATV into a "magic AI Box" won't need anything else except "just" upgrading the CPU from A-series to M-series. Actually the A18 Pro would be enough, it's already used for local inference on the iPhone 16 Pro.
Enthusiast/Prosumer/etc. market is generally still usually highly in most markers even if the revenue is limited. e.g. if hobbyists/students/developers start using Intel GPUs in a few years the enterprise market might become much less averse to buying Intel's datacenter chips.
