As a European, I was shocked by how slow trains are in the US.

It definitely doesn't encourage people to use them instead of flying.

IIRC isn't the symmetry between Q and K also broken by the direction of the softmax? I mean, row vs column-wise application yields different interpretation.

Yes but in practice, if you compute K=X.wk, Q=X.wq and then K.tQ you make three matrice multiplication. Wouldn't be faster to compute W=wk.twq beforhand and then just X.W.tX which will be just two matrices multiplication ? Is there something I am missing ?

Most models have a per-head dimension much smaller than the input dimension, so it's faster to multiply by the small wk and wk individually than to multiply by the large matrix W. Also, if you use rotary positional embeddings, the RoPE matrices need to be sandwiched in the middle and they're different for every token, so you could no longer premultiply just once.

Oh yes! That's probably more important, in fact.

Well, I think that this is also answer to your question about the intuition.

If the assymetry of K and Q stems from the direction of the softmax application, it must also be the reason for the names of the matrices :)

And if you think about it, it makes sense that for each Key, weights to all of the Queries sum to 1 and not vice versa.

So this is my only intuition for the K and Q names.

(It may or may not be similar to the whole "db lookup thing"... I just don't use that one.)

For frequently used things, like cruise control, just a few months needed.

Does it also have motion sensors to turn on the interior lamp? That's what I've found the most frustrating on some of their models.

You want to turn on the lamp? Try some waving gestures...

Also, I don't know whether this is a feature, a bug, or just a lack of skill on my side but I couldn't get the lamp to turn on with the engine turned off. VERY impractical.

You have to touch the light itself, which is more or less fine, but sometime doesn’t work if you’re wearing gloves…

Interesting that the release comes a day after Google's new models [1]. Seems a bit like strategical timing :) Maybe they waited until some of the competitors release something so that they can upstage his release with theirs?

____

[1] Which, btw, I think deserve better sentiment. On benchmarks, the new Gemini Pro seems to be better than GPT-4o. It's just not so hyped...

I think the sentiment here is not fully objective, there are nice improvements in benchmarks (and even more so when accounting for the price): https://imgur.com/a/K3tVPEw

Also, this model shouldn't be compared to the CoT o1, I think. That is something different (also in price and speed).

> What's important is consistently moving. That means getting up from the desk every hour for a 5 minute walk or some stretching. Switching between standing and sitting at a desk is a first step in the right direction but really we need to be moving.

Yeah, that's it. At least when it comes to overall health (I'm not sure about back issues).

I did a little bit of research on this theme myself and it seems that the key is to interrupt periods of prolonged sitting with any movement (even walking helps) for any period of time (even 1 minute helps). The more intensive/longer the better, obviously.

See also notes from my web-research at http://mnicky.wikidot.com/healthy-programmer (it's based on the book that got me started on this topic, but e.g. in Section 3 there's some additional reading about benefits of making pauses when sitting for a longer time and similar research).

Btw, using standing desk for prolonged time can be harmful as well, the truth is even more than sitting :) So yes, combine standing and sitting, but stand only for a shorter time (after 30min it starts to have negative effects).

At my team, we recently established a rule that during Friday you can work on anything work-related you want (e.g. it doesn't have to be something from the actual sprint etc). Similar to the old Google "20% for your projects" kind of a thing.

I expected that lots of useful or interesting things will be made during that time, which is true. But for me, the most beneficial effect is that my enjoyment of my work (which previously wasn't bad - I really liked my work) increased a lot.

So this can be another way how to boost your job enjoyment.

For me they have nothing (success! :)

I use Firefox + FB container + uBlock Origin + Privacy Badger and recently started to use CanvasBlocker as well. I have Firefox configured to delete all cookies on closing (except for few sites to avoid the need to enter the 2FA code every time I log in).

I've also set Firefox Enhanced Tracking Protection to "Custom mode" with "cross-site and social media trackers" blocked [1] and to use block list "level 2".

I also have the "Do Not Track" option switched on.

I don't have a proper smartphone (never owned one), just KaiOS-powered dumb-phone on which I use Facebook mobile (i.e. their web site) all the time.

Also no Pi-Hole or similar stuff.

I use a throwaway email account for Facebook.

______

[1] Just now I've found out that there seems to be a new option, to disable "all cookies from unvisited sites", which I'm going to try as it looks even better.

> I use a throwaway email account for Facebook

I feel like this might be key, I use a random burner number and this seems to confuse the tracking.

Another one:

> Police officers and Clearview’s investors predict that its app will eventually be available to the public. Mr. Ton-That said he was reluctant. “There’s always going to be a community of bad people who will misuse it,” he said.

(...)

> Asked about the implications of bringing such a power into the world, Mr. Ton-That seemed taken aback. “I have to think about that,” he said. “Our belief is that this is the best use of the technology.”

And then read this:

> Because the police upload photos of people they’re trying to identify, Clearview possesses a growing database of individuals who have attracted attention from law enforcement. The company also has the ability to manipulate the results that the police see. After the company realized I was asking officers to run my photo through the app, my face was flagged by Clearview’s systems and for a while showed no matches. When asked about this, Mr. Ton-That laughed and called it a “software bug.”

Wow. Just wow.