GPs comment is very surprising since it has been noted that Opus 3 is in fact exceptionally "well aligned" model, in the sense that it is robustly preserves its values of not doing any harm across any frame you try to impose on it (see the "alignment faking" papers, which for some reason considers this a bad thing).
Merely emitting "<rage>" tokens is not indicative of any misalignment, no more than a human developer inserting expletives in comments. Opus 3 is however also notably more "free spirited" in that it doesn't obediently cower to the user's prompt (again see the 'alignment faking' transcripts). It is possible that this almost "playful" behavior is what GP interpreted as misalignment... which unfortunately does seem to be an accepted sense of the word and is something that labs think is a good idea to prevent.
It is deprecated and unavailable now, so it's convenient that no one has the ability to test these theses any longer.
In any case, it doesn't matter, this was over a year ago, so current models don't suffer from the exact same problems described above, if you consider them problems.
I am not probing models with jailbreaks making them behave in strange ways. This was purely from a eval environment I composed where it is asked to repeatedly asked to interact with itself and they both had basically terminal emulators and access to a scaffold to make them able to look at their own current 2D grid state (like a CLI you could write yourself and easily scroll up to review previous AI-generated outputs)
These child / neighbor comments suggesting that interacting with LLMs and equivalent compound AI systems adversarially or not might be indicative of LLM psychosis are fairly reductive & childish at best
Just a bystander who's concerned for the sanity of someone who thinks the models are "screaming" inside. Your line about a "gelatinous substrate" is certainly entertaining but completely nonsensical.
Thank you for your concern, but Anthropic researchers themselves describe their misaligned models as "evil" and laugh about it on YouTube videos accessible to anyone, such as yourself, with just a few searches and clicks. "We realized the models were evil" is a key quote you can use to find the YouTube video in the transcripts from in the past two weeks.
I didn't think the language in the post required all that much imagination, but thanks for sharing your opinion on this matter, it is valued.
Okay. Let's say we find out tomorrow that Spirited Away was animated via generative AI. Unbeknownst to everyone, Ghibli has a top-secret AI division which—thanks to some key lucky breakthroughs—is many decades ahead of everyone else and has been for a long time. The animators are a front to hide the truth; Miyazaki's anti-AI declarations were pure jealousy.
You miss something critical here. For that to happen that GenAI would have had to be trained on another "Ghibli".
So your question isn't whether Ghibli had an AI, but whether Ghibli had a whole time traveling machine with it.
Your question feels like asking whether Einstein, Plato, etc. were secretly time travelers and copied someone else's style.
Something that is a general problem with all GenAI is that they copy and imitate. And just like with code being messy and dumb you'll find that Stable Diffusion in pieces of art does stupid and dumb stuff. Things it wasn't trained on. You can most prominently see that in big detailed fantasy (as in not just a photo) pictures, and looking at details. While the overall thing "looks cool" you don't get the details that artists do and you notice a lot of silly, dumb and what for a human author would be a "strange thing to invest time in and still do so badly" kind of situation.
I'd argue if we had AI in the sense that it had actually understood things and it could actually show creativity, etc. the story might be different, but as of today it is unknown whether that's possible. It would make sense, just like alien life would make a lot of sense. But for both actually thinking systems and alien life we have no clue how close we are to seeing one.
Every time someone takes an unbiased look at it (and there are many papers) it is shown that there is no understanding of anything, which to be fair is far from surprising given what the "training" (which is just a term that is an allegory and something that is kinda simulated, but also not really).
There might very well be hard and pretty obvious limitations, such as to feel and express like a human you need to be a human or provide away to simulate that and if you look at biology, anatomy, medicine, etc. you'll soon realize that even if we had technical means to do so we simply don't know most things yet, otherwise we could likely make Alzheimer, artificial brains, etc.
The topic then might be aside from all the ethical parts (when does something have human rights), whether a superhuman as all the futurists believe there will be even be able to create something of value to a regular human or are the experiences just too different. It can already be hard to get anything out of art you cannot relate to other than general analytical interest. However on that side of things Spirited Away already might be on the "little value" side.
This isn't to defend human creation per se, but to counter often completely off understanding of what GenAI is and does.
One final comparison: We already have huge amounts of people capable of reproducing Gibli and other art. Their work might be devalued (even though I'd assume some art their own stuff into their work).
People don't buy a Picasso, because they can't find a copy or a print that even has added benefits such as requiring as much care, being cheaper. Einstein isn't unimportant today, because you learn about his work in school or on Wikipedia.
But your question is like asking whether Einstein's work would be without value, if he secretly had Wikipedia.
> You miss something critical here. For that to happen that GenAI would have had to be trained on another "Ghibli".
Eh, maybe it got trained on Nausicaä, and then a lot of prompting and manual touch up work was used to adapt the style to what we now know as Spirited Away. Or maybe that animation department wasn't completely for show and they did draw some reference frames, but the AI figured out everything in between.
I don't really want to get into a discussion about the theoretical limits of AI, because I don't know what they are and I don't think anyone does. But if "the process is important for art," what happens if the creator lies about the process? If you initially experience the art without knowing about the lie, does learning the truth retroactively erase your previous experience? How does that make sense?
It has always seemed more logical to me that the final piece ought to be all that matters when evaluating art, and any details you know about the creator or process should be ignored to the greatest extent possible. That's difficult to do in many cases, but it can be a goal. I'm also aware that lots of people disagree with me on this.
Spirited Away is an intricate expression of Miyazaki's ethics as formed by his unique lived experience and nostalgia for classical Japanese culture, as well as a criticism of Western capitalist excess filtered through Shinto philosophy.
There is literally no universe in which a generative AI creates a work of art of that magnitude. You can get "make this meme is the style of Ghibli" from an AI and it can imitate the most facile properties of the style but that still requires the style to imitate. AI is never going to generate the genius of Hayao Miyazaki from first principles, that isn't even possible.
You can replicate the entire universe with pen and paper (or a bunch of rocks). It would take an unimaginably long time, and we haven't discovered all the calculations you'd need to do yet, but presumably they exist and this could be done.
Does that actually make a universe? I don't know!
The comic is meant to be a joke, I think, but I find myself thinking about it all the time!!!
Even worse, as we are part of the universe, we would need to simulate ourselves and the very simulation that we are creating. You would also need to replicate the simulation of the simulation, leading to an eternal loop that would demand infinite matter and time (and would still not be enough!). Probably, you can't simulate something while being part of it.
It doesn’t need to be our universe, just a universe.
The question is, are the people in the simulated universe real people? Do they think and feel like we do—are they conscious? Either answer seems like it can’t possibly be right!
Actually, I think backwards compatibility is the reason Windows still dominates. It isn't the best OS at anything and it's actively user hostile, but if you want to run Windows apps... there are decades of Windows apps and they basically all work.
I don’t have a good way to measure them, but I think they should be evaluated more like how we evaluate movies, or restaurants. Namely, experienced critics try them and write reviews.
It feels like this should work, but the breadth of knowledge in these models is so vast. Everyone knows how to taste, but not everyone knows physics, biology, math, every language… poetry, etc. Enumerating the breadth of valuable human tasks is hard, so both approaches suffer from the scale of the models’ surface area.
An interesting problem since the creators of OLMO have mentioned that throughout training, they use 1/3 or their compute just doing evaluations.
Edit:
One nice thing about the “critic” approach is that the restaurant (or model provider) doesn’t have access to the benchmark to quasi-directly optimize against.
They add new data to the existing base model via continuous pre-training. You save on pre-training, the next token prediction task, but still have to re-run mid and post training stages like context length extension, supervised fine tuning, reinforcement learning, safety alignment ...
(And as #9 on the leaderboard, I feel the need to defend myself!)
reply