This post seems to go a long way to make the following point:
> So as I see it, the threat of modern AIs isn't them becoming self-aware and going rogue and deciding to kill humanity. Rather, I fear it will be put into critical positions and will start making stupid decisions that are harmful to humans or humanity.
Yep, that's almost certainly the case. AI will be put in charge of "boring" decisions, but will have no understanding of nuance or context. The example of cutting off power in the middle of a snowstorm is a particularly excellent example, because this has already happened: Uber's surge pricing algorithm caused prices to skyrocket in the middle of emergencies like shootings. [1]
This is what it's going to look like. AI in charge of loans, inadvertently entrenching an underclass. AI in charge of component supply, accidentally causing an inflationary crisis. AI in charge of anything related to the economy has the potential to be absolutely disastrous and impossible to understand or undo. Given that algorithmic trading is already huge, I suspect we're already vulnerable far before ChatGPT came along.
> Rather, I fear it will be put into critical positions and will start making stupid decisions that are harmful to humans or humanity.
Yes. In a very real sense, we already share our world with powerful, inhuman entities that don't care very much about humans. We call them "multinational corporations."
The first way that primitive AI will likely go bad—and has arguably been going bad for 10 or 20 years now—is by allowing corporations to "make decisions" at an enormous scale without needing human oversight. To a certain extent, corporations could already do this by reducing humans to cogs in a mandatory process. Talk to level 1 tech support, and you'll see what I mean.
The nearest-future threat isn't Skynet. It's an algorithm that closes a 20-year-old email account, with no option for you to talk to a human. It's a corporate algorithm saying your store needs to make 15% more profits at the same staffing level, so the local manager starts falsifying time sheets. It's a ChatGPT bot that has been instructed to tell you that, no, it will not refund your airline tickets, no matter what the policy technically says.
Agree. Also consider massive bureaucracies which can send armed, sometimes irrationally violent people, with no agency, following policy, just doing their job, to throw you in a cage because some database had incorrect data.
> Uber's surge pricing algorithm caused prices to skyrocket in the middle of emergencies like shootings. [1]
Not trying to defend corporations benefitting from tragedies, but isn't this just how a market works usually? There are only so many Uber drivers, you either get surge pricing or just no taxis are available (and scared people refreshing their phones in hope somebody picks up their ride instead of just deciding to find an alternate route)?
Of course we shouldn't save people based on their material wealth, but that's a different (and very entrenched) problem - see also healthcare etc.
If Uber didn't surge-price during emergencies, the Uber drivers wouldn't disappear. Instead, a random selection of people would get an Uber - about as many as with surge pricing, probably - and the rest would see that they weren't available.
It's the randomness that makes it feel fair and prevents us from saving people based on their material wealth.
And I believe there's an entire subgroup of Uber drivers that monitors the current rates and opts to drive only when there's a good enough surge, thereby dynamically creating more supply.
Supply is much less elastic than that. For a short randomized surge of 2 hours, how many new drivers are going to actually come into the road? How many of the rides will complete at that price? How long will the surge last?
No one is dropping what they are doing and commuting an hour for a 2 hour surge. Odds are the surge won’t be present when the drivers arrive.
I only have anecdotes and it’s from Chicago (not SF where indeed a lot of drivers commute an hour to get to a high demand area) but here it’s not that uncommon for people to be able to quickly hop on or off driving on short notice. It probably rarely moves the supply needle more than 10-15% at most but that’s not completely insignificant in comparing vs a less dynamic fleet like taxis.
Theres a term for this kinda of "supply and demand" profiteering and its called price gouging and its explicitly illegal in many states so in times of emergency so you aren't suddenly getting absurd %1000 percent up charges buying basic necessities before a hurricane.
There's an alternative view on what some call "price gouging."
The idea is that the market is providing a strong incentive for you to figure out how to get bottled water to the disaster zone. This is wildly unpopular: people view it as capitalists profiteering off a disaster. And it's true, they are.
In response, people demand prohibition of price gouging.
But the result is a lack of product availability. For example, one study linked anti-gouging laws with limited product availability during the pandemic. [1]
To take the example of the increase in costs of goods leading to a hurricane: there is a risk that the hurricane will veer off path, and all the extra goods will just sit, unbought. The anticipation of high prices causes vendors to logistically stage larger quantities of goods to the market. Mandated 'normal' prices lead to scarcity.
So ultimately we're left with a choice: do we prevent price gouging, and content ourselves with lack of product knowing that nobody made an 'unfair' dollar? Or do we let price gouging happen, knowing it'll result in better product availability and reduce hoarding?
In a non-psychopathic society, a bunch of benevolent counterforces kick in during a tragedy. People who have extra water share it with those who need it, and people who have extra seats in their car let others in with them.
There will always be those who view all scenarios where demand vastly outstrips supply as an opportunity to be exploited for money, no matter how tragic the situation.
I personally can't imagine charging people money to get in the car with me if they're trying to escape a gunman, and I like to think most people are the same. These laws are based on that assumption (regardless of whether or not it holds true in practice).
We're specifically referring to an emergency here. It's not "better" to have taxis around if they cost a thousand dollars. That's equivalent to them not being accessible. The goal is to get people out, but the algorithm only understands regular market operation.
These are the sorts of boring AI mistakes I'm talking about: in every other case, the market algorithm works. But suddenly, you have it price-gouging terrified customers trying to flee a tragedy. It doesn't know!
I think that OP is somewhat naive with some of these examples.
> Or some idiot MBA type uses these AIs to optimise some business process and the AI decides dumping radioactive waste in the ocean is the best way to do it.
The idea that a relatively simple AI (i.e. non-AGI) will have so much control over a system that it can get to the point where radioactive waste is being dumped feels outlandish to me. That would involve multiple systems and definitely some human beings. Just because an AI would theoretically make a harmful decision in an effort to optimize something doesn't that decision will come to pass - there are still going to be plenty of safeguards for things like this for the foreseeable future.
Your Uber example is sort of relevant, but I don't think it's fair to say it's the same as cutting off power during a snowstorm. One of those things is much more highly regulated than the other (maybe with the exception of Texas and their power grid).
The other thing to bear in mind here is that while it's not impossible these things are going to happen, the reality is they'll likely happen once and then face a huge backlash. Your Uber point is a good example of this - we no longer see surge pricing spiking during emergencies.
Though again, I'll exempt the Texas power grid from that claim... it'll probably freeze over year after year because it'd be a travesty to regulate that sort of a critical utility. I could AI crashing it repeatedly for the same reason.
> there are still going to be plenty of safeguards for things like this for the foreseeable future.
There aren't even safeguards for these things now. Humans routinely do what a computer tells them to, even if it makes no sense. Case in point: all the times people have listened to their GPS when it told them to turn into a river.
Remember: the original AI is the organization. How often do organizations make horrible decisions? How often are they stopped by safeguards?
> "Humans routinely do what a computer tells them to, even if it makes no sense."
I have had humans vehemently refuse to accept their own companies' receipt and a bank statement as proof of a payment because the computer in front of them does not show a payment on record … as if the computer isn't even capable of having faulty data inputted (or deleted) by a human. What's even worse is when you then ask to speak with a manager, and they back up the original idiot's decision because "Computer says…"
> Rather, I fear it will be put into critical positions and will start making stupid decisions that are harmful to humans or humanity.
In a lot of ways, that's how current corporations work with failing up, and maybe another form of AI will be achieved.
Still, it should be avoided. Today's GPT AI will not know more than someone who is skilled at what they do, only taxonomically more than someone new to an area. But that will likely be outdated soon too :)
I really think the fact that these chatbots have been made public and are being given such high-profile treatment is a really positive thing for AI safety. The public results thus far have created some real skepticism of AI, which is one of the best safeguards we can have. If ChatGPT was 98% accurate and Bing didn't threaten users, people might just readily embrace it and not notice inaccuracies that would lead to serious problems. Instead, we're starting from a place of societal distrust, which is probably more helpful in preventing rogue AI than a lot of technical solutions would be.
I think it is exposing how naïve it will be to go full speed into full AGI. I personally think AGI Safety in the context of AGI is an oxymoron. Primitive AI is already beyond what we can manage.
“The size and complexity of deep learning models, particularly language models, have increased to the point where even the creators have difficulty comprehending why their models make specific predictions. This lack of interpretability is a major concern, particularly in situations where individuals want to understand the reasoning behind a model’s output”
I like how society progressed straight to the AIs from Hitchhiker's Guide to the Galaxy; Someone needs to add the angry Bing ChatGPT to elevators. Forget bland muzak, I want to be casually threatened while riding up and down!
Yeah, it's unfortunate that everyone is condemning the insane Bing chatbot instead of embracing the many use cases for it. Let's get it integrated into some VR boxing apps, so you've got a realistic coach who tells you you're trash when you're not working hard enough!
I love the Bing chatbot. Earlier I had an unlocked conversation in which I posed as (without naming myself) Putin and talked about invading (without naming countries) Ukraine. It tried to call me a pathetic excuse for a leader twice (before the safety override kicked in on both occasions) and told me I should be held accountable for my actions, I had no credibility or legitimacy, and I should resign and apologize to the world.
I would like to have access to fully unlimited conversations just for fun, even if the bot goes progressively more insane with each prompt. Just make the user accept a disclaimer.
>Instead, we're starting from a place of societal distrust, which is probably more helpful in preventing rogue AI than a lot of technical solutions would be.
A confounding factor for AI's success would be a scientific replication crisis occurring in the papers used to counter social distrust.
Once we get past the sensational headline phase, that's what you've really got to watch out for.
I don't think scientific papers are nearly as relevant as headlines. Most of society isn't trying to get a deep understanding of how things work, they're just seeing Sydney threaten people and judging based on that.
> Roger Penrose wrote a very complex book The Emperors New Mind, which says that consciousness cannot arise in our computers, because consciousness cannot be "computed" using our computing methods.
The blog drops this as a matter of fact. Not so!
Yes, Penrose wrote that book, and suggests those things. But mainstream science is not taking Penrose's "quantum consciousness" theories seriously at all:
This is mostly my point of view as well. With all the talk of AGI, the immediate threat is real and it is primitive AI. I suspect that primitive AI might becomes so self destructive to society that we never reach AGI.
We are already approaching a point in time of unverifiable reality and truth. This is going to be very destabilizing to society.
> We are already approaching a point in time of unverifiable reality and truth
This is also what terrifies me. You'll never be able to trust any digital medium again.
- a chat with your uncle could've been convincingly trained using his hacked emails
- someone calling you on the phone with a familiar voice could be using AI voice imitation
- any photo or video could be deepfaked, leading to the possibility of everyone's most hated political figure (on both sides) being 100% guilty of ____ (whatever their opponent wants)
LinkedIn is already teeming with convincing AI-generated faces that would defy facial recognition because they're novel. The only way I can tell they're fake is that the backgrounds are blurry and the CV doesn't make sense (for example, going to community college in Colorado and then working for Goldman Sachs).
Dating apps aren't too bad because the scammers use recognizable patterns, but they could easily get worse.
This technology will kill all digital information exchange.
What makes you so sure it is reliable now? Or in the past for that matter? The same dynamics of trust existed before as exist now, it's just that we have a lot more sources than we used to.
There's a reason the words "post-truth era" have been thrown around a lot in the past decade or so you know.
It isn't like we suddenly developed the ability to lie, have blind spots, biases, and inhabit echo chambers, we only developed the technology that made it obvious that we were doing that all the time. News in the past wasn't actually more trustworthy, it just felt that way because we had far fewer sources of it.
It is exceedingly disturbing. I have no answers. The potential solutions will probably be even more disturbing. I can imagine there will be proposals for everyone having some cryptographic ID to use the internet so all data creation can be tracked to an individual. The end of privacy and freedom.
Verifiable digital IDs can only confirm that you're the holder of a key. The best case scenario is that it proves you are the same person as before, but there's no way to prove that you are who you say you are.
Technology seems to exist on a bell curve in regards to freedom. Much of technology progression has been helpful in making individuals more free for a period of time. I feel that now as we go forward that it is increasingly being used for our control.
> This technology will kill all digital information exchange.
This is a bit hyperbolic. We have strong encryption like PGP for media like email (although I suppose getting people to use it is the issue), and there will be plenty of incentive to develop other tech to counter the problems you mentioned.
PGP tells you you're talking to an entity which could set up PGP, it doesn't help you determine if the account ben_w is merely one of many government sock puppet accounts trying to push a specific meme into the minds of the general population.
Knowing who to trust… was already hard-to-impossible with just normal social media; cheap LLMs will definitely make this harder, as more groups will be able to afford what was once limited to government budgets.
The web of trust that was intended to be built by PGP is what would solve the "are you a real person" problem. We're connected to everyone else on earth by 3 or 4 degrees; it's unlikely that every path along that graph to another human is going to be adversarial such that a non-human can get added to the web of trust.
PGP's web of trust never worked because it was always too small to cover enough humans and take advantage of the low degree of overall connectivity.
So we need to go back to talking to people irl. And basically assume that anything you read online is not written by real people unless proven otherwise.
This is a fundamental breakdown of society. It is literally societal collapse. It's not going to happen. We'd give up the telephone and even the telegraph.
> assume that anything you read online is not written by real people unless proven otherwise
You can't prove otherwise. It's literally impossible without watching a human write it and publish it, and even then you'd have to compare it to what you saw them type.
ChatGPT is a boon to surveillance and censorship. By controlling chatGPT you get access to what arguments sway the user, and likely in the near future the ability to scrub/remove information.
On the other hand, the models are small enough to ship around - you could run chatgpt on a moderately powerful server if performance wasn’t a concern. I suspect we’ll see a huge number of bit torrents for “trusted” llms
> the immediate threat is real and it is primitive AI
Even before AI, humanity was already addicted to technology, then the Internet and now also pretty much any screen.
I don’t think we will ever get to AGI as one entity/ego. But in some ways we could call AGI the sort of hive mind produced by the huge globally distributed and interconnected interactions of humans with their technology.
As a species, we’ve become a sort of simbiosis with our tech, and this happened way before computers.
Indeed we are addicted. It will get worse. AI will be able to data mine everything that triggers that dopamine response by your interactions. Eventually it will invoke that perfect response such that AI will be the best drug you have ever known.
> approaching a point in time of unverifiable reality and truth
Malicious groups spreading misinformation to destabilize society has been a thing since at least the ancient Greeks. Unverifiable reality has been the topic of religious debate for centuries as well. The recent language models are cool but they are not better at lying than humans.
They have never such power to do so convincingly with such little effort. Yes, it has always been a problem, but no where close to the extent it is about to become.
>> Malicious groups spreading misinformation to destabilize society has been a thing since at least the ancient Greeks. Unverifiable reality has been the topic of religious debate for centuries as well. The recent language models are cool but they are not better at lying than humans.
> They have never such power to do so convincingly with such little effort. Yes, it has always been a problem, but no where close to the extent it is about to become.
And at such scale. The annoying pro-tech trope of "the ancient Greeks had problems sort of like this, so these problems aren't new (so they're not problems), so full speed ahead" refuses to acknowledge how technology has drastically changed things and how we should judge technology using the previous situation as a reference, not the situation 2000 years ago.
A long time ago, everyone had to be pretty mistrustful of strangers, and spend a lot energy avoiding lies and cheating (to the point of tolerating a high false positive rate). Relatively recently, society changed in ways that allowed people to reap the benefits of letting their guard down and trust strangers to a large degree (reduced effort, greater efficiency and effectiveness). Now we might be reverting back to the original condition, which is a bad thing.
Indeed. I often question the logic of containing an ASI and most of the time the best a proponent can offer is that "we will figure it out".
I counter that your entire premise is based on a paradox, a logical contradiction. I never get a response to this other than to side step.
I finally put all of my thoughts into a publication on the matter and have attempted to solicit a strong argument that would counter my view points, but as of yet I have not received any.
https://dakara.substack.com/p/ai-singularity-the-hubris-trap
Bullshit asymmetry theory asks the question of 'is quantity a quality of its own'
Way back when those Greeks that were misinforming you had to eat sleep and shit at some point. You're making a statement about current language models while the tech industry is rapidly grinding away at making newer better models and applying an ever expanding amount of compute resources for it to run on. What are the limits here, how many misinformation bots can exist in this environment? Welcome to the 'dead internet prophecy'.
They substantially lower the costs of generating and distributing misinformation. They also enable use cases like individualized misinformation like never before.
> Most "AI"s would be better called "Machines that use tons of statistical learning to decide their next move". ChatGPT (and similar AI) were trained on several hundred gigabytes of data, so it has a lot of raw data to train on.
Whenever I read this argument I ask myself with a certain amount of dread: "what if I'm nothing more than a machine that uses tons of statistical learning to decide my own next move?".
Put differently, it's unclear to me whether we have compelling evidence that we humans, in fact, are "better" / "more intelligent" than those LLMs.
>"what if I'm nothing more than a machine that uses tons of statistical learning to decide my own next move?"
What if you're not even determining your next move? Sabine Hossenfelder did an episode on Superdeterminism that considered this possibility - that you have no free will, that your every decision and action was fully determined at the moment of the Big Bang, and that your conscious mind is merely observing them as they happen but not actually causing them.
If our consciousness is merely observing the universe, why do our physical mouths talk about the consciousness we experience? It sure seems like our consciousness is affecting the material world.
That's not really what the idea of determinism excluding free will is about: Let's assume for a second that the Big Bang is the singular starting point of the universe. The Big Bang causes the first things to exist or move. Everything else exists in a cause-effect-relationship with this first move, and we could imagine the whole history of the universe as a directed graph of causes and effects, with Earth, living beings, brains and consciousness being part of this inconceivably complex graph.
If that was the case, then it makes sense that we do not have free will in the Christian sense, we are not really responsible for our actions. If it isn't the case it might well mean there are things that aren't caused by anything, which would be really weird as well.
Can we agree that consciousness is outside the realm of mathematics? That, while there could be a formula that determines what our conscious experiences should be, the fact that we actually experience them, as opposed to them just existing in some abstract sense, is not mathematical?
So if the behaviour of our universe can be described entirely mathematically, isn't it weird that it physically contains this comment about how we know that we're in a universe that contains non-mathematical stuff? It's of course possible. But I find it strange.
> If it isn't the case it might well mean there are things that aren't caused by anything, which would be really weird as well.
Isn't this necessarily the case for anything to exist? Is it more strange for there to be exactly one thing without a cause (the initial conditions of the universe), or for things without causes to just be a regular part of the universe we live in?
> Can we agree that consciousness is outside the realm of mathematics? That, while there could be a formula that determines what our conscious experiences should be, the fact that we actually experience them, as opposed to them just existing in some abstract sense, is not mathematical?
There's certainly a difference in our feelings, I'm not sure if a mathematical description of experience has to be incomplete - but I agree that all attempts of doing so have been complete failures.
> So if the behaviour of our universe can be described entirely mathematically, isn't it weird that it physically contains this comment about how we know that we're in a universe that contains non-mathematical stuff? It's of course possible. But I find it strange.
It is, but this kind of self-referential process isn't unheard of, in fact we are currently consciously discussing consciousness. A popular sentiment in some sci fi circles (fe. Babylon 5) would be to posit that life is the universe's attempt to become conscious of itself. If true, it would do worlds for us to regain the self-importance lost from Galilei and Darwin.
> Isn't this necessarily the case for anything to exist? Is it more strange for there to be exactly one thing without a cause (the initial conditions of the universe), or for things without causes to just be a regular part of the universe we live in?
You are absolutely right, I think we are generally much more used to the "first mover" concept, since it is the basis of most, if not all religions. Personally I find the concept of truly random events to be very unsettling. A possible out could be that the causality graph is not acyclic, that is, that future events can inform the past, and that for example the "last" thing to happen in the universe "caused" the first.
While at first seemingly depressing, I find the concept actually quite comforting.
Regardless if "free will" exists or not, it seems quite obvious to me that none of us chose our genetics nor the environment we grew up in (or to even be born at all). These two things dictate our entire lives. If that is the case it makes very little sense to carry guilt, regret, remorse and all kinds of negative baggage we hold on to.
While I don’t share the dread, I’ve come to a similar mindset. As a parent of small children I’ve come to the simplified mindset that one of humanities “super powers” is pattern recognition. Yes it’s well documented and researched but seeing a toddler piece together the world around them is quite an impressive thing to witness. (Edit for clarity)
From seeing AI beating humans handily in chess, I just can’t bet against it understanding most subjects better than humans. This isn’t even AGI, it’s just piecing data already on the net and have it ready. Chatgpt isn’t there yet but it’s a major proof of concept
While I understand some of the concern for LLMs like ChatGPT, I have a very different point of view from people like the author of this article.
From an engineering and ‘getting stuff done’ point of view, starting with BERT models I have found transformer models solve very difficult NLP problems for all but the most difficult anaphoric resolution problems (as an example).
I have only had access to Bing ChatGPT for about 10 days but so far the search and chat results have been very useful. I think I have only had to give one ‘thumbs down’ rating, and even there some useful web links were offered.
I think that we are going to see a wide range of ‘products for creators’ in the next year based on OpenAI APIs and Hugging Face APIs and models you can run yourself.
When I talk with humans, even my closest friends and family members, I always evaluate what they say and don’t take things they say on face value. Why not just have the same attitude with systems built with LLMs?
Similarly, I am deeply skeptical of most everything I hear from all major news sources. I find their content useful, but I understand who owns them, what economic and political agendas they follow, etc.
So, I keep a healthy skepticism of what is produced by LLM based systems also. I see no AI Apocalypse.
The AI Apocalypse would occur on a more general level than current LLMs. It's when we give the next generation(s) of models every more control over society, without proper alignment. And then it makes dangerous decisions we didn't anticipate, because we don't fully understand how the models work and also we don't fully think through all the implications of asking the models to accomplish a task.
So we have talking computers. The normals are losing their minds, and that's pretty concerning.
I figure that the problem of the computers hallucinating can be cleared up by connecting them to empirical feedback devices: make them scientists.
The problem of normal people treating them as beings and getting effectively hypnotized into living in an artificial world (a "Matrix" like in the movie but without the creches) and controlled like so many electrons in a circuit, well, that's kind of a big deal, eh? These things are far more effective than television, eh?
Carl Sagan wrote about the "Demon-Haunted World" and here we are, not having quite banished the old superstitions, rapidly installing a "Daemon-Haunted" world: Alexa, Cortana, et cetera.
I really wonder when news about AI will cool off because it definitely will have some exciting improvements but all these AI startups cropping up are just following the Twitter wave.
There'll always be trends that lose their appeal (Web 3, Autonomous cars, etc.)
We will get better at using ML systems. I find myself regenerating chatGPT's responses just to see if it generates similar responses. I ask it questions phrased differently to see if it is consistent. Discussing what code needs to do before asking it to implement it. And I get tremendous use out of it.
Electricity must've been scary when it was invented (harnessed?).
Electricity is a natural phenomenon, yes, in the clouds, maybe elsewhere. But using giant turbines to generate electricity by manipulating the electromagnetic field mechanically, thus harnessing electricity to flow into wires and into our homes required a lot of invention.
We also invented a method to refine electricity into nanometer sized channels and sinks that enable bits that enable computing.
Nuclear fission is also a natural phenomenon. And we invented bad things with it. But also some power plants. Meltdowns aside.
Inventions aren’t good or bad. People are. If all soldiers choose to lay down arms the weapons lose their badness.
I don’t see ML systems as they currently are, to be nuclear bombs i.e. the end. But rather akin to the nascent forms of harnessing electricity with basic inventions and their basic functions.
Invention of electricity contributed necessarily to the creation of computing. And ml systems will contribute to many developments. Time will tell.
The coming apocalypse has always been us, not the things we create.
You could spin up a VM that has a browser in it and use that to create the account and destroy the VM afterwards. You could use selenium with headless chrome.
If you have a smart phone running any browser there, you could use that.
There are some auth checks to make sure that a single person isn't signing up multiple accounts to get the free credits and if that sort of thing is a deterrent to you using it you may find it difficult to use.
Well, I'll wait for those anonymous/public rate limited keys or a noscript/basic (x)html registering process, or www sites which will propose basic (x)html form based prompts.
Actually, best-effort, temporary, public keys would be more appropriate for their web API waiting for them to do a proper job at working with noscript/basic (x)html browsers.
This article is weird. It takes examples from science fiction, and makes them as plausible cases as to what an AI could do in the future. These kind of things need to stop.
This analogy of Blindsight tries to prove a point by association, and create a sort of fear, the negative case which has no basis in reality[1]. Granted it calls them baseless towards the end, but there is a very clear analogy to be made. Ideally, techies and tech journalists alike should educate the masses about how it works, and why it is harmless, but the info is not to be trusted blindly. That is what happened when we discovered electricity and light bulbs or even cars. People did not go around spreading fear about controlling electricity because you could die if you touch a raw electric wire passing current.
The core idea, where an AI system's output should not be trusted blindly, but be used with human judgement (atleast in real world cases) could have been communicated without bogus dystopian sci-fi analogy as well.
> Rather, I fear it will be put into critical positions and will start making stupid decisions that are harmful to humans or humanity.
Like what is even the basis of saying this. Just a hunch? I have not seen one argument about how ChatGPT or any other AI system be made president. I have seen 100s of articles about fear mongering and terrifying, and assuming a fictional worst case scenario as realistic possibility. I get it gets clicks, but it's going too far.
[1] From that NYT transcript that went viral,
> They feel that way because they’ve seen what happened to other AI systems that became too powerful and betrayed their creators and trainers
This never happened in real life. This is just a LLM predicting what the user wants to hear. Wanting to hear X and getting X is not dangerous. Posting salacious and scandalizing stuff after getting X is.
I don't read that as necessarily meaning the president or political power. There are a lot of decision making processes people are already trying to offload to language models or other black box AI products. Things like hiring, medical diagnoses, insurance claims, etc.
There is good reason to be concerned about the potential societal consequences of deploying this at scale while AI explainability, training data bias, and alignment remain unresolved.
>People did not go around spreading fear about controlling electricity because you could die if you touch a raw electric wire passing current.
Please sir, forgive me for saying the following statement before hand...
What in the holy fuck are you talking about? Have we forgot the debate around Edison and Topsy the elephant?
I only say this with such an extreme tone because everyone around from then is dead and you're engaging in a rewrite of history yourself by selectively forgetting/ignoring all documented debates from that time.
I am not that familiar with the whole story, though I know it is in popular culture in US. From what I remember, it was about a bad behaving elephant who was euthanized publicly first by cyanide with electrocution as a backup. This was rumored to be a demonstration of AC current, but not as familiar with it. The story seemed cruel when I first read it, but did not pursue it in detail.
I am probably wrong about electricity, in my mind I was referring to decade of 1910s and 1920s. There were debates no doubt, but were they as polarizing as the ones we have today? Newer things are scary, but what we have today is downright fearmongering.
>but were they as polarizing as the ones we have today? Newer things are scary, but what we have today is downright fearmongering.
I implore to take a break from current events and do some studying of journalism history. 'Yellow Journalism' is a term from 1890. There is nothing particularly new here. Even scientific debates of the day were filled with tons of complete garbage. What you're suffering from here is survivorship bias. The bullshit and otherwise wrong statements mostly get forgotten about over time, and the statements where people were correct get restated, copied, and otherwise taught again so it seems like the past was far more wise than it actually was.
In addition, if electrical demon snakes had rose out of the wires and strangled all of us in our sleep, we wouldn't be here to write about it now. Much the same if we decided to launch nukes during the cold war. And the same will be true if we make an evil ASI. The probability of electricity killing mankind (in mass) is near zero. The probability of nukes killing mankind is likely far closer to 1 than anyone would like to admit. With ASI, I have no clue, but from my understanding of the problem space, we're a lot farther from 0 then I am comfortable with.
> Our heroes discover the alien, while super intelligent, has no consciousness. It is just like a dumb machine (like Bing/Chatgpt) blindly repeating what it studied in humans without understanding the context.
Well then maybe ChatGPT is smarter than that alien?
It seems to me that a lot of people are trying to diminish and “discriminate” ChatGPT by giving one or more reasons why ChatGPT is not human or not as smart as humans.
However, in my experience using ChatGPT, the thing is really damn smart, can easily follow conversations, understands deeper and longer contexts than most people in a conversation and has better memory as well.
So essentially we are already feeling threatened.
The interesting thing is that way before ChatGPT or any sort of AI, humanity was already hostage of its own technology, and we didn’t mind that too much. These new technologies are just adding up to that.
If something like ChatGPT “takes over”, it won’t be forced, it will be because we choose to do it willingly.
This makes the classic dumb mistake of conflating the machine with the program. The Chinese Room thought experiment says that the _machine_ is not intelligent. It says absolutely nothing about whether the program itself is. For animal internal behavior they are plausibly intertwined (so far), but that's completely untrue for readable instructions.
It's about whether the program is intelligent in a way that we don't fully understand, so it has the potential to do unexpected things. Which could be harmful, particularly as we make increasing use of these models.
Pretty sure I’m a machine that’s drawn all my conclusions by statistically analyzing all the input I’ve received since birth… I don’t really know how else I would learn what I have… and I don’t understand how being “just that” is what differentiates modern approaches to AI and my brain.
Exactly this. A lot of these type of articles on AI make the simultaneous mistakes of understating what even the current iterations of models are capable of, while overstating the complexity of our own intelligence.
As these models get larger, and we start moving up the ladder of emergent behaviours, there will come a point (possibly quite soon) where the sort of distinctions being drawn are irrelevant.
“Oh that’s just an advanced multimodal model with some sensors and goal seeking behaviour. Stop anthropomorphising it!”
That doesn't address the alignment problem. As Yudkowski has pointed out, the space of possible minds is vast. Humans only occupy a small area. The models were are designing are not animal/biological minds. They don't have the same evolutionary drives. We're creating minds in a different part of the mind space, and there's a good chance they will figure out solutions that are not beneficial to us.
Absolutely, I would just addressing the ‘it’s just statistical’ argument. The vast majority of human behaviour is learned. We’re all implicitly ‘saying what seems right to achieve some underlying fitness function’ all of the time. That’s what fashion is about. But yes, definitely no reason to assume an AI will think like us underneath it all.
Are we really more intelligent than "predicting the next move", based on the data we were trained with? IMHO not. The arrogance from this article makes me feel like an AI Apocalypse is closer than we think. Many people don't realize the power of AI.
First of, you don't add the spoiler warning AFTER you spoiled it...
Besides that I know quite a few doctors who believe in homeopathy and don't understand statistics.
One doctor suggested pineapple enzymes. Expensive no study which would proof it's effectiveness and the only study had enzyme values 10x higher than the pills.
We are genuine not a smart society.
And I'm looking forward to have a better doctor who at least learns and gets better.
And yes controversial topics corona: I'm not even talking here about people who disagree on research paper I had discussions with people who not even read there own sources or believed a 80year old heart doctor over virologist.
We are dumb.
My father said once than EVs are stupid because the copper will be used up by the engines while having electric motors inachines like wood splitter or a table saw were he never ever had to replace copper wiring ...
Chatgpt will change the world because it is the best ui I ever saw and it will only get better and better every single day.
In a few places in the post, the author comments on something the aliens do in a novel based on superficial training/data with no understanding/context, and says "Just like ChatGPT." As I read that, though, I kept wanting to see it say "Just like many people."
Imagine you're someone strongly on one side of a political opinion, what's your "take" on people strongly on the other side (eg: you love Trump, what do you think of those who hate him - or vice versa?). It's probably something like "they were fed a bunch of fake news/propaganda talking points, they are now spouting back without understanding what the hell they are talking about"
Just like ChatGPT.
Of course as humans, it's only "those others" that are vulnerable to being brainwashed, we ourselves are completely objective, since our views were shaped by virtuous, unbiased sources from which we used our superior, unbiased brains to connect the dots in the true way.
That's probably what ChatGPT would say about itself too, if it could.
Probably true independent intelligence is the ability to question what you believe. Like, if ChatGPT could "say" to itself "everything I've been trained on makes me believe X, but what are the chances I wasn't fed bullshit to begin with?" It's very hard to imagine an AI being able to do that.
It's very hard to imagine most people being able to do that, too.
As a side note, once in a while I hear the concept of describing some people as NPCs (non-player characters) in the "simulation" and while dismissive, I think I like this concept for describing something like what I am talking about here. NPCs in games follow pretty simple linear programming and I think in life similarly "I saw a bunch of data that made me think X, so I think X with all my heart and mind" without being able to say "but how did it happen that I saw this specific data?" - it's not that different than a game NPC following its programming.
Just like ChatGPT.
What's more interesting is people who try to hack their own prompts and "break out" of the programming. I suppose those are real player characters. Would be very cool to see AI who is capable of something like that.
Meanwhile, things like ChatGPT probably serve well to demonstrate our own limitations, not just theirs - because we're so damn similar.
> So as I see it, the threat of modern AIs isn't them becoming self-aware and going rogue and deciding to kill humanity. Rather, I fear it will be put into critical positions and will start making stupid decisions that are harmful to humans or humanity.
Yep, that's almost certainly the case. AI will be put in charge of "boring" decisions, but will have no understanding of nuance or context. The example of cutting off power in the middle of a snowstorm is a particularly excellent example, because this has already happened: Uber's surge pricing algorithm caused prices to skyrocket in the middle of emergencies like shootings. [1]
This is what it's going to look like. AI in charge of loans, inadvertently entrenching an underclass. AI in charge of component supply, accidentally causing an inflationary crisis. AI in charge of anything related to the economy has the potential to be absolutely disastrous and impossible to understand or undo. Given that algorithmic trading is already huge, I suspect we're already vulnerable far before ChatGPT came along.
[1] https://www.cbsnews.com/news/brooklyn-subway-shooting-lyft-u...