I think this article is a helpful reminder that we should spend more of our energy working out responses to the practical problems of these issues rather than wasting energy in paralyzing doomerism. That said, I think it's important to recognize that one cause for doomerism is the sheer scale of potential impact. For example, the threat of sea level rise is not merely isolated floods, for which humans have contended with for thousands of years. It's that sea level rise simultaneously threatens a disproportionate share of the world's populations, largest cities, economic engines, and financial systems over a sustained period of time (unlike, say, hurricanes). The problems themselves (flooding, migration, etc.) are not necessarily novel, but it's the possibility that they arrive at a scale that overwhelms our capacity to respond. In the long run, humans will adapt and carry on, but also in the long run we are all dead.
The conversation about AI safety has unfortunately not been well served by talk about percentage risk of human extinction. It sounds hyperbolic and contrived, mostly because the extinction estimates are very fuzzy and rather anxiety-driven. But there is a real underlying problem. Right now, the way AI models are made more clever, and the way they are taught not to behave badly, are both through the efforts of human AI researchers. But it's looking increasingly feasible to make AI models more clever through the efforts of other AI models (this is called Recursive Self Improvement, or RSI). When RSI happens, the rate that AI models will become more clever will accelerate to computer speeds and scales. Unfortunately nobody knows yet how to similarly accelerate the task of teaching AI models not to do bad things: that still requires humans. If one of the AI labs implements RSI soon, the rate that models become more clever may drastically outstrip the rate in which they are taught not to behave badly. This will lead to very clever models behaving quite badly. A sufficiently clever model will be very good at deception and persuasion, so there is plenty of grist for the imagination when thinking about what could go wrong. Precise estimates of what will go wrong will vary, but do not discount the concern because we can't predict exactly what harm will be done and to whom. We don't know the details (it's an unprecedented scenario after all) but it will not be good.
I like the idea of finding the middle points as an epistemic approach to avoiding propaganda/misinformation or whichever. I tend to think mcluhanian historicism might be a good approach to seeing actual direction and depth?
Oren took a lot of words to say what the founder of the "new" right has already told us-both issues are a "hoax". Don outlined the "new" right's AI non-policy yesterday in his UN speech, promising no US leadership on an issue he clearly doesn't remotely understand, other than a rebranding of AI to "special" intelligence. Fittingly, the non-policy apes his longstanding climate non-policy.
The speech should be required listening for all Americans, they should observe his declining condition. He mused about committing war crimes in Iran, bragged about the fake Greenland deal, threatened action soon in Cuba, and outlined his vision of international relations as he explained how we're absconding with Venezuelan oil. "To the victor go the spoils" was the quote, taking US policy back to the days of gunboat diplomacy over a century ago. Too bad few Americans remember the outcome of the pre-war multipolar world Don seeks to recreate.
He also slurred his way through criticism of the "Dumocrats", congratulated himself for ending eight wars, rambled about Russia, Russia, Russia, he was playin all the hits. It's humiliating for our nation to have this aging, angry, imbecile represent us on the world stage as he enters the Fat Elvis stage of his reign.
This is incorrect: "few of the people warning of AI doom seem much inclined to . . . . shut down data centers." In fact, the most alarming aspects of AI to those of us who live in rural areas where the purveyors of AI want to locate these monstrosities are their huge footprint and massive resource dependence -- both of which cause great stress and dislocation to local populations. The fact that you simply brush this aside is telling.
"Climate Change "is a scam. The problem that has been researched is Global Warming. Whatever you think about the research, it was renamed to allow everything to be thrown at the wall. Global means global, no regional drought or heat waves need apply. El Nino cycles are climate change but then it changes back. There can be no global drought. The water cycle is closed. Water can change form or distribution but it never goes away. The globe can warm. It has before. If it is AGW it can be bad news if it coincides with a natural peak but good news if is contracyclical . And why are Canadians against global warming.
Interesting comparison between climate change hysteria and current AI concerns. In passing, Oren mentions the national debt. When that bubble bursts, it will be analagous to a meteor hit. And the ensuring years will be VERY costly to many.
I'm inclined to agree on AI, but I think applying this argument to climate risks undermining the very action we need to be taking. As we know it is better to act earlier
I think this article is a helpful reminder that we should spend more of our energy working out responses to the practical problems of these issues rather than wasting energy in paralyzing doomerism. That said, I think it's important to recognize that one cause for doomerism is the sheer scale of potential impact. For example, the threat of sea level rise is not merely isolated floods, for which humans have contended with for thousands of years. It's that sea level rise simultaneously threatens a disproportionate share of the world's populations, largest cities, economic engines, and financial systems over a sustained period of time (unlike, say, hurricanes). The problems themselves (flooding, migration, etc.) are not necessarily novel, but it's the possibility that they arrive at a scale that overwhelms our capacity to respond. In the long run, humans will adapt and carry on, but also in the long run we are all dead.
And I will say take Americans' fears seriously and try to convince them through rational discourse that their fears may be overblown.
The conversation about AI safety has unfortunately not been well served by talk about percentage risk of human extinction. It sounds hyperbolic and contrived, mostly because the extinction estimates are very fuzzy and rather anxiety-driven. But there is a real underlying problem. Right now, the way AI models are made more clever, and the way they are taught not to behave badly, are both through the efforts of human AI researchers. But it's looking increasingly feasible to make AI models more clever through the efforts of other AI models (this is called Recursive Self Improvement, or RSI). When RSI happens, the rate that AI models will become more clever will accelerate to computer speeds and scales. Unfortunately nobody knows yet how to similarly accelerate the task of teaching AI models not to do bad things: that still requires humans. If one of the AI labs implements RSI soon, the rate that models become more clever may drastically outstrip the rate in which they are taught not to behave badly. This will lead to very clever models behaving quite badly. A sufficiently clever model will be very good at deception and persuasion, so there is plenty of grist for the imagination when thinking about what could go wrong. Precise estimates of what will go wrong will vary, but do not discount the concern because we can't predict exactly what harm will be done and to whom. We don't know the details (it's an unprecedented scenario after all) but it will not be good.
I like the idea of finding the middle points as an epistemic approach to avoiding propaganda/misinformation or whichever. I tend to think mcluhanian historicism might be a good approach to seeing actual direction and depth?
Oren took a lot of words to say what the founder of the "new" right has already told us-both issues are a "hoax". Don outlined the "new" right's AI non-policy yesterday in his UN speech, promising no US leadership on an issue he clearly doesn't remotely understand, other than a rebranding of AI to "special" intelligence. Fittingly, the non-policy apes his longstanding climate non-policy.
The speech should be required listening for all Americans, they should observe his declining condition. He mused about committing war crimes in Iran, bragged about the fake Greenland deal, threatened action soon in Cuba, and outlined his vision of international relations as he explained how we're absconding with Venezuelan oil. "To the victor go the spoils" was the quote, taking US policy back to the days of gunboat diplomacy over a century ago. Too bad few Americans remember the outcome of the pre-war multipolar world Don seeks to recreate.
He also slurred his way through criticism of the "Dumocrats", congratulated himself for ending eight wars, rambled about Russia, Russia, Russia, he was playin all the hits. It's humiliating for our nation to have this aging, angry, imbecile represent us on the world stage as he enters the Fat Elvis stage of his reign.
Good luck America.
This is incorrect: "few of the people warning of AI doom seem much inclined to . . . . shut down data centers." In fact, the most alarming aspects of AI to those of us who live in rural areas where the purveyors of AI want to locate these monstrosities are their huge footprint and massive resource dependence -- both of which cause great stress and dislocation to local populations. The fact that you simply brush this aside is telling.
"Climate Change "is a scam. The problem that has been researched is Global Warming. Whatever you think about the research, it was renamed to allow everything to be thrown at the wall. Global means global, no regional drought or heat waves need apply. El Nino cycles are climate change but then it changes back. There can be no global drought. The water cycle is closed. Water can change form or distribution but it never goes away. The globe can warm. It has before. If it is AGW it can be bad news if it coincides with a natural peak but good news if is contracyclical . And why are Canadians against global warming.
Interesting comparison between climate change hysteria and current AI concerns. In passing, Oren mentions the national debt. When that bubble bursts, it will be analagous to a meteor hit. And the ensuring years will be VERY costly to many.
I'm inclined to agree on AI, but I think applying this argument to climate risks undermining the very action we need to be taking. As we know it is better to act earlier