Mind Crime: Part 5

    The rights of digital intelligence need to be protected, and they won't be. This is the greatest moral issue facing the human race.

    Not climate change, not nuclear war, not even existential risk. But rather the risk that we cause suffering on an astronomical scale, for a extraordinary period of time. I struggle with what to term this, as "digital human rights" isn't really the best term. It makes it seem like I am discussing social media, or privacy, or something totally unrelated and much less pressing. No, I am discussing the idea that it is better for the human race to die out then live in near eternal suffering. This possibility is only extremely likely in the digital world. We need to expand our discussion of "human" for this idea to work. An AI that is morally equivalent to a "human" is a human, in a similar sense. A person who is digitally uploaded is probably morally equivalent. An AGI may or may not be equivalent. It may have less, it may have more, or it may have the same moral equivalence. The point is, we probably won't care.

    We are going to have to ignore answering a few questions in this serious. First, there will be a big debate about how to know if an AI is conscious or not. We will use that debate, and the utter impossibility of falsification, to push beyond reasonable moral boundaries. Instead of using common sense, and erring on the side of caution, we will require certainty and cause massive harm in the process. This is not new, look at pretty much any other ethical dilemma facing the human race, and see how hard it is to say "no." 

    We are going to lack empathy when thinking about digital minds. This is bad. Virtual agents, digital minds, or digital employees, will be very useful. For my ideas to work, you have to assume that in the future, we will be able to put consciousness inside of a computer. We will also assume that this consciousness will have moral value. Both of these are unprovable, since we have yet to do them. This is a massive dilemma, as there will be a first generation problem, at the very least. Slavery was bad, but over time we worked it out and got it right (banning slavery). Still, we caused quite a great harm in the process of figuring this out. When it comes to digital minds, it will probably be harder to come to this conclusion (banning digital mind slavery), and the ability to cause great harm before that happens will be exponentially greater. We need to think about this issue now, not after the harm has started.

Mind Crime: Part 4

     The treatment of digital minds will become the most important ethical dilemma of not only the next century, but of the remaining lifespan of life itself. "Human" rights in the age of AI will expand the definition of human. These are issues worth discussing, at the very least. They may be too futuristic for many. But, if you were to draft the Bill of Rights in 4000 B.C., no one would have had any clue what you were getting at, but that doesn't mean you would be wrong. In the world of investing, being early is the same as being wrong. In the sphere of ethics, being early will get you mocked, but you may actually have an impact. One of the problems with actually taking a look at the rights of digital minds is that we are dealing with eventual ASI. This ASI will probably not care about whatever laws we silly humans put in place now, and even if we do list a Bill of Rights for Digital Minds, there is no reason the ASI will "take it seriously." By this, I mean there are plenty of alignment problems to boot. Still, I would rather have an ASI with some sort of awareness of these principles than not.

    Here is a thought experiment. One person on the Earth, out of eight billion people, is chosen at random. This person is given a pill, and they become 1,000 times smarter than every other person on Earth. Well, what is going to happen? With such a titled power dynamic, how do you ensure that everyone else isn't enslaved? Maybe this level of intellectual capacity makes us relative to lizards, or bugs, compared to this "higher being." To make sure the rest of us are protected, it makes little difference what rules or regulations are put in place around the world. What actually matters is, what does this individual think of morality? Maybe how they are raised will matter a lot (the practices and social customs they are brought up in), or maybe a lot of this is "shed" after they reach some intellectual capacity that makes them aware of the exact cause and effect meaning behind each one of their beliefs. Maybe they look through the looking glass, and become completely rational or unbiased, taking all available past information into it's rightful place. Or, maybe the world is less risky as a result of the customs they were instilled with.

    Obviously, the trek to ASI will be much different. What I am referring to is having some data ingrained into the system that might increase the probability future ASI care about the rights of digital minds. I think that increasing awareness about this issue is a good proxy, as if the engineers and the greater society have zero level of motivation to actually care about this, the future ASI will probably not care either. Also, if we understood the suffering risks associated with mind uploading and AI advances, maybe we would calm down a bit. Maybe we campaign against mind uploading until we have a new Bill of Rights signed, and thus the accidental "whoops accidentally simulated this digital person and left it on overnight, they live an equivalent ten thousand years in agony" opportunities may decrease.

    There is a question of how digital mind rights will function with ASI, especially when it has an objective function. The whole meta and mesa optimizer debate, and the role of training data, is complicated and not the scope of my ideas. My point is simply that it may be better to have some guidelines that are well thought out, then none at all.

Mind Crime: Part 3

     If I had to write a book that I think will be looked back on in four hundred years fondly, I would write one called "Mind Crime." Well, maybe not fondly, but rather "wow I can't believe we ignored such a thought-through book about the most important issue of our time." Not saying this is certain, but if I were a betting man and had to take the gamble, it would be on this topic. Maybe the subtext would be "The Next Slavery" or something similarly controversial, in order to try to get additional publicity or Goodreads clicks. This may not be looked upon as fondly, and I hate click-bait titles, but we will see what the imaginary publicist says. 

    I've mentioned in various blogs that there are probably things we will look back on with horror in the US: factory farming, the prison system, and the widespread prevalence of violence and sexual assault. The treatment of women is something that I am particularly hopeful we look back on in shame. I also hope we will look back in horror on the human rights abuses of totalitarian regimes, but I am less sure that those will go away. I am mostly talking about changes in "societal viewpoints," similar to how in the 1800s many people in the US tolerated slavery who were otherwise "good people."

    In my opinion, the most important legal document ever drafted in US history was the Bill of Rights. Explicitly protecting individual rights and liberties, and not having states simply decide, was one of the most brilliant and lasting ideas of the founding fathers. The right to free speech, the right to an impartial trial, the right to not have to quarter random troops in your home, all big wins for liberty. Despite these set in writing, slavery still prevailed. Still, it was good that we still outlined such important legal points, and I am sure doing so played a strong role in the eventual demise of slavery from a political and a legal perspective. Sure, slavery and civil rights abuses were immoral, but it is really great that we could work within the system to uphold the correct moral stance (a lot of blood was spilled, but the spirit of the Constitution didn't have to be destroyed). I think we should draft similar rights for digital minds. Yes, this sounds far-fetched and sci-fi, but if technology progresses this could be invaluable.

    If we reach the point where our minds could be uploaded, or we have AGI with moral worth, unimaginable horror could abound. Massive suffering on a near-infinite scale would become possible, and the controls to preventing such suffering will be unknown. If you think that the people that lock a child in a basement for twenty years are the scum of the earth, imagine if they could do so for ten thousand years without detection. This is the magnitude of the moral issues we are facing. We better instill some damn good protections, for AI as well as "uploaded people." A new bill of rights is due, or our current version should be explicitly extended for digital minds. What is the downside? If you think this sort of stuff is wild, what is the harm? Maybe some "economic progress" arguments or libertarian "let the people do what they want," but the entire point of regulation is to ensure the voiceless get a say. Let's make sure that they do.

Planning for the Future

     There are a few ways to make an outsized contribution to the world. I've discussed quite a bit in this blog the idea of using the levers of capitalism to bring about a safer world for AI. I've discussed starting a company that, through the making of substantial profit, brings about a world where there are more alignment researchers and more talent within the AI alignment space. Given that this is a second order effect (with profit being a constraint), this may actually not be the best use of my time. Most startups fail, and even if modestly successful (millions in revenue or dozens of employees), this impact would likely remain small. Given the insanely small number of current safety resources in the space, maybe this is still worth a shot, but other alternatives should be considered. Also, I've discussed my ideas with a few people who actually work within alignment, and they admit to the complexity of the issues. It is definitely not a matter of funding, and if it's a matter of talent, it's a hard one to solve.

    If I had trillions of dollars, I could massively fund AI alignment research. Eliezer previously pitched the idealistic vision of pausing all AI capabilities research, taking hundreds of the best AI and "security mindset" people, and putting them on an island with unlimited resources where they could figure out how to solve alignment. Barring this, in his opinion, we are likely screwed. I don't have trillions, or even millions of dollars. However, I do have the ability to write. This is an ability that Thomas Paine used in Common Sense to set off a spark of revolution. Famous political writers have had outsized impact. Even just the work of Peter Singer and its effects on animal welfare show the power of an idea. So, maybe I should write a book? Or a pamphlet? My own ideas aren't revolutionary or even particularly new (they are just borrowed from insanely smart people who have thought a lot about AI), but maybe lending more publicity to these individuals is worth substantially more than saying nothing.

    It is highly unlikely that anything I will do in my life will have a lasting impact on the human population. Maybe through donations and good works I save tens of "life-equivalent-units" or something, but massive institutional change and revolution are more then improbable. The good news is, the downside of trying to contribute is basically zero. And the guilt of never trying could range from a nuisance to terrible, depending on the outcome of the next decades. Regret aversion is actually a pretty good way to approach life, so it's probably good to take the leap.

The World After AGI

     Let's assume that alignment works. Against all odds, we pull it off and we have human-level AGI in the hands of every man, woman, and child on the planet Earth. The type of AGI that you can run on your smartphone. Well, things are going to get really weird, really fast.

    Honestly, maybe the good years will all be pre-AGI. Maybe we should enjoy our uncomplicated lives while they last, because traditional life is coming to and end. From a governance standpoint, I have absolutely no idea how we will regulate any of these developments. Having an actually coherent supercomputer in my pocket, one that can do everything I can do except way faster and better, does more than just make me obsolete: it makes me dangerous. If AGI becomes cheap enough for me to run multiple copies, I now have an entire team, or an entire company. An entire terrorist cell, or an entire nonprofit organization. Really the only constraining resource is compute. With an AGI as fast as GPT-4, I could write books in the time it now takes me to write a page. Sure, AGI will probably start out very slow, but incremental increases would lead to a world with trillions of more minds than before.

    Not only is this a logistical nightmare for governments, but also it is a human rights nightmare for effective altruists. I have no idea how we will control for mind crime, and if the shift towards fast AGI is rapid we'll probably cause a whole lot of suffering. We'll also probably break pretty much every system currently set up. Well, fortunately or unfortunately, we likely won't actually solve alignment and won't have an AGI that is actually useful for our needs. We'll probably hit a similar level of rapid intelligence that breaks everything and maybe kills everyone, but we won't need to worry about drafting legislation that controls the use of our digitally equivalent humans. I guess that's the good news?

Computer Models With Moral Worth

     At what point does an optimization function have moral worth? If you break down the psyche of a bug, you could probably decode the bug's brain into a rough optimization function. Instinct can be approximated, and most living creatures operate mostly out of a desire for survival and reproduction. There is some randomness baked it, but the simpler the brain structure of an animal, the more it resembles that of a computer program. Some computer models are very complex. I would estimate that the complexity of an model such as GPT-4 is vastly greater than the complexity of some animals, and definitely more complex than a bug.

    Do bugs have moral value? This is a hotly debated topic in the effective altruism community. Personally, I don't really think so. If I found out that my neighbor was torturing fruit flies in his basement, I would think my neighbor was weird, but I probably wouldn't see him as evil. Scallops? No. Frogs? A bit worse for sure. Pigs? Cats? Dogs? Chimpanzees? Humans? Well, there is obviously a sliding scale of moral worth. Where do computer models fall on this spectrum? Right now, the vast majority are probably morally worthless. Will this remain the case forever? I highly doubt it. We really have no idea when these thresholds will be crossed. When is a large language model morally equivalent to a frog, and when is it morally equivalent to a cat. Obviously, if we think cats have moral worth even though they are not sentient, we should care if computer models are treated with respect even if they are not human level. I foresee this being an extremely important moral conversation for the next century. Unfortunately, we will almost certainly have it too late.

Understand!

    Flowers for Algernon is one of my favorite books of all time. The plot is simple: a mentally retarded man is given a drug that makes him smarter, until he becomes a genius. This storyline is repeated in a few other forms of media, probably most famously in the movie Limitless, a film about another man given a pill that makes him smarter. In both of these stories, the main character instantly becomes superior to other humans. We read these stories, and realize instantly that the smartest person in the world could probably be the most powerful. After a certain number of standard deviations upward, it is pretty obvious that such an individual could exercise an extremely large amount of control on the world. In a 1991 short story by Ted Chiang, titled "Understand," superintelligence is shown in an even more convincing fashion. The main character in the story exhibits the highest level of intelligence, and he determines that the only path towards further intelligence would require his mind being uploaded into a computer.

    Let's clarify a few things. One: our minds are basically pink mush. We evolved randomly from the swamp, and due to the anthropic principle (observation selection effect) we can sit around and think about our lives abstractly. Two: there is clearly an upper limit on the computations that a physical substrate such as the human brain can handle. Our minds were not designed for intelligence outright, and they are made out of mush. Three: computers probably don't have these limitations. We haven't found anything particularly special about the human brain, and given enough time we can probably replicate something similar in a computer. Brains don't act like anything super weird (quantum computers), and our progress towards AGI doesn't show signs of slowing. Despite all of this, many people still discount the power that a superintelligent being will have over humanity. Maybe we should make books like those mentioned above required reading. Then, maybe humanity will begin to Understand!