notes-singularity2

the singularity might be horrible, not wonderful.

if humans consciousness is simulatable then what's to say someone or something won't simulate you and torture you? keeping you "alive" for millions of years in this fashion?

there are many awful possibilities that are not as crude as this. imagine that you woke up one day to find the world gone and only a changing geometric pattern before your eyes. there is a noise that sounds sort of like a cross between electrical feedback and a weird drum machine. almost the same noise repeats over and over again, annoyingly. you shout out but you can't hear yourself. but sometimes the noise or the geometric pattern changes somewhat; it seems to be correlated to your thougts or actions, but in some complicated, subtle way. this continues for millions of years, unchangingly -- you are thoroughy insane or at least miserable after just the first 1000 years or so. why would anyone want to simulate you this way? perhaps they are measuring your reactions to simple stimuli in a methodic fashion, just as we might expose a circuit to single frequencies one at a time in order to build a linear model of its response. This scenario is not that much different to what we do today to non-human primates in science experiments regarding vision. Or, perhaps your role is more like a single neuron in a brain; perhaps whoever it is is simulating a million of you at once, and linking them together, in such a way to produce a composite entity that itself is conscious -- but you'll never know.