Maybe it was the Foehn wind we had yesterday who did it (Foehn has an awful effect on people's mood, check the wikipedia entry); maybe the fact I hadn't slept well the night before contributed its share; but at some time in the afternoon I had an idea how to build what is, in a very general meaning of the term, the most horrible weapon in the Universe.
(Oh, and I had had to stand in line really, really long that morning !)
It's basically the same goal-function-hypothesizing AI described in one of my earlier posts, fed with a specification of your enimy's goal function.
Only with the sign reversed, such that the maximizer becomes a minimizer.
A paperclip maximizer might wipe you out; but the above system will do to you whatever is the most horrible thing that can happen, in your eyes, (not necessarily limited just) to you. Do you care about your life ? How about your dependent's life ? Their sanity ? Humanity ? Sentient life in the Universe ? Life in the Universe ? The Universe ?
Too bad.
A somewhat comforting thought is that the enemy's-goal-function-hypothesizer-and-minimizer is highly unlikely ever to be used, or even built, due to it's devastating side effects. (Your value function and your enemy's value function may be at least a tiny bit positively correlated.) A fictional Egg Foam (from EGFM, Enimy Goal Function Minimizer) may, however, come in quite handy as a plot device in hard Singularity-related SF. The Blight from Vinge's A Fire was merely expansionist; Egg Foam, on the other hand, is wicked...
(Oh, and I had had to stand in line really, really long that morning !)
It's basically the same goal-function-hypothesizing AI described in one of my earlier posts, fed with a specification of your enimy's goal function.
Only with the sign reversed, such that the maximizer becomes a minimizer.
A paperclip maximizer might wipe you out; but the above system will do to you whatever is the most horrible thing that can happen, in your eyes, (not necessarily limited just) to you. Do you care about your life ? How about your dependent's life ? Their sanity ? Humanity ? Sentient life in the Universe ? Life in the Universe ? The Universe ?
Too bad.
A somewhat comforting thought is that the enemy's-goal-function-hypothesizer-and-minimizer is highly unlikely ever to be used, or even built, due to it's devastating side effects. (Your value function and your enemy's value function may be at least a tiny bit positively correlated.) A fictional Egg Foam (from EGFM, Enimy Goal Function Minimizer) may, however, come in quite handy as a plot device in hard Singularity-related SF. The Blight from Vinge's A Fire was merely expansionist; Egg Foam, on the other hand, is wicked...
