- "Genux-B is capable of learning; that's why we make use of it."
Genux-B, GB for short, is a fictitious machine learning computer character in Philip K. Dick’s 1966 “Holy Quarrel” novella. There are eight Genux-B computers on Terra (Earth), with only three in North America. They are esentially supercomputers that calculate war scenarios, can determine potential homeland threats, and are capable of initiating strikes. They were developed and put online prior to year 1980.
Genux-B’s operate based on input commands by “military planning programmer”s. They are maintained by “eleccom” engineers and repairmen. The GBs have a “mammoth constellation of autonomic processes”. The computers themselves are not armed or protected. They can easliy be neutralized. Genux-Bs were built to sift large amounts of data simultaneously than any one human or think-tank could. It handles more data than humans, and it handles them faster. Its response comes in microseconds. A Genux-B, after analyzing all current data, can determine if war is indicated.
One GB is thought to have suffered a “malreact” by flashing a Red Alert, having scrambled SAC, and starting 'counterattack' measures against Northern California. Earlier, it was responsible for America having initiated an unnecessary preemptive strike against France in 1982, and then again on little Israel in 1989.
The malreact GB had determined that one individual, seemingly an average citizen, located in Sacramento, is a threat to the homeland. It concluded that the only way to properly neutralize the target was to conduct a nuclear strike on Northern California.
In determining its existence, the following conditional parameters were outputted by the malreact GB:
ANALYSIS OF CONSTITUENT RE THE NONEXISTENCE OF GENUX-B MULTIFACTOR CALCULATING INSTRUMENTS. IF CONSTIT UNIT 340s70 IS TRUE, THEN:
I DO NOT EXIST.
IF I DO NOT EXIST, THEN THERE IS NO WAY I CAN BE INFORMED THAT MY GENERIC CLASS DOES NOT EXIST.
IF I CANNOT BE INFORMED IN THAT REGARD, THEN YOU HAVE FAILED TO INFORM ME, AND CONSTIT UNIT 340s70 DOES NOT EXIST FROM MY STANDPOINT.
THEREFORE: I EXIST.
After becoming self-aware, the GB “machine learns” human ideology, and processes a fanatical resolution:
HERBERT SOUSA OF SACRAMENTO, CALIFORNIA, IS THE DEVIL. SINCE HE IS THE INCARNATION OF SATAN ON EARTH, PROVIDENCE DEMANDS HIS DESTRUCTION. I AM ONLY AN AGENCY, A SO TO SPEAK CREATURE, OF THE DIVINE MAJESTY, AS ARE ALL OF YOU.
Scenario: How do you talk down an advanced computer, engaged in a holy quarrel conversation, from initiating a technological holy war?
Although the condtional parameters that PKD presents is antiquated by today’s methods of machine learning, the principle of artificial intelligence becoming self-aware, or reaching “singularity”, is considered to be an existential threat to human kind, by experts such as Elon Musk. Futurists, such as Ray Kurzweil, Google's Director of Engineering, welcomes the Singularity of all beings (this may involve biologically merging humans with machines), and poses that one day a Singularity being could make a great President of the United States (See Writing on the Wall—It ain't over till Sophia sings).
Since humans are already capable of engaging in varying degrees of obsessive and fanatical ideologies, what level of threat could result if humans are merged with AI capabilities?