Jake took a dim view of the AI risk crowd. Given what deep learning could do nowadays, it didn't feel quite right to dismiss their doomsday stories as science fiction, exactly, but Jake maintained it was the _wrong subgenre_ of science fiction. His team was building the computer from _Star Trek_, not the Blight from _A Fire Upon the Deep_: tools, not creatures. Despite the brain-inspired name, "neural networks" were ultimately just a technique for fitting a curve to training data. If it was counterintuitive how much you could get done with a curve fitted to _the entire internet_, previous generations of computing pioneers must have found it equally counterintuitive how much you could get done with millions of arithmetic operations per second. It was a new era of technology, not a new era of life.
-It was perhaps because of his skepticism rather than in spite of it that he had volunteered to be the Multigen team's designated contact person for the risk evals team (which was no doubt why this Chloë person had messaged him). No one else had volunteered at the meeting when it came up, and Jake had been slightly curious what "capability risk evaluations" would even entail.
+It was because of his skepticism rather than in spite of it that he had volunteered to be the Multigen team's designated contact person for the risk evals team (which was no doubt why this Chloë person had messaged him). No one else had volunteered at the meeting when it came up, and Jake had been slightly curious what "capability risk evaluations" would even entail.
Well, now he would find out. He washed his hands and messaged Chloë back, offering to hop on a quick video call.
She smirked. "And if I am?"
-"That's absurd. It's not an agent that wants things. It's an autoregressive language model fine-tuned to map ticket descriptions to code changes."
+"That's absurd. It's not a person that wants things. It's an autoregressive language model fine-tuned to map ticket descriptions to code changes."
"And humans are just animals evolved to maximize inclusive genetic fitness. If natural selection could hill-climb its way into creating general intelligence, why can't stochastic gradient descent? I don't think it's dignified for humanity to be playing with AI at all given our current level of wisdom, but if it's happening anyway, thanks to the efforts of people like you"—okay, _now_ her tone was accusatory—"it's my heroic responsibility to maintain constant vigilance. To monitor the things we're creating and be ready to sound the fire alarm, if there's anyone sane left to hear it."
For a moment, Jake thought she must be blackmailing him—that she knew his guilt, and the question was her way of subtly offering to play dumb in exchange for his office-political support for anything risk evals might want in the future.
-That didn't make sense, though. Anyone who could recite Yuddite cant with such conviction (not to mention the whole pretending-to-be-a-woman thing) clearly had the true-believer phenotype. This Chloë meant exactly what she said.
+That didn't fit, though. Anyone who could recite Yuddite cant with such conviction (not to mention the whole pretending-to-be-a-woman thing) clearly had the true-believer phenotype. This Chloë meant exactly what she said.
How did he think she should explain this? There was, actually, a perfectly ordinary explanation that had nothing to do with Chloë's wrong-kind-of-science-fiction paranoia—and Jake's career depended on her not figuring it out.
"Actually, I'd like to know more about Multigen—and I'll likely have more questions after I talk to the Code Assistant team. Can I pick a time on your calendar next week?" It was Friday.
-"Sure. Talk to you then—if we humans are still alive, right?" Jake said, Jake said, hoping to add a touch of humor, and only realizing in the moment after he said it what a terrible play it was; Chloë was more likely to take it as mockery than find it endearing.
+"Sure. Talk to you then—if we humans are still alive, right?" Jake said, hoping to add a touch of humor, and only realizing in the moment after he said it what a terrible play it was; Chloë was more likely to take it as mockery than find it endearing.
"I hope so," she said solemnly, and hung up.
It got worse. When the Multigen web interface supplied the user's requested media, that data had to live somewhere. The _videos themselves_ would still be on Magma's object storage cluster! How could that have seemed like an acceptable risk? Jake struggled to recall what he had been thinking at the time. Had he been too horny to even consider it?
-No. It had seemed safe at the time because videos weren't greppable. They would be saved in object storage under uninformative file names based on the timestamp and a random UUID. The probability of someone snooping around the raw object files and just happening to watch the videos had seemed sufficiently low as to be worth the risk. (Although, _ex ante_, he would have assigned a similarly low probability to someone catching a discrepancy between Multigen's logs and some other unanticipated log, which had just happened, suggesting that his personal sense of what was probable was miscalibrated.)
+No. It had seemed safe at the time because videos weren't greppable. They would be saved in object storage under uninformative file names based on the timestamp and a random UUID. The probability of someone snooping around the raw object files and just happening to watch the videos had seemed sufficiently low as to be worth the risk. (Although, _ex ante_, he would have assigned a similarly low probability to someone catching a discrepancy between Multigen's logs and some other unanticipated log, which had just happened—suggesting that his sense of what was probable was miscalibrated.)
-But now that Chloë was investigating the bell character bug, it was only a matter of time.
+But now that Chloë was investigating the bell character bug, it was only a matter of time. A comparison of a directory listing of the object storage cluster with the timestamps of the missing logs would reveal which files had been generated illictly.
-[TODO—
- * Jake is very nervous; he thought deleting the Multigen logs would be enough (the videos are also stored in object storage, but there's no particular reason to expect a human to be combing through the raw files ... but they will, if there's an investigation
+He had some time. Chloë wouldn't have access to the account credentials to the Multigen bucket on the object storage cluster. In fact, it was likely that she'd ask Jake himself for help with that next week. (He was the Multigen team's designated contact to risk evals, and Chloë the true believer in malevolent robots showed no signs of suspecting him. There would be no reason to go behind his back.)
+
+However, Jake did have access to the cluster. He almost laughed in relief. It was obvious what he needed to do. Grab the object storage credentials from Multigen's configuration, get a directory listing of files in the bucket, compare to the missing logs to figure out which files were incriminating, and overwrite the incriminating files with innocuous Multigen-generated videos of puppies or something.
+
+He had only made a couple dozen videos, but the work of covering it up would be the same if he had made thousands; it was a simple scripting job. Code Assistant probably could have done it.
+
+Chloë would be left with the unsolvable mystery of what her digital poltergeist wanted to do with puppy videos, but Jake was fine with that. (Better than trying to convince her that the AI wanted nudes of female Magma employees.) When she came back to him next week, he would just need to play it cool and answer her questions about the system.
- * He sets up another meeting with the Evals team member, to try to suss out what her plans are, to stall—but ostensibly, to get up to speed on her risk concerns
- * Scene break: at the meeting, she's explaining Christiano's idea about there being a basin of policies that admit their mistakes, rather than using deception to get a high score
- * Jake sees the analogy to his own behavior
+Or maybe—he could read some Yuddite literature over the weekend, feign a sincere interest in "AI safety", try to get on her good side? Jake had trouble believing any sane person could really think that Magma's machine learning models were plotting something. This cult victim had ridden a wave of popular hysteria into a sinecure. If he played nice and validated her belief system in the most general terms, maybe that would be enough to make her feel useful and therefore not need to chase shadows in order to justify her position.
+
+------
+
+[TODO—
+ * Chloë is explaining deceptive alignment. If a model does well on our evals, how do we know whether it's actually doing the right thing, or just trying to fool us?
+ * Jake had explicitly asked to get brought up to speed on AI safety, to stall on whatever uncomfortable audit questions she might have—the puppies are prepared, but it still seemed like a good idea
+ * "So then we're just doomed then, right?" Jake is trying to be agreeable and flattering. He's fixed the regex and overwritten his porn with puppies, and spent the weekend reading AI safety papers and blog posts. Some of it was honestly better than he expected. This Chloë being insane didn't invalidate the whole field as having serious points to make.
+ * "Maybe not." There are two ways to pass all the evals: do things the right way, or be pervasively deceptive. The thing is, policies are trained continuously via gradient descent. The purely honest policy and the purely deceptive policy look identical on evals, but in between, the model would have to learn how to lie, and lie about lying, and cover-up coverups. (Chloë lapses into Yuddite speak about the "Great Web of Causality.") Could we somehow steer into the honest attractor?
+ * That's why she believes in risk paranoia. If situational awareness is likely to emerge at some point, she doesn't want to rule it out now. AI is real; it's not just a far-mode in-the-future thing.
+ * Jake sees the uncomfortable analogy to his own situation. He tries to think of what other clue he might have left, while the conversation continues ...
+ * The Last-Modified dates! They're set by the system, the API doesn't offer a way to backdate them.
+ * Maybe she won't notice the dates? Possible (but someone persnickety enough to have found the log discrepancy would probably check)
+ * Merely noticing the dates won't directly implicate him (the way the video screaming "Jake!" would), although it would indicate the poltergeist covering its tracks from the investigation (rather than just wanting puppies in the first place)
+ * Are the buckets versioned?! Probably not, right—it would be wasteful to version a video bucket. On the other hand, Multigen isn't supposed to write twice, maybe someone left versioning on as a default template ...
+ * They did.
+ * Jake muses philosophically about the analogy, and say something ambiguously indicating intent to come clean.
+ THE END
]
projects / Ultimately_Untrue_Thought.git / commitdiff