It had been obvious since DALL-E back in 'twenty-one—earlier if you were paying attention—that generative AI would reach this level of customization and realism before too long. Eventually, it was just a matter of the right couple dozen people rolling up their sleeves—and Magma's willingness to pony up the compute—to make it work. But _it worked_. His awe at Multigen's sheer power would have been humbling, if not for the awareness of his own role in bringing it into being.
-Of course, this particular video wouldn't be showcased in the team's next publication. Technically, Magma employees were not supposed to use their cutting-edge generative AI system to make custom pornography of their coworkers. Technically (what was probably a lesser offense) Magma employees were not supposed to be viewing such content during work hours. Technically—what should have been a greater offense—Magma employees were not supposed to covertly introduce a bug into the generative AI service codebase specifically in order to make it possible to create such content without leaving a log.
+Of course, this particular video wouldn't be showcased in the team's next publication. Technically, Magma employees were not supposed to use their cutting-edge generative AI system to make custom pornography of their coworkers. Technically (what was probably a lesser offense) Magma employees were not supposed to be viewing pornography during work hours. Technically—what should have been a greater offense—Magma employees were not supposed to covertly introduce a bug into the generative AI service codebase specifically in order to make it possible to create custom pornography of their coworkers without leaving a log.
<span style="float: right; margin: 0.4pc;">
<a href="/images/fake_deeply-jake.png"><img src="/images/fake_deeply-jake-smaller.png" width="450"></a><br/>
The profile text indicated that Chloë was on the newly formed capability risk evaluations team. Jake groaned. _Yuddites._ Fears of artificial intelligence destroying humanity had been trending recently. In response, Magma had commissioned a team with the purpose to monitor and audit the company's AI projects for the emergence of unforeseen and potentially dangerous capabilities, although the exact scope of the new team's power was unclear and probably subject to the outcome of future intra-company political battles.
-Jake took a dim view of the AI risk crowd. Given what deep learning could do nowadays, it didn't feel quite right to dismiss their doomsday stories as science fiction, exactly, but Jake maintained it was the _wrong subgenre_ of science fiction. His team was building the computer from _Star Trek_, not [AM](https://en.wikipedia.org/wiki/I_Have_No_Mouth,_and_I_Must_Scream) or [the Blight](https://en.wikipedia.org/wiki/A_Fire_Upon_the_Deep): tools, not creatures. Despite the brain-inspired name, "neural networks" were ultimately just a technique for fitting a curve to training data. If it was counterintuitive how much you could get done with a curve fitted to _the entire internet_, previous generations of computing pioneers must have found it equally counterintuitive how much you could get done with millions of arithmetic operations per second. It was a new era of technology, not a new era of life.
+Jake took a dim view of the AI risk crowd. Given what deep learning could do nowadays, it didn't feel quite right to dismiss their doomsday stories as science fiction, exactly, but Jake maintained that it was the _wrong subgenre_ of science fiction. His team was building the computer from _Star Trek_, not [AM](https://en.wikipedia.org/wiki/I_Have_No_Mouth,_and_I_Must_Scream) or [the Blight](https://en.wikipedia.org/wiki/A_Fire_Upon_the_Deep): tools, not creatures. Despite the brain-inspired name, "neural networks" were ultimately just a technique for fitting a curve to training data. If it was counterintuitive how much you could get done with a curve fitted to _the entire internet_, previous generations of computing pioneers must have found it equally counterintuitive how much you could get done with millions of arithmetic operations per second. It was a new era of technology, not a new era of life.
It was because of his skepticism rather than in spite of it that he had volunteered to be the Multigen team's designated contact person for the risk evals team (which was no doubt why this Chloë person had messaged him). No one else had volunteered at the meeting when it came up, and Jake had been slightly curious what "capability risk evaluations" would even entail.
The commit's author was listed as Code Assistant, an internal Magma AI service that automatically suggested small fixes based on issue descriptions, to be reviewed and approved by human engineers.
-That part was mostly true. Code Assistant had created the logging change. Jake had added in the bell character backdoor and attributed it to Code Assistant (`GIT_COMMITTER_NAME="Code Assistant" GIT_COMMITTER_EMAIL=codeassistant@magma.com git commit --amend; git push --force-with-lease`), gambling that whichever of his coworkers got around to reviewing Code Assistant's most recent change submissions would rubber-stamp them without noticing the bug. (Who reads regexes that carefully, really?) If they did notice, they would blame Code Assistant. (Language models hallucinate weird things sometimes. Who knows what it was "thinking"?)
+That part was mostly true. Code Assistant had created the logging change. Jake had written the bell character backdoor and [melded it onto Code Assistant's change](https://www.atlassian.com/git/tutorials/rewriting-history#git-commit--amend), gambling that whichever of his coworkers got around to reviewing Code Assistant's most recent change submissions would rubber-stamp them without noticing the bug. (Who reads regexes that carefully, really?) If they did notice, they would blame Code Assistant. (Language models hallucinate weird things sometimes. Who knows what it was "thinking"?)
Thus, by carefully prefixing his requests with the bell character, Jake could make all the custom videos he wanted, with no need to worry about explaining himself if someone happened to read the logs. It was the perfect crime—not a crime, really. A precaution.
Jake's blood flash-froze. "Wh—what?"
-Chloë pasted two more links into the chat, this time to Magma's log viewer. "Requests go through a reverse proxy before hitting the Multigen service itself. Comparing the two, there are dozens of requests logged by the reverse proxy that don't show up in Multigen's logs—starting just minutes after the bug was deployed. The reverse proxy logs include the client IP, which is inside Magma's VPN, of course"—Multigen wasn't yet a public-facing product—"but don't include the request data or user auth, so I don't know what the client was doing specifically—which is apparently just what they, or it, wanted."
+Chloë pasted two more links into the chat, this time to Magma's log viewer. "Requests go through a reverse proxy before hitting the Multigen service itself. Comparing the two, there are dozens of requests logged by the reverse proxy that don't show up in Multigen's logs—starting just minutes after the bug was deployed. The reverse proxy logs include the client IP, which is inside Magma's VPN, of course"—Multigen wasn't yet a public-facing product—"but don't include the request data or user auth, so I don't know what the client was doing specifically. Which is apparently just what they, or it, wanted."
Jake silently and glumly reviewed the logs. The timestamps were consistent with when he had been requesting videos. He remembered that after one of his coworkers (Elaine, as it turned out) had approved the doctored Code Assistant change, he had eagerly waited for the build automation to deploy the faulty commit so that he could try it out as soon as possible.
That didn't fit, though. Anyone who could recite Yuddite cant with such conviction (not to mention the whole pretending-to-be-a-woman thing) clearly had the true-believer phenotype. This Chloë meant exactly what she said.
-How did he think she should explain this? There was, actually, a perfectly ordinary explanation that had nothing to do with Chloë's wrong-kind-of-science-fiction paranoia—and Jake's career depended on her not figuring it out.
+How did he think she should explain this? There was a perfectly ordinary explanation that had nothing to do with Chloë's wrong-kind-of-science-fiction paranoia—and Jake's career depended on her not figuring it out.
-"I don't know," he said. It suddenly dawned on him that staying in this conversation was not a smart play. "You know, I actually have another meeting to get to," he lied. "I'll fix that regex today. I don't suppose you need anything else from me—"
+"I don't know," he said. He suddenly remembered that staying in this conversation was probably not in his interests. "You know, I actually have another meeting to get to," he lied. "I'll fix that regex today. I don't suppose you need anything else from me—"
"Actually, I'd like to know more about Multigen—and I'll likely have more questions after I talk to the Code Assistant team. Can I pick a time on your calendar next week?" It was Friday.
"I hope so," she said solemnly, and hung up.
-_Shit!_ How could he have been so foolish? It had been a specialist's blindness. He worked on Multigen. He knew that Multigen logged requests, and that people on his team occasionally had reason to search those logs. He didn't want anyone knowing what he was asking Multigen to do. So he had arranged for his requests to not appear in Multigen's logs, thinking that was enough.
+_Shit!_ How could he have been so foolish? It had been a specialist's blindness. He worked on Multigen. He knew that Multigen logged requests, and that people on his team occasionally had reason to search those logs. He didn't want anyone knowing what he was asking Multigen to do. So he had arranged for his requests to not appear in Multigen's logs, thinking that that was enough.
_Of course_ it wasn't enough! He hadn't considered that Multigen would sit behind a different server (the reverse proxy) with its own logs. He was a research engineer, not a devops guy; he wrote code, but thinking about how and where the code would actually run had always been someone else's job.
It got worse. When the Multigen web interface supplied the user's requested media, that data had to live somewhere. The _videos themselves_ would still be on Magma's object storage cluster! How could that have seemed like an acceptable risk? Jake struggled to recall what he had been thinking at the time. Had he been too horny to even consider it?
-No. It had seemed safe at the time because videos weren't greppable. Short of having a human (or one of Magma's more advanced audiovisual AI models) watch it, there was no simple way to test whether a video file depicted anything in particular, in contrast to how trivial it was to search text files for the appearance of a given phrase or pattern. The videos would be saved in object storage under uninformative file names based on the timestamp and a unique random identifier. The probability of someone snooping around the raw object files and just happening to watch Jake's videos had seemed sufficiently low as to be worth the risk. (Although, _ex ante_, he would have assigned a similarly low probability to someone catching a discrepancy between Multigen's logs and some other unanticipated log, which had just happened—suggesting that his sense of what was probable was miscalibrated.)
+No. It had seemed safe at the time because videos weren't searchable. Short of having a human (or one of Magma's more advanced audiovisual AI models) watch it, there was no simple way to test whether a video file depicted anything in particular, in contrast to how trivial it was to search text files for the appearance of a given phrase or pattern. The videos would be saved in object storage under uninformative file names based on the timestamp and a unique random identifier. The probability of someone snooping around the raw object files and just happening to watch Jake's videos had seemed sufficiently low as to be worth the risk. (Although, _ex ante_, he would have assigned a similarly low probability to someone catching a discrepancy between Multigen's logs and some other unanticipated log, which had just happened—suggesting that his sense of what was probable was miscalibrated.)
But now that Chloë was investigating the bell character bug, it was only a matter of time. Comparing a directory listing of the object storage cluster with the timestamps of the missing logs would reveal which files had been generated illicitly.
projects / Ultimately_Untrue_Thought.git / commitdiff