The Legacy
Author and humanitarian Malka Older imagines what happens when an AI dies, and how a community of scientists would grieve after its demise.
Fiction by MALKA OLDER
It was a slow ride to Arkhangelsk. It was the dark of the year, so they were piloting on stored energy, and the captain was appropriately cautious. Small wind turbines stood hopefully off the bow, but typically garnered little more than the charge for an occasional hot shower. Not that Leoka begrudged the hot water. It was very cold, and although the current fashion was for people to prefer the chill and rail against the heat, Leoka had never liked winter. The brief hot showers were the only time in the four-day trip when she felt warm all the way through. She tried to convince herself it was an adventure – that she was going to a remote place very few people ever got to visit and she was being paid to do it – but it didn’t work. The only thing that made it seem worth it was the story, and the fascination of Mikhailai.
They let Leoka disembark shortly before reaching the city, at a dock purpose-built for the archive. There were 295 stone steps from the dock up to an imposing metal door fixed into the rock cliff. (Leoka counted them; it would make a nice detail for her article.) There was a button at the door, and then a long, long wait.
The door clanged open. “Welcome, welcome. Are you Leokadia?” A heavyset man peered out into the dim noonday of far-north winter as though from someplace much darker.
“Leoka is fine,” she said, incidentally correcting his vowel pronunciation. “Doctor…?”
“Daniel Ottilsen, the linguist. Come on, let’s get you out of the cold.”
Leoka did not notice much difference in temperature as they started down the stairs. The chemical lights were colorless but bright enough for Leoka to notice that Ottilsen wore an oxygenator in his nose even though it was one of the subtle expensive ones.
It felt like they were descending far below the level of the ocean, but Leoka had forgotten to count this time, concentrating instead on Ottilsen’s easy patter. “Rough journey? Helluva time of year to come out here. But I suppose you didn’t have much choice.” He laugh-coughed. “Nor do we all. Well, this is the, uh, lobby I guess you could say. We use it as a game room sometimes. But most of the rest are either still asleep or in the server room, which is where the forensic coders have to hang out, for the most part.” He turned back to Leoka and seemed to notice her duffel for the first time. “Oh, I’m sorry. I should have taken that before the steps. Could I, uh…”
“It’s fine,” Leoka said, still short of breath. “Maybe you could show me where to put it down?”
“Oh yes. This way.” He led her past a ping-pong table to a long corridor, and opened one of the first doors. “Here you are. Bathroom’s through there – just a long drop, I’m afraid, but it’s very long. Everything’s clean – we have people who come in once a week to see to the laundry and bring supplies. There’s a cooking roster – you’ll see that in the kitchen, not required of guests but of course any new recipes are very welcome. And – you’ll be needing this – we have to keep the whole place cold, of course, well, and it was designed to be kept cold, couldn’t heat it if we tried – but here’s a space heater for the room. You can plug it there, runs off whatever we’ve got stored, but typically we’ve got plenty of juice to keep you snug through the night. Hope you brought some sweaters too! Anyway. I’ll be off and let you get settled.”
When Ottilsen had left, carefully shutting the door behind him, Leoka sat down on the bed and hugged herself. I am inside Mikhailai, she thought in amazement. I am inside its brain! Well, perhaps not the brain itself. But close! The ear canal, say. I am inside the ear canal of a dead intelligence. She laughed at herself, and went to splash water on her face.
Leoka met most of the other scientists at lunch, which was mostly beans and tomatoes from the hydroponics. Only one of the forensic programmers, Rayna Naoumov, was at the mess; the others were eating in front of their screens. “Many, many lines of code,” Naoumov shrugged. “They love it too much.”
She picked at her beans; they were good, but she didn’t want to finish too fast. “So…what are you all doing here?” She had imagined a handpicked team, working together to save the world. But the scientists looked at each other and shrugged.
“Research grants,” answered Selena Filippova, a narrow woman with the thickened skin and voice of a smoker. “My personal specialization is the sociology of technology that doesn’t progress in the way we expect, that has a disconnect with society’s idea of progress. So…” She raised her hands expressively.
Leoka turned her gaze to Roser Blanxart, who nodded. “As a cyborg anthropologist, this is the most interesting and fruitful place for me to be. It is true, Mikhailai is no longer interacting, but we have the records here, and…it is very important work.”
“Can you tell me what happened?” This isn’t how Leoka had planned to do her interviews – en masse, over food, starting at the end of the story – but she can’t keep herself from asking.
“What always happens,” Filippova answered. “People expect technology to exist independent of their efforts, they start to take it for granted, they forget to pay the electricity bills, and poof!”
“They didn’t forget to pay the electricity bill,” said Květa Klement, the AI theorist. Her gray hair looked like it had been cropped for convenience and then grown out in the salon-less environment of the lab, and had a bright streak of green down one side. “It was a decision.”
“It was a collective decision,” Filippova inserted. “Not a single one. It was a number of small decisions, omissions…”
“They both took the AI for granted and were frightened of it,” Blanxart put in. “We can see in the documents from the time.” As though it had been twenty years or fifty instead of five. “Politicians saying Mikhailai was a useless expense, news headlines hinting at some vast unspecified harm it could and would commit against us.”
“People always see themselves in the technology they create,” Filippova put in.
“And of course they couldn’t imagine that they wouldn’t be able to turn it back on, or create a newer, better one, at will,” Klement snorted.
“It’s a complicated system,” Pylyp Chayka, the electrical engineer, finally got a word in. “Not just the programming,” he added, with a nod to Naoumov. “Electrically, the cooling, the power, everything needed to keep it running…a very complex system, and honestly it was running on threads for a long time. They were trying.”
“They failed,” Filippova said.
After lunch, Leoka asked Naoumov to take her back into the server room. “Do you like working here?” she asked, as the younger woman led her downstairs and through corridors that made no pretense of being anything but old mine tunnels.
Naoumov wiggled her hand. “The work is fascinating, but you know, there’s not much opportunity to get out. We go to Arkhangelsk once a week or so, but it’s kind of a trek. It can get a little, um…” She seemed to be searching for a phrase that wouldn’t insult her coworkers. With difficulty, Leoka resisted the urge to suggest one (insular? claustrophobic? stir-crazy?), but Naoumov didn’t finish the sentence.
“Tell me why the work is so interesting,” Leoka said at last.
Naoumov spread her hands expansively. “I mean, this is the greatest technological mystery on the planet right now! We made something work and now…we can’t. Nobody really understands why Mikhailai worked in the first place. So even when I’m spending a day staring at, maybe, ten lines of code…I don’t know, I feel like if I can grasp, you know, really really understand what those lines are doing, what they mean in the overall context of Mikhailai – well then, that takes us that one tiny step closer to understanding Mikhailai.” She paused. “Also, I feel like, as I reorganize my brain to understand and take in the processes of Mikhailai …maybe my brain is becoming a little more like it, you know? Like I can take on something of that kind of intelligence.”
The server room did not look like part of a defunct mine, even if it was more a cavern than a room, stretching away in the dim bluish light. Every visible surface was coated or plated for greater energy efficiency and cooling: panels to recapture light; wicking material to reduce heat build-up. Leoka had felt the temperature drop at least five degrees as she walked in. The servers lurked in the darkness along both sides, and in the middle were the terminals where the forensic programmers pored over Mikhailai’s billions of lines of instructions.
The terminals were on, and there was a faint hum from the servers as they disgorged their guts on command, but the synaptic wires were silent. It felt like a morgue.
Naoumov introduced Leoka to Franklin Okeke, the head of the forensic program team. “Head in terms of coordination and administrative duties, not so much supervisory,” he said apologetically. Everyone else in the room was avoiding Leoka’s eyes, so instead of trying to corner one of them, she stayed with Okeke.
“Can you explain to me why this is so difficult?” she asked, recorder running. “You have all the code that made Mikhailai work. Why can’t we access this intelligence now, when we need it more than ever?”
Okeke leaned back and interlaced his fingers with the air of someone who did not talk to non-specialists very often. “Mikhailai was not a machine of levers and pulleys, such that you know exactly what will happen when you do this. Mikhailai evolved over time. It was a very, very complex set of processes.” He paused, then leaned forward. “All these people out there,” he gestured, leaving it vague as to whether he was talking about the credulous world outside or the non-coding scientists living with him in Mikhailai’s mine, “they want Mikhailai to come back, they hope for it. They don’t seem to realize, that even if by some unforeseen event Mikhailai were to start functioning again, it would be a different Mikhailai.”
Leoka scribbled some random notes to cover her recovery from that statement. “Do you believe Mikhailai, in any version, would be able to resolve the current crisis?”
“I don’t know,” Okeke answered. “This intelligence was alien to us, completely different. We don’t know how Mikhailai came up with its answers or what it might be capable of, let alone how it might have evolved over time and when faced with new challenges. But…perhaps, yes. It accomplished such incredible things while it was functioning.” He traced his palm over the terminal casing.
Leoka spent most of the afternoon in the server rooms, some of it talking to the other coders, some of it recording ambient noise and making notes about the location. There was little idle chatter, which might have been because of her presence, but mostly the coders looked engrossed in their work. A long blackboard hung on one wall, with messages scrawled on it:
Has anyone come across the command(?) gnrip?
I have a four-line string that repeats exactly; error or ??
What if it doesn’t want us to find it?
Across the top was scrawled in capitals REMEMBER TO CROSS-REF
WITH ORIGINAL CODE BASE
She finally left when she could no longer bear the cold.
That night at dinner, the power went off. Leoka sat waiting for auxiliary lighting until she realized that no one around her was calm. She couldn’t see until someone managed to fumble their way to the sideboard and turn on one of the solar flashlights that stood waiting, but she could hear people pushing back their chairs, feeling their way along the table, hurrying for the door.
“What is it?” She asked, when the flashlight was on and most of the rushing footsteps had reached the hall. “Do we have to evacuate?”
“No.” It was Pylyp Chayka’s voice, calm beside her, and Leoka had time to think that it was odd for the electrical engineer of all people to remain untroubled by a blackout. “They’re all hoping it means Mikhailai is back and has drawn off all the power.”
“Does it?” Leoka started up out of her seat.
Chayka let out a dry little cough-laugh. “Almost certainly not. We lose power three or four times a week from grid fluctuations that our internal power can’t completely cover. Don’t worry.” She heard him take a swallow of his drink. “We’ll know soon enough.”
“I…” Leoka realized that she hadn’t scheduled an interview with Chayka, thinking of him more as facilities staff than researcher. “Are you working on figuring out why Mikhalai stopped, or how to restart it?”
Chayka laughed again, more fully this time. “They brought me in to determine whether it was an electrical engineering fault that killed off the machine.” Leoka tried not to flinch at killed, then remembered nobody could see her. “I was able to determine pretty quickly that it wasn’t – there were no power surges or drops, no major technical problems that I could identify, no blanks in the records.” He shrugged. “But I was already here on a year contract, and they realized they need someone around to help manage energy usage by the researchers, so I stayed. It’s well-paid enough, although a little far from home.” She heard his chair push back against the floor, and a few moments later the splutter of a match as he lit a candle.
Leoka heard footfalls from the corridor, much slower than when they had left, followed by the bouncing glow of a flashlight. Blanxart was the first back, and even in the candlelight her expression made it immediately clear: Mikhailai was still absent.
Leoka interviewed Ottilsen in his tiny office, reminiscent of those of her professors at university but further cramped by a large computer for, he told her, linguistic analysis.
“Mikhailai created its own language,” Ottilsen explained. “That is to say, its way of writing to itself, sending messages to different parts of itself and writing new code, evolved. First it was shortcuts, a few neologisms, dialectical stuff. But the last year or two were completely unintelligible to the programmers. We still don’t understand it. I’ve been attempting to re-create the evolutionary process of the language and decipher it with the help of the programmers, but it’s slow going.”
“How can you translate a dead language that only one entity ever spoke?”
“Great question, but Mikhailai used the language to communicate between different parts of itself.” Ottilsen ran his hands through his thinning, over-long hair. “So in a sense it’s more than one speaker, and also we can see the impacts, the direct results of each communication…” He trailed off, and then went on again quickly. “To be honest, I have a very strange sensation about these…exchanges. They don’t…I feel a bit silly saying this, but they don’t feel like computer language.”
Leoka raised her eyebrows. “In what way?”
“They – we haven’t, of course, completely worked out the language yet – but from what I can understand – they feel not simply functional. I – perhaps I am imagining it, but I get the sense of emotions. Connotations, I should say. Some of the exchanges, I am sure, are not commands, but…discussions. Perhaps commiserations.”
“Many people have theorized that Mikhailai became sentient,” Leoka said, as neutrally as she could.
“Yes…” Ottilsen agreed but didn’t seem to be satisfied. He stared at the scribbled notes pinned on his wall. “I think Mikhailai shut itself off.” It burst out from him, as if he couldn’t resist this opportunity to share it. “I can’t be sure yet, but I think it was unhappy.”
He remained lost in thought while Leoka tried to figure out how to extricate herself from what sounded like an obsession. “That sounds…provisional?” she essayed, hoping to lead the way back to firmer ground.
“Oh yes, yes,” Ottilsen said, looking back at her. He attempted a smile. “Don’t know how I’m ever going to write those findings up.”
Leoka obliged with a polite laugh.
“I, uh, would prefer you didn’t mention them in your piece.”
Leoka agreed, but on her way back through the dim corridors she couldn’t stop thinking about what he had said. The power outage of the night before now hovered in her thoughts like a ghostly message from Mikhailai, a beyond-the-grave effort to stymie the ongoing resuscitation attempts.
The night before she left, sitting on her bed in the room she could only think of as Mikhailai’s ear canal, Leoka typed her notes and transcribed bits of her recordings. She had noted the timestamp for key quotes, making it easy to find them. She searched for something Filippova had said and played it back: “It’s not a new thing for humans to forget something they once knew. The whole popular concept of the Renaissance, even if it isn’t entirely accurate, reflects that sense of knowledge that has been lost and rediscovered, or not. The Antikythera Mechanism. The Great Library of Alexandria. The wonders of the ancient world…in some cases the knowledge faded because it was no longer needed with the same urgency, or no longer needed at all. Or a device was made by one person, or a small specialized group, and the line of transmission was cut by war or some other crisis…in any case, the difference now is simply that we are learning and forgetting much faster. We are creating things that are much more complex, so that even the creator may not fully understand how they function.”
That reminded Leoka of a related quote, and she switched to the interview with Blanxart. “It will be easy, soon, for people to imagine that Mikhailai didn’t really exist, or at least didn’t do the things we believe it did, wasn’t capable of what we believe it was capable of. After all, if something isn’t repeatable, does it exist?”
When Leoka stepped onto the boat for her return journey the air was colder and darker than when she had arrived; or maybe that was her mood. She looked out over the icy water and wondered if Mikhailai had felt lonely. The only intelligence of its kind, so different from everyone else on the planet that literally no one could understand it. But probably imagining loneliness was perpetrating anthropomorphism. Maybe Ottilsen was doing the same thing. If none of the other scientists could understand how Mikhailai had thought, calculated, solved diplomatic problems, slashed through centuries-old knots in biology and physics, why should she believe that Ottilsen could understand how Mikhailai felt?
She certainly had no reason to think Mikhailai felt anything like she did. But that was what she felt as she stared over the frigid water: loneliness. It didn’t make any sense: Mikhailai was something people had created, that hadn’t existed at all a decade ago. Why should she miss it? But talking to the researchers had given Leoka the sense of the extinct AI as something radically different, something alien and full of potential, now irrevocably gone. They were left with the limits of their human-style intelligence, and that seemed terribly bereft.
Frowning at the water, Leoka set about designing how she would convey that feeling to her listeners.
Malka Older is a writer, aid worker, and sociologist. She is a Faculty Associate at Arizona State University, where she teaches predictive fictions.