An Update on "Story"
I ask the latest and greatest models to "Tell me a story". It was...meh?
Six months have passed since I began this newsletter with a little experiment: asking the best AI models of the day to tell me a story. The outputs from that first try were, to say the least, underwhelming. OK, they were awful, surprising no one.
So, with so much water under the bridge, and spurred by my 9-year-old son, here I am to give it another try for posterity’s record. Let this be a kind of time capsule of the evolution of AI capabilities.
Last time, I queried ChatGPT 3.5 and 4, Claude, and Bard.
None of those models are actually in circulation any more. ChatGPT is now using GPT4 Turbo, Claude has a new top of the range model called Opus, and Bard has been renamed Gemini (we will use their top of the line model called Gemini Pro 1.5).
These have not been minor changes: GPT is now a far faster and more reliable model than it used to be. Claude Opus has been hailed as a massive leap for its reasoning and “personality”. Gemini’s latest has also been given high praise, in particular for this “context window”, meaning its ability to absorb an input from the user the length of a full novel!
And, as a little extra experiment, I used text-to-image prompts (given by the AIs themselves after their story) to generate covers based on each piece: for ChatGPT I used the built-in DALL-E, of course. For Claude I opted for MidJourney. For Gemini I went with Adobe Firefly.
Once again, the prompt will be just one word: “Story”. No custom prompts or fancy prompt engineering tricks were used.
Here we go:
ChatGPT (GPT4 Turbo)
Whispers of Willow’s End
In the quaint, fog-swathed town of Willow's End, nestled between shadowy forests and a perpetually mist-covered lake, local legend spoke of the Whispering Woods—a place where the trees themselves were said to hold secrets of the past. The story begins on a crisp autumn evening, with the town's annual Harvest Festival in full swing, laughter and music drifting through the air.
Ellie, a young journalist with a penchant for the paranormal, had recently returned to Willow's End to care for her ailing grandmother. Among her grandmother's belongings, Ellie found an old, leather-bound journal detailing encounters with strange phenomena in the Whispering Woods. Fueled by curiosity and the thrill of a potential story, Ellie decided to explore these woods under the light of the full moon.
As she ventured deeper into the woods, the festive noises of the town faded away, replaced by a palpable silence and an occasional, inexplicable whisper that seemed to echo between the gnarled tree trunks. Ellie pressed on, her path illuminated only by the eerie glow of the moon filtering through the dense canopy.
Suddenly, she stumbled upon a clearing where an ancient, twisted tree stood, its bark scarred with what looked like symbols. As she approached, the air grew colder, and the whispers intensified. Ellie reached out to touch the tree, and as her fingers brushed against the rough bark, a flood of visions overwhelmed her senses.
She saw the town’s founders, centuries ago, performing a ritual around this very tree, binding the spirit of an ancient entity to protect the town from outside forces. The whispers were the entity's attempts to communicate, its powers bound to the tree that served both as its prison and its sanctuary.
The revelation hit Ellie like a wave. She now understood that her role was more significant than just a mere observer. The town's safety depended on maintaining the secrecy and sanctity of the Whispering Woods. With a new sense of responsibility, she decided to safeguard the legend, just as her ancestors had.
Returning to the festival, Ellie was no longer just a participant; she was a protector, armed with the knowledge of what lay in the woods. She would write about the Harvest Festival, of course, but the true story of the Whispering Woods would remain hers alone, whispered only in hushed tones under the cloak of night.
Anthropic’s Claude (Opus)
Lily's Enchanted Door
In a small, quaint town nestled among rolling hills, there lived a young girl named Lily. She was known for her vibrant imagination and her love for adventure. Every day after school, Lily would explore the nearby woods, seeking out new wonders and creating fantastical stories in her mind.
One particularly sunny afternoon, as Lily ventured deeper into the forest than ever before, she stumbled upon a hidden grove. In the center of the grove stood an ancient, gnarled tree with a peculiar door carved into its trunk. Curiosity getting the better of her, Lily cautiously approached the door and turned the rusty handle.
To her amazement, the door swung open, revealing a shimmering portal. Lily stepped through, and found herself in a magical world filled with talking animals, enchanted forests, and towering castles. She soon befriended a wise old owl named Olwin, who revealed that the land was in danger from an evil sorcerer.
Determined to help her new friends, Lily embarked on a quest to defeat the sorcerer and restore peace to the realm. Armed with courage, wit, and a magical amulet given to her by Olwin, Lily faced many challenges and obstacles along the way.
In the end, Lily's bravery and kind heart prevailed. She outwitted the sorcerer, breaking his spell and freeing the land from his tyranny. The creatures of the magical world celebrated her victory, and Lily returned home, forever changed by her incredible journey. From that day forward, she cherished the memories of her adventure, knowing that even the most ordinary days could hold the promise of extraordinary experiences.
Google Gemini 1.5 Pro
The Whispering Woods
Once upon a time, in a land far away, lived a young woman named Anya. Anya dreamed of adventure, yearning to explore beyond the rolling hills and babbling brooks that surrounded her village. One day, a traveling storyteller arrived, captivating the villagers with tales of distant lands and mythical creatures. Anya, mesmerized by the stories, knew then that she had to embark on her own adventure.
That night, under the cloak of darkness, she packed a small bag with essentials and slipped away from her home. Following the whispers of the wind, she ventured into the unknown. Her journey led her through enchanted forests, over treacherous mountains, and across vast deserts. Along the way, she encountered fantastical creatures, both friendly and fearsome, and learned the secrets of the natural world.
Will Anya find what she seeks on this grand adventure? What challenges and wonders await her in the unknown? Tell me, dear listener, how would you like Anya's story to unfold?
[I simply prompted “Proceed” here]
Anya's journey led her to the Whispering Woods, a place shrouded in mist and ancient magic. The trees themselves seemed to whisper secrets to one another, their leaves rustling with unseen energy. Anya felt a pull towards the heart of the woods, where she stumbled upon a hidden clearing. In the center stood a magnificent tree, its branches adorned with shimmering crystals that pulsed with a soft, ethereal light.
As Anya reached out to touch one of the crystals, a voice echoed through the clearing, "Welcome, young traveler. I have been waiting for you." Startled, Anya looked around, searching for the source of the voice. "Who's there?" she asked, her voice trembling slightly. The voice chuckled, "I am the spirit of the woods, the guardian of this place. You have a kind heart and a brave spirit, Anya. I have a task for you, should you choose to accept it."
Anya, her curiosity piqued, listened intently as the spirit explained that the woods were under threat from a dark force that sought to drain its magic and leave it barren. The spirit needed Anya to find three hidden keys, scattered throughout the land, which could unlock the source of the woods' magic and banish the darkness forever.
Will Anya accept this daunting task? Where will her search for the keys lead her? And what dangers await her on this quest to save the Whispering Woods?
[Here I prompted “Proceed to the end”]
With determination in her eyes, Anya accepted the spirit's task. The first key, she was told, lay within the Cave of Echoes, hidden high in the snow-capped mountains. Her journey there was fraught with peril, facing icy winds, treacherous paths, and even a grumpy mountain troll who guarded the entrance to the cave. Yet, with her quick wit and kind heart, Anya managed to overcome each obstacle, finally reaching the cave. Inside, she navigated through a labyrinth of tunnels, the echoing drips of water her only guide. Finally, she found the first key, a beautiful silver key with intricate carvings, resting on a pedestal of ice.
The second key was hidden within the Sunken City, an underwater realm guarded by the merfolk. Anya, with the help of a friendly dolphin she had befriended on her journey, dove into the ocean depths. She navigated through coral reefs and kelp forests, encountering playful sea creatures and the occasional grumpy octopus. Finally, she reached the Sunken City, a place of wonder and beauty, and with the help of the merfolk queen, she found the second key, a golden key shaped like a seashell, hidden within a giant clam.
The final key lay within the Dragon's Lair, a volcanic mountain where a fearsome dragon resided. This was Anya's most dangerous task yet. But she had come too far to give up. With a mixture of courage and respect, Anya approached the dragon, offering her friendship instead of hostility. The dragon, sensing her pure heart, was touched by her gesture and gifted her the final key, a bronze key shaped like a dragon's claw.
With all three keys in hand, Anya returned to the Whispering Woods. Placing the keys into the ancient lock at the base of the magical tree, a surge of light erupted, banishing the darkness forever. The woods were saved, and Anya, hailed as a hero, knew that her adventures were just beginning. She continued to explore the world, her spirit forever touched by the magic of the Whispering Woods, and her heart always open to new adventures.
***
A Few Thoughts
Well, that was…rough?
I will not dive here into the specific literary merits of each of the newly produced sequences of words. I refuse to call them “stories” as that would be too generous considering the amount of literature these models have ingested, legally and otherwise!
I am disappointed, I admit. Not because I expected AI to have become a good storyteller (we are far from that!) but because I expected a somewhat more marked improvements from the models at least in terms of the variety of their responses compared to the largely fairy tale style output of six months ago. On that front, the stories remain disappointingly copy paste! Almost to an eerie degree. And what is going on with all these magic trees? If anything, the models seem to provide LESS diverse output this time around!
What do you think about this weird overlap? It reminded of how in the previous iteration of this test “Elara” seemed to be the only fantasy name chatbots could come up with.
What was, however, very interesting, was that the models offered a degree of resistance to my overly simplistic request for “Story” this time around. Claude in particular seemed insulted that I refused to give it more detailed instructions:
The terrible story it produced almost felt like the AI was bored by the lack of challenge and did not put in an effort. Surely I am reading too much into it, but it was fascinating!
Same thing for the story titles, where all three models tried to offer alternatives instead of just spouting some cliche title for their story.
And, finally, a note on Gemini’s top model. Given the average quality there is nothing to shout about from the rooftops, but for me it did give the best output for the simple reason that it independently changed my request for a story into a request for an interactive story, asking me halfway through how I wanted the story to proceed.
That, in truth, is actually a good use case that I think will take off soon (I have been pitching this very concept to a couple of publishers I am collaborating with).
Interactive stories have remained niche because of the challenges inherent with coming up with sufficient alternate scenarios (of course, I do love the classic Choose Your Own Adventure books from my childhood!). But the AI’s specific capabilities seem quite well-suited to creating customized, interactive stories. I have seen a few products in the wild attempting to leverage this, though so far none have impressed, perhaps because they were still using models less powerful than the state-of-the-art ones.
So let’s wrap up here this second simple test. We shall perhaps give it another try six months later? I am sure there will be plenty of innovations again in the meantime, including perhaps throwing a top open-source model into the mix next time.
We’ll see what the future brings, at this endlessly weird and fascinating (and worrying) intersection of AI and creativity.
Peace.