I'd say AI should be able to write a book about 2.5 years ago:

https://lifearchitect.ai/books-by-ai/

For what it's worth, I experimented a little with ChatGPT to write a 7000 word hard-boiled detective story. My main rule was that I couldn't write any original content myself - I was exclusively the editor.

I started with eliciting the stages of a detective story and then got ChatGPT to detail a plot based on those. Then I asked it to write individual scenes for each plot beat. Then I had it revise to correct inconsistencies and get the hard-boiled style down.

The process took maybe two or three hours. Compared to a human writer the result was mediocre and full of cliches. But... it was a recognizable story with plot and characters and it was mostly coherent. It was also a lot of fun to coach into existence.

From that experience, well, we're definitely not at the point where something like ChatGPT can write fiction coherently on its own for longer than a few pages and even with significant coaching, the results are "low human." But from this point to super-human? Who knows?

Might not take that long...

(Turns out that the full story was 4438 words.)

We are in 2023, Id say close to 2025. It only need a few more evolutions before I can write a good book with good human complexity style of writing.

It can already to that. It's just not available to the public as a free tool.

I’d say 3-5 years before it’s writing genuinely impressive original stuff on the level of pro novelists. But even next year it should be much better.

What if as it writes, it compresses the previous chapters to a summary with only facts that are key for the following chapters?
I think it works kinda like that in AI Dungeon, but you have to fill in those "memos" yourself.

Does novelai not count?

So here's the feature I think you need to make the tool work:

right now, the machine works by:

<current symbol buffer> + neural network -> <current symbol buffer> + 1 symbol

It needs to become

f(all previous sessions symbols) = salient context

<salient context> + <current symbol buffer> + neural network -> <current symbol buffer> + 1 symbol + <updated salient context>

"Salient context" is whatever the machine needs to continue generating text to match to something like a detective story. So it needs to remember the instructions, the main character's names, and so on. It does not need to remember every last word previously in the story.

To make it really good it needs to be aware of metrics of quality. Amazon/royalroad number of reviews and review ratings. Number of copies sold of the novel on the market. Etc. This way it can weight what it learns from text by how much humans liked that particular structure of text.

After that you'll need the AI to generate many stories, get user feedback, and iterate. I think eventually they will be good, and at some point past that it may discover ways to make them REALLY good that humans have not.

You can ask chatGPT to write a summary of the story first, then the chapter names and chapter summaries, then each chapter one at a time. Try it! This hierarchical method can already achieve some of what you’re talking about.

So essentially it would just do something similar automatically.

Did you story have the poor and down on his luck detective who is overqualified for the job, his ratty run down office, and the clients as suspiciously attractive women?

Oh and by the way, using human ratings of the model output is exactly how ChatGPT is trained. Human-in-the-loop reinforcement learning.

Correct but this was done at a small scale by chatGPT employees. I am saying we look at every novel that has data on its sales, every story on a site that has metrics of views or other measurements of quality and popularity, etc.

This might give the machine more information on what elements work that people like. Maybe enough to construct good stories.

NovelAI is a great tool, I have already used it a lot in the past, but I meant more like a book made out of a prompt, with little to no human interaction to create something good. And sadly NovelAI is still a bit behind of GPT-3, even with their GPT-Neox model

Quite a long time, maybe. One of the key limitations of GPT-4, as well as other language models in its class, is the context window.

In the case of GPT-3, the context window is approximately 2,048 tokens. This means that when generating text, GPT-3 can only consider the 2,048 tokens immediately preceding the point at which it is generating text. This can make it difficult for the model to maintain coherence and consistency when generating longer texts, such as a book.

When GPT-4 is developed, it'll likely still be limited by the context window . Researchers would need to develop new architectures that are able to take into account a larger context window. This is complex as hell, and it is not clear if it'll be possible with our current hardware limitations.

Thank you for the list! But I meant a book being written entirely by an AI, not just it being the co-author

Arguably it can be done now, you just need to take the architecture of gpt3 plus davinci and allow it to process enough tokens to write a full book without forgetting things from the start. I actually worked with it to create a concept for a game idea I had but after a certain amount of words, it gets confused. I was able to adjust for this by giving it summaries of where the story was upto and reminding it of points that we had already incorporated.

So tldr, 1) allow it to process more tokens, or 2) compensate for this with human guidance.

That's interesting! I also played a bit with ChatGPT, trying to write a long story. Although even with me trying to correct and guide it, the story quickly became inconsistent and almost everytime the AI tried to end the story, finishing with the cliche ending of a fairy tale.

This is the way.

I've said it before, and I'll say it again. ChatGPT reminds me of a 13 year old who can never accept when they are wrong, has full access to the internet by thought and is very forgetful.

NovelAI can write a book now. It's not very good, somewhere on badly written fanfiction porn level, but it's a book alright.

The real question is - when can AI write a best seller book?

That was definitely the default but I asked for something a little different and I got a dedicated female investigative journalist trying to track down a serial killer. One interesting twist is that the killer turned out to be a police officer himself. The climax of the story was the killer literally tackling the reporter as she burst into the police station with proof of his crimes!

And then what happened?! She gets arrested for resisting arrest and assaulting an officer? All the evidence she had on her disappears?

She spends time in jail until her defense attorney presents a cloud backup of her data to the DA? The charges get dropped but no one is punished for their actions but the serial killer?

Looks like I overestimated the word-count. The full story is 4438 words and here is the last 469 words:

>...As she approached the police station, she heard the sound of footsteps behind her. She turned to see the killer closing in on her, a cold glint in his eye.
>
>Kate's heart raced as she took off running, the killer hot on her heels. She weaved through the crowded streets, her breath coming in short gasps as she tried to outrun him. She knew that she couldn't let him catch her, not with the evidence she had against him.Finally, she spotted the police station in the distance. She redoubled her efforts, her legs pumping as she sprinted towards safety. She could hear the killer's footsteps growing closer and closer, and she knew that she was running out of time.
>
>Just as she reached the doors of the police station, the killer tackled her from behind. She went down hard, her head hitting the pavement with a sickening thud. She struggled to get up, but the killer was too strong.
>
>She heard the sound of sirens in the distance and knew that help was on the way. She just had to hold on a little longer. She summoned all of her strength and managed to break free from the killer's grasp. She scrambled to her feet and took off running, her heart pounding in her chest.
>
>She burst through the doors of the police station, her breath coming in ragged gasps. The officers on duty rushed to her aid, their faces grim as they took in the sight of the killer hot on her heels.
>
>Kate collapsed onto the floor, her body shaking as the officers apprehended the killer. She looked up at the chief of police, her eyes filled with tears of relief. The sight of the grim-faced officers and the smell of stale coffee in the station added to the tense atmosphere.
>
>###
>
>Kate presented the evidence to the chief of police, who was shocked and outraged by the revelation. The chief of police sat at his desk, his face grim as he listened to the recording. "This is serious," he said, his voice laced with disgust. "A police officer using his badge to commit murder and cover it up. It's an outrage.
>
>"What are we going to do?" Kate asked, her voice trembling.
>
>"We're going to do our jobs," the chief said, his voice firm. "We're going to arrest this killer and make sure that he pays for his crimes. And we're going to do everything in our power to restore trust in the department and ensure that something like this never happens again."
>
>Kate nodded, her heart heavy. She knew that this was a blow to the department, but she also knew that they couldn't let the actions of one corrupt officer tarnish the reputation of the many good officers who served with honor and integrity.The chief stood up, his face determined. "Let's go bring this killer to justice," he said, his voice filled with conviction.
>
>The killer was arrested and Kate's story was front-page news. The sound of her own relieved sigh filled the room as the chief congratulated her on a job well done.
>
>Kate's career was revitalized and the community was safer because of her bravery and determination. Despite the personal sacrifices she had made, Kate was proud of what she had accomplished. She had brought a killer to justice and exposed corruption in the police department. As she walked out of the police station, the warm sun on her skin and the sound of the city's hustle and bustle were a welcome reminder of the life she had fought to protect.

10 years, maybe??? 🤷‍♂️

I think what you are asking is how long until researchers find out a way of expanding the input size of the model? Because there ways you can get it write a book right now. For example, you can tell it to write the first page, then a summary of it. Then feed the summary and let it write another page. And so on.

As to increasing the input size, if no new way comes around, it will get bigger as the transformers get more efficient, which they will. Look up flash transformers for example.

Already can….

Unrealistic but I mean, if you compare the AI to a high school student this is pretty good.

Exactly - and the fact that it's this good is pretty amazing. There's a real danger of goalpost moving.

Yeah. I would generalize to "most educated adults can't do better in their lifetime". I can maybe write a slightly better story since I've read a lotta stories, but not by a huge amount.

[deleted]

AI will never surpass human authors.

Literally never

It's pretty stunning what the GPT models can / will be able to do with language, but we're still a long way from competing with professional writers. There's a big difference between coherent prose (which anyone with a bit of experience can write) and creating compelling original characters or tapping into the human condition. Great novels accomplish a lot more than just "good writing".

That being said, there is a lot of trash that gets published anyway! The AI may not be coming for Stephen King, but it will be coming for a lot of the mediocre stuff on Amazon.

People said the same thing about Go.

But when it comes to art, what does "surpass" even mean? Did Steinbeck surpass Tolkien? Did Picasso surpass Klimt?

It's a meaningless question. What matters is whether AI will be able to create works that are meaningful and moving to humans - and I believe that it will.

That's the same way I've used it to architect and write out larger programming projects. When you're stuck with a limited context window, a top-down approach is king.

Famous last words, and I write.

It can be done already, but the quality of the book will always be in question. You just break down what ChatGPT is doing into small consumable tasks. You don’t say, “Write me a book.” You say, “Please give me an outline for a book and list 12 chapters with titles relevant to the plot. Then you ask it to write each chapter. I’d recommend you also edit as you go. ChatGPT is a tool to save labor, but you have to give it reasonable, consumable inputs versus your expectations.

It literally can already write very large books this is nothing new

Denial

Did you even read twilight?

Realism

What have you written?

This is crap by high school standards, let alone college.

Go is solvable algorithmically.

Creating characters, plots and themes isn't. This will never happen.

After messing with it, my impression was it does feels far off, but it may honestly just be 1 or 2 more orders of magnitude away for computing power and it could happen fast. At the end of the day, I was extremely impressed.

I don’t think it’s unrealistic to imagine AI becoming 100x more powerful than today, more a question of when not if. Just my opinion!

I think the novelty alone might clinch it even if it kinda sucks tho

Can you imagine how wealthy someone would get if they created an AI author that makes a themed series of books while imitating a certain human persona and maintaining a social media presence?

Computing power isn't the barrier.

The barrier is that really good writing relies on experiences and emotions. I don't doubt an algorithm could churn out a passable Hardy Boys mystery by 2050. A procedural that sticks to a basic template with basic characters and no themes to speak of.

But an AI writing something along the lines of Blood Meridian, I am quite confident saying this will literally never happen. Not unless we get fully sentient robots like Data from Star Trek.

ChatGPT has improved on context and memory quite a bit over GPT-3 and I assume it will be the same for GPT-4. You can have quite a long conversation with it remaining consistent but plotholes still appear after 1-2 pages of words (and the story’s are dull). Increasing the coherency of a response is definitely a focus at the moment so I wouldn’t be surprised if GPT-4 has the capability to write 5-10 pages which form a believable story. So it all depends on when GPT-4 is going to release which I’m guessing will be Q3 this year.

Should be possible at some point. But since GPT is so general, I think you have to coax it a little first, to get in the "mood" to be an author, or at least to make it be consistent in its style. Perhaps retrain or feed it with crime novels from a particular author to better understand what constitues a GOOD story written in a certain style.
Sort of like the tricks people use to make ChatGPT write code. ChatGPT isn´t (wasn´t) willing to write code, you had to sort of get it to accept certain rules first, a sort of framework, but when that was done it could output code.

AI is based on what it was trained on. The majority of stories are pretty dull and cliche. I’m guessing if you train an AI model on great stories it could generate great stories too.

Imagine an AI writing a story with Tolstoy, Kafka, Le Carre influences. Bonkers.

That's already happened. The question is how long it'll take for them to be able to write one that's any good. The hard thing here is figuring out how to teach it to do something that very few people really have an understanding of.

> and is very forgetful

To be more precise it has no long-term or procedural memory at all and it can't learn (from your interactions).

> <salient context> + <current symbol buffer> + neural network

That's RNN (recurrent neural network). As far as I know LSTM is still state of the art for them. And it struggles with long-term dependencies.

[He checks papers]

It looks like combination of transformer and LSTM does provide some benefits, but nothing groundbreaking yet.

Writing is editing. AI is not good at line editing or structural editing. It beats humans on quantity, but it hasn't been taught the basics: show not tell, avoid adverbs, avoid long sentences, try not to repeat yourself, keep the story consistent.

AI has been trained to not be biased. This is good of course, but when you write a story, you have to choose a side. You need to choose a coherent setting and cast of characters. So in the absence of personal preferences and history, AI can only choose the most prevalent patterns in the training data or random stuff. This means clichés or something incoherent. Also the training data is not up to date with modern ideas AFAIK. It leans toward books written before 1920.

So I think the question should be when can AI edit properly? When and how can it build a personal style and history? I don't think that throwing money and data at this is enough. Knowledge of psychology, neuromorphic hardware, spiking neural networks, extreme learning machines, or something else entirely is required. So a decade is probably insufficient.

[deleted]

Would love where we can read the book and give the a.i tasks of removing or changing things we don´t like.

We convinced sand, rocks and electricity to write an 'ok' story.

Louis CK Everything Is Amazing And Nobody Is Happy

https://www.youtube.com/watch?v=nUBtKNzoKZ4

Creative writing is a learned skill. It's not some magic thing that's unachievable by AI.

Deniers act like ChatGPT is where AI development ends, as if what we have right now is proof that we are incapable of making something more complex. This is just a step to something bigger.

It can already do that. What you might want to be curious about is when will it formulate its will to write that book.

It's partially learned, partially talent, partially experience.

Same approach I’ve taken with moderate success. After awhile though, you can definitely tell it’s lost some of the context.

Currently, AI can generate text that is coherent and reasonably well-written, but it is not yet capable of writing a book that is comparable in quality to one written by a human. GPT-3, the most advanced AI language model available to the public, can generate text that is relatively coherent and well-written, but it is still limited in its ability to create truly original and engaging stories.

It is difficult to predict exactly when an AI will be able to write a book that is comparable in quality to one written by a human. The development of AI and its capabilities is a rapidly evolving field, and it will likely depend on advances in natural language processing, machine learning, and other related technologies.

It's also important to note that while AI can generate text that is coherent and relatively well-written, it still lacks creativity and originality that are essential elements of a good book. It is unlikely that an AI will be able to write a truly great book without human-like creativity and understanding of human emotions.

And what else can I tell you honestly, with the utmost sincerity!!! Don't use articles written by AI... no matter how good the program is. Google prints you and penalizes you. And if you use them, use them only as ideas, nothing more. Professional advice. Google is smart as hell.

It can basically do that right now, it’s just not very good at it yet.

Currently you can co-author with chatGPT and get a book of arbitrary length with enough revising, re-generations, and trial and error. The book will be ok, albeit cliche-ridden and surface level in a lot of areas but it'd be readable and worse books would certainly exist. Will we ever get to the point that we can do it with the click of a button? Probably. But that alone is super human. If you were told to write a book about a topic it would take you a lot of revising, rewriting, and trial/error too.

GPT 4 - 2024

Nice, then you can have like a fully impenetrable bubble to live in... or to force others to live in. Fun times.

I'm about to release a mobile app that uses GPT and Dall-e to create illustrated stories for kids. If you're interested, check out the link: https://play.google.com/store/apps/details?id=not.a.bug.storia. I'm also releasing a test version soon, so if you're interested in trying it out, feel free to DM me.

That is also used by human writers, it is called the snowflake method.

Can we see it? Is it posted anywhere?

No - it was just for fun! But you can see the last few hundred words below somewhere in a separate comment.

That’s what the artist said

Kind of proves my point. AI art programs are good at making pretty renders. It's on par with calendar art. They aren't doing anything with any kind of message or thematic depth and almost certainly never will. You can get away with that in visual media, but not if you're trying to write a whole fucking novel.

Lmao, sure bro RemindMe! 2 years

I will be messaging you in 2 years on 2025-01-19 02:18:32 UTC to remind you of this link

CLICK THIS LINK to send a PM to also be reminded and to reduce spam.

^(Parent commenter can ) ^(delete this message to hide from others.)

2 years. Be seeing you.

👍

Perhaps, or one could use the same AI to have any kind of book you wish. I fail to see the use of the ‘real’ author, since I doubt they could dictate the direction I’d want a book better than an AI that adjusts in real-time my preferences whenever I open up my magic book full of anything I want it to be. You’re correct, so long as we don’t have the freedom to make the book ourselves. In fact, it may be seen as limited to even buy a book except for historical or novel purposes (pun intended).

Agreed, this is more like middle school standard in regards to narrative, although with perfect grammar and spelling.

I think by this time next year it will be capable of writing one. But we'll see.