Trialled by Daniel Zuboff
Strategy at Loomery
One of the most popular use cases for generative, text-based AI is correcting and improving text you’ve already written. Notion has used data from customers to refine their integrated AI writing tools as discussed in How good is Notion’s built-in AI?, and as a result, they’ve made their editing with AI features easy to find and use. But, of course, Notion is not the only service offering such tools, and I’m curious to pit some of the most highly-rated products against each other and see which ones come out top. The quality of the writing produced is very important, but the ease of integration and the level of polish of the service are also important factors to take into account.
Most if not all of the writing improvement models are going to be based on GPT 3. Presumably, they’re all using OpenAI’s API to input a user’s text into their LLM along with a certain prompt, and then they output whatever is produced. Because of this, I’ll compare their more recent offering (GPT 4) against these services, and find out whether their careful prompting of GPT 3 beats my prompting of a more advanced generative model.
The products we’ll be testing are Wordtune, Copy.ai, Notion, Rytr, Grammarly, and GPT 4. To make this vaguely scientific, we’ll offer the same input to each service. I’ll make sure it’s got plenty of mistakes.
Input text (an extract from an English essay about Frankenstein):
I hate wet and reiny days.
It rained a lot in 1816.... a lot - like everyday; the weather in Europe was abnormally wet because it rained in Switzerland on 130 out of the 183 days from April to September. If I was Mary Shelley I might decide to write a book too. Afterall, it was the onnly thing you could do without TV or anything. She said that she "passed the summer of 1816 in the environs of Geneva...we occasionally amused ourselves with some German stories of ghosts... These tales excited in us a playful desire of imitation" So, people were stuck inside and bored. Mary Shelley decided to write a book becuase it was so awful outside. I can totally see her point, you know? I guess I would write a novel if there was nothing else to do.
This is an extreme example but we’re seeing how powerful these tools are. Whilst these sites may have plenty of offerings to help with idea generation and text creation, we’re only interested in how good they are at correcting and improving existing text.
Let’s see how Notion handles this. I just need to highlight the text, go into the ask AI subheading and click on fix spelling and grammar: