A test to decide whether we should consult AI-generated writing or not

Question

I know mentioning AI often triggers people here. I totally understand that because I've seen AI answer my questions in the worst way possible; it provides, with confidence, incorrect information in terms of word and phrase choices. However, there's no denying that AI is one of the greatest tools for English learning considering the immediacy of the responses it gives and the range of knowledge it has, of course, only if you know how to use it wisely.

I don't automatically trust AI, that's why I have asked dozens of questions here even after consulting AI. However, I have noticed that AI is more likely to provide you with naturally written answers when you tell it to do an open-ended task, meaning a task that does not involve following too many different requirements from you at the same time. A great example of this kind of task is telling it to write an essay.

But this is simply a theory. Therefore, it would help us learners so much if native speakers could access Gemini, a free and highly accessible AI, and tell it to write an essay on any topic to evaluate the accuracy of its language. I would love to know whether its choices of words and phrases are unnatural and if it writes like a native speaker or not. If its writing is good, it will be an invaluable source for us English learners to learn new vocabulary in context.

Below is an example of an essay question that you can copy and paste to check its writing:

Write an IELTS essay sample for this topic: Education for young people is important in many countries. However, some people think that the government should spend more money on education for the adult population who cannot read and write. To what extent do you agree or disagree?

Answer 1

Every creative writing AI I've used --including Gemini-- has pretty much perfect grammar and vocabulary. I've never noticed a mistake or a misplaced word or phrase. In my opinion, it's safe to assume everything is used naturally, and I would allow my own students to use AI-generated essays to discover new vocabulary, expressions and grammar structures. BUT ONLY THOSE THINGS.

You already know that AI invents information, but AI writing style is also worth mentioning here. It's verbose and flowery, on the formal side, and feels generated from a fill-in-the-blanks template. This might be useful for English learners preparing for standardized English exams like IELTS, but elsewhere, like in university, my feeling is English essays written in the style of AI would have a ceiling of B+, and get graded down from there based on the content*. It's high school-quality writing at best, and I wouldn't print one of that style even in a community newspaper*.

_{* I've never been a professor. I have never run a community newspaper.}

Answer 2

AI is more likely to provide you with naturally written answers when you tell it to do an open-ended task

AI is trained on what can already be found on the internet and most popular AI also searches the internet for information.

Think about what happens when you use a search engine. Search for one or two words and you'll get a broad range of results. The more specific your search, the less results you'll get. If you insist on exact matches to a particular string, that really limits the possible results.

So the same with AI - it has the whole internet to work from. The more fuzzy your request, the broader its knowledge base, and it can produce something that seems more original because it can splice together information from so many different sources. The more specific you get, the less it has to work with. With fewer resources to consult, its results are going to more closely resemble what has already been written.

If its writing is good, it will be an invaluable source for us English learners to learn new vocabulary in context.

The standard of grammar produced by AI is very high. It seems to strictly follow a style guide of some sort - notice, for example, the way ChatGPT punctuates quotations and formats lists. Always the same. Very consistent.

Because it follows a style, this helps to give the impression of originality. It will always take what is already available and rewrite that in its own style. This is how other sites can identify AI-written material with a reasonable degree of accuracy.

The issue that many see with the increasing use of AI for internet content - and this is already becoming apparent - is that the more AI-produced content appears on the web, the more AI content is readable by AI. Right now, when you ask AI to write an essay on Macbeth, it finds ones written by humans, but in a few years, the essays it finds could be overwhelmingly written by AI. And because those essays are already likely to be in its style, it won't change them too much. It will give you back the same essay. It's already happening with AI art - at first, it was producing original pieces from elements of human art, but now the internet is filling up with AI art the same pieces are starting to come up again and again.

From the perspective of an English language learner - you probably want to be able to speak and write in everyday styles as well as formal styles. AI will by default lean towards academic styles of writing, and that could become more and more the case as time passes.