Many books have emerged about the virtues and vices of AI. One is The ChatGPT Revolution by Donna McGeorge. She used Chat GPT to write it and promotes it as a good way to correct grammar and spelling. However, I noticed several grammar issues in the book. That’s not unusual, I see obvious grammar mistakes in published books quote frequently. This seems to happen more now than in the past. This is why this time I am comparing how ChatGPT 3.5 (the free version) and Google’s Bard attempt to correct grammar. ChatGPT does not give reasons for any of its changes and Bard does.
What I did
I trialled one passage of academic writing describing a procedure which contained three grammar errors. The prompt was “Correct the grammar in this:” followed by a paragraph of text. I then conducted another trial with a simple passage with many grammatical errors using the same prompt.
What ChatGPT did
Trial 1: This AI reproduced one of the obvious original grammar errors and correctly altered two other ones. It split a long paragraph into two shorter ones and altered a long sentence, making it two shorter ones, which was appropriate. It also substituted synonyms for three verbs in the original (which it was not asked to do). ChatGPT changed the order of ideas in the last part of the text.
Trial 2: The AI corrected all grammatical errors.
What Google Bard did
Trial 1: This AI gave a list of some of the alterations made and a reason for each. It said it removed commas in one particular sentence when it actually made no change. It added a comma in another sentence when it was ungrammatical to do this. It made fewer changes than ChatGPT.
Trial 2: All grammatical errors were corrected. Bard corrected two errors which were not included on its list of explanations for its corrections. It also “explained” it took an action which it did not actually take.
What I learned
Both types of AI took unexpected actions when I asked them to correct the grammar of the complex academic text about a procedure. For the simpler text, both corrected all the basic grammatical errors. In my experiment ChatGPT did not offer any explanations for the changes that it makes to a text. While it seems a good idea for an AI to give explanations, this function in Bard is not working well. It is interesting to compare how the two versions of AI appear to the potential user. Bard is presented as an experiment and the user is thanked for participating. ChatGPT 3.5 was released with a very successful media campaign and now ChatGPT 4 is for sale by subscription.
(If you want to read about my experiment on summarising with ChatGPT, you can find it here.)