January 25, 2023

GPT-3 vs. Human: Who can write a better article on community building?

I wanted to see how GPT-3 performed when I asked it to rewrite my latest article on community building. Here are my notes.

ChatGPT is all the rage on the Internet these days. Finally, we have an AI Assistant that is actually intelligent. People are astounded by it’s capabilities of comprehension and writing.

So, I decided to see how it performed when I gave it the same prompt as my last article’s title:

Unintuitive things I’ve learnt about building an online community

I used Notion’s AI Assist as my interface (which is based on the same GPT-3 API like ChatGPT - meaning that Notion AI and ChatGPT have the same brain). Here’s the short article that it produced:

Building an online community can be a tricky proposition. Even after years of experience, there are still lessons to be learned. Here are some of the unintuitive things I've learnt about building an online community.

1. It’s more important to focus on the people who are already part of the community than it is to focus on attracting new members.
2. The most successful communities are those where members feel like they are part of something bigger than themselves.
3. While it can be tempting to focus on the “big numbers” of how many people are in the community, it’s more important to focus on quality over quantity.
4. The key to keeping people engaged in a community is to create a sense of belonging.
5. It’s important to be flexible and willing to adapt to meet the needs of the community.
6. It’s important to make sure that everyone feels included, regardless of their background or interests.
7. To keep the community vibrant and active, it’s important to create opportunities for members to interact with each other.
8. It’s important to foster an environment of mutual respect, where everyone is respected and differences are celebrated.
9. It’s important to create a safe and welcoming space where people can be open and honest.
10. Finally, it’s important to remember that building an online community is a long-term commitment, not a short-term project.

I observe 3 things of note here:

  • It’s not wrong
  • It’s not novel
  • It has no limits
Created using Playgroundai.com - an image generation app that uses GPT-3's sibling API

It’s not wrong

One of the most common criticisms against GPT-3 is how it can be confidently wrong. But I didn’t find it to be true in my use case.

Not only did it write meaningful sentences without any grammatical errors, it’s also not wrong about anything it says about community building. Atleast not objectively.

Sure, you could argue against the point about making sure that “everyone feels included, regardless of their background or interests” but that would be up for debate. I can understand how it may be wrong when you ask it technical questions like debugging a piece of code or a trivia question but it is remarkably correct for non-technical topics.

It’s not novel

Even though it’s not wrong, nothing it says about community building is particularly novel. It feels like something an “SEO expert” would write to cover target keywords without having any real insights into the subject.

That’s not surprising to me because novelty comes from experience.

Great writing includes anecdotes and stories. It gives you contradictions and transformation. It is created by people who do things, solve challenges and in the process, discover something interesting to say.

Just like a bad SEO content strategist, the AI obviously cannot have any insights into the subject at hand. It has read a bunch of content on the Internet and can only regurgitate it in different forms.

It has no limits

This is the part that scares me but also holds the most promise.

This AI can generate correct, non-novel content infinite number of times. The excerpt above took me less than 3 seconds to generate. But that’s not it. With another 2 seconds, I was able to get it to expand upon the first line.

Prompt: “Why is it important to focus on the existing community members?
GPT-3: “It is important to focus on the existing community members because they are the foundation of any successful online community. They are the ones who are already familiar with the platform, the content, and the people. They are the ones who have already built relationships and trust with other members. Focusing on the existing community members is also essential for expanding the reach of the community. They are the ones who will help bring in new members, as well as promote the community to their friends, family, and colleagues. It is also important to focus on existing community members because they can provide valuable feedback and insights into how the community can be improved. By engaging with existing community members, it is possible to gain valuable insights into what works and what doesn’t work for the community.”

Remarkably correct.

And the limiting factor here was actually my own ability to think and write the prompt.

That’s what scares me. If you’ve hated the "SEO-fication" of search engine results till now, expect a tsunami of such surface-level content in the future. Especially around non-technical disciplines like community building, sales, marketing or philosophy.

But on the flip side, I can also sense an unprecedented opportunity here. The ability to generate intelligible content, at will, in an instant, feels like a superpower. Like something out of a science fiction movie.

Combine with this the fact that this is only version 3 of the GPT model. We’ll soon have GPT-4, GPT-5 and so on. One can’t even imagine how those would be even better.


Prompt: “What would Steve Jobs say about building an engaged community?”
GPT-3:

"If Steve Jobs were alive today, he might have some interesting thoughts on building an engaged community. After all, he was a master at gathering people around a shared vision and inspiring them to take action. He was known for his ability to bring people together and make them believe in a cause. Jobs believed that having a passionate, engaged community was essential for success and he often spoke about the importance of creating a sense of belonging. He might have said something like this:

"Building an engaged community requires creating an environment where people feel included and valued. It's about creating an environment where people can come together and share ideas, collaborate, and create something bigger than themselves. When you build an engaged community, you create a powerful force that can help you achieve your goals."”