I‘ve tweeted nearly 5,000 times in my life, which certainly feels like a lot.
But last week I did something on Twitter I’ve never done before: I used artificial intelligence to help me decide what to tweet. More specifically, I used a service called Post Intelligence, which recommended links and photos to post, suggested the time of day I should post to get the best engagement, and even estimated the popularity of my tweets before I sent them based on the language I used in the tweet.
To do this, Post Intelligence, which used to be called MyLikes and has raised $11 million from Khosla Ventures, uses algorithms similar to those Twitter and Facebook use to determine what you see in your feed.
The company analyzed my Twitter account to determine the topics I tweet about most and calculated which of those topics also perform well with my followers. Then the AI went out and found tweets about those topics that were performing well on Twitter and suggested I share them, too.
The results: The AI-suggested tweets performed better than my normal ones. Kinda.
I sent 24 tweets over a span of 9 days, 12 that included media suggested to me by Post Intelligence and 12 that including content I found on my own. (This excludes a lot of replies to tweets that I sent, and the three times I tweeted my own stories from Recode.)
The AI-powered tweets received an average of 7.2 favorites and 2.2 retweets apiece. My “original” tweets received 5.0 faves and 1.5 retweets, on average. On the surface, the AI appeared to be a noticeable help.
I also added 70 new followers in the 9-day stretch; I had averaged just 118 new followers per month in the six months prior.
But the engagement data is skewed: The AI suggestions led to my most popular tweet of the period, this gem about BBC girl and how she would be a badass reporter (or badass anything, from the looks of it), which generated a whopping 33 favorites and 11 retweets.
Without that outlier, my AI-suggested tweets averaged 4.8 faves and 1.4 retweets, on average, almost exactly the same as the tweets I sourced on my own.
I have a theory for why the AI didn’t significantly improve my tweeting: The stuff the AI recommended to me was the same type of stuff I already see and share on Twitter. Post Intelligence was using an algorithm to personalize content for me; Twitter does this, too. So while I certainly came across specific items I might not usually see in my feed, they were the same kind of items I am used to.
There are still ways the AI could have helped, though it’s impossible to know if it did. For example, I tweaked the language on a number of my tweets to try and make them more popular based on Post Intelligence’s prediction score. I have no idea how those tweets would have performed had I simply stuck with my original language.
Same thing goes for posting tweets at certain times recommended to me by the algorithm; it’s impossible to know how they would have done had I posted them when I wrote them instead of scheduling them for later.
But it does seem clear that there are certain things a computer just can’t know, like offline social dynamics.
Here’s an example: This tweet poking fun at my colleague, Jason Del Rey, performed the same as this other tweet about Donald Trump golfing, even though the Trump tweet was predicted to perform better.
This didn’t surprise me. I know that my colleagues love to tease each other online, and mentioning Jason in a tweet was bound to generate at least a few faves. Golfing, on the other hand? Not really a sport my followers tend to care about, especially if Trump is involved.
Eventually, I imagine the AI will learn these kinds of nuances, though it obviously hasn’t figured it out just yet. And in defense of Post Intelligence, my sample size here is probably much too small. These algorithms learn over time. Nine days and just 24 tweets is not really sufficient.
I still believe that AI can improve our tweeting, though it feels early. There seems to be an obvious opportunity to build a tool like this inside of Facebook or Twitter, especially for new users. When you create a profile for the first time, these services try and help you connect with or follow others you may know. But getting people to take that plunge and actually post something is still a challenge.
Post Intelligence CEO Bindu Reddy agrees. “[It’s better] for people who don’t know what to say,” she explained. If social networks could encourage people to post by using AI predictions to help them craft something successful, it might help alleviate some of the fear of hitting that publish button.
I want to keep using Post Intelligence to see if a larger sample size changes my performance (or my mind). You can try it here as well. If I learn anything fun, I’ll be back to share it with you all.
This article originally appeared on Recode.net.