After the 2008 financial crisis, many skeptical investors began to bypass Wall Street analysts and turn to stock analysis and advice sites, like the Motley Fool and Zacks Investment Research. Seeking Alpha was only a few years old at the time, but the site would become one of the largest platforms of its kind, with more than 18,000 authors producing articles about stocks that often spike trading for the companies mentioned.
But with its authors quickly adopting generative artificial intelligence in their writing, would the platform continue to provide its readers with the same high-quality articles? Also, how would AI use affect author productivity?
With the development of generative AI, the worry is that over time, all of these viewpoints will converge to one view—the bot view.
“Seeking Alpha has grown to be the No. 1 platform for investors based on each article providing a unique thesis to the reader,” says Harvard Business School Assistant Professor Yuan Zou. “With the development of generative AI, the worry is that over time, all of these viewpoints will converge to one view—the bot view.”
As most industries grapple with the best ways to harness AI—and when not to use it—Zou and her coauthors explore the risks and opportunities gen AI tools present for analyzing financial data in the working paper, “Generative AI Use by Capital Market Information Intermediaries: Evidence from Seeking Alpha,” released in September. Content has proliferated with the rise of gen AI, but the research finds that AI generally provides inferior information to investors. However, it can be considered useful in providing greater exposure for stocks.
AI's lower-quality investment content
Zou has long explored how the quality of financial information affects investors’ decisions. Then came the launch of ChatGPT three years ago.
“On the one hand, gen AI is very powerful in its ability to generate text, but on the other hand, it can be untrustworthy and may lack people’s own opinions,” she says.
Based on those risks, Seeking Alpha equated the use of AI to plagiarism and banned contributors from using ChatGPT to write articles on its platform starting in late 2023. That gave Zou an opportunity to analyze the effects of using gen AI on the platform before and after the ban with her fellow researchers—HBS predoctoral fellow Chenyang Ma, now a doctoral candidate at Duke University, and Boston College professors Mark Bradshaw and Benjamin Yost.
In terms of the quality of articles, the researchers found:
Lower trading volumes for stocks covered, with smaller average returns compared with human-written articles.
Fewer comments from readers. The articles were also less likely to be recognized as “editor’s picks.”
“We interpret these results as showing, on average, that AI-generated articles are less impactful and have less informative content than human-written articles,” Zou says.
After all, she says, it’s hard for AI to match the original creativity of articles created by humans.
“When I first started looking into this, I was impressed by the unique insights of articles published on the platform,” Zou says. “In addition to occasional personal insights from company visits or product use, a large share of the articles focused on detailed fundamental analyses, offering a mix of perspectives that went beyond simple news summaries.”
One article about the limitations of Airbnb in foreign markets, for example, was based on the author’s own experiences traveling internationally. “This personal experience is something very special that makes articles more convincing to read, and at least as of now, gen AI doesn’t have the ability to do this.”
The tradeoffs of quantity and quality
How does AI use affect author productivity? Following the initial launch of ChatGPT in November 2022, the share of AI-generated articles rose sharply from less than 2% in pre-ChatGPT months to a peak of 13% of all articles in October 2023, then fell to about 4% after Seeking Alpha enforced the no-AI policy.
Zou and her coauthors also classified Seeking Alpha contributors into AI adopters and non-adopters and compared changes in productivity for the two groups over time. Following the launch of ChatGPT, AI adopters experienced a sharp productivity boost relative to their peers, publishing 55% more articles—about two additional pieces per month—and expanding coverage to more new firms. Much of this gain, however, dissipated once Seeking Alpha began enforcing its no-AI policy.
Based on conversations with content creators, Zou and her colleagues found that some writers used ChatGPT judiciously, perhaps to summarize a piece of research or craft part of an article based on bullet points they provided, rather than asking gen AI to draft an entire article.
To get a sense of the quality of these articles, the researchers looked at the articles flagged by the AI detection software written by contributors with more than 18 months of experience with Seeking Alpha before the launch of ChatGPT. They found little difference in the quality produced by these authors compared to the human-written articles.
“We cannot observe directly how people are using AI,” Zou says. “But we conjecture that these authors care about their reputation on the platform and know what good quality means and are using AI differently.”
By contrast, they also looked at authors who joined the platform within the first three months of ChatGPT’s launch, figuring that at least some authors were looking to opportunistically crank out articles to make money. These authors produced the lowest-quality articles.
With AI, firms get attention
Although the researchers found that AI articles are less informative than human articles, they also found that AI articles contain some information relevant to investors. “So the AI articles, even if they are of lower quality, can help generate interest in companies,” Zou says. “Even low-quality articles are better than no articles.”
The researchers documented a rise in the monthly share of firms receiving coverage following the launch of ChatGPT, which then receded once Seeking Alpha began enforcing its no-AI policy. To assess the broader capital market effects of this AI-driven expansion in coverage, they analyzed bid–ask spreads, illiquidity, and the speed of price discovery around firms’ earnings announcements.
At least for now, there’s no substitute for human creativity.
Their findings indicate improvements across all three dimensions—narrower spreads, reduced illiquidity, and faster price discovery—after ChatGPT’s launch. However, these gains dissipated once AI use was curtailed and coverage declined. While AI generally results in weaker content, those articles can be a useful supplement to human-generated articles, according to the findings.
“Gen AI might be a low-cost tool to help information intermediaries cover a wide range of stocks and aid in the price discovery process,” Zou says. At the same time, she says, “at least for now, there’s no substitute for human creativity.”
Image: HBSWK, created with assets from AdobeStock
