Traffic to programmer help site Stack Overflow has been dropping “by an average of 6% every month since January 2022, and was down 13.9% in March,” according to stats from web traffic analysis company SimilarWeb, a likely reason being increased use of AI coding assistants such as GitHub Copilot and ChatGPT.
Stack Overflow, a programmers’ question and answer site, has long been among the first stops for countless developers when stuck on a coding issue or forgetting the right syntax or arguments for a common task. Pop-up help in editors like Visual Studio Code is another way of getting this kind of assistance, and the quality and scope of this coding help has greatly increased in the last couple of years, thanks to use of AI and the arrival of services such as Copilot, AWS CodeWhisperer, and most recently, Google Bard.
Copilot first entered public preview in June 2021 and was generally available from June 2022. OpenAI’s ChatGPT, which shares some technology with Copilot, can also write code on request. CodeWhisperer was first previewed in June 2022, and Google introduced code generation with Bard last month.
Website traffic fluctuates for many reasons, not least constant changes in search engine algorithms, and it is hard to match up exactly how increased use of AI coding corresponds to declining use of Stack Overflow. Stack Overflow publishes its own traffic statistics and analysis of these in conjunction with Web Archive does not altogether match up with the SimilarWeb stats. That said, it does seem notable that today the site is reporting 5.6m visitors per day and 3.5k questions per day, whereas in June last year that was 7.4m visitors and 5.5k questions.
DevClass addressed this exact question with Stack Overflow CEO Prashanth Chandrasekar in November last year, when he made the point that developers need to understand the code they are writing. “At some point you’re going to need to know what you’re building. You may have to debug it and have no idea what was just built, and it’s hard to skip the learning journey by taking shortcuts,” he told us.
The counter point is that a developer might copy and paste code from Stack Overflow without understanding it either – though the crowdsourcing effect means that popular code snippets get a lot of eyes inspecting them for quality. The same will not be the case for more niche questions, many of which may not be answered at all.
Chandrasekar has now publicly acknowledged that AI coding poses a challenge for Stack Overflow. Last week he posted a blog about a 10% or 58 employee reduction in the workforce, and made an undertaking to launch “AI/ML focused offerings” in the coming months. In a separate post Chandrasekar writes, “we’ve got a dedicated team working on adding GenAI to Stack Overflow and Stack Overflow for Teams”, the latter being a private, paid-for version of the site.
Perhaps as one might expect, Chandrasekar highlights limitations of AI coding. “One problem with modern LLM systems is that they will provide incorrect answers with the same confidence as correct ones, and will ‘hallucinate’ facts and figures if they feel it fits the pattern of the answer a user seeks,” he states. A “symbiotic relationship between humans and AI that ensures the ongoing relevance of community-driven platforms like Stack Overflow,” he insists.
According to its own stats, there are 24m questions on Stack Overflow, 69 percent of which are answered. That forms an invaluable programming resource that is hard for competitors to emulate; yet in a short time AI technology is eroding that advantage. Even if Stack Overflow successfully adds some AI magic of its own, it will be hard for it to match the convenience of coding assistance right there in the editor, informed by the developer’s own context and coding patterns. The consequences of that for code quality are open for speculation, but the trend will be hard to resist.