“Within 6 months of ChatGPT’s release, activity on Stack Overflow decreased by 25% relative to its Russian and Chinese counterparts, where access to ChatGPT is limited, and to similar forums for mathematics, where ChatGPT is less capable.”
1 Like
Unfortunately they could not measure if the activity that remained was more sophisticated than before ChatGPT. It doesn’t strike me as bad that those kinds of forums have less activity, given how obvious a lot of the questions and answers are.
Having used stack overflow in the past and present, a lot of the answers merely redirect you to some documentation or feature, which you then evaluate if it solves your problem or not, and for those use cases, a LLM is perfectly usable and often times faster and more precise than filtering through multiple posts to find your answer.
In the end if “basic” activity is reduced, it means that people are having their problems solved, and for that we should all use the best tool for the job(if on the job 
) regardless if it’s “AI” or not.
4 Likes
It seems they did try to measure this via a couple of proxies but, yes, not directly:
“We observe no large change in social feedback on posts, measured using votes, nor in the experience composition of posting users following ChatGPT’s release. These results suggest that average post quality has not changed, nor has ChatGPT replaced only the new and inexperienced users.“
3 Likes