On September 29, changesets 12097 and 12098 were committed to the Meta Trac repository for wordpress.org by Scott Reilly (Audrey Capital). These changes remove the Active Install Growth chart from the plugin repository’s “Advanced” section on individual plugins. The only explanation given by Scott is “insufficient data obfuscation.”
These changes are simply a reversion of the code that Alex Shiels (Automattic) added to create the section with active install data back in 2017. Alex’s ticket for improvements to the “Advanced” view (#3106) includes other additions to public plugin data that were never implemented due to concerns over the potential for them to be “gamed.”
Plugin Owners Respond
On September 30, Mark Zahra opened Trac ticket #6511 “Bring back the active install growth chart” to express its importance to plugin owners and to request improvements rather than deletion. Discussions were already taking place in Make WordPress #meta, on Twitter, and in Post Status #business, where Mark shared his ticket. At Post Status, a discussion was already happening about the challenges of entering the plugin repo and succeeding there since Vito Peleg had just launched Atarim’s freemium Visual Collaboration plugin at wordpress.org.
Abuse Comes to Light But Details Aren’t Clear Yet
For some time now, it’s been a common and reasonable assumption in the WordPress business community that some of the bigger plugin owners have very accurate data on their plugins’ usage as well as their competitors — enough to create a “leaderboard” to assess the effectiveness of sales campaigns or to see a buying opportunity in others’ declining install figures.
Abuse of the active install growth chart has been an issue in the past. Barış Ünver’s article, The Decline of Speed Booster Pack, touches on an incident where several hundred plugins had their install stats artificially inflated, mostly to mask the perpetrator.
Abuse seems to be an issue again but of a different type.
Josepha Haden Chomphosy joined the #meta discussion and said Marius Jensen was approximately correct in guessing “the active install growth could be (was being?) used to determine near exact numbers, making the intended obfuscation pointless.”
The data shared is always a bit obfuscated so that it’s harder to “game the system” — the same reason we don’t have running leaderboards for contributions.
Josepha haden Chomphosy <https://wordpress.slack.com/archives/C02QB8GMM/p1664559896577109>
Security or Privacy? Or Security and Privacy
In reply to Mark’s ticket, John James Jacoby affirmed there had been, in Michael Nelson’s words, a “closed-door security or privacy decision taken by a larger group.” The active install growth charts were not pulled on a whim by a single individual.
Sponsored by Awesome Motive, John is a full-time contributor to the Core, Documentation, and Meta teams. He says he independently reviewed the code generating the data for the active install chart which he noted “is outside of the Meta repository” with other “code responsible for keeping WordPress.org running.” None of this code is “publicly available,” but he “independently identified precisely why these charts were removed the way that they were,” and he “would not have made any different decisions had [he] been in on the decision-making process.”
John added he doesn’t “have any doubts that [improvement and not removal] is the long-term goal” for the active install growth chart. Further, he added that fast, private action was called for and “is not intended to hurt your community of users.” Instead, the intention is to “[exclude] many people to protect them from some people.”
Later, in #Meta, John specified two future options for the return of Active Install data:
Private by default, and optionally made public by plugin authors on a per-plugin basis.
A GUI could be invented to allow plugin authors to add usernames with access to the stats, similar to how it works for the support forums.
Why Active Install Data Matters
Active installs are the only way plugin owners can estimate the number of sites that are using their free plugin — and assess growth or decline over time. As Joost de Valk noted on Mark’s ticket, “The trends in this data are super important for plugin developers, as seen by the many many people that have responded to this in [WordPress] Slack.”
By monitoring many plugins across the repo, market trends can be assessed as well. Doing this work manually for even a single plugin is quite a chore. Iain Poulson, who is now at WP Engine, automated and improved on that process for Plugin Rank, which is now an Awesome Motive product. wpMetric offers a similar analysis. So does WPDesk’s Active Installs.
For some time now, it’s been a common and reasonable assumption in the WordPress business community that some of the bigger plugin owners have very accurate data on their (and their competitors’) plugins’ usage — enough to create a “leaderboard” to assess the effectiveness of sales campaigns or see a buying opportunity in others’ declining install figures. Josepha and John are clearly opposed to that happening, certainly in a public way.
Healthy Co-opetition Is Not a Leaderboard
Reflecting on some early BuddyPress history, John shared how that project’s “original primary concern” had been:
…revealing active install charts & graphs for all plugins & themes may not actually be healthy for the entire community, because it is impossible to resist using that single number to speculate about things those numbers may or may not imply – quality, security, performance, earnings, success, etc… and when that scales to inevitably comparing data across multiple plugins & themes, is any of that actually healthy, positive, or a real goal?
John also addressed Joost’s sharpest criticism that “Automattic has an unfair competitive advantage because they have access to more accurate stats,” while everyone else is now fully in the dark. (Joost made this claim on several channels, and it’s a continuing thread in #meta.)
I will go one step farther and say, that if a goal with any data is to be fair to each other with it, that includes a responsibility to serve up the same data with the same interface to everyone, and to prevent people from accessing it in any way that is unintended or unfair.
JJJ <https://meta.trac.wordpress.org/ticket/6511#comment:5>
…which is essentially what has happened, here.
Open to all — for all who want to be open — seems to be the way the issue will be resolved, probably to a near consensus.
But will enough data be available to satisfy those who feel like “Second-Class Third-Party Developers,” as Iain Poulson put it in his WP Trends newsletter this morning?
The deepest issues will likely remain divisive — perceived competitive advantages, the definition of healthy competition in an open source ecosystem, and who gets to referee these things.
Josepha emphasized that “suggestions are welcome for how to get some data for you all while doing our best to stick with a ‘co-opetition‘ mindset.”
Reply on Mark’s ticket if you have helpful contributions to make toward that goal.
Worthwhile questions that (re-)emerged in these events:
- Can active install data collection be improved and explained sufficiently to indicate what it measures and how accurate it is — without revealing too much information that could be abused? What is the best balance of care, openness, and awareness that abuse will always happen, sooner or later?
- Can decisions like pulling active install data (along with the people and processes involved in those decisions) be more transparent and publicly defined to avoid the confusion, injury, and distrust that often results? As a matter of internal public relations, could there be people tasked with explaining delicate issues that can’t (immediately) be explained fully in a public way? (As opposed to hints and guesses in Slack.)
- Same question for WordPress.org #forum policies on moderation, reviews, replies to reviews, and other things plugin owners care about. Without being too open (due to the risk of “gaming”), can clearer guidance be given to onboard new plugin owners at .org? E.g, how reviews are validated, why they may be removed, and how to appeal their removal? Or how plugin owners and their support staff should avoid or deal with being put under moderation?
- What are winning growth strategies for plugin owners that aren’t dependent or overly focused on single measures of success — whether they use the .org repo or not? What are the best ways to go all-in with .org? Or is that simply a mistake?
- How does/doesn’t the inclusion of all wp.org plugins in the new wp.com marketplace affect active install stats? Are these combined or separate numbers? What data is available from wp.com to free and commercial plugin owners about their installs there?
