Page MenuHomePhorge

Voluntary Usage Survey App
Open, Needs TriagePublic

Description

I had a thought about gathering information about installs in the wild.

In Phabricator, there was some talk about an opt-in "call home" feature that would collect statistics and system info (Number of active users, version of php/git...) to provide inputs for making support decisions ("can we drop support for git 1.4?"). The "call home" feature has some major downsides, including getting InfoSec people alarmed.

Here, I'd like to talk about an alternative approach:

Have an app installed on this instance, and ask users to manually report the relevant information:

  • Organization type and (optionally) name
  • Contacts - phorge.it users that are involved in the install
  • Branch used (stable/master/phabricator/very-old)
  • Versions of PHP, git, other stuff. Linux distribution name
  • rough numbers (users, tasks, revisions...)
  • how much code they changed in the core, and how much extensions code they have.
  • probably more stuff

The information would be exposed only to Community Members.

This would allow us to have some idea on what is being used, and allow each install to decide how much information they wish to share.

What do we think? Would installs maintain their information? Would we get enough information to make good decisions?

Event Timeline

Uhm. Maybe we can create a script or a frontend button to auto-answer most of these questions.

Personally, I'd be willing to manually fill out a survey. I'd heavily oppose anything that phones home in any capacity.
An app/script which generates some report which can be manually copy-pasted sounds reasonable.

In general I am advising against any operative measures to "drop features or support" based on surveys. There can potentially be thousands of installs which might never show up on your/our radar.

Premising that a CLI script could be not enough, probably better a frontend button so to be able to know if it's mod_php or PHP-fpm etc.

How would we as phorge upstream use the info? Would it inform development? Be just informational?

For support of old software versions is it sufficient to say that we will support whatever the upstream devs still support? E.g., as long as git version X.Y is still receiving security updates from either its core team or through something like a RHEL distribution, we will continue to guarantee support for it. But like, PHP 5.X which Phab nominally still supports has been EOL since 2019. I don't think that's compatibility that the project gains anything from continuing to support

How would we as phorge upstream use the info? Would it inform development? Be just informational?

Fair question :)

The main use-cases I was thinking about:

  • Occasionally, a new feature of PHP or Git is enticing for us to use, but would break older versions. Having some idea about what is actually in use will be helpful in deciding if we should adopt it. Now, we err on the conservative side (which is why we still support php 5)
  • Performance optimizations: We want Phorge to be fast, so sometimes we make some wacky optimizations. But without real-world data about, e.g. "how deep do the sub-tasks graphs get", it's hard to optimize the right thing.

    For some of our features - like Workboards and Audit and Phriction we hardly use at all, and some installs use very heavily. For others (Slowvote? Countdown? Calendar? Fund?) maybe nobody is using at all.
  • It's sometimes useful to say "hey, anybody here manages the WMF install? I have some questions"

One of my favourite updates in my workplace was to store counts of feature usage per month in a table. Move forward 3 years, and whenever we need to update some code in a feature, we check for it in the table, and if it's not been used in 3 years we just scrap the feature.

So I would do something similar here. Have Phorge store anonymous usage data, and then every 6 months have an administrator notification giving them the option of emailing in the usage data (showing what will be sent), or declining for another 6 months.

Downside is this will involve a new table.

I think that is complex because some admins are not well active on their platforms, so as said before it can be underdimensioned

There is a significant amount of Phabricator dark matter out there - companies/people using the software, it works well enough, not really easy to know they exist or anything about their usage. I'm sure at least some of them have moved to Phorge. Automattic/wordpress.com have moved to Phorge and I wasn't even aware that they were using Phabricator before that. This is despite the fact that I did a pretty extensive amount of research to identify every company using Phabricator back in ~2019 as part of my work for Wikimedia, with the goal of reaching out and trying to organize an informal Phabricator users group. We had the idea that the various corporate users probably had good reasons to be collaborating and at least talking to each other since most of them were not active in the upstream project. Anyway, that never really panned out, although it did trigger a flurry of interest and some ongoing discussions via email (maybe even one video meeting but I can't remember the details now.)

I guess this is just to say that many Phab/Phorge users are pretty slow adopters and fairly busy focused on other goals so the response rates won't be really high, however, I'm sure there are some details still to be learned that could help inform decisions going forward.

When you have questions about the Wikimedia install, my past experience is probably still relevant for the most part, along with @aklapper who has been involved with Phab @ Wikimedia since before I was even hired.

I do find it interesting that different orgs use phabricator so very differently. Wikimedia probably has the largest Maniphest database by far (maybe Facebook is some competition there but I never learned much about the scale of their internal Phabricator and i don't know if they still use it extensively or at all). A few places just use Differential and that's pretty much the end of it. Some shops are all about mercurial or even (ghasp) svn. It's really amazing how versitile the project really is and how well it can be tailored to specific use-cases. Really one of the most adaptable collaboration tools around anywhere.