Page MenuHomePhorge

Voluntary Usage Survey App
Open, Needs TriagePublic

Description

I had a thought about gathering information about installs in the wild.

In Phabricator, there was some talk about an opt-in "call home" feature that would collect statistics and system info (Number of active users, version of php/git...) to provide inputs for making support decisions ("can we drop support for git 1.4?"). The "call home" feature has some major downsides, including getting InfoSec people alarmed.

Here, I'd like to talk about an alternative approach:

Have an app installed on this instance, and ask users to manually report the relevant information:

  • Organization type and (optionally) name
  • Contacts - phorge.it users that are involved in the install
  • Branch used (stable/master/phabricator/very-old)
  • Versions of PHP, git, other stuff. Linux distribution name
  • rough numbers (users, tasks, revisions...)
  • how much code they changed in the core, and how much extensions code they have.
  • probably more stuff

The information would be exposed only to Community Members.

This would allow us to have some idea on what is being used, and allow each install to decide how much information they wish to share.

What do we think? Would installs maintain their information? Would we get enough information to make good decisions?

Event Timeline

Uhm. Maybe we can create a script or a frontend button to auto-answer most of these questions.

Personally, I'd be willing to manually fill out a survey. I'd heavily oppose anything that phones home in any capacity.
An app/script which generates some report which can be manually copy-pasted sounds reasonable.

In general I am advising against any operative measures to "drop features or support" based on surveys. There can potentially be thousands of installs which might never show up on your/our radar.

Premising that a CLI script could be not enough, probably better a frontend button so to be able to know if it's mod_php or PHP-fpm etc.

How would we as phorge upstream use the info? Would it inform development? Be just informational?

For support of old software versions is it sufficient to say that we will support whatever the upstream devs still support? E.g., as long as git version X.Y is still receiving security updates from either its core team or through something like a RHEL distribution, we will continue to guarantee support for it. But like, PHP 5.X which Phab nominally still supports has been EOL since 2019. I don't think that's compatibility that the project gains anything from continuing to support

How would we as phorge upstream use the info? Would it inform development? Be just informational?

Fair question :)

The main use-cases I was thinking about:

  • Occasionally, a new feature of PHP or Git is enticing for us to use, but would break older versions. Having some idea about what is actually in use will be helpful in deciding if we should adopt it. Now, we err on the conservative side (which is why we still support php 5)
  • Performance optimizations: We want Phorge to be fast, so sometimes we make some wacky optimizations. But without real-world data about, e.g. "how deep do the sub-tasks graphs get", it's hard to optimize the right thing.

    For some of our features - like Workboards and Audit and Phriction we hardly use at all, and some installs use very heavily. For others (Slowvote? Countdown? Calendar? Fund?) maybe nobody is using at all.
  • It's sometimes useful to say "hey, anybody here manages the WMF install? I have some questions"

One of my favourite updates in my workplace was to store counts of feature usage per month in a table. Move forward 3 years, and whenever we need to update some code in a feature, we check for it in the table, and if it's not been used in 3 years we just scrap the feature.

So I would do something similar here. Have Phorge store anonymous usage data, and then every 6 months have an administrator notification giving them the option of emailing in the usage data (showing what will be sent), or declining for another 6 months.

Downside is this will involve a new table.