Page MenuHomePhorge

Phorge mail may not meet Google's email sender guidelines
Open, Needs TriagePublic

Description

Background:

A large Phorge instance can easily generate more than 5000 messages a day from Maniphest tasks. Wikimedia's likely sends somewhere in the 10-15k range, at least. While not all of those go to Google-controlled domains, a good many of them must.

From the sender guidelines:

Make it easy to unsubscribe

Always give your recipients an easy way to unsubscribe from your messages. Letting people opt out of your messages can improve open rates, click-through rates, and sending efficiency. One-click unsubscribe makes it easy for people to opt out. If you send more than 5,000 message per day, your marketing and subscribed messages must support one-click unsubscribe.

To set up one-click unsubscribe, include both of these headers in outgoing messages:

  • List-Unsubscribe-Post: List-Unsubscribe=One-Click
  • List-Unsubscribe: <https://solarmora.com/unsubscribe/example>

When a recipient unsubscribes using one-click, you receive this POST request:

"POST /unsubscribe/example HTTP/1.1
Host: solarmora.com
Content-Type: application/x-www-form-urlencoded
Content-Length: 26
List-Unsubscribe=One-Click"

Learn more about List-Unsubscribe: headers in RFC 2369 and RFC 8058.

Mail from Maniphest currently includes a prominent link to email preferences, but does not set List-Unsubscribe or List-Unsubscribe-Post headers as far as I can tell.

I am guessing, but have not yet investigated, that setting a simple List-Unsubscribe would be relatively trivial, while the one-click version could be quite a bit more involved.

I also don't have a clear picture of what else besides tasks might send a lot of mail, or if it would be necessary to make changes in more than one place.

Event Timeline

  1. Would adding List-Unsubscribe: https://we.phorge.it/settings/panel/emailpreferences/ header be enough to solve this?
  1. Does this actually apply to "transactional" emails?
  1. Installs can check how much mail they're sending out by using bin/mail volume, to see if they need to worry about this. It just collects all mails created in the last N days and counts them by user.

Re: 1), the doc makes it sound like not, but I'm not entirely sure.

Re: 2), I'm personally very unclear. Will try to dig up more info.

Re: 3), Thanks - will give a shot.

This is promising, from the Email sender guidelines FAQ:

Do all messages require one-click unsubscribe?

No. One-click unsubscribe is required only for marketing and promotional messages. Transactional messages are excluded from this requirement. Some examples of transactional messages are password reset messages, reservation confirmations, and form submission confirmations.

Senders that already include an unsubscribe link in their messages have until June 1, 2024 to implement one-click unsubscribe in all commercial, promotional messages.

Bugmail clearly isn't "marketing and promotional", and I doubt a lot of users are deliberately flagging it as spam, so we might be in the clear on this requirement. (Though I guess I can imagine the absence of those headers tipping a sender over the edge in some circumstances.)

using bin/mail volume, to see if they need to worry about this. It just collects all mails created in the last N days and counts them by user.

side comment, I tried this on the Wikimedia server and after running for a while it ended like this:

@phab1004:/srv/phab/phabricator/bin# ./mail volume

Killed
In T15719#15276, @Dzahn wrote:
@phab1004:/srv/phab/phabricator/bin# ./mail volume

Killed

haha, makes sense :)

Try maybe with --days 2 (instead of the default 30).

  1. Would adding List-Unsubscribe: https://we.phorge.it/settings/panel/emailpreferences/ header be enough to solve this?

No. Per their FAQ,

Can a one-click unsubscribe link to a landing or preferences page?
Senders should follow the specifications for one-click unsubscribe that are defined in RFC 8058, and add List Unsubscribe headers to all outgoing promotional messages, as described in our Email sender guidelines.

Email admins should use RFC 8058 as the reference for implementing one-click unsubscribe. One-click unsubscribe links that link to a landing or other type of web page don’t comply with RFC 8058.

The link will be accessed by Gmail, the user is not shown anything. It's essentially an API call to unsubscribe the user. No auth cookies will be sent, either.

  1. Does this actually apply to "transactional" emails?

No but it's pretty unclear what they consider transactional. The guideline says "Marketing messages and subscribed messages must support one-click unsubscribe"; if you get notifications for a task, is that subscribed mail (because you subscribed one time in the past and then it keeps coming) or transactional mail (because another user has to do something to generate an email)? I think the common-sense reading is that "transaction" refers to something done by the user getting the email, not another user.

On the other hand, the FAQ says

What happens to messages that don’t meet the one-click subscribe requirement for promotional, marketing, and commercial messages?
We don’t automatically reject messages or mark messages as spam when they don’t meet the one-click unsubscribe requirements in our Email sender guidelines. However, unwanted messages that don’t use one-click unsubscribe are more likely to be reported as spam by recipients. An increase in messages marked as spam increases the chances that future messages from the same sender are delivered to spam.

We prioritize mitigation support for email delivery issues for senders that meet all requirements in our Email sender guidelines.

which is basically a long-winded version of "we won't do anything about it".

yes, but:

Do all messages require one-click unsubscribe?
No. One-click unsubscribe is required only for marketing and promotional messages. Transactional messages are excluded from this requirement. ...

and:

How is the distinction made between promotional and transactional messages
The distinction between promotional and transactional messages can vary depending on industry and applicable regulations. Message recipients, not Google, determine the nature of the messages they receive. ...

(My highlights)

This suggests to me that there isn't a third category - everything is either promotional or transactional.

I'd argue that none of our emails are "marketing" or "promotional" (or "commercial"), even if it's hard to claim they are "transactional".
But as long as 98%[1] of the users agree that our mails aren't commercial, we don't need the one-click solution.

I'm left wondering when a "list-unsubscribe" header is appropriate - looks like anything that would need it will also need the one-click?


To be clear, I'm against implementing the one-click solution on grounds of Product: when a user invokes the one-click, it's not clear if we should unsubscribe them from the particular object that created the email, or from all-emails-except-password-reset, or something in the middle; Any choice will make will make some users unhappy, probably loudly so.


[1] a number I just made up

This suggests to me that there isn't a third category - everything is either promotional or transactional.

I'd argue that none of our emails are "marketing" or "promotional" (or "commercial"), even if it's hard to claim they are "transactional".
But as long as 98%[1] of the users agree that our mails aren't commercial, we don't need the one-click solution.

From Google's FAQ, "Some examples of transactional messages are password reset messages, reservation confirmations, and form submission confirmations.", i.e. transaction messages are those messages that are part of some transaction initiated by a user somewhere else. Most Phorge emails are caused by a user subscribing to events and those types of emails would fall into the other bucket of promotional or marketing email I believe.

To be clear, I'm against implementing the one-click solution on grounds of Product: when a user invokes the one-click, it's not clear if we should unsubscribe them from the particular object that created the email, or from all-emails-except-password-reset, or something in the middle; Any choice will make will make some users unhappy, probably loudly so.

In some instances the workflow would be helpful. If I receive an email from a phorge about a bug report I now longer care about, it would be nice if hitting the unsubscribe button in my email client, unsubscribed me from further updates to that bug.

I'm left wondering when a "list-unsubscribe" header is appropriate - looks like anything that would need it will also need the one-click?

It's definitely a valid thing to do - it's defined in RFC 2369 section 3.2 and it doesn't require any specific behavior or semantics.

Here is an example from how Gmail shows the unsubscribe option for a mailing list with List-Unsubscribe but no List-Unsubscribe-Post (the link near the bottom):

Képernyőkép 2024-01-24 094259.png (410×589 px, 67 KB)

and I think clicking on it will just open the link in a new window. (All the examples I can find use an email address for unsubscribing, not an URL, and Gmail has special handling for those, so not 100% about this.)

With List-Unsubscribe-Post, you get a more prominent button like this:

Képernyőkép 2024-01-24 093517.png (103×480 px, 10 KB)

and if you click on it, you get a popup like this:
Képernyőkép 2024-01-24 095307.png (219×568 px, 19 KB)

also if you click on "Report spam", you get a dialog offering to unsubscribe instead:
Képernyőkép 2024-01-24 095449.png (261×568 px, 31 KB)

So old-style List-Unsubscribe is less useful but still somewhat useful.

To be clear, I'm against implementing the one-click solution on grounds of Product: when a user invokes the one-click, it's not clear if we should unsubscribe them from the particular object that created the email, or from all-emails-except-password-reset, or something in the middle; Any choice will make will make some users unhappy, probably loudly so.

Given how it's displayed on the Gmail UI, IMO unsubscribing from all emails or at least all emails of the given type (e.g. all Maniphest emails if it's a Maniphest notification) is the only thing that makes sense. If I mark an email as spam, and then use the unsubscribe option instead, I would not expect to continue getting identical email from all the other tasks.
No idea how (or if) other email clients handle the header, though.

(Having a one-click per task unsubscribe link somewhere in the email body would be nice, of course, and there it's easy to characterize what exactly the link will do. But for the header, I think something wider makes more sense.)

The Google FAQ has some very vague commentary on this:

One-click unsubscribe doesn’t automatically remove the recipient from all messages from the same sender. When you implement one-click unsubscribe according to RFC 8058, by using List Unsubscribe headers as described in our Email sender guidelines, the recipient can be removed only from the mailing list associated with the message. One-click unsubscribe lets you control which mailing lists recipients are removed from.

In T15719#15304, @Tgr wrote:

Given how it's displayed on the Gmail UI, IMO unsubscribing from all emails or at least all emails of the given type (e.g. all Maniphest emails if it's a Maniphest notification) is the only thing that makes sense. If I mark an email as spam, and then use the unsubscribe option instead, I would not expect to continue getting identical email from all the other tasks.
No idea how (or if) other email clients handle the header, though.

I agree, as a user of GMail, when I click the unsubscribe I pretty much want to never get any email from that sender ever again. Unsubscribing from a narrow subset isn't enough.

I also agree with this:

To be clear, I'm against implementing the one-click solution on grounds of Product: when a user invokes the one-click, it's not clear if we should unsubscribe them from the particular object that created the email, or from all-emails-except-password-reset, or something in the middle; Any choice will make will make some users unhappy, probably loudly so.

If we were to implement oneclick then I think the most appropriate behavior would be adding their email to a separate unsubscribed list that prevents us sending any notifications. Basically the all-emails-except-password-reset case. But given that there might be disagreement about the desired behavior, and the complexity of making it flexibly configurable, I think I'm leaning towards "don't do it unless we absolutely have to."

The one situation where I can see it being very useful to the user is if they lose access to their account and fixing email preferences requires them to log in. Say the phorge instance has multi-factor enabled and they lose their authentication factor, and are unable (or unwilling) to contact the site admins to regain access to their account. Now they are potentially stuck receiving notifications for an account they can't use and can't fix the preferences. In this situation a global one-click unsubscribe with no login requirements would be very nice to have.

Five months later it seems that we survived. Anything left to do?