[tor-project] disruption in email services

Mailing Lists tor-project
1

Hi everyone,

We've had reports of increasing delivery failure rates at other
providers, particularly gmail, from multiple parties in the course of
the last few weeks / months.

It might be worth falling back to an alternative provider if you have
access to one, for your @torproject.org email, otherwise you might miss
some email. Inversely, if you have a hard requirement to reach
@gmail.com people right now, you may want to use another provider or use
a gmail account directly.

We're working on emergency mitigations for this problem. I should come
up with a proposal tomorrow and work should start next week.

In the meantime, status.torproject.org has been updated. Feel free to
circulate outside of this space, since many people here might not be
able to read this message in the first place, of course.

a.

···

--
Antoine Beaupré
torproject.org system administration

2 Likes
2

SPF record :wink: :wink: :smiley:

Cheers!
David

···

On 30 Nov (15:12:52), Antoine Beaupré wrote:

Hi everyone,

We've had reports of increasing delivery failure rates at other
providers, particularly gmail, from multiple parties in the course of
the last few weeks / months.

It might be worth falling back to an alternative provider if you have
access to one, for your @torproject.org email, otherwise you might miss
some email. Inversely, if you have a hard requirement to reach
@gmail.com people right now, you may want to use another provider or use
a gmail account directly.

We're working on emergency mitigations for this problem. I should come
up with a proposal tomorrow and work should start next week.

In the meantime, status.torproject.org has been updated. Feel free to
circulate outside of this space, since many people here might not be
able to read this message in the first place, of course.

--
l2kj7J1p2QON/ZOIT8ZzZhbo5MU1g3dQ6yKqBtMmxhM=

3

Status update, day three:

We now have "soft" SPF and DMARC records on all mail servers and DKIM
signatures on the three major mail servers.

This will probably impact users currently sending mail from gmail and
riseup, as their reputation will suffer from not being in the allow
lists. If this is a problem, an `include:riseup.net include:google.com`
mechanism could be added to the top-level SPF policy.

Next step is to finish the DKIM deployment (#40989) and then make the
DMARC and SPF records "hard", which will happen once we are more
confident this will cause more good than harm.

Work on the mail exchanger may also start soon, alongside other
mitigations for problems we may encounter from here on.

Do let us know here if you encounter problems or improvements. As usual,
you can follow our work in GitLab in:

Comment there or file new issues in:

If all fails, contact us at:

Onwards!

···

--
Antoine Beaupré
torproject.org system administration

1 Like
4

Status update, week two: TL;DR: situation mostly fixed. Next work in 2023.

Things seem to have gone back to normal. In fact, I've been a little
hesitant in sending this email, because I'm worried that people will
suddenly remember all those issues they've been having with email
forever and remind me how everything *else* is broken, but dang, it
looks like we kind of made it?

As far as I can tell, emails send through the submission server have
stopped bouncing (#40640 closed!), authentication problems for messages
actually delivered at gmail have stopped (#40765 closed!) and other
outright gmail bounces seem to have also stopped (#40959 closed!).

That is a *major* improvement, in such a short time span. There are
still issues with our mail services, of course... In particular, email
forwards are still in somewhat of a gray zone. They are *technically*
not allowed, especially for senders with a hard SPF policy, but that was
already a problem before. We *may* see delivery problems for mails sent
*internally*, but, again, probably not worse off than what we were
before.

And, for the record, we are well within my time estimates for the labor
on this. We still have a *lot* of work to do to complete (even a plan
for) the mail infrastructure, but I'm going to call this one as fixed
for now, as the next steps require further thinking, and even more major
architectural changes. So I'm going to reserve those for that
non-proverbial next year.

In the meantime, I'll work on another one of my famous
proposal. Apparently, ChatGPT can generate those for me now, so maybe
I'll give that a shot.. :wink:

"Make a proposal for torproject.org mail services in the style of
anarcat's TPA-RFC" please?

A.

···

--
Antoine Beaupré
torproject.org system administration

1 Like
5

A huge thank you for jumping up and making changes when we noticed a downturn in delivery on our fundraising emails during a critical time. Many rounds of applause and beverages on me to you and the TPA team the next time we meet in person--thank you!!

Al

···

On 12/15/22 12:32 PM, Antoine Beaupré wrote:

Status update, week two: TL;DR: situation mostly fixed. Next work in 2023.

Things seem to have gone back to normal. In fact, I've been a little
hesitant in sending this email, because I'm worried that people will
suddenly remember all those issues they've been having with email
forever and remind me how everything *else* is broken, but dang, it
looks like we kind of made it?

As far as I can tell, emails send through the submission server have
stopped bouncing (#40640 closed!), authentication problems for messages
actually delivered at gmail have stopped (#40765 closed!) and other
outright gmail bounces seem to have also stopped (#40959 closed!).

That is a *major* improvement, in such a short time span. There are
still issues with our mail services, of course... In particular, email
forwards are still in somewhat of a gray zone. They are *technically*
not allowed, especially for senders with a hard SPF policy, but that was
already a problem before. We *may* see delivery problems for mails sent
*internally*, but, again, probably not worse off than what we were
before.

And, for the record, we are well within my time estimates for the labor
on this. We still have a *lot* of work to do to complete (even a plan
for) the mail infrastructure, but I'm going to call this one as fixed
for now, as the next steps require further thinking, and even more major
architectural changes. So I'm going to reserve those for that
non-proverbial next year.

In the meantime, I'll work on another one of my famous
proposal. Apparently, ChatGPT can generate those for me now, so maybe
I'll give that a shot.. :wink:

"Make a proposal for torproject.org mail services in the style of
anarcat's TPA-RFC" please?

A.

_______________________________________________
tor-project mailing list
tor-project@lists.torproject.org
tor-project Info Page

_______________________________________________
tor-project mailing list
tor-project@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-project

2 Likes
6

Thank you, Thank you and Thank you TPA team :slight_smile:

···

On Thu, Dec 15, 2022 at 5:35 PM Al Smith <smith@torproject.org> wrote:

A huge thank you for jumping up and making changes when we noticed a
downturn in delivery on our fundraising emails during a critical time.
Many rounds of applause and beverages on me to you and the TPA team the
next time we meet in person–thank you!!

Al

On 12/15/22 12:32 PM, Antoine Beaupré wrote:

Status update, week two: TL;DR: situation mostly fixed. Next work in 2023.

Things seem to have gone back to normal. In fact, I’ve been a little
hesitant in sending this email, because I’m worried that people will
suddenly remember all those issues they’ve been having with email
forever and remind me how everything else is broken, but dang, it
looks like we kind of made it?

As far as I can tell, emails send through the submission server have
stopped bouncing (#40640 closed!), authentication problems for messages
actually delivered at gmail have stopped (#40765 closed!) and other
outright gmail bounces seem to have also stopped (#40959 closed!).

That is a major improvement, in such a short time span. There are
still issues with our mail services, of course… In particular, email
forwards are still in somewhat of a gray zone. They are technically
not allowed, especially for senders with a hard SPF policy, but that was
already a problem before. We may see delivery problems for mails sent
internally, but, again, probably not worse off than what we were
before.

And, for the record, we are well within my time estimates for the labor
on this. We still have a lot of work to do to complete (even a plan
for) the mail infrastructure, but I’m going to call this one as fixed
for now, as the next steps require further thinking, and even more major
architectural changes. So I’m going to reserve those for that
non-proverbial next year.

In the meantime, I’ll work on another one of my famous
proposal. Apparently, ChatGPT can generate those for me now, so maybe
I’ll give that a shot… :wink:

“Make a proposal for torproject.org mail services in the style of
anarcat’s TPA-RFC” please?

A.

tor-project mailing list
tor-project@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-project

tor-project mailing list
tor-project@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-project

2 Likes