Over the past few weeks, I’ve seen the following occasionally when I try to send an email to someone who uses gmail, I get a “Undelivered Mail Returned to Sender” with the following in the message:
host aspmx.l.google.com[xxxxxxxx] said: 550-5.7.1 [xxxxxxxxx] Our system has detected that 550-5.7.1 this message is likely suspicious due to the very low reputation of 550-5.7.1 the sending IP address. To best protect our users from spam, the 550-5.7.1 message has been blocked. Please visit 550 5.7.1 Why has Gmail blocked my messages? - Gmail Help for more information. xxxxxxxxxx (in reply to end of DATA command)
If I resend the email, it typically goes right through.
Sending email to my own personal Gmail account and select “show original” from the menu, I see:
So it seems my mail server is set up fairly well. Perhaps my SMTP credentials have leaked, but my server is not showing up on any of the DNS blacklists.
The hazards of running your own email server – the fundamental problem may be that the big email providers make it difficult to send email to them …
For the past two days, late in the day, email to google/gmail recipients are getting blocked. Earlier today, it was working fine. Even sending to my own gmail address is now blocked. Guess I’ll try again tomorrow.
Same pattern holds – last evening, I could not send email to any gmail recipients, even my own gmail account. This morning, it is working fine. This seems a little more complicated than a simple blacklisting – perhaps some automatic algorithms have run amok.
postmaster.google.com still does not show any data from my domain. Looking at this thread, it seems there needs to be a certain volume of mail required:
The Gmail PMT does require a certain volume to analyse and display the details. Hence I prefaced with “if you happen to send out a fairly large number of emails to Gmail users…”. Hopefully, with time, that volume will be reached and the details displayed.
As for the message, it is impossible for me or anyone else to accurately discern the suspicious component because that it determined by references to Gmail’s own algorithm and information from reputed blacklists, crowd-sourcing and user preference not to mention the details included in the guidelines help article, plus the content of not just the message, but also anything included in the signature.
It seems if my server was being used to send spam, than there would be plenty of email volume sent to gmail users.
digging through the SMTP server logs – there are only 3 users, a wordpress site, and a git server that use this mail server. In the last month, there were only 244 messages sent. Most of the messages were business related messages sent from me. It does not appear that the SMTP credentials have leaked or anything like that.
This is why EVERYONE tells you not to selfhost email.
However, email is about the only widespread decentralized communication platform, so it seems healthy to keep it this way, instead of everyone routing their mail through two large companies. So I’m a bit stubborn in this regard, and perhaps a bit of an idealist …
This issue is also a potential problem for google mail users – if people can’t contact you (especially potential customers, etc), is that a problem? In our effort to stamp out every little last bit of spam, there will also be collateral damage.
I just learned every time my mail gets blocked, the transaction is using IPv6 addresses. So not sure why it uses IPv4 sometimes and IPv6 at other times. This is very likely the issue …
Gmail was blocking email again today, so I added the following to my NixOS config on the mail server:
networking.enableIPv6 = false;
Have not had any failures since.
It is interesting that most of the time my mail server used an IPv4 connection to Gmail, but occasionally (which seemed to happen mostly in the afternoons), it would use IPv6. Not sure if this is something Google turns on randomly, or if that is a routing/DNS thing.
IPv6 address space is too large to reasonably track. Like, so large, everyone could potentially have an internet of internets to themselves. I have not tried to purchase ipv6 address space, but it’s probably so cheap that there is no economic incentive to even maintain reputation on blocks. Like, pennies or dollars. They’re so cheap that AWS doesn’t even charge for them, which is saying something. So it’ll be very cheap to get addresses and trash them and move on. That’s from the spammer side.
From the operator’s side, let’s say that you wanted to keep a blacklist/whitelist of all IPs in IPv4. That’s about 4B addresses. Each address takes 4 bytes. Let’s also be generous and say we’ll keep a 32-bit float to store a “score.” – you can keep track of all 4B addresses with a very high-precisions score in 32GB (16GB for the addresses, 16 for the score). That’s easily stored/updated in RAM. It’s actually a manageable size to keep lots of detailed history.
IPv6 has 2128 addresses. That’d require more storage to track than all of the hard drives currently in existance, a quick google says that’s about 64 zetabytes (276 bytes).
Basically, tracking by IPv6 address is such a large address space, and the identity so unreliable, that domain reputation will be what is used for the majority of delivery decisions (combined with stuff like DMARC policies, ARC verification, etc.). Even now, gmail (and Office 365, and other large ISPs) use domain and sender reputation rather than IPs.
Today, MS is rejecting my emails to live.com addresses. This is not critical, so I can take my time to work on this. It appears that MS has a service where they provide feedback on the reputation of an IP address:
This appears to be similar to the postmaster tools Google provides, which to date still display the following:
No data to display at this time. Please come back later.
Postmaster Tools requires that your domain satisfies certain conditions before data is visible for this chart. Refer to the help page for more details.
