Real Free Websites Real Free Websites

Dropped Emails: Yahoo’s Gain?

Friday, February 15th, 2008

As spam levels continue rising, all of the big players (Yahoo, Hotmail, AOL and Gmail) have been ramping up their spam detection and becoming more aggressive in their rejection of emails. Undoubtedly people have benefited from receiving fewer spam emails, but it seems at least in Yahoo’s case that valid emails are also dropped. Unbelievably, Yahoo stands to gain from overly-strict email verification.

Background: aggressive email verification

There are a number of competing email verification systems out there. Omar’s blog details what Yahoo, Google and Microsoft are up to. Yahoo instead has chosen to use a different verification system, called Domainkeys. The systems are not mutually exclusive, and none is supposed to be absolutely required for an email provider to accept an email as non-spam. In time one verification system may be accepted as the standard, but this is still early days for email verification systems.

The problem with delivery of emails to Yahoo-provided email seems to have begun for many around the end of October 2007. However, more recently Yahoo has continued to step up their spam defences. In this blog, Yahoo “Spam Czar”, Mark R., writes

“…at Yahoo! we’re tightening up on our spam controls in ‘08, and in fact we’ve already begun rolling out a significant new defense system (and it’s only January 4th!)”

This sounds like good news for Yahoo users: less spam is good news for everyone. But read on!

Results of the changes

My attention was drawn to Yahoo’s system changes when a client queried the failure of an email to arrive at Yahoo and BTinternet emails (Yahoo provides email service for BT). In recent months emails were consistently failing to arrive and what’s more, my client explained:

“…people we have been emailing with regularly for months are now saying they are no longer receiving our messages.”

By examining server logs, you can trace an email’s status: in this case and others I have now seen, it was clear that Yahoo’s servers were not accepting the email, but were deferring it:

2008-01-16 17:48:28.129600500 new msg 14206764
2008-01-16 17:48:28.129627500 info msg 14206764: bytes 22222 from <xxxxxx@xxxxxx.co.uk> qp 28514 uid 504
2008-01-16 17:48:28.129644500 starting delivery 40718: msg 14206764 to remote xxxxxx@yahoo.co.uk
2008-01-16 17:48:28.381553500 delivery 40718: deferral: Connected_to_195.50.106.7_but_greeting_failed./Remote_host_said:_421_Message_from_(216.35.197.131)_temporarily_deferred_-_4.16.50._Please_refer_to_http://help.yahoo.com/help/us/mail/defer/defer-06.html/
2008-01-16 17:55:09.427873500 starting delivery 40932: msg 14206764 to remote xxxxxx@yahoo.co.uk
2008-01-16 17:55:10.426180500 delivery 40932: deferral: 195.50.106.7_failed_after_I_sent_the_message./Remote_host_said:_421_Message_temporarily_deferred_-_4.16.51._Please_refer_to_http://help.yahoo.com/help/us/mail/defer/defer-06.html/
2008-01-16 18:15:08.176657500 starting delivery 41304: msg 14206764 to remote xxxxxx@yahoo.co.uk
2008-01-16 18:15:12.860120500 delivery 41304: deferral: 195.50.106.7_failed_after_I_sent_the_message./Remote_host_said:_451_Message_temporarily_deferred_-_[140]/
2008-01-16 18:48:29.365991500 starting delivery 41811: msg 14206764 to remote xxxxxx@yahoo.co.uk
2008-01-16 18:48:29.599861500 delivery 41811: deferral: Connected_to_195.50.106.7_but_greeting_failed./Remote_host_said:_421_Message_from_(216.35.197.131)_temporarily_deferred_-_4.16.50._Please_refer_to_http://help.yahoo.com/help/us/mail/defer/defer-06.html/
2008-01-16 19:35:09.055033500 starting delivery 42669: msg 14206764 to remote xxxxxx@yahoo.co.uk
2008-01-16 19:35:09.380619500 delivery 42669: deferral: Connected_to_195.50.106.7_but_greeting_failed./Remote_host_said:_421_Message_from_(216.35.197.131)_temporarily_deferred_-_4.16.50._Please_refer_to_http://help.yahoo.com/help/us/mail/defer/defer-06.html/

This deferral requires that the sending mail server resend the email, and this cycle continues usually until a point (often 48 hours later) when the sending server admits failure and bounces the email back to the sender, or the receiving server (in this case Yahoo) accepts the email.

Prioritization of emails

Strangely, in the light of the above, Yahoo deny that they use a process called “greylisting”, which involves rejecting an email and not accepting it until it is resent, on the assumption that spammers will not resend emails. The language they use to describe greylisting is this:

“[greylisting is] where an SMTP server will reject every message the first time it is attempted”

Well, maybe they do not reject every message - for example, Yahoo to Yahoo emails go through quickly - but there are consistent delays from some other domains. So consistent, that it seems like some messages from non-priority servers are always delayed. In fact one user was told by Yahoo that:

“…all shared hosting IPs are instantly flagged as suspect by them.”

Some have suggested that Yahoo servers have been overloaded with emails, which may have caused Yahoo to defer emails. However, if this were the case, then all emails would stand an equal chance of deferral. In reality, we see emails from some domains or mail servers that arrive almost instantly, whereas other emails are be placed in a cycle of deferrals. There is obviously some prioritization going on here Yahoo, why not admit to it?

Emails sent to spam folders

Even when these emails get through, usually hours or sometimes days later, the emails are often directed by Yahoo’s SpamGuard into spam folders. This is obviously no welcome for an email that has spent days trying to be accepted. These “false positives” might not be so frustrating if SpamGuard did not let through so many junk emails. Why SpamGuard singles out some legitimate emails for their spam folder and lets through obvious spam really needs to be addressed.

Still more worrying, there are whispers and rumours of Yahoo accepting emails, and then “silently dropping” them. This refers to the email being “lost” somewhere between Yahoo’s servers accepting it and the user: the email does not go to their inbox, but neither does it make it to their spam mail folder. It seems that Yahoo is simply deleting emails, without the email even entering the spam mail folder. Has anyone had any experience with this?

Who is affected?

Businesses
Obviously, businesses that have domain names that are not prioritized suffer. There is nothing surer than losing a customer when that customer thinks you have not replied to them.
Yahoo users
There are many domains that send relevant, useful email to Yahoo users. These people are not receiving emails and may know nothing about it.

How Yahoo gains by not delivering emails

Anyone would think that Yahoo’s business would also suffer if it did not perform the service of effective email delivery. However, because of their market position, as the biggest email provider, they are in a strong position. Businesses cannot afford to ignore users that have Yahoo email accounts. What is the best way to ensure your business email arrives at Yahoo email inboxes? Make sure your email is sent by Yahoo mail servers! So you can either open a Yahoo email account for your business or you can move web hosts to Yahoo’s web hosting! Sounds like good news for Yahoo…

This could be speculation at Machiavellian levels, and hopefully Yahoo do not think they would be able to go through with this without damaging their reputation. However, Microsoft’s recent bid to buy out Yahoo indicates that business at Yahoo is far from healthy. Hopefully, this is not a last desperate attempt to save business.

How to make sure your email gets through

So, if we do not want to put our proverbial tails between our legs, what can we do to avoid migrating to Yahoo web hosting? Not an easy one, I am afraid. However, here are some suggestions:

  1. Make sure you know how other mail servers see your email. There is a very good service to check what verification your email has as it is sent out, provided by Port25. All you have to do is send a test email addressed to check-auth@verifier.port25.com and you are sent back a reply indicating the verification that your mail server provides and also a spam-likelihood score based on the open source mail filter SpamAssassin. This can be very helpful in identifying any faults in your mail server verification and the analysis provided by SpamAssassin can also be helpful in seeing whether your emails are innately “spammy” or note. Amusingly, this is a domain that seems to suffer delays when sending to Yahoo mail servers: try sending an email from a Yahoo account.
  2. Send your email as plain text rather than HTML. Spam is less likely to be plain text and therefore spam filters are more likely to let plain text messages go through. You will lose some functionality in your emails though, such as no images, and font formatting.
  3. Watch the content of your emails and try and reduce their “spamminess”. This should help you at least with mail filtering, and keep your email out of the spam folder. As well as the Port25 email test above, you can test your email’s spamminess by emailing it to spamcheck@sitesell.net (make sure you include the word “TEST” in the subject. Again, Yahoo users, be prepared to wait.
  4. Make sure you keep your domain out of any spam filter blacklist by ensuring any bulk emails have an option to leave the mailing list, and make it clear that you will not be passing on the recipient’s email address. Most people have now heard that spammers trick people into trying to opt out, but that they use this to identify real email addresses. So it is all too easy for someone to send your bulk newsletter to their spam folder to get rid of it, if they do not trust you to opt them out.
  5. This is more for server managers, but you could try and persuade your host to see if they can get get your domain whitelisted on Yahoo
  6. Again for server managers, install Yahoo’s email verification system, Domainkeys, although the trouble involved in adding this functionality to the server, added to the possibility that it will not make a difference, may be off-putting for most mail server managers.
  7. Installing one or more of the other competing email verification systems (e.g. SPF, CSV, DKIM, and SenderID is in some cases, easier than DomainKeys, and may help delivery, but if using DomainKeys will not guarantee it, then this is even less likely to ensure delivery.
  8. Goodmail Systems will certify your email (assuming you manage to pass their stringent accreditation checks), allowing you to receive a delivery confirmation:
    CertifiedEmail, but your business headquarters will have to be in the US, Canada or UK.
  9. Perhaps one of the best solutions, is to use registered email, which gives you legal proof that your email has been sent, received and read. However, you will have to pay a price for this piece of mind (approximately $0.60 per email at time of writing): RPost. This will not guarantee your email will arrive, or that it will avoid being deferred. However, it will allow you to know for sure whether it has or has not arrived.

Any other ideas?

Caveat

Having said all this, I admit to being a long-term satisfied Yahoo mail user, and I think the service beats Hotmail hands down and has held its own against Gmail. The real bad guys here are the spammers: if it were not for them…

Furthermore, it could be argued that a free email service owes nothing to its users, although a counterargument might centre around the monetization of this service through advertising. What is clear is that this recent turn of events is worrying, and may mean my dependence on my Yahoo email account goes into decline…

Related Posts

  • No Related Post

12 Responses to “Dropped Emails: Yahoo’s Gain?”

  1. Tim Says:

    I for one put a block on yahoo’s spiders. If they do not wish to be RFC compliant, others should not allow their spiders access to your servers making their search engine worthless. I have no trouble getting email into any other domain as I do with yahoo, so I will do all I can to spread the word to make yahoo spiders as worthless as their email service.

  2. Real Free Websites Says:

    Tim,

    That’s a very principled and brave stance, but not many websites can afford to turn down organic traffic from Yahoo - even if they are not the dominant search engine. Hopefully, you have not seen a downturn in visitors as a result.

  3. Rob Says:

    Great resource! Thanks for putting it together. Nothing more annoying than missing emails…

  4. Miriam "Kelly" Duncan Says:

    I cannot check my e-mails and that makes me sooooooooooooooooooooooooooooo mad. I have got other e-mail accounts but I love this one because This was my first e-mail address.

  5. Murphy Says:

    Yahoo is damaging my business. Everytime I see a customer with a yahoo.com email address, I know I’m not going to get their business because they’ll never see my response. Furthermore, I send my emails from my yahoo.com email address also!! I’m forced to fill out the captcha challenge and my emails still get deleted or put into the spam box.
    I’ve chatted with yahoo help and filled out their forms and jumped through their hoops to fix the problem with no success of any kind.

  6. Grace Says:

    I’m getting really mad at Yahoo! now. I forgot my first e-mail’s passwrod so I nee to get another one. I began transferring my accounts into that e-mail (like crunchyroll, friendster, youtube) but I haven’t received all the verification mails I need. I thought it’s because of the new default setting of Yahoo! since my other old e-mail can receive verification mails. Do you know how to fix this? Will Yahoo! loosen the SpamGuard a bit one day? Please reply.

  7. Real Free Websites Says:

    Hi Grace,
    Sorry, we don’t have any influence over Yahoo or their policies! Maybe it’s time to try another service provider? E.g. Gmail.

  8. Amr Aboelmagd Says:

    Hi,

    Gmail is worse. I use both of them. May be we should search for a reliable email provider which is not Yahoo neither Gmail.

  9. Why you should dump your Yahoo! mail? « words.anotheredge.com Says:

    […] Article from realfreewebsites.com: Dropped Emails: Yahoo’s Gain? […]

  10. zephania omondi Says:

    Has had yahoo since 2003 to date. Just the othe week lost all my 2009 and 2008 email and not the other years before. I reaaly have some important information and contacts that i would never know how to get back. What should i do???????????

  11. allam Says:

    Hi, I’ve had problems with customers claiminbg they do not receive my emails when i send to their yahoo accounts. Today ste up my own yahoo email account to see what was happening, and sure enough, after 4 hours my email eventually arrived straight into spam folder !!

    looks to me like yahoo have made a real dogs dinner of their spam filters.
    would never use a company like that.

  12. Rod Latchford Says:

    How i agree with Grace!i could not remember my password so Yahoo gave me a verification code but i had to sign in to get to it,as i could not remember the correct password i could not reach the verification code to get a correct password!but i found a way round it,so armed with the new password i tried to sign in again but Yahoo did not recognise my address so am going round in circles with Yahoo! they owe me over 80 emails including my phone bill!so if you can read this Yahoo,get your finger out!!

Leave a Reply