Thursday, October 18, 2012

Email Deliverability in the Age of Spam

Non-profit organizations of all sizes are facing considerable challenges as they reach out to their supporters using email. The advantages of this communication method - instant, low-cost access to an organization’s audience, with personalization and tracking capabilities unavailable via direct mail or telemarketing - remain compelling, but are being undermined by spam.

Recent market researches currently estimate that more than 73% percent of all email is spam. As the volume of unsolicited, objectionable, and fraudulent email has increased, recipients are understandably raising barriers to protect their inboxes. Internet Service Providers (ISPs) now compete for subscribers based on their ability to control spam and provide a secure online communications experience.

As email recipients have moved to rapid adoption of spam controls, messaging by non-profits encounters an increasing number of delivery barriers. Legitimate mail may be refused for delivery, or routed to bulk spam folders, or mangled to remove images or disable hyperlinks. Such “collateral damage” is too often considered an acceptable cost of controlling the spam flood. Organizations need to understand how technical trends in spam controls will impact the overall deliverability of their email communications, and what they can do to ensure their messages reach a supporter’s inbox. Strict adherence to best practices in regard to permission-based list building and emailing will be critical if non-profits want to continue to enjoy the significant benefits of this form of communication.

Who Controls Email Delivery?

Contrary to the expectations of many email publishers, there is no technical or legal requirement that ISPs or other mail gateways accept every email sent and ensure delivery to intended recipients. Each mail system can establish its own rules governing access to its inboxes, and then enforce these rules using any of the anti-spam solutions available on the market. While email recipients are gaining access to tools that can effectively control spam, this actually increases deliverability challenges for legitimate email publishers. You have probably already encountered delivery barriers created by “solutions” that rely on “challenge-response,” “whitelists,” “blacklists,” content filters, distributed spam detection networks, etc. Barriers may be raised at multiple points along the path an email takes to the inbox, making it difficult to ascertain exactly where or why your messaging is being blocked.

Because few of the systems in use are inter-operable, solving a delivery problem associated with one product or getting whitelisted by one mail system does not necessarily help with the next barrier. While the delivery landscape is growing increasingly complex, most anti-spam systems actually rely on a relatively small number of parameters to sort the good mail from the bad. Most important is the reputation of the mail server sending messages: Has that IP address or domain sent spam in the past? Is it compliant with Internet mail standards? Is it on whitelists that the recipient system uses to identify good mail, or on blocklists used to identify spammers? Some aspects of a sender’s reputation are purely technical, but reputation is now primarily based on the number of spam complaints an email publisher generates. The greater the number of spam complaints, the more likely your email will encounter delivery barriers across the entire range of anti-spam systems. Ultimately, control over delivery is being distributed to recipients, and this is good news for permission-based email publishers: if you keep your complaint rates low, your communications will encounter far fewer delivery barriers.

Navigating New Paths to The Inbox: What Should You Do?

• Adhere strictly to permission-based messaging. The best defense against a poor mail reputation and public stigmatization of your organization as a spammer remains strict adherence to opt-in list-building practices.

• Review your spam complaints and unsubscribe requests regularly to identify any email acquisition methods that fail to obtain explicit permission, such as co-registration, list sharing, etc. Don’t be tempted by CAN-SPAM’s legitimization of unsolicited messaging and opt-out—ISPs are not required to deliver CAN-SPAM compliant email and will raise delivery barriers to any unsolicited mail, whether or not it is commercial spam. Non-profit or political communications are subject to the same set of rules: If a mailing is unsolicited, it will generate complaints; if it generates complaints, delivery barriers will go up, and your mail reputation will go down.

• Pay attention to all spam complaints. Do not ignore complaints from recipients that your messaging is unwanted. More and more complaints are being fed into sophisticated, distributed spam identification networks, which are then used to guide further spam control efforts. AOL, for example, keeps a daily report card on the number of complaints that messages from your domain generate, and then utilizes these complaint rates to determine whether to raise additional delivery barriers. Even a few abuse complaints can generate serious consequences; systems like SpamCop can trigger blocklisting based on very low complaint thresholds (on the order of one complaint in one million messages). Complaints are indicators of problems in your organization’s list-building or communication practices. Investigate why recipients are reporting that your messages are spam: Do they feel they did not opt in to your email list? Are they objecting to the frequency of your messaging? Are they signaling that they no longer find your content or engagement opportunities compelling?

• Partner with an email provider who understands the landscape. If your organization is a relatively high-volume mailer (i.e., sending more than 10,000 messages per month), the current delivery landscape is almost certainly too complex to navigate on your own. It’s likely that you are not even fully aware of all the barriers your current messaging encounters. If you are relying on metrics like bounce rates to evaluate your delivery, you are operating in the dark.

Most ISPs and spam control systems do not use bounces to notify email publishers that their mail has been rejected or routed to a spam folder, so more sophisticated inbox monitoring based on seed lists is required to get a true picture of your current delivery performance. While it may be relatively easy to send email from your own IT infrastructure, getting the mail delivered to your supporters’ inboxes is another story. Few organizations have the in-house resources to manage ISP relations or investigate and resolve blocklist or other delivery problems.

It is important to partner with an email service provider that has a comprehensive delivery assurance program. The service provider’s top deliverability objectives should be to get your email into your supporters’ inboxes and to protect your mail reputation. Only consider service providers that have a good mail reputation themselves and can document their delivery performance. Remember that service providers typically send multiple clients’ email from shared mail servers, so select a provider that only works with clients sending permission-based email.

The service provider should operate all of the following essential components of a delivery assurance program:

Email Infrastructure Operated to Support Delivery
To ensure delivery, it is critical that your email provider operate in compliance with a wide variety of ISP technical requirements, including sender authentication, bounce management standards, and integrated spam complaint feedback loops. Since these requirements vary across ISPs, a sophisticated infrastructure is required to enable different interactions with different receiving domains.

“Whitelist” Status at ISPs and Recipient Domains
Whitelisting is a designation that allows high-reputation mailers to automatically bypass anti-spam filters and avoid security-related message mangling (e.g., suppressed images or deactivated hyperlinks). Some recipient domains (e.g., AOL and Yahoo!) operate their own whitelists, while others (e.g., MSN) rely on third-party whitelists developed by commercial firms (e.g., Habeas SafeList). Whitelisting enables a service provider to maximize delivery of messages to the inbox, and to reduce the list hygiene problems created by deceptive bounce messages.

“Blocklist” Detection and Resolution
Blocklisting disrupts delivery at any recipient domain that uses a specific list to control entry into its network. There are literally hundreds of blocklists, operated by Internet community projects and commercial firms. Your email provider should be regularly monitoring for the presence of its mail servers on various blocklists, and have procedures in place to get such blocklistings lifted.

Email Software Optimized to Facilitate Permission-Based Communications
Your email service provider’s application should automatically facilitate best-practice subscription management, including multiple registration modes (e.g., single opt-in, confirmed opt-in, or double opt-in) and easy-access, user-friendly subscription management. Offer your subscribers a one-stop location where they can review their current subscriptions, modify their email preferences, and maintain their profile. Don’t select a service that makes it difficult for subscribers to remove themselves from your email list, as recipients will be more likely to report your messages as spam just to get off your list.

What to Expect in the Future
Organizations will increasingly be held accountable for their own mailing practices. Generalized whitelists are falling out of favor at ISPs because the list-building and messaging practices of individual clients vary, and generalized whitelists can allow organizations with poor practices to hide among legitimate mailers to get unwanted mail delivered. Email service providers are under increasing pressure to use a dedicated IP for each client, so every organization can be held accountable for its own mailing practices.

In the last few years, a consensus has emerged that email abuses can only be controlled if the existing email infrastructure is enhanced to support authentication of a sender’s identity, construction of a sender’s mail reputation, and certification of a sender’s permission-based list-building practices. Technical efforts are proceeding rapidly on all these fronts, and have already begun to affect email publishers. As part of a larger effort to control online identity theft, brand spoofing, and phishing, sender authentication systems are being rapidly adopted by major ISPs like AOL, MSN and Yahoo! Reputation services like senderbase.org or trustedsource.org now track the mail volume and spam complaint rates of all email publishers. Companies like Habeas are providing third-party certification of the opt-in status of email lists to support various whitelisting programs.


No comments:

Post a Comment

Thank you for posting your comment. All comments will be moderated prior to posting them to the blog.