ListMailPRO Email Marketing Software Forums
ListMailPRO Email Marketing Software Forums => Server Optimization, Tweaks => Topic started by: BoomPop on June 15, 2007, 01:23:58 am
-
Hello Dean (and everyone).
I've been running a cPanel box for years, but I just received confirmation that they (cPanel) will not be supporting DomainKeys for at least 6 more months. And considering that Yahoo & AOL already trash over 90% of the emails coming from my server (and I don't send newsletters or mass mailings of any kind), I'm looking to make some drastic changes.
I'm looking hard at Plesk. I know you recommend it Dean. But do you have any experience getting DomainKeys to work with it? I've read around some of the forums and it looks like it can (possibly) be done, but I'm not sure I'm up to the task.
REALLY not looking forward to learning a new control panel, but will suffer if it means some of my mission-critical emails will actually get delivered!
Thanks.
-
Greetings,
I've tried DomainKeys and cannot recommend it, nor am I able (after many, many efforts) to confidently add it to Plesk qmail alongside other important patches... When I did get it patched in (minus certain other conflicting patches) the main problem I had with it was the bounce email and the "send from" email for each message had to be exactly the same for the DomainKeys signature to be added. This means you would not be able to customize the "send from email" for each list for display purposes.
Furthermore, I do not believe DomainKeys guarantees you will get more messages delivered to Yahoo. They are doing some strict rate-limiting these days... I'm guesstimating but I doubt you can get more than about 20,000 messages to Yahoo per day, per server, on average.
You might have far better success increasing delivery to Yahoo by contacting them. Check out the following thread for more details, particularly the last post which contains a link with more information about solving rate-limiting-rejected email.
http://listmailpro.com/forum/index.php?topic=1579.0
You can phone AOL and have them run a check on your server IP and/or domain name for blocks. It may be possible to re-assure them of your professionalism and have your complaint thresholds increased. They are very friendly.
http://postmaster.aol.com/contact
You can also set up a script to automatically remove complainants, so they are less likely to complain more than once.
http://listmailpro.com/forum/index.php?topic=721.msg3592
First you would need to set up a "Feedback Loop" to a certain email address, then you would set up an alias on the same address, pointing to the script.
Regards
-
This means you would not be able to customize the "send from email" for each list for display purposes.
Wow! Am I glad I came and asked you before I decided to do anything drastic. Never would have thought of this minor (but not so minor!) detail, It's very important to know.
I'm guesstimating but I doubt you can get more than about 20,000 messages to Yahoo per day, per server, on average.
Heck... I'd settle for 10 or 20! Much less 10 or 20 thousand. I hate Yahoo for what they're doing. Putting such high priority on a system that 95% (I guessed at that figure) of the webmaster population CAN'T implement. The only people who can afford to play exactly by their rules are the spammers...people who do nothing but figure out how to bypass/circumvent the spam safeguards. The rest of us small timers are just caught in the muck.
Thanks again, Dean. I appreciate your other tips as well. It's just that my main competitor has beautiful DomainKeys in the headers of every email he sends. And every email he sends is getting delivered (from the testing I've been able to do, he he he...) And my thinking has been, "if he can do it, then so can I!" But I guess maybe not...
So now I have to figure out what I'm gonna do. Perhaps have someone build a pure RH server for me with no control panel at all.
No that won't work either.
God I hate this.
Still love, love, love ListMail though.
-
Ok... thinking some more DW. You said:
When I did get it patched in (minus certain other conflicting patches) the main problem I had with it was the bounce email and the "send from" email for each message had to be exactly the same for the DomainKeys signature to be added. This means you would not be able to customize the "send from email" for each list for display purposes.
Does this mean that you DID get it working happily, even though it crippled your ability to use ListMail to its fullest extent? If that is the case, I might consider getting a second server just specifically for this one mission-critical domain/email address. It would be a not-so-pretty work-around. And at this point I'll do just about anything. Fingers crossed. :wink:
-
Hi BoomPop, DW,
Hmmm, yes this is a 'royal pain' - I'm going through it now myself. After setting up a dedicated mail server, I've spent the last week or so trying to get back (I was on previously) AOL's whitelist - it turns out the whitelisting only lasts for 6 months - at least that is what one of their techs told me on the phone.
AOL does seem very agreeable - and reachable, I'm not sure yet about Yahoo, but I planned on implementing their 'domain keys' just for the purpose you mention BoomPop - GETTING DELIVERED...
It's worth it, I bet you could hack the headers yourself in LMP - with DW's approval, but it sounds like you have to give up having a seperate 'send from' from each list being different than the 'bounce to' address. This wouldn't bother me in the bit, and I'd gladly give that up to get things delivered to Yahoo.
So, a few questions... perhaps we can collaborate on some ways to get Domain Keys working and provide some alternative to others using LMP? I'm really curious BoomPop, you say: . And every email he sends is getting delivered (from the testing I've been able to do, he he he...) And
How in the world do you test your competitor getting their emails delivered - unless you're seeding their list with your own (dummy) email addresses and then watching them for deliverability?
So now I have to figure out what I'm gonna do. Perhaps have someone build a pure RH server for me with no control panel at all.
What's a 'RH server', and as you say, I don't think control panel has anything to do with it does it - I believe the DomainKeys are only header calls that can be implemented in the MTH (Mail Transfer Handler) on your server. For example my server runs Courier Dot Mail, there are specific calls that can be added to implement Domain Keys for this MTH, there must be similar ones for other mail server MTH's, just find out which MTH you are running and track down how to implement DomainKeys, or change the headers to build something similar. Yahoo itself has a page on it, here's a link to it:
http://domainkeys.sourceforge.net/
As it seems my mail handler isn't on there... well, I'll keep digging because as you mention, it's worth it... We need to be prepared for any of these type of 'header' validations that may take place - if you want to get your email delivered!
Let us know what you find, and what works for you - anyone else - please post your own solutions for implementing Yahoo DomainKeys with details - it will help others get it working...
-
BoomPop,
Does this mean that you DID get it working happily, even though it crippled your ability to use ListMail to its fullest extent?
Yes, and actually I could work on it again and -maybe- get it patched in with all my other favourite patches... it might be easier the second time around considering I've gained experience with the other patches since last time I tried DomainKeys.
Brett,
I bet you could hack the headers yourself in LMP
While creative, I think this would be too much work. Basically you'd need a full DomainKeys implementation in PHP... I'm not even sure this is possible considering the complexity (message length calculations, etc). There are easier ways, as you mention later on in your reply, where this is implemented at the MTA level.
What's a 'RH server'
It's the OS - RedHat linux - Fedora Core 1-5, RHEL (Red Hat Enterprise Linux), etc. Most of my servers run RedHat and I prefer it, while some others say it is not the most efficient OS. :)
I believe the DomainKeys are only header calls that can be implemented in the MTH (Mail Transfer Handler) on your server.
Yes, exactly - thanks for providing a link to more information on this at Yahoo!
In my experience, once it's set up in the MTA / MTH the DomainKeys signature/header should be automatically added, provided the Bounce and Send From emails match.
Regards
-
DW,
Thanks for looking at this, I wonder a few things about Domain Keys.
1.) After looking closer at the Sourceforge page I notice there is only support for a few MTH's/UA's: Sendmail, Qmail, Ironport, Mdaemon. So, I wonder which one do you use, which one is best to use? Is it best to go 'low level' if possible, and how low can you go - is Mdaemon the lowest level listed here?
I believe all must use Sendmail or Qmail, but I'm not sure, I thought my own server is using Courier Dot Mail as the MTH, are Sendmail/Qmail MTH's? If so, I don't see any way to install the Domain Keys in Courier Dot Mail, so do I still install for Sendmail/Qmail, and does Courier Dot Mail just pass the mail to Sendmail/Qmail? I believe it possibly does (I'm going to confirm this with my host).
2.) I find this in the documentation for Mdaemon: *Exclusive mode will not work with mailing lists so use this with caution
, you can read this yourself on this page: http://www.altn.com/Support/KnowledgeBase/KnowledgeBaseResults/?Number=KBA-01744
so apparently you can't go into 'exclusive mode' at all using Domain Keys if you are using a mailing list, instead you must go into ' NEUTRAL or RELAXED (signature optional)' mode, and then what does this do for deliverability?
DW, do you have any way of testing a few of these different DK modes and possibly checking on how deliverability is affected? I figure with your clout as developer you might be able to contact Yahoo directly to get an answer on where to install DK for it to be most efficient - and how it affects deliverability if 'relaxed mode' is used?
-
Here's my 2 cents... Forget Domain Keys, you DO NOT need them.
Simply setup SPF records for your domain and be happy. It's VERY VERY VERY simple and it's what I use and get all ( well of course never all ) of my mail through to aol and yahoo...
-
Hi Mike,
Thanks for the feedback, I was just looking at setting up DomainKeys (DK) and DKIM.
You say:
Simply setup SPF records for your domain and be happy. It's VERY VERY VERY simple and it's what I use and get all ( well of course never all ) of my mail through to aol and yahoo...
Ok, it seems you have verified that SPF records are enough to allow Yahoo to accept email, is there some way to validate/check this from my own server?
One further thing, I did setup the SPF records, but I have a little different situation in that my server is referred to in to different ways:
1.) mydomain.com
2.) server.mydomain.com
I know this seems a little goofy, but I went round/round with the techs at my host and it is required for the CPanel to work correctly, I've seen it described at other large host services as well. Also, when the mail is delivered I notice it says the mail is coming from server.mydomain.com - so it does seem it is using that to deliver the email.
When I setup my SPF records I did it like this:
mydomain.com. IN TXT "v=spf1 a mx ~all"
server.mydomain.com. IN TXT "v=spf1 a ~all"
Does that look right to you? Can you post what your SPF records look like - just so I can make sure I've got it right? I also wondered one thing, maybe you know, when I look at the records it indicates there is a mail server at mydomain.com (because of the MX record), but it does NOT indicate there is a mail server at server.mydomain.com, I've wondered if I need to also add a MX record for server.mydomain.com also, and then indicate that it has a MX in the SPF record too, like this:
mydomain.com. IN TXT "v=spf1 a mx ~all"
server.mydomain.com. IN TXT "v=spf1 a mx ~all"
What do you think? Do you have any ideas/suggestions? If I use the 'SawMill' as DW suggested will I be able to see if my email is being delivered properly to Yahoo? How do you check it Mike - do you use software to verify delivery?
-
Well my spf record is very simple:
mydomain.com. IN TXT "v=spf1 ip4:68.79.xxx.0/24 -all"
It's simple because I own that 0/24 block of IP's. I used a tool on the web to help me but I can't remember where. And somewhere it said you should use "-all" instead of "~all" for some supposedly good reason.
Well for me verifying delivery to yahoo I simply use a yahoo account honestly. I am safelisted with yahoo so supposedly my emails goto the inbox, but I like to check for sure on occasion.
But for what you got in your SPF record, I'd say it should be "ok". You could add the mx for server.mydomain.com, it shouldn't hurt anything at all and might help depending on how your dns is setup I guess. Of course if you have a static IP then just use it. The only thing with doing that is if you change IP's you have to change the SPF record.
-
Hi Mike,
Thanks for the input...
I used a tool on the web to help me but I can't remember where. And somewhere it said you should use "-all" instead
Most likely you used the free tool at OpenSPF at this website:
http://www.openspf.org/Tools
I used it too, but I just noticed that they also have an SPF sender check. It worked great, in fact the one I used, sending an email to: check-auth@verifier.port25.com gave me a full report including DomainKeys and DKIM.
Of course my domain passed on the SPF and the SenderID check but came back 'neutral' for DK and DKIM because the message was not signed. I still think it might be an advantage to try to run DomainKeys signing, but then again, there is also an overhead using this method as every email that goes out must run a process to 'sign' the email...
How about using something like 'Habeas', have you ever thought of that, I know that Ralph Wilson over at WilsonWeb uses it, here's info on habeas:
http://www.habeas.com/
What do you think?
-
How about using something like 'Habeas', have you ever thought of that, I know that Ralph Wilson over at WilsonWeb uses it, here's info on habeas:
http://www.habeas.com/What do you think?
I have seen a few different variations of services similar to this and I believe they could possibly help with delivery ( I don't think to yahoo, aol or msn ) but probably to other places.
The other ones I saw were fairly costly for the amount of email I send, I couldn't find pricing on this one, but it is worth at least looking into.
-
Hi Mike.
send, I couldn't find pricing on this one, but it is worth at least looking into.
I'm not sure the pricing either, I did get their free 'analysis' report which shows that I'm valid with SPF, reverse DNS, etc. I may consider the next step which is to request an audit, and then they probably give you a price, but you are correct - I believe many of these type of services are priced based on how many emails you send/month so it's probably not going to pay for me to even consider it at this point.
Actually, I think it might be some type of yearly fee, but I'm not sure.
I'll let you know if I hear anything else... I just got this from them, check out the links at the bottom of the page there is a free online seminar their doing on deliverability:
http://www.habeas.com/en-US/News/press-releases/Habeas-Wins-2007-American-Business-Award/
-
Sorry I missed this conversation guys. I didn't have reply notifications turned on.
I'm seriously about to blow my brains out on this whole spambox/undelivered/delayed email thing. I cannot BELIEVE how much time and money I've wasted(?) on it. Since I started this thread, I've set up a new server with a new host (who promised one thing and is now delivering another). So I'm going absolutely nowhere fast. :roll:
Here's my 2 cents... Forget Domain Keys, you DO NOT need them.
Simply setup SPF records for your domain and be happy. It's VERY VERY VERY simple and it's what I use and get all ( well of course never all ) of my mail through to aol and yahoo...
mike2... you say that DomainKeys are not necessary. Well, I have SPF records. ReverseDNS. PTR. etc, etc. My IP is squeaky clean... never been on any spam lists anywhere. In fact, I've never sent a single mass mailing of any kind. Not once. Yet Yahoo and AIM never delivers my Listmail confirmation emails to the inbox. Well actually, when testing the other day, I *did* have 2 test messages make it to the yahoo inbox, but they were both delayed by nearly 72 hours (under Yahoo's fairly new delay procedure). These are confirmation emails I'm talking about, not newsletter mailings. As far as I can tell, this started in October or November, 2006.
So back to your quote above... if DomainKeys aren't the problem, what in the heck could be it?
My main competitor has it set up exactly right, and it works every time that I've tested it.
His setup is:
=======================
NS1.HISDOMAIN.COM (separate IP)
NS2.HISDOMAIN.COM (separate IP)
NS3.HIS-HOST.COM (separate IP)
NS4.HIS-HOST.COM (separate IP)
mail.hisdomain.com (separate IP)
www.hisdomain.com (same as NS1 IP)
Send & Bounce email is the same: support@hisdomain.com
Zero errors on DNSReport.com
=======================
This is the setup I'm trying to mimic. But I can't find a host that knows what the h3ll they're doing enough to make it happen. Actually, to be honest, I did find one that *could* set it up, but they won't because they're ultra-paranoid about security... so they only do things their way (meaning I cannot designate my own servername or dedicated mailserver IP).
And for anyone reading this... if you have a cPanel server, you can forget using DomainKeys... for at least 6 months...probably much longer. And it sounds like it will work on Plesk, but only with some significant caveats (as DW pointed out earlier). So as I mentioned earlier... I think the only way to do it is to setup a plain, stripped down Unix/Linux box and build the system from scratch. That way you can configure all the settings yourself and tell all the naysayers (and crappy, incompetent webhosts) what they can do with themselves.
Can you tell I'm furious and not thinking so clearly anymore? LOL...
-
I just got this from them, check out the links at the bottom of the page there is a free online seminar their doing on deliverability:
http://www.habeas.com/en-US/News/press-releases/Habeas-Wins-2007-American-Business-Award/
Thank you for all this info about Habeas, Brett. I wasn't familiar with them at all.
-
Boom - Even with SPF records and Reverse DNS and clean IP's your email can and will still be delivered to the junk folder for other reasons ( I'd guess mainly for content at this point even if it's just confirmation emails... )
The only other thing I can suggest is try to get "Safelisted" with yahoo honestly. They don't use a feedback loop ( that I know of ) like MSN or AOL, but they will put your IP(s) on a test for up to 14 days to monitoring and then can "White or Safe list" you if they feel appropriate.
-
BoomPop,
Yet Yahoo and AIM never delivers my Listmail confirmation emails to the inbox.
Here's something I can recommend:
Manually check the "Return-path:" header of a confirmation email sent by ListMail. If this is not bounce@example.com (Your ListMail "bounce to email") then this may play a part in how your message is classified by Yahoo.
Some servers require the following to be added to config.php to properly set the Return-path with PHP mail()-sent messages. (Welcome, confirmation and some notifications)
$phpmailf = 1;
While this may not be all of the problem it could be part of it. I think the best solution is, as Mike suggested, to talk to Yahoo directly. They are one of the more aggressive spam fighters these days and are, in fact, the source of most of my own headaches.
Regards
-
Manually check the "Return-path:" header of a confirmation email sent by ListMail. If this is not bounce@example.com (Your ListMail "bounce to email") then this may play a part in how your message is classified by Yahoo.
Thank you for the follow-up Dean.
Yes, the return path is showing up exactly right. The one thing that I've been concerned about though, is that I'm seeing 2 "Received: from" fields in the headers, with some strange conflicting info...
Received: from 72.xx.xx.xxx (EHLO srv1.NEW-hostname.com) (72.xx.xx.xxx)
by mta202.mail.re4.yahoo.com with SMTP; Fri, 15 Jun 2007 02:01:22 -0700
Received: from nobody by srv1.OLD-hostname.com with local (Exim 4.63)
(envelope-from <support@mydomain.com>)
And actually, I have seen a *few* Yahoo messages finally show up in the inbox, but nearly 72 hours has passed before they were finally delivered.
Regardless, as you both suggested, I think my best bet is to contact Yahoo directly.
BUT... does this mean I should just give up on DomainKeys? After all the blood, sweat, tears, and cash I've invested to make them work? Ugh... As I said before, I hate whoever decided to put so much weight on them when only a tiny percentage of webmasters have the infrastructure in place to implement them. :roll:
-
Hi,
Interesting, BoomPop you're right, my Confirmation Emails to Yahoo are delayed for hours, I tested it myself, here are my headers:
X-Apparently-To: myemail@yahoo.com via 206.190.58.151; Tue, 26 Jun 2007 10:38:23 -0700
X-YahooFilteredBulk: xxx.xx.xx.xxx
X-Originating-IP: [xxx.xx.xx.xxx]
Return-Path: <nobody@server.mydomain.com>
Authentication-Results: mta131.mail.re4.yahoo.com from=mydomain.com; domainkeys=neutral (no sig)
Received: from xxx.xx.xx.xxx (EHLO server.mydomain.com) (xxx.xx.xx.xxx) by mta131.mail.re4.yahoo.com with SMTP; Tue, 26 Jun 2007 10:38:23 -0700
Received: from localhost (localhost [127.0.0.1]) (uid 99) by server.mydomain.com with local; Tue, 26 Jun 2007 11:41:32 -0400 id 000233AD.468133AC.00004E6A
So, it appears that I need the phpmailf variable DW recommends because my return-path is NOT equal to my bounce address, instead it is nobody@server.mydomain.com.
Look at the dates/times of send/deliver,
SENT:
Tue, 26 Jun 2007 10:38:23 -0700
DELIVER:
Tue, 26 Jun 2007 11:41:32 -0400
I thought it was even longer than that, but it's at least over 1 hour, this is too long and can really slow down people 'confirming' - but I thought I had watched the 'Bulk Folder' at Yahoo for at least 4-6 hours and not seen the email - so, the delay is disconcerting and a problem. Either way the email ended up in the 'Bulk Mail' folder.
I'll try adding the phpmailf and see if that changes anything.
Regardless, as you both suggested, I think my best bet is to contact Yahoo directly.
If not, HOW do you contact Yahoo, is there an appropriate 'Postmaster' email address/telephone number I can use? What do you recommend DW, BoomPop?
-
It is not uncommon for Yahoo mail to be delayed. This is the rate-limiting I've been spouting on and on about lately. :D Imagine trying to send 500,000 messages.... some messages wouldn't get through for many days and would eventually be lost AND you run the risk of crashing your server by overloading the mail queue/hardware...
Here's a link with more information about contacting Yahoo, which seems to be the only way out of this mess for legitimate marketers:
Why am I getting “451 Message temporarily deferred” or "421 Message from x.x.x.x temporarily deferred" errors when sending mail to Yahoo!? (http://help.yahoo.com/l/us/yahoo/mail/original/abuse/abuse-60.html)
-
BTW... has anyone else come to the same conclusion that I have: That ehem... Yahoo Email SUCKS! :roll:
Thank you for the tips in this thread guys. Working on it now.
-
DW, BoomPop, Mike
I've send contact info to Yahoo to see what can be done (white listing, etc).
My question to DW then - have you considered using VERP (Variable Envelope Return Paths) as suggested at yahoo website?
List managers should remove addresses that generate bounces. A particularly popular technique for managing bounces is to use VERP to identify the recipient address that has failed.
Here's the URL for more info on VERP:
http://cr.yp.to/proto/verp.txt
The suggestion to use VERP appears on this page at Yahoo:
http://help.yahoo.com/l/us/yahoo/mail/original/abuse/abuse-58.html
I've double checked and found that some email I receive at my own Yahoo account uses Domain Keys and does NOT get delivered to the inbox, while other email does not use Domain Keys and does get delivered to the inbox... So, it seems that Domain Keys for now is NOT the best way to guarantee delivery to Yahoo....
Has anyone heard back from, or considered Habeas as I mentioned earlier?
-
With AOL it is incredibly easy to get whitelisted officially.
Just call them up on the phone via their 800 number and talk to their postmaster department.
Then get them to add you to their feedback loop.
Then, you will get "SCOMP" message alerts every time someone clicks to "spam" your email.
This way you can remove those people from your lists, and then AOL will quickly and easily deliver the messages.
-
I have been considering VERP. There does seem to be support for almost all MTAs however in many cases administrator assistance may be required to set up the alias. That's not to say I'm not seriously considering implementing this. I think it's a great idea that can potentially solve the issue of "which email bounced" and also open up some improvements such as my originally unsuccessful implementation of resending bounced messages (ie. no increase in seq #). I don't see why we can't pass a message ID and the email or userid. :D