ListMailPRO Email Marketing Software Forums

ListMailPRO Email Marketing Software Forums => Server Optimization, Tweaks => Topic started by: DW on May 23, 2005, 09:30:24 am

Title: Sending to AOL, HotMail, Yahoo and other free services
Post by: DW on May 23, 2005, 09:30:24 am
It is recommended that anyone sending large amounts of email to AOL apply to become whitelisted, set up a feedback loop to be notified of spam complaints, and adhere to the guidelines.

http://postmaster.info.aol.com/tools/whitelist_guides.html

I have created a script that can be used to process AOL spam complaints to ListMail-sent email.  It will basically "click" and confirm the remove link in your message automatically to remove the user.  I will reply to this post with the (Perl) script in case you want to use it.

HotMail's official Postmaster site:
http://postmaster.msn.com/

Some interesting discussion about HotMail blocking:
http://www.webhostingtalk.com/showthread.php?threadid=224942
http://forums.devshed.com/showthread.php?t=109369

I am still continuing my efforts and have not fully analyzed Yahoo, gMail, or other free service providers.  Please post any information you have about email being blocked by free services!

Regards
Title: Sending to AOL, HotMail, Yahoo and other free services
Post by: DW on October 04, 2005, 11:38:00 am
Set up an email alias/forwarder/pipe forwarding email sent to your AOL Feedback Email to this script, which should be saved with the .cgi extension and stored in your cgi-bin folder.

Updated: June 24, 2007
Code: [Select]
#!/usr/bin/perl -w

# hostname - shown in notifications
$hostname = 'MyServer';
# notify? 1 or 0
$notify = 1;
# email to notify
$my_email = 'test@example.com';

# get piped message
$em = '';
while($l=<>){
 # replace confirm links to allow removal
 $l =~ s/confirm.php/rem.php/gi;
 $em = $em . $l;
 if(index($l,'rem.php?')!=-1){
  $rl = substr $l, index($l,'http://');
  $epos = index($rl,'"');
  if($epos==-1){ $epos = index($rl,' '); }
  if($epos==-1){ $epos = index($rl,"\n"); }
  if($epos==-1){ print "eol not found.."; return 1; }
  $rl = substr $rl, 0, $epos;
 }
}

$apos = index($em,"scomp\@aol\.net");
if($apos!=-1){
 if(!$rl){ print "AOL COMPLAINT but NO REMOVE LINK"; exit; }
 print "AOL COMPLAINT - Link Found! $rl\n";

 #randgenerator
 sub rsgen{  my $rslen=shift; my @ch=('a'..'z','A'..'Z','0'..'9','_'); my $rs; foreach (1..$rslen){ $rs.=$ch[rand @ch]; } return $rs; }

 # follow remove link
 my $rs=&rsgen(8);
 $cmd = "curl -o /tmp/aol$rs \"$rl\&dodel=1\" 1\> /dev/null 2\> /dev/null";
 system($cmd);
 # notify
 if($notify==1){
  $cmd = "mail -s \"$hostname - AOL Spam Complaint\" $my_email < /tmp/aol$rs";
  system($cmd);
 }
 system("rm -rf /tmp/aol$rs");
}
Title: Shared mail server and AOL feedback loop?
Post by: BGSWebDesign on November 08, 2005, 02:32:31 pm
Hi DW,

I had a BIG problem today with AOL.... for some reason, today is the day I started getting spam reports sent to my shared host, note I've been sending email from a shared host for years!  AOL could not confirm the problem from my DOMAIN IP address, they said all is fine... after further checking it seems that it's the shared mail domain IP that is blacklisted with MY DOMAIN indicated to be blocked... so apparently all email blacklisted from this shared mail server - but BECAUSE of my domain sending...   after calling AOL back they confirmed the shared mail server was blacklisted.

I came up here looking for an answer.... now, my question regarding this, does this 'feedback loop' script REQUIRE a dedicated server?  Here's the info on the feedback loop at aol:
Quote
AOL offers a feedback loop service, free of charge, to parties sending large amounts of mail to AOL members. The feedback loop (FBL) will forward any mail reported as spam originating from the associated IP addresses back to the listed email address. We highly recommend the use of a dedicated e-mail address for this purpose.


So, I wonder what happens if I enter the shared mail server IP address in the IP address box for the feedback loop?  Does that mean that ALL bounced email from that shared IP is sent to my email address, or does AOL filter out the feedback by REPLY-TO first and only feedback the message to MY FEEDBACK Email address?

Do you know how this might work DW, or anyone???  Urgently need to answer this...
Title: Sending to AOL, HotMail, Yahoo and other free services
Post by: DW on November 09, 2005, 12:30:33 am
Brett,

I host a number of clients running mailing lists and AOL, in my experience, has been very quick at responding to phone calls to their postmaster hotline.  I'm usually unblocked instantly and back on the whitelist shortly thereafter.  This may be in part due to the fact that I assure them I'm in control of the clients on my server and am (now) automatically processing feedback with the above-mentioned script to remove complaining users to my clients' lists.  HotMail and Yahoo can't seem to figure to offer a feedback loop, unfortunately..

Quote from: "webshaman"
So, I wonder what happens if I enter the shared mail server IP address in the IP address box for the feedback loop? Does that mean that ALL bounced email from that shared IP is sent to my email address, or does AOL filter out the feedback by REPLY-TO first and only feedback the message to MY FEEDBACK Email address?

I believe AOL can send reports to anyone regarding an IP.  I am not sure if you will receive reports for anyone mailing from the same IP.  What I recommend is:

Hope that helps! :)

Regards
Title: Sending to AOL, HotMail, Yahoo and other free services
Post by: BGSWebDesign on November 09, 2005, 07:05:40 am
Hi DW,

Thanks for your excellent reply...  one question, you say:
Quote
The application will be scanned and if rejected you might be instructed to phone the hotline.


I have setup the feedback script as you recommended, and now have an aol_complaint email address forwarding to that script for removal...
1.) How do I test it?
2.) They unblocked me immediately, but say it may be up to 72 hours for White Listing to be turned on, how long did you find it normally takes for you to be scanned, and put into White List, my host has given me until Friday...
Title: Sending to AOL, HotMail, Yahoo and other free services
Post by: DW on November 09, 2005, 09:31:28 am
Quote
I have setup the feedback script as you recommended, and now have an aol_complaint email address forwarding to that script for removal...
1.) How do I test it?

By default the script sends an email with the output of the HTML remove success page.
Code: [Select]
$cmd = "mail -s \"AOL Spam Complaint\" $my_email < /tmp/aol$random
_string1";
system($cmd);
Set the $my_email variable to your email address.
Quote
2.) They unblocked me immediately, but say it may be up to 72 hours for White Listing to be turned on, how long did you find it normally takes for you to be scanned, and put into White List, my host has given me until Friday...

I'm afraid I can't be of much more help in this area.  I didn't pay specific attention to how long each of my whitelist requests took.  I've never heard of anything about scanning.  If your host is making an issue of this and pressuring you I would find it unfair as there are other users on the server who could be causing complaints, too.  Complaints are not uncommon even with completely opt-in lists as subscribers often forget they subscribed.

Regards
Title: Sending to AOL, HotMail, Yahoo and other free services
Post by: mike2 on November 22, 2005, 12:02:42 pm
Having a problem with this script....  My feedback loop is my main email, so I  setup an email called aolcomplaint @ mydomain, when I get a complaint from aols loopback, I just forward it as a bounce to aolcomplaint @ mydomain...

This should work I believe, but here's what I get with ever email I've tried so far (about 5 or 6) and I have manually checked to see if the email address was in my database and it was there, so I'm not sure whats happening, any suggestions?

Here's what the message I get looks like, which basically says it's not finding the address, but it's looking for ( !email ), which doesn't seem to be breaking down to the users email:

Code: [Select]
<html>
<head>
<title>Email Address Not Found!</title>
</head>
<body bgcolor=white>
Your email address (!email) was not found in our database!  This error should never happen, please
contact the administrator of this web site for a manual removal.<br> </body> </html>


UPDATE:

I found what may be the problem for me, I'm not sure.  I stopped the script from deleting the temp files and noticed the one looks like this:  

Code: [Select]
<a href=3D"http://www.my-domain.com/m/rem.php?u=3D6cf977e">http://www.my-domain.com/m/rem.php?u=3D6cf977e</a>


I'm guessing it's supposed to be just one URL, not an HTML like statement.  
Let me know if this is the problem and how I'd fix it, thanks.
Title: Sending to AOL, HotMail, Yahoo and other free services
Post by: DW on November 23, 2005, 12:40:23 am
Mike,
Quote
<a href=3D"http://www.my-domain.com/m/rem.php?u=3D6cf977e">http://www.my-domain.com/m/rem.php?u=3D6cf977e</a>

The script simply finds the remove URL and 'follows' it.  It would appear that the link is coming through incorrectly, possibly by your sending of HTML mail :?:  The "3D" should not be in the string.  I suspect this link would work, with the 3D removed:
http://www.my-domain.com/m/rem.php?u=6cf977e

We need to determine where the message is being malformed.  Do you also receive the AOL email in it also being processed by the script?  Can you send me what an (HTML?) remove link looks like in that message?

The 3D should actually be %3D, I believe, and the % is being lost somewhere.  If we can find the string WITH the % we should be able to do a PHP "urldecode" or something else to format the string/message properly. :)

Regards
Title: Sending to AOL, HotMail, Yahoo and other free services
Post by: mike2 on November 23, 2005, 08:30:17 am
I'm actually not sending HTML e-mail.  A long time ago I set an option to automatically add the HTML part for aol addresses and have never changed it.

What I send my message it looks like a normal link:
http://ma-enews.com/cgi-bin/to.cgi?l=maadop

But upon examining the email returned by AOL's feedback loop, every URL in my message looks like:
<a href=3D"http://ma-enews.com/cgi-bin/to.cgi?l=3Dmaadop">h=
ttp://ma-enews.com/cgi-bin/to.cgi?l=3Dmaadop</a>

So, I'm guessing this has to be Listmail adding this.  I know I really no longer need that option because most Newer AOL browsers actually make the hyperlinks work properly by default, but I can't seem to find that option anywhere anymore...
Title: Sending to AOL, HotMail, Yahoo and other free services
Post by: BGSWebDesign on November 23, 2005, 08:55:23 am
Hi DW,

I'm also curious as to what Mike is experiencing, maybe I have the same problem?

Quote
Set the $my_email variable to your email address.


I've done that, and the script seems to be setup properly 755 permissions, etc... but I NEVER receive an Email to my email address - wouldn't I see this subject: "AOL Spam Complaint"?

I never get any... but AOL is not complaining and they have white listed me, and my host is not complaining either so I'm assuming the feedback loop must be working?

Still, I'd like some assurance it is... anything else I should try... can I run this script from the command line to test it?

How did Mike look at the local temp files?
Title: Sending to AOL, HotMail, Yahoo and other free services
Post by: mike2 on November 23, 2005, 12:32:25 pm
webshaman,

I am receiving the email from the script just fine.  But my email is showing that the email address isn't getting removed due to my URL not being formatted properly.  (Which I believe is because of an older setting that I set a while back, but can't find out how to un-set.)

I looked at the /tmp files because I run my own server(s).  I doubt if you are in a shared environment you can do this (or at least you shouldn't be able to).  

But if you look at the script, that is where it stores the files (email) for processing.  You can probably change the directory to a temporary directory under your home directory and then stop the script from deleting the files.  You would/should then be able to see them.  

As for your problem, make sure the email address you set $my_email to is correct and you kept the "\" before the "@" sign else it won't work.
Title: AOL complaint Script problem, and testing the script...
Post by: BGSWebDesign on November 28, 2005, 08:49:35 am
Hi Mike, DW,

I think this needs to be looked at in detail...  what I found is that when copying/pasting the script from above it didn't come through correctly in my editor...

This line was not correct:
Code: [Select]
 $cmd = "mail -s \"AOL Spam Complaint\" $my_email < /tmp/aol$random_string1";
since the ramdom _string1 were separated by a space!

Quote
make sure the email address you set $my_email to is correct and you kept the


I have this setup correctly...  

Thanks for the tip Mike, I've completely updated the code to write the tmp files to another temp folder on my server - and NOT to delete them, I'll let you know if I now get the emails, and what the files are showing... but still the question for DW...

How would anyone test this without re-writing the script like I have done?  It doesn't seemt his script can be run from the command line - since it's looking for input...
Title: Sending to AOL, HotMail, Yahoo and other free services
Post by: DW on November 28, 2005, 03:56:18 pm
Set up a file test with the following in it, including the top line break:

Code: [Select]

From: <scomp@aol.net>

http://example.com/mail/rem.php?u=test123

Replace "example.com/mail" with your ListMail domain and path.

Now you can test the AOL script from an SSH shell with the following command.

./aol-notify.pl < ./test

Regards
Title: Sending to AOL, HotMail, Yahoo and other free services
Post by: mike2 on December 09, 2005, 02:33:32 pm
Ok, I am still having a problem with this script for some reason...

I just updated to 1.85 because I thought that removing the undocumented URL feature to aol would fix my problem but it hasn't.

Now all I get back in my email from the script is this:
Quote
admin-60-Incorrect information in file: './maoptin/lm_chtmld.frm'


I looked at a raw view of the email I get back from aol and in every URL right after the equal sign is "3D", looks like this:
Quote
http://ma-enews.com/cgi-bin/to.cgi?l=3Dmaadop


Somehow somewhere things are getting converted to ascii or something, because when looking at the raw view of the message I see this:
Quote
=93The Short Book on the Offer in
Compromise and IRS Liens and Levies!=94


The =93 and =94 above were originally quotes ( " " ).

I am using Pegasus for my email program, could this be the problem?  I am thinking someone Pegasus is doing this and of course when I then bounce it to my aolcomplaint email address it's not right.

I know I could change my aol white list address, but it'd be easier to fix this problem (maybe), because I want to see the original message sometimes.

I am going to telnet into my server and read the email in pine next time to see if I can tell if pegasus is adding the 3D.  Otherwise I'll just need to remove the "3D" string from the URL in the script.

UPDATE:

Ok, when I bounce this mail to my yahoo email address or back to myself, the "3D" isn't there, now I am lost...

On a side note Dean, when testing a link tracking link before saving it, I get the same error:
Quote
admin-60-Incorrect information in file: './maoptin/lm_chtmld.frm'


Well, now I can't re-duplicate that error, but it showed up once when I first started setting up "Link tracking", which was just today after updating to 1.85.
Title: Sending to AOL, HotMail, Yahoo and other free services
Post by: BGSWebDesign on December 09, 2005, 02:46:22 pm
Hi Mike,

Quote
I know I could change my aol white list address, but it'd be easier to fix this problem (maybe), because I want to see the original message sometimes.


Did you try the test above from DW?

I don't believe mine is workign correctly either, and I have not had time to run the test as outlined...  but I'm not getting the proper bounced messages and I know my email address is setup correctly...

There might be some ASCII conversion going on, most likely it's in the browser, or your reader, you would need to ENCODE/DECODE when it comes off the command line to see it correctly - and that could be your problem... so it probably looks alright to the script, but that doesn't explain why I'm not getting any AOL bounce messages...  I'll run the test in the next day or so and let you know what I find... I suggest you try running the test too - from your command line...  let us know what you come up with?
Title: Sending to AOL, HotMail, Yahoo and other free services
Post by: mike2 on December 09, 2005, 03:18:50 pm
Quote from: "webshaman"
Did you try the test above from DW?


I just tried his test and I get nothing at all, no email, nothing.

Also, can you please check this out as I am getting the same kind of error I describe above when clicking on a "confirm link", which is serious.

Here's the thread I started in the general forum:
http://listmailpro.com/forum/index.php?topic=1005.0
Title: Sending to AOL, HotMail, Yahoo and other free services
Post by: DW on December 09, 2005, 09:05:37 pm
Mike,

It's really hard to say what's going on without access to your server...  I wonder if you are still experiencing the error you reported before with HTML special characters.  If you would be willing to give me access I could test this further.

Regards
Title: Sending to AOL, HotMail, Yahoo and other free services
Post by: mike2 on December 10, 2005, 12:37:10 pm
Ok I went ahead and submitted my server info for you to check out.  Please let me know what you find asap, thanks.  Go ahead and check out this problem as well, thanks.
Title: Return-Path on signup confirm
Post by: alan on December 11, 2005, 05:49:36 am
I get lots of these from AOL:

    **********************************************
    **      THIS IS A WARNING MESSAGE ONLY      **
    **  YOU DO NOT NEED TO RESEND YOUR MESSAGE  **
    **********************************************

The original message was received at Sun, 11 Dec 2005 03:55:35 -0500
from localhost.localdomain [127.0.0.1]

   ----- Transcript of session follows -----
... while talking to mailin-04.mx.aol.com.:
>>> DATA
<<< 421-:  (DYN:T1)  http://postmaster.info.aol.com/errors/421dynt1.html
<<< 421 SERVICE NOT AVAILABLE
... while talking to mailin-02.mx.aol.com.:
>>> DATA
<<< 421-:  (DYN:T1)  http://postmaster.info.aol.com/errors/421dynt1.html
<<< 421 SERVICE NOT AVAILABLE
... while talking to mailin-01.mx.aol.com.:
>>> DATA
<<< 421-:  (DYN:T1)  http://postmaster.info.aol.com/errors/421dynt1.html
<<< 421 SERVICE NOT AVAILABLE
... while talking to mailin-03.mx.aol.com.:
>>> DATA
<<< 421-:  (DYN:T1)  http://postmaster.info.aol.com/errors/421dynt1.html
<<< 421 SERVICE NOT AVAILABLE
<"snipped"@aol.com>... Deferred: 421-:  (DYN:T1)  http://postmaster.info.aol.com/errors/421dynt1.html
Warning: message still undelivered after 4 hours
Will keep trying until message is 5 days old

The people at AOL have confirmed this to be because of improper headings:

X-ClientAddr: 127.0.0.1
Return-Path: <apache@secure.blueplatehost.com>

It's the user apache in the address above that gets the confirmation email blocked. Return-Path for mailings are set to bounce@xxx.com so there's no roblem there.

How can I correct this for confirm & welcome emails?
Title: Sending to AOL, HotMail, Yahoo and other free services
Post by: DW on December 11, 2005, 09:03:00 pm
You may have success with the extra 5th paramater to PHP mail() which can correctly set the Return-Path on problematic servers.

In config.php set the line:

$phpmailf = 1;

I will be implementing an option / detection for this soon.

Regards
DW
Title: Sending to AOL, HotMail, Yahoo and other free services
Post by: mike2 on December 12, 2005, 06:32:30 am
Dean,

Any luck finding out anything on my server?

This is kind of critical, I mail everyday and people are going to start getting mad if they can't unsubscribe.   Please let me know something asap!
Title: Sending to AOL, HotMail, Yahoo and other free services
Post by: alan on December 12, 2005, 06:43:53 am
Thanks Dean, that did the trick.

I had already been aware of that 5th parameter and use it in my other PHP scripts. I was going to recommend this but did not know how you were implementing a correct return path on the Mailings but not on the Confirm and Welcome emails.

It's good to know you were ahead of the game with that config parameter.

FYI, I also had to modify /etc/mail/trusted-users and add 'apache' as a trusted user to prevent Sendmail from adding the 'X-Warning' header to the emails.
Title: Sending to AOL, HotMail, Yahoo and other free services
Post by: DW on December 12, 2005, 07:45:20 am
Alan,

I'm happy to hear that worked for you!

Mike,

I think I'm going to need more access to troubleshoot the issue.

Regards, DW
Title: Sending to AOL, HotMail, Yahoo and other free services
Post by: mike2 on December 12, 2005, 07:52:01 am
Yes I saw the ticket and just updated it with some additional stuff, check it out and let me know, thanks.

At first glance I wonder if its my version of mysql, as its 3.23.58 ???

I'm thinking some of the upgrade commands didn't work (not supported), but it's just a guess.
Title: Sending to AOL, HotMail, Yahoo and other free services
Post by: DW on December 12, 2005, 07:59:29 am
The upgrade was tested with this MySQL version and a number of people have upgraded to or installed the database set up in the update.  The MySQL error you are experiencing (http://listmailpro.com/forum/index.php?topic=1005.0) is definitely not normal...

Let's keep talk in this thread to the AOL notify script.  To test this I will require access to modify the notify script forwarded to by your email alias.

Regards
Title: Sending to AOL, HotMail, Yahoo and other free services
Post by: DW on March 29, 2006, 02:46:01 pm
Some unexpected line breaks seem to have appeared in my posting of the AOL code above (http://listmailpro.com/forum/index.php?topic=721.msg3592#msg3592).  I think I have fixed it now...
Title: Sending to AOL, HotMail, Yahoo and other free services
Post by: DW on June 21, 2006, 08:52:32 am
Working on a client server today I'm again looking into being blocked by HotMail.  It looks like they've updated their postmaster (http://postmaster.hotmail.com) page.  The most relevant information for getting unblocked seems to be at this url (http://postmaster.hotmail.com/Troubleshooting.aspx) in the "Sender services, tools, and issue submission" section.

Hope this helps someone. :)
Title: Sending to AOL, HotMail, Yahoo and other free services
Post by: DW on July 27, 2006, 03:10:18 am
AOL seems to encode certain characters when it returns the complaint-generating message. I finally fixed the script (http://listmailpro.com/forum/index.php?topic=721.msg3592#msg3592).  It now replaces =3D with = and escapes ampersands to accomodate shell commands calling the new-style remove links with more than just the "u" var value.

Let me know if you have any troubles with it.

Regards
Title: Sending to AOL, HotMail, Yahoo and other free services
Post by: BGSWebDesign on August 04, 2006, 11:13:56 am
Hi all,

I just heard about this and it looks important, the Sender Policy Framework will help email get through to major ISP's such AOL, Yahoo, etc, here's info, you'll need to use the wizard there to setup a SPF (click on What to do, Publish a Record - Wizard):
http://www.openspf.org/

Do it now, you'll be happy you did  :wink:

Here's 2 useful articles on it:
http://clickz.com/showPage.html?page=3388371
and
http://www.clickz.com/showPage.html?page=3389391
(also includes info on increasing AOL delivery rates).
Title: ARF Format - abuse reporting format?
Post by: BGSWebDesign on June 08, 2007, 11:36:41 am
Hello,

I finally got my server setup and setup the Feedback loop, now a question.

I just submitted for white list status at AOL, they are now suggesting an 'ARF' format to be used for receiving their complaints.  I'm not certain, but it mentions something about the email address (at AOL) that is complaining NOT appearing in the reports, so I wonder..

DW - do you know if your aolnotify.pl script will work with this ARF format?

You can read more about ARF format here:
http://postmaster.info.aol.com/fbl/arfinfo.html

Here's exactly what the reply webpage showed after I submitted the whitelist request:
Quote
Once approved you will start receiving feed back reports to the e-mail address you submitted. Also note these emails will originate from Scomp@aol.net. The AOL member's email address will not be included in the reports.


Does anyone know if this works with the AOL notify script for LMP?
Title: Sending to AOL, HotMail, Yahoo and other free services
Post by: DW on June 09, 2007, 02:23:06 am
Brett,

Email addresses in message headers, and possibly the body, are indeed filtered (not sure why they do this).  The AOL complainant removal script  checks for the remove link, which is not removed from each returned message.  ARF is fine - they still return the original message in it's entirety including the remove link.

Regards
Title: Sending to AOL, HotMail, Yahoo and other free services
Post by: DW on June 15, 2007, 02:10:30 am
The AOL auto-removal code (http://listmailpro.com/forum/index.php?topic=721.msg3592#msg3592) has been updated to handle more types of remove links.
Title: Sending to AOL, HotMail, Yahoo and other free services
Post by: BGSWebDesign on June 16, 2007, 08:38:46 pm
DW,

Please check your code, this new code did NOT work for me, even though the old code worked just fine!  I tested it many ways, but I can tell you this, the old code runs fine and takes about 5-6 seconds to process, the new code runs immediately (1 second), reports no errors, and does NOT send an email notifying me, nor does it remove the invalid User ID.  

Another thing to note, when running the test, you MUST find the UserID (UID field), and NOT the ID field.  Here is how you test it.  You must create a test file, call it 'aoltest.txt', mine is below:
Code: [Select]


From: <scomp@aol.net>

http://mydomain.com/mail/rem.php?u=ef9b530



Make sure you replace mydomain.com with your own domain, AND that you go into PhpMyAdmin and locate a test email address you want to remove - after locating a test address, do a browse and find that address in your MySQL database, then replace the 'ef9b530' shown above with the UID column that shows in your MySQL database for the user you want to do a 'test remove' - make sure that user has a CNF field of '1' (active) - otherwise the remove will not work and report an error (to your email address) that the user is not in the database.

Then, save it as ASCII text file and upload it to the folder you will place the aolnotify.pl script (shown below).  Then Login to your shell account and enter this at the command prompt (from the folder where you have placed the aolnotify.pl file):
Code: [Select]

./aol-notify.pl < ./aoltest.txt


You should receive an email address in the address you specified indicating that the user has been removed (see below - replace aolnotify\@mydomain.com with the email address you want to be notified at at your own domain.  You can also verify that it removed the address by reloading the MySQL database in PhpMyAdmin and looking for the email address that you used (in aoltest.txt), after you locate it, the CNF field should be changed to '2' if that address was successfully removed.

For anyone looking, here's the 'old code' (aolnotify.pl) that I used that worked:
Code: [Select]

#!/usr/bin/perl -w
# config
$my_email = "aolnotify\@mydomain.com";

# get piped message
$em = '';
$hd = '';
while($l=<>){
 $em = $em . $l;
 if((index($l,'From: ')!=-1 && index($l,'scomp@aol.net')==-1) || (index($l,'To: ')!=-1 && index($l,'Delivered-To:')==-1) || index($l,'Subject: ')!=-1){
  $hd = $hd . $l;
 }
 if(index($l,'rem.php?')!=-1){
  # print "l [$x]: $l"."l [$x]: 0123456789012345678901234567890123456789012345678901234567890123456789\n";
  $rl = $l;
  $rl = substr $rl, index($rl,'http://');
  $epos = index($rl,'"');
  if($epos==-1){ $epos = index($rl,' '); }
  if($epos==-1){ $epos = index($rl,"\n"); }
  if($epos==-1){ print "eol not found.."; return 1; }
  $rl = substr $rl, 0, $epos;
  $rl =~ s/=3D/=/g;
  $rl =~ s/\&/\\\&/g;
 }
}
$qstr = "\nFrom: <scomp\@aol.net>";
$apos = index($em,$qstr);

if($apos!=-1){
 if(!$rl){
  #print "AOL spam complaint but NO REMOVE LINK... $hd";
  exit;
 }
 #print "AOL SPAM COMPLAINT - Remove Link FOUND! $rl\n";
 sub rsgen{
  my $rslen=shift;# the length of
  my @ch=('a'..'z','A'..'Z','0'..'9','_');
  my $rs;
  foreach (1..$rslen){ $rs.=$ch[rand @ch]; }
  return $rs;
 }
 my $rs=&rsgen(8);
 $cmd = "curl -o /tmp/aol$rs $rl\\&dodel=1 \> /dev/null 2\> /dev/null";
 #print "\ncmd=$cmd\n";
 system($cmd);
 sleep(5);
 $cmd = "mail -s \"AOL Complaint Processed\" $my_email < /tmp/aol$rs";
 system($cmd);
 system("rm -rf /tmp/aol$rs");
}



After you test it, report if it works, than try it with the new version DW has updated, does it work for that version too?  I can't get the new version to work for me...  

Any ideas DW?
Title: Sending to AOL, HotMail, Yahoo and other free services
Post by: DW on June 18, 2007, 02:28:41 am
Brett, thanks for reporting this (and for your very helpful response) - the script was totally broken.  Indeed, I just grabbed what I thought was my latest revision and didn't test it. :oops: I have again updated the code (http://listmailpro.com/forum/index.php?topic=721.msg3592#msg3592) - let me know if you have further troubles.
Title: Sending to AOL, HotMail, Yahoo and other free services
Post by: BGSWebDesign on June 19, 2007, 08:45:43 am
DW,

Quote
I have again updated the code - let me know if you have further troubles.


That still doesn't work?  I get the same response, it returns immediately (< 1 second) and it doesn't set the CNF field in the database, it also does not send me any notfication.

Also, another problem.... the original code you posted (which I posted above) works when I run it from the command line, but I'm now getting emails from AOL and it doesn't appear to be working because I click the 'REMOVE' links in the attached messages I receive from AOL and these users are not removed from the database!

Shouldn't the feedback loop remove them from ALL lists immediately when it runs?

Here is a paste in of one of my messages I receive from AOL:
Code: [Select]
This is an email abuse report for an email message received from IP address
xxx.xx.xx.xxx on Tue, 19 Jun 2007 08:06:04 -0400

For information about this format please see
http://www.mipassoc.org/arf/

For information about AOL E-mail guidelines, please see
http://postmaster.aol.com/guidelines/

If you would like to cancel or change the configuration for your FBL please
contact our postmaster help desk. Up to date contact information for the
postmaster help desk is located at
http://postmaster.aol.com/contact



--------------------------------------------------------------------------------
Attachment: message/feedback-report (0.2K)


Display / Download
 


The message also contains an attachment, which I downloaded (appears below) followed by what looks like the entire original message, here's the attachment:
------------
Code: [Select]
Feedback-Type: abuse
User-Agent: AOL SComp
Version: 0.1
Received-Date: Tue, 19 Jun 2007 08:06:04 -0400
Source-IP: xxx.xx.xx.xxx
Reported-Domain: server.mydomain.com
Redacted-Address: redacted
Redacted-Address: redacted@



The entire message then appears after this - the message appears to include the correct header info, and the end of the message with the removal code, here's a paste in of most of the message:
------

Code: [Select]
From:  "My Domain" <brett@mydomain.com>
To:  redacted@aol.com
Date:  19 Jun 2007, 08:05:08 AM  
Subject:  Holleigh your *new Destiny Astrology Reading...  
           
 

--------------------------------------------------------------------------------

Hi Holleigh,

It's been a really crazy year, so I Had to email you
right away and ask you:

Do you want to have a better idea of what lies ahead for
you?

MESSAGE TEXT REMOVED.....

To be canceled from this mailing, choose the link below:
http://mydomain.com/mail/rem.php?u=8addu8v

To be canceled from ALL mailings at My Domain, choose:
http://mydomain.com/mail/rem.php?c=13&u=8addu8v


Can you tell me what is going on?  Will you provide a fix for this, for now it looks like I'm going to have to go through hundreds of messages every time this comes in and remove the users by hand, unless there is some other way to remove these users?

I keep getting this sent to my email notify address:
Code: [Select]
<html>
<head>
<title>Email Address Not Found!</title>
</head>
<body bgcolor=white>
Your email address (!email) was not found in our database!  This error should
never happen, please contact the administrator of this web site for a manual
removal.<br>
</body>
</html>


I've uncommented the print statements and the actual remove link looks fine, here's what I see from the shell:
Code: [Select]
./aolnotify.pl < ./entiremessage.txt
RL=http://mydomain.com/mail/rem.php?c=13&u=8addu8v
APOS,BPOS=-1,638
AOL COMPLAINT - Link Found! http://mydomain.com/mail/rem.php?c=13&u=8addu8v
CMD=curl -o /tmp/aolMoBhnb_p http://mydomain.com/mail/rem.php?c=13&u=8addu8v&dodel=1 > /dev/null 2> /dev/null


Do you know why this is not working???  It's strange because I can run the exact command (found above in the curl statement on the URL of my browser and it works)?  I also tried replacing the curl command with wget and it still doesn't work???
Title: Sending to AOL, HotMail, Yahoo and other free services
Post by: DW on June 20, 2007, 03:04:10 am
Brett,
Quote
It's strange because I can run the exact command (found above in the curl statement on the URL of my browser and it works)? I also tried replacing the curl command with wget and it still doesn't work???

What happens when you run the full curl command from an SSH shell?  (You can skip the 1> /dev/null 2> /dev/null part for more info)

I'm thinking that the solution is either to replace ? with \? and & with \& or, I believe, surround the URL in double quotes, as I have done in the latest code...

Also, I may have messed this part up in the latest revision:
Code: [Select]
$apos = index($em,"\nFrom: <scomp>");
I am waiting for a complaint on a test server to double-check this.  I've changed it to this just in case:
Code: [Select]
$apos = index($em,"\nFrom: <scomp");
Quote
Shouldn't the feedback loop remove them from ALL lists immediately when it runs?

Not necessarily - the script simply follows whichever remove link is found in the message.

Regards
Title: Sending to AOL, HotMail, Yahoo and other free services
Post by: BGSWebDesign on June 20, 2007, 12:36:28 pm
DW,

Quote
What happens when you run the full curl command from an SSH shell?


It acts like it runs and reports that it is calling the page, but then when I look in the database the CNF field has NOT been changed to '2', instead it is still '1' - so I don't have a clue what is going on here?

Quote
am waiting for a complaint on a test server to double-check this.


I've been debugging it myself and spent a few hours on it yesterday, what I did was just saved the entire message that AOL sent to me and I use that for testing.

You're right about the <scomp you had it right the second time, you have to check it both WITH and WITHOUT the angle brackets as the mail I'm getting now from AOL is without the angle brackets so it's like this:
Code: [Select]

From: scomp@aol.net


Quote
with \? and & with \& or


Yes I tracked down that problem too - I had to removed the extra \ in front of the &, so it's like this:
Code: [Select]

#  $rl =~ s/\&/\\\&/g;


As you see I commented out that line.

But, either way, even when I get the CORRECT SYNTAX and call the thing with CURL or with WGET - either way it does not work... so, I had a thought last night anyway - there's no reason to call this out on the command like that - it would be better anyway to just include the lmdelete.pl file and then call something similar to the lmdelete function but to call it with the UID field instead of with email - so, instead of calling it like it is called now:
Code: [Select]

LMdelete('9,10,11,13,14,15,16,24,27,28,29,32',$lmpemail);


Writing and calling a different function I'll call it LMdetelALL with the UID like tihs:
Code: [Select]

LMdeleteALL($lmpuid);


So, I'm off to look at that, if you have time, and add the function let me know, I might not have time to get to it by tonight as I'm busy on something else.   Let me know - but I think this sounds like a better way to do it - there's too much to worry about with calling WGET or CURL.

What do you think?
Title: Sending to AOL, HotMail, Yahoo and other free services
Post by: DW on June 24, 2007, 06:38:05 pm
I will consider how to better improve this script such as by removing the user from all lists.  (A simple GET var added to the command-line by the AOL script could tell a modified rem.php to remove from all lists).  Considering how infrequent complaints will be I think wget/curl should work fine - I don't see a need to go to a Perl method.

For now I have updated the code again (http://listmailpro.com/forum/index.php?topic=721.msg3592#msg3592) with one important change to avoid the problem of varying "From" and "Return-path" headers.  (I also noticed a serious bug in the last post....)

The search line is now as follows:
Code: [Select]
$apos = index($em,"scomp\@aol\.net");
Regards
Title: Sending to AOL, HotMail, Yahoo and other free services
Post by: BGSWebDesign on July 02, 2007, 12:16:34 pm
DW,

Quote
For now I have updated the code again


Congrats, it works fine now.  I did have to add in some of my own code to parse out another type of mailing I do, but all works now.