Author Topic: bounced processing - is it affected by resume.php? (Read 7777 times)

BGSWebDesign · « **on:** April 18, 2006, 12:54:26 pm »

Hi,

On another note, as you know I've been having problems with my resume.php script keeping the mailing going out - I noticed this:
My Bounced Mailbox is over 30 MBytes!

This doesn't sound right - DW - have you tested that when resume.php continues on and picks up with Dailymail that it also runs the BOUNCE processing sript that Dailymail would do?

It doesn't seem like the bounced processing is getting run properly - as the size of this bounced mailbox is huge...

Can you tell me - is there an easy way to run just the Bounce processing part of DailyMail and NONE of the other parts - can I turn OFF all the other parts except Bounce Processing and run it that way?

mike2 · « **Reply #1 on:** April 18, 2006, 01:06:29 pm »

If you un-check all other options on the "daily mail" page, then that should be the only thing that happens...

Of course you may want to wait for DW to answer that 100%...

Btw, why are you using a bounce mailbox as opposed to doing it with bounce.cgi on the fly? Just curious.

BGSWebDesign · « **Reply #2 on:** April 18, 2006, 02:08:29 pm »

Hi Mike,

Quote

why are you using a bounce mailbox as opposed to doing it with bounce.cgi on the fly?

I've found that the overhead of processing on the fly with bounce.cgi slows the system too much and prevents the rest of LMP (link tracking) etc. from running properly - especially when a large mailing goes out.

It's much better for system overhead to handle it all at once with Dailymail sometime in the middle of the night when there are fewer visitors and the system is a little more idle.

DW · « **Reply #3 on:** April 19, 2006, 11:22:56 am »

The problem with mailbox processing is that with just a few bounces (~5000) it can take HOURS to process. This is, I believe, because of hard drive limitations on large numbers of files in folders.

You may want to disable bounce processing until I rewrite bounce.cgi in 100% Perl, which will avoid an unnecessary web call through wget.

Regards

BGSWebDesign · « **Reply #4 on:** April 19, 2006, 11:38:11 am »

DW,

Funny thing, have you ever ran Dailymail with ALL options checked except for Bounce Processing - it doesn't seem to run and gives me this message:

Code: [Select]

Nothing to do!

Possible bug? I think it is, I had to track down several lines in dailymail.php here is what I changed to get Bounce Processing to run by itself with all the other options turned 'Off' (checked boxes).

Line 94:

Code: [Select]

if($errchk) echo "<input type=checkbox class=chkbox name=nobounce value=1> Skip bounce via mailbox<br>"; added 'class=chkbox'

Line 154:

Code: [Select]

if($nomaint && $nosched && $nofups && $nobounce && $noscfup) echo "Nothing to do!<br>";
Replaced: ($errchk <> '1' || $nobounce) with '$nobounce'

Line 301:

Code: [Select]

if(!$nobounce){
Changed it from: if($errchk=='1' && !$nobounce){

Well that about does it, DW you'll have to go over those changes to see if it's all ok - but for now I had to do that to get Bounce Processing to run on it's own with all the other item CHECKED, I did NOT have the top two checkboxes checked (they were left with UNCHECKED): Allow LMP to Run Automoatically, and Update Last Execution Time.

Quote

mailbox processing is that with just a few bounces (~5000) it can take HOURS to process

Are you sure of this? I've processed 1,000's of bounces before and it didn't take that long - maybe 30-50 minutes - not hours, how did you come up with this number, maybe it's not like that on my server - I don't remember it taking that long... ever, but this time I apparently have a LOT of message - I'm not sure how to check how many are sitting in there - but I'd like to read the mailbox and see how many replies are there.

No - I've confirmed it DW - on my server I just processed over 7,000 messages in 20 minutes, here's the output:

Code: [Select]

Connecting to bounced message server... Connected!
Verifying username and password... Verified!
Processing bounced messages, please wait... Done!
7834 bounced messages processed.

Quote

until I rewrite bounce.cgi in 100% Perl

When will that be? Rough estimate? Maybe it's something I could do on my own - as I need this 'yesterday'

BGSWebDesign · « **Reply #5 on:** April 21, 2006, 02:02:49 pm »

bump...

DW - see code above to fix problems in dailymail.php which prevent it from running for bounce processing only...

DW · « **Reply #6 on:** April 21, 2006, 05:36:31 pm »

Got it, Brett and I have made the changes - thanks. Since I seemed to have not been retrieving the "errchk" variable before dailymail this may have been preventing mailbox bounce processing from running at all, manually or automatically. The $errchk variable still must be used so that bounce messages are not processed for those with the option disabled.

Quote

Are you sure of this? I've processed 1,000's of bounces before and it didn't take that long - maybe 30-50 minutes - not hours, how did you come up with this number, maybe it's not like that on my server

I did one test with upwards of 15,000 messages and it took several hours to clear out the mailbox (using Outlook). I am not sure if this will be the same on all servers.

Regards

DW · « **Reply #7 on:** April 21, 2006, 05:44:28 pm »

I believe the correct fix for the bounce mailbox processing, so that it is still skipped when dailymail is run automatically for those with bounce mailbox disabled, is as follows:

Line 117:

Code: [Select]


  // check last ran time
  if(!$manual){
   $crow = mysql_query("select daylast,dayopt from $ctable where 1");
   list($dl,$do)=mysql_fetch_row($crow);

to

Code: [Select]


  $crow = mysql_query("select daylast,dayopt,errchk from $ctable where 1");
  list($dl,$do,$errchk)=mysql_fetch_row($crow);

  // check last ran time
  if(!$manual){

With the single change above the rest of your suggested changes should be unnecessary.

The $errchk var was not being loaded before dailymail execution, therefore I think that bounce mailbox processing has NOT been working as expected on upgraded installations. You seem to have confirmed this with your recent report of a plus-sized bounce mailbox, which indicates the messages were not being processed.

Regards

BGSWebDesign · « **Reply #8 on:** July 11, 2006, 10:39:05 am »

Hi DW,

Is this fixed in the latest version?

One other thing, you mention:

Quote

several hours to clear out the mailbox (using Outlook

Of course, Outlook is the problem here, you can't expect that software to process fast - it's probably running on the Access engine which is very, very slow!

As I stated above, I just noticed several hundred message and processed them quickly (see below):

Code: [Select]

Connecting to bounced message server... Connected!
Verifying username and password... Verified!
Processing bounced messages, please wait... Done!
1403 bounced messages processed.

I'm curious if this problem is fixed where it will process the bounced mail, with the Auto-resume feature AND using the email account bounced messages processing option (in configuration)?

DW · « **Reply #9 on:** July 12, 2006, 02:14:14 pm »

Yes, the mailbox bugs are fixed in v1.86b v1.87. Mailbox processing during auto-resume is not currently a feature, nor was it planned... I like it and will consider it - moreso, of course, as it becomes a necessity. If yours are processing quickly on a daily basis there shouldn't be a need for it, as bounce (removals) are also calculated on a daily basis.

Regards