ListMailPRO Email Marketing Software Forums
ListMailPRO Email Marketing Software Forums => General Help & How-To => Topic started by: DW on September 29, 2005, 03:32:16 am
-
I'm working on the next update and am thinking about providing more individual message options (headers (http://www.cs.tut.fi/~jkorpela/headers.html)). For example:
1. Message Importance: high, medium (normal), or low
2. A List-Unsubscribe: header containing a remove URL
3. Return receipt? Note: There is no Internet standard for email return receipts, thus the functionality of delivery acknowledgment is unpredictable.
Perhaps some additional headers, such as Subscriber IP and RefURL could be options as well.
"Send From" emails on a per message basis are also under consideration.
A number of other features and improvements are also being worked on - I may post here for further feedback.
Regards
-
Hi DW,
Great to hear the new version is under way!
"Send From" emails on a per message basis are also under consideration.
Headers are a good idea, but individual 'per message' headers are not necessary, and will cause overhead that will slow the mailings... please think again about this... how many people really need individual headers?
Here's some of the features I need most (IN ORDER OF IMPORTANCE) - others please chime in...
My Top 10 (in order of importance - 1st is most important)
------------------------------------------------------------------
1.) A background mailer, that 'kick starts', or 'removes bad email addresses', and continues on - REGARDLESS of what happens, I know that you've done a lot of work on this, as the v. 1.84 is MUCH better than 1.7x as far as mailings going out, but from time to time I still get a hangup, and there's no way to restart unless I manually restart the mailing (by clicking).
2.) A TRUE list copy function, that also copies my Followups... this would also let me do an IMPORT to a new list, and then copy the Followups from another list into this one, without having to copy/paste them all by hand.
3.) Split Tested Mailings (as well as Split Test link tracking), please see my last post - with full examples here:
http://listmailpro.com/forum/index.php?topic=455.0
This includes a way to enter multiple messages (4-6) that will be sent splitting them between the list being mailed to, also the links provide a way to split where they are sent to, enter a new link to be tracked, and then enter 4-6 different URLs to send the user to...
4.) Along these lines, I love your new 'Extract Links' code, and use this to extract all clicks to new tables, but it's time consuming... I send different links (for each of my user selections) in 5-8 different mailings, I would like to be able to combine 5-10 different LINKS and extract a GROUP of links into one NEW LIST.
5.) Custom HTML that allows for different Unsub ALL, Unsub ALL Confirm, Unsub ALL Error, etc... for more info see this link:
http://listmailpro.com/forum/index.php?topic=873.0
6.) Profile editing - with form validation - something like the new Profile page you sent, but that will allow admins to enter field types that can be validated for correct values (including a date entry validator). For more info see this link:
http://listmailpro.com/forum/index.php?topic=900.msg3525
7.) A way to NAME my messages, that is NOT just the Subject of the message, provide an Admin NAME that will display, copy in the Subject when the Admin enters it, and then let the Admin change this to anything they want.
8) Disabling/Enabling Followups in each list based on whether they are activated or not (checkbox on), and a generall all-around better Follow-up editor... for example, I'd like to be able to copy followups from one list to another, by clicking of a checkbox, or selecting a group of them, and then 'Copy' to list xx.
9.) A BULK remove function, display a box for me to copy/paste a group of addresses that need to be removed, and then click Remove to have them all removed from 1 list (or ALL if all checkbox is turned on).
10.) A GLOBAL blacklist, enter a list of Emails that are NEVER sent to...
11.) Backup functions working properly, including gzipping the database for easy download.
That's all I can think of at the moment (oops that's 11), I'll post more as I think of them... good luck, and please keep us informed, or let me personally know about how things might work so we can provide input - you've been great about that in the past :D
Thanks for all of your work on LMP, it's really proving itself every day...
-
Thanks for your input and kind comments :D
Many of your requests are a priority for me as well - I'll see what I can do. 8)
Regards
-
This is GREAT news...I love the progress this program is making.
I like the suggestions that have already been made but the only one feature that I personally like to add is:
http://listmailpro.com/forum/index.php?topic=815.0
Being be able to ADD followups BETWEEN existing followups.
A) Users on a followup BEFORE the one added proceed as normal, receiving the new followup in turn.
B) Users on a followup AFTER the one added proceed as normal and never receive the new message.
That's it for now...Your doing a great job Dean...Thanks!
Don
-
"Send From" emails on a per message basis are also under consideration.
This is something I need. To clarify, it is not that every single message gets a different "Send From", but rather when you send a message, you can override the from address so that it comes from a single address which is different from the default address set in the list settings. I have needed this because my list address might be sales@ but occasionally I need to send a message where I want the return address to be support@.
I could also use the ability to put "NOT" in user selection rules.
Thanks!
-
I'm working on it. I'm sure I got the "Send From" feature from you in the first place, chufford. :)
The first thing I did was put in a "text-only" domain option so that I could remove the undocumented AOL code (where text-only was sent and links were made "clickable" the "old" way for AOL members). Due to known problems with HTML messages (from some people.. others reported no problems..) aol.com will appear as default in this feature. Links will no longer automatically be made "AOL-clickable" and I may not provide an option unless it's requested as I have heard that this is no longer an issue with later versions of AOL.
Next, I think I will be working on custom fields and providing the ability for you to select which type of data to accept (ie INPUT or SELECT). These types will later be used in the Signup Code generator and a future User Profile page.
Regards
-
Some discussion about the User Profile feature and potential improvements to Custom Fields:
http://listmailpro.com/forum/index.php?topic=900.msg3525#msg3525
-
Hi DW,
I am hesitant to use JavaScript validation as it can be defeated easily. For example, someone could simply set up a form to post to your script to submit unexpected data.
Hmmm... not sure how it is defeated, can you explain? I see your point about someone submitting unexpected data, that will have to be written into the script server side - that's right, but still, Javascript validation is the fastest/easiest method I know of to validate user entry on forms, so I'm curious how someone would defeat it... unless it's a hacker, in which case the validators you build in server side will prevent that... so you probably need both.... have a look at Javascript validators, they are really powerful.
While I'm at it, I should also make custom fields unlimited. Instead of changing the user database table, breaking everyone's custom scripts, I could deprecate the user1 thru user10 fields and remove them later on with fair warning.
Hold on a minute... I use ALL of them... so if you're going to do that, just provide us with a SQL script to move them over to the new tables if that's what you choose! Yes, I can see that having them in the user table is limiting, and really not the best way to do it... so put them in a separate table, just provide us OLD users a quick way to update to the new tables and I'll be happy :D
-
Hmmm... not sure how it is defeated, can you explain?
As I mentioned, someone could simply set up a form to post to your script to submit unexpected data bypassing all JavaScript. I know JavaScript is powerful and can probably do the same regular expressions (used for validation) as PHP, and look nicer, without having to "submit" the page to see an error, but it would be redundant along with server-side.
... so if you're going to do that, just provide us with a SQL script to move them over to the new tables
I don't expect you to move the data yourself. Instead of providing a script I will build this right into the upgrade.
Unlimited custom fields with styles and options is a tough feature that will take several days coding to get right. I plan to allow you to set your own variable names (with a few exceptions). For compatibility with the old-style, where each list can have variables named user1 thru user10, I plan to make the feature similar to message codes, with Global and list-specific fields. I am still debating as to whether or not to allow customization of the first name, last name, and email fields. Default text, a few options (auto capitalization, for example) could be implemented for these. I could even allow list-specific fields for them to complete the customizability. It doesn't make a lot of sense to visit the "Custom Fields" page for these standard fields, however.. Of course, it would be useful as far as functionality is concerned.
Work, work, work :) ListMail is going to be the hands-down best mailing list manager available. :D
I'm still working on the new site (http://lmhost2.com), too. I plan to change licensing. I'd like to offer bundles of licenses at a discount that can be applied to your account, for additional sites/domains (when applicable), or turned into NEW accounts, for resellers!
Regards
-
Hi,
I see your point on the Custom Fields...
Back to my Top 10, I have one thing to add here about Split Testing...
Please, please also provide a way that the SUBJECT of the Email can be entered as part of the split tested message. That way if we do NOT want to change the BODY of the Email, we can just type in new Subject lines in the block of Subject/Body provided for Split Testing and get an Email that will have the SAME body, but different subject lines!
Right now this is a big pain, as I have to setup Link Tracking, and try to send to small groups on my list, to determine the best Subject Line...
Subject Line testing is an integral part of Split Testing.
Thanks, and keep us posted on the update...
-
Hi Dean,
I need the first feature you mentioned in the original post.
1. Message Importance: high, medium (normal), or low
One client often wants me to mail out an urgent email to his list and I can't accomodate this request when he does.
Thanks,
David
-
Hi DW,
I'd like to add another feature request, one that I forgot earlier:
------------------------------------------------------------------------
BOUNCE LIST MANAGEMENT: I'd like to see a much better way for managing Bounced Users, for example, I'd like a utility to MOVE them to another 'Bounced List' at regular intervals, or manually, AND, when that happens their Bounces are REMOVED, and they are back to ZERO bounces, so I can try mailing them again, with a 'Lost Email'message or something similar, this is way to attempt to recover bounced users...
Also, regarding: One client often wants me to mail out an urgent email to his list and I can't accomodate this request when he does.
I believe that 'Header Controls' would let you do that as well, without the need for Urgent messages, I can't see much value in 'Urgent Messages', I've never used them, or known them to work at all, perhaps you should ask your client why they find value in using them?
-
My client is not highly technical. He just knows that when someone sends him an important message, the little red ! shows up in the email list of his Outlook program. If there is another way to accomplish the same thing, that is fine with me.
David
-
Hey Dean,
Awhile back, I posted something like this image (form). Can it be done in 1.85?
Thanks,
Greg
(http://camdenboatworks.com/custom.jpg)
-
Hi,
Here's another one I thought of, I really hope you can implement this in the next release, as it's very important to my marketing schedule.
I really need to have Day of Week scheduling, for example, schedule a Followup on Monday, that means that if the User joins the list on Sunday, they won't see their first message until Monday, likewise if someone joined up on Monday (after the followup went out) - they wouldn't see their first followup until ONE WEEK later, on the next Monday...
Followups after that also proceed on DAY OF WEEK, for example, next followup on Wednesday, then on Sunday, then Friday, etc...
Also, if you could provide CALENDAR date scheduling that would be great too, for example, Followup1 goes out on March 3rd, Followup 2 on March 10th, and Followup3 on April 3rd.... This type of scheduling would NOT indicate the Month, but instead the Calendar Day of the month. Perhaps a schedule Column titled 'Calendar Day', selectable as 1-27 (not sure how you'd handle 28-31 because of February and other months less than 31)... I don't really need them, but others might?
I see these as being TWO SEPARATE schedule types, for example EITHER one, or the other, but NOT both at the same time, for example, if you Choose the DOW (day of week) scheduling, you cannot ALSO choose the Calendar Day scheduling, and likewise if you choose the Calendar Scheduling you cannot also choose DOW scheduling.
That's it... the list is getting longer, I know, sorry about that... keep chugging along Dean, maybe release this one as two separate releases, so you're not trying to fit it all in one?
-
I've been a little busy with some custom work - one of which is a new feature in ListMail. :-) A client let me know how to do the "Not Active" query for user selection, but only on MySQL 4. I tried to work it into User Selection but it became way too confusing to create rules. What I did was simply added a feature that allows you to enter completely CUSTOM SQL for user selection instead of rules. How 'bout that?
Greg, I will have to consider how to best implement your feature. I'm a little swamped right now - I've got to stop jumping on these free modifications all the time and getting sidetracked. :) A small donation could get me moving in your direction sooner. ;)
I've got some testing to do as far as optimizing how fast each message is processed and built (a client sending large messages reported slow sending speed) and will then be again working hard on the update.
Brett - another good suggestion and one I have found myself thinking about as I plug away here. I envision daily, weekly and monthly emails with a start day/date for weekly and monthly mailings. A monthly mailing with a start date would eliminate the need for many options (drop down to choose the "day" of each month to send) and problems with months having different numbers of days. I could implement this into the current interface on the Send Email and Scheduler pages. The options for sending could read "Now", "Date", "Daily" (with days of the week beside it), "Weekly" (with the "start date" beside it), and "Monthly" (also with the "start date" beside it).. Not too hard, really. I'll see what I can do. :) If I missed something let me know.
I'm really looking forward to getting all these features into the program and making ListMail #1!
Regards
-
Hi DW,
added a feature that allows you to enter completely CUSTOM SQL for user selection instead of rules.
Wow, I love it! Thanks, just be sure to include plenty of SAMPLES/Examples on how to do the difficult queries... otherwise this is great!
"Now", "Date", "Daily" (with days of the week beside it), "Weekly" (with the "start date" beside it), and "Monthly" (also with the "start date" beside it).. Not too hard, really. I'll see what I can do. If I missed something let me know
Ok, here's the trick with this, I'm not sure if your idea will work with how I need to do it, but you can fill me in... these are NOT weekly, or Monthly, or Daily, or Date, instead they are appearing on a DAY OF WEEK (DOW)
Followup1 - Monday (regardless of when user joined list)
Followup2 - Wednesday (after the Sunday mailing - followup1)
Followup3 - Sunday (after the Wednesday - of followup2)
Followup4 - Thursday (after the Sunday of followup3)
---- NOTE change of scheduling here ----
Followup5 - Monthly on 3rd of Month
Followup6 - 5 days after Followup5
Followup7 - Monthly on 3rd of Month (hits the next month)
Followup8 - 5 days after Followup7
.... etc, etc, with these, until possibly RESTART With Followup5 (but NOT restarting from the beginning, and NOT including Followup1-4 in the Restart).
Ok, hope that's clear... I know may be interesting to program...
I've got to stop jumping on these free modifications all the time and getting sidetracked.
Yes, you do, they'll eat you up... Please tell me how much you would need to get started on this Scheduling Update, I need it and it will save me time if you can finish it within a week or so?
Good luck, and keep us posted on how it's going... I'd really like to see a nice 'ETA Post' - with possible idea of when certain features may be ready.... these scheduling features are becoming more important for me.... hope you can help.... man, you're going to have to increase price on 1.85... that's for sure ; )
-
Dean,
First of all...thanks for your constant improvements.
I would really enjoy seeing even small updates at a time...it's fun and very useful.
The people that use your program everyday, such as myself, really look forward to any new features and updates...it makes our day. Even one new feature every so often is great...it doesn't have to big a whole new version.... Please upload updates as they come along ...
I am very happy to say that I made a great investment when I purchased your program.
Thanks for all the hard work!
Don
-
Don, thanks for your suggestion! I will consider releasing less, more often, in order to keep the hype alive.
Brett,
I can add DOTW, no problem - I forgot that one. :)
Followup1 - Monday (regardless of when user joined list)
Followup2 - Wednesday (after the Sunday mailing - followup1)
Followup3 - Sunday (after the Wednesday - of followup2)
Followup4 - Thursday (after the Sunday of followup3)
---- NOTE change of scheduling here ----
Followup5 - Monthly on 3rd of Month
Followup6 - 5 days after Followup5
Followup7 - Monthly on 3rd of Month (hits the next month)
Followup8 - 5 days after Followup7
.... etc, etc, with these, until possibly RESTART With Followup5 (but NOT restarting from the beginning, and NOT including Followup1-4 in the Restart).
This could be a bit tricky.
Let's see - we can accomplish followups 1 thru 4 with a simple little addition to signup-xtra.php setting the users' Seq#1 delay to a certain value based on the day of the week.
Followups 5 and 7 are straightforward, a monthly email on a certain day of the month.
As for 6 and 8 - this is the tricky part. :) I have an idea: "sub"-scheduled messages - basically a followup attached to a scheduled message... The difficulty / downside of this approach is that we would have to track who received what message and when. I am not sure I can do this without creating a new table to hold the data. I wonder how well this type of table data would perform on large (or even medium..) installations where millions upon millions of messages are sent. :( I have seen MySQL slow down accessing tables this large without proper server optimizations. Many servers are not optimized so this may not be feasible for the majority. Then again, we might be able to store less data somewhere else, such as keeping the user IDs of people due to receive certain messages instead of ones that have already been sent. While this will still take up space (and a new table) it will perform better under load. Is it worth creating a whole new table for this one sub-scheduled followup feature? :?:
On a side note: Considering monthly emails further, it IS going to be necessary for extra options to deal with the fact that each month has a different number of days.
ie:
[Send] -> [Monthly]
1st of the month thru 28th (?) of the month
Last day of the month
Last day -1 thru Last day -28
1st Monday thru Sunday
2nd Monday thru Sunday
3rd Monday thru Sunday
4th Monday thru Sunday
That's an overwhelming number of options. What if I simply used a 3 parameter cron-style timing? (omitting hour and minutes or, in the future, just minutes) :) That would be a heck of a lot easier to program and work into the interface and would offer far more diverse options that would be exhaustive for me to implement manually (such as every second Sunday in December). As per my proposal, sub-scheduled messages might also be able to be added to scheduled messages that use cron-style timing.
Regards
-
Don, thanks for your suggestion! I will consider releasing less, more often, in order to keep the hype alive.
Dean,
That's a great way of saying it..."Keep the hype alive". This kind of stuff does get around on the net, and could help increase sales...that's how I heard about your program to begin with. Hype on a forum about how good "your personal" service was. and the quality of your program.
Can't wait for the next upgrade..."releasing less, more often"...great idea :D
Don
-
Hi DW,
What if I simply used a 3 parameter cron-style timing? (omitting hour and minutes or, in the future, just minutes) That would be a heck of a lot easier to program and work into the interface
I think you're taking this approach way too far...
I can see this being done with simple programming code in the front end of the Admin interface that ADDS the followup to your followup list. The second scenario (MONTHLY scheduling) is more important to me right now, the DOW scheduling has some value, but a little further down the road... ok, so maybe implementing the MONTHLY scheduling now and leaving the signup-xtra.php as something to come, or give me quick idea on how to do that, and I'll code it up...
Now to the second scenario (MONTHLY scheduling)
As for 6 and 8 - this is the tricky part. I have an idea: "sub"-scheduled messages - basically a followup attached to a scheduled message...
I don't see why you would require 'sub' scheduling here? Basically you just need to look at when the next followup occurs and calculate the number of days until WHEN the next monthly date shows up - why would you need to 'sub schedule'? and HOW do you see this:
I am not sure I can do this without creating a new table to hold the data
working, HOW would you see this new table working, and what layout would it be? I have a BSCS degree with concentration in Database Design, so doing things like Normalizing tables for performance are what I'm used to working with... if you tell me what you're tyring to do, I'll spend some time optimizing how you could do that with the tables... though, I don't believe it's necessary, tell me first why you need to 'sub schedule'?
-------------
UPDATE
------------
What if I simply used a 3 parameter cron-style timing?
Ok, I've thought about your 'CRON' approach a little further, and if that helps you and is EASY to implement then GO FOR IT! I can work with the cron style format with no problem, and if you can pull it off that way, GREAT!!!! Basically I guess you're calculating the number of seconds until the next FOLLOWUP based on the CRON type entry - that's fine... let's see a working example, etc... as I mentioned, I need this in the next few weeks, if it's possible, let me know, if you need a donation to get moving on this?
-
Brett,
ok, so maybe implementing the MONTHLY scheduling now and leaving the signup-xtra.php as something to come, or give me quick idea on how to do that, and I'll code it up...
The signup-xtra.php would be pretty easy. You can get a numerical representation of the day of the week with the php date() function. The users' first delay would be calculated to land on the day you like and the rest of your followups would be set, via standard delay, to go out on certain weekdays afterwards.
Basically you just need to look at when the next followup occurs and calculate the number of days until WHEN the next monthly date shows up
Okay, so the followups table could be modified to contain the id of the scheduled message to send after. The users' 'delay' would be calculated upon reaching that followup based on the followup delay (# days after scheduled message to send) and days until the scheduled message is to be sent. The followup sequence would pause until that message is sent. I can see this working.
Ok, I've thought about your 'CRON' approach a little further, and if that helps you and is EASY to implement then GO FOR IT!
I think the cron approach would give us some extremely powerful timing options and be reasonably quick to implement. With it you will be able to do daily, monthly, every 3rd of the month (except february), etc etc. Eventually I could even work some of it into an option-rich interface, using the cron string representation as a back-end, so it's easy for people with little technical knowhow to set up complicated schedules.
As I mentioned, I need this in the next few weeks, if it's possible, let me know, if you need a donation to get moving on this
This is very doable. I can always use a little extra money (still workin' my way out of a small debt but am making progress!) so it would be great if you could pay me for this modification. It will benefit all ListMail owners so I'll do it cheap - $50 (http://tinyurl.com/9fyx5)? I can start Wednesday and if I work fast I could have it done by Friday/Monday. 8) Did you want the followup modification too?
Maybe I should release this and the few other features I've implemented (http://listmailpro.com/changes.php) before unlimited custom fields..
After that, I should finish and release the new site (http://lmhost2.com) (set your forum theme to Classic) and work those changes (new help page format, logo, etc) into the next update.. :o
I'm optimistic, motivated, and working hard - I'm sure I'll be out of the red soon :)
Regards
-
Hi DW,
pay me for this modification. It will benefit all ListMail owners so I'll do it cheap - $50? I can start Wednesday and if I work fast I could have it done by Friday/Monday. Did you want the followup modification
Sure thing, I've sent the money for update... thanks a million... regarding 'Did I want followup Modification'? What does that mean, I thought that's what we were doing?
Anyway, I've sent a Contact Email about it to you personally.... Really need the CronStyle Followups, AS LONG AS they can work in succession, from one to the next... etc :) I'm sure you get the picture... from the details laid out above... and in email...
I think the Email I sent was lacking something... really what I need is as follows:
Followup1: First Tuesday of Month
Followup2: 3 days after Followup1
Followup3: First Tuesday of Month (month AFTER Followup1)
Followup4: 5 days after Followup3
etc, etc...
With possibility to CYCLE the followups BACK to First Followup in the list at any time.... but probably at the END of the Followups... How about a 'RECYCLE Followups' option....
That's it.... email me directly for more, and good luck...
-
I've been working my butt off on this one (Cron Followups). The interface is done; The error checking is done; The help page (http://listmailpro.com/help/followups.html) is (mostly) done; I have a great cron class all ready to implement and initial testing is showing success. All I have to do now is put it into dailymail.
I will be putting a few more features in before releasing this to the public as it requires database changes. I may find a way to release it to Brett first, since he paid me for it.
With possibility to CYCLE the followups BACK to First Followup in the list at any time.... but probably at the END of the Followups... How about a 'RECYCLE Followups' option....
This is possible with a small custom addition noted in this post (http://listmailpro.com/forum/index.php?topic=232.0). I will make sure to include a "Reset Followups" button to reset all users to the start of the sequence. I am very likely to put in a loop feature but it might not be right away.
Regards
-
Hi DW,
:D Sounds promising... had a look at the Help Page and I'm very pleased...
One question:
Example: 1,2 3,4 5,6
Result: The 1st or 2nd of March or April, but only if it's a Friday or Saturday
I guess this means I can also do something like:
1-7 * 1-5 which would hit the FIRST (and only first) week of any month, only if it's Monday-Friday... so, the first Weekday of the month?
Let's say that I want to hit the FIRST Weekday of the month, so I need Monday-Friday, then, I want the NEXT Followup to also hit the NEXT following FIRST Weekday (of the following month).... how would I do that, would the next followup Cron be the exact same as the Cron above?
One further question... since this is how I'll really need it, the First Weekday of any month to START the sequence, and then a delay of 5-7 days for Followup2, and Then a CRON for the Next First Weekday of the Following Month for the Followup3, and then 5-7 days after for the next Followup4....
Thanks again for your great work, I can't wait to see the update... and yes, I would appreciate getting it first :)
-
1-7 * 1-5
which would hit the FIRST (and only first) week of any month, only if it's Monday-Friday... so, the first Weekday of the month?
Yes, that's right - although, the weekend can only be 2 days long so you should also be able to use 1-3 * 1-5
Let's say that I want to hit the FIRST Weekday of the month, so I need Monday-Friday, then, I want the NEXT Followup to also hit the NEXT following FIRST Weekday (of the following month).... how would I do that, would the next followup Cron be the exact same as the Cron above?
Yes, that's right, but if the two crons are right next to each other you will need to put in a delay of a few days on the second one to get around the days following the first one.
One further question... since this is how I'll really need it, the First Weekday of any month to START the sequence, and then a delay of 5-7 days for Followup2, and Then a CRON for the Next First Weekday of the Following Month for the Followup3, and then 5-7 days after for the next Followup4....
How is the delay 5-7 days calculated? Will this just be at your discretion? There should be no problem doing this. You can use a cron followup to start it off, a delayed followup second, another cron, and another delayed.
I'm working on getting this feature finished. Here's my latest issue (I'd like your feedback).
The cron feature matches between given "Last" and "Now" times. I was thinking that I do not necessarily need to modify the user table to store the "last followup sent" date/time for each user. Since dailymail is supposed to run every day without fail I could simply run the cron with the "last" time exactly 24 hours before the execution time. This should work perfectly. The downside of cron-based followups used in this fashion, however, is that if your dailymail fails to execute on a given day someone could be waiting a while to receive a message they should have received a week or a month before. Storing the "last followup sent" time for each user will partially prevent this problem because the match will occur next dailymail no matter what. The remaining problem is if dailymail fails for a number of days in a row and the date skips past matching TWO cron-style followups - again, the user will have to wait until it matches again... So both methods have a similar drawback but one does not require modifying the user table and has a somewhat higher chance of failure. What do you think? Should I avoid modifying the lm_users table? (I'd like to) Is this the ultimate drawback to cron-style followups? I think this will always be a problem when trying to send on specific days. The dailymail failure issue might also affect (future) scheduled messages as I don't see how to send "any set for today or BEFORE" with a monthly/weekly mail.
Maybe there's a feature I can create to help smooth things over when dailymail fails. If I stored when dailymail ran I would be able to tell when it didn't run, and could cycle through previous days "mimicking" the execution on alternate days! Hmmm... it's like the anti-skip feature for portable CD players. :) "[X] Automatically "catch-up" when Dailymail fails" on Config page? :D
I guess I will proceed without modifying lm_users and consider putting in the catch-up feature ASAP.
Regards
-
Hi DW,
Sounds great, ok, here's a few things, first you ask:
How is the delay 5-7 days calculated?
This is an interesting question, initially I thought, well, 5-7 days from actual first send, BUT, if I don't know which day IS the first weekday of th e month, that means I also will NOT know the number of followup days to provide for hitting the NEXT Weekday a week ahead followup, so really what I need is Followup1 - First Weekday of Month, Followup2 - Next available Weekday of Month ONE WEEK LATER (only if that is at least 5 days)... does that make sense, for example, if this month, I hit First Weekday on Tuesday - November 1, 2005, THEN, I want Followup2 to hit next week on Monday November 7, 2005, BUT only IF the number of days between Followup1 and Followup2 is AT LEAST 5 days... so in this example it would be at least 5 days and so ok, so you tell me - would I use CRON and also include DELAY to do something like that - I think I figured it out, see next paragraph...
For another example, if the First Weekday of Month was on a Thursday, for example Thursday December 1, 2005, THEN since the next followup Weekday one week ahead from that is Monday December 5, 2005 IS LESS THAN 5 days out, I need to make an adjustment and wait another day. oh I think I have it, instead I would use a CRON such as this:
"7-13 * 1-5" for the second followup which would guarantee that it occurs in the Second Week but only a weekday : ) Great!!!
... So both methods have a similar drawback but one does not require modifying the user table and has a somewhat higher chance of failure. What do you think? Should I avoid modifying the lm_users table? (I'd like to) Is this the ultimate drawback to cron-style followups?
I think it would be best if you did NOT modify lm_users, adding data there will only INCREASE exponentially the amount of total disk space I use, I have enough subscribers that adding anything there adds quite a bit to my database size....
I was thinking that I do not necessarily need to modify the user table to store the "last followup sent" date/time for each user. Since dailymail is supposed to run every day without fail I could simply run the cron with the "last" time exactly 24 hours before the execution time.
Yes, I think this is a better option, and I like what you say at the end BEST:
If I stored when dailymail ran I would be able to tell when it didn't run, and could cycle through previous days "mimicking" the execution on alternate days! Hmmm... it's like the anti-skip feature for portable CD players. "[X] Automatically "catch-up" when Dailymail fails" on Config page?
This is BRILLIANT, good job.... yes, I believe this is the best solution, you STORE when DailyMail last ran - date/time or whatever you need (IN CONFIG or somewhere other than in lm_users, store it only once), then provide the 'catch-up' option, but tell me HOW would the 'catch-up' option try to 'catch-up'? You have to be very careful here, does it try to RUN twice instead of ONCE every time it runs, or what do you have in mind for how DailyMail tries to 'catch-up'?
Also, since you're storing when dailymail runs, why not also show on the Config page (Last DailyMail Execution: dayofweek mm/dd/year hh:mm), and if possible, even allow Admin to MODIFY that, that way Admin would have complete control for modifying when next DailyMail runs... or HOW quickly the catch-up tries to 'catch-up'.... is this possible, in other words, if Admin sees DailyMail didn't run for 7 days, being worried about next execution THROWING everything off, I go into Config, turn Catch-Up ON (if it's off), and then set Last Daily Mail Execution 1 MONTH behind... this provides me with a faster 'catch-up', and GUARANTEES that any CRON style Followup will be caught up exactly to the point where I am now :D
One requirement of allowing Admin to personally set the Last DailyMail Execution is that they MUST get the correct dayofweek OR it will cause problems... unless you do NOT let them set that, that would be best, show the last Daily Mail Execution as indicated above, let Admin set the mm/dd/year hh:mm, but do NOT let Admin set dayofweek... then you calculate dayofweek when you do your 'catch-up'...
Ok, that's about it... sounds great, when will it all be ready/fully tested... I'm happy to wait until Monday next week if required? Or, if you want me to setup some of my own TESTING here, I'll test it out on my TestList?
Let me know either way...
-
I think I have it, instead I would use a CRON such as this: "7-13 * 1-5" for the second followup which would guarantee that it occurs in the Second Week but only a weekday : ) Great!!!
Yes, it could work this way or you might be able to do it with a delay of 4 or 5 and a cron task set for any weekday. I'm tired right now it's hard to imagine. :) I should have this ready for you to play with tonight.
HOW would the 'catch-up' option try to 'catch-up'? You have to be very careful here, does it try to RUN twice instead of ONCE every time it runs, or what do you have in mind for how DailyMail tries to 'catch-up'?
I was thinking dailymail would run or loop multiple times faking the 'current date' for all missed days and finally running the current days processing. This would result in sending multiple followups, in order for all missed days, at next execution of dailymail. I could extend the 'date' feigning to message codes so date codes are processed based on the previous day(s). This might be an important option for people with time-based messages. For example, a "10 day course" where in each message they project the same end date (when you can project dates with the formatted date message code - soon:).
Also, since you're storing when dailymail runs, why not also show on the Config page (Last DailyMail Execution: dayofweek mm/dd/year hh:mm), and if possible, even allow Admin to MODIFY that, that way Admin would have complete control for modifying when next DailyMail runs... or HOW quickly the catch-up tries to 'catch-up'...
Changing this will not effect when the actual server dailymail cron task runs the script. Apart from being able to modify the 'catch-up' feature (to send the wrong messages) I don't think there is a benefit to allowing modification of the last dailymail time.
if Admin sees DailyMail didn't run for 7 days, being worried about next execution THROWING everything off, I go into Config, turn Catch-Up ON (if it's off), and then set Last Daily Mail Execution 1 MONTH behind... this provides me with a faster 'catch-up', and GUARANTEES that any CRON style Followup will be caught up exactly to the point where I am now
The next execution could easily throw everything off without catch-up on. I think setting the last execution far before the missed dates would only cause problems. The catch-up would run with 'incorrect' dates which I envision, as above, also using for message codes. With catch-up on, running a loop through the days missed will result in the sending of any followups, cron-style or delay-based, just as if dailymail had run on each day. Cron-style followups should be guaranteed to be caught up correctly.
Ok, that's about it... sounds great, when will it all be ready/fully tested... I'm happy to wait until Monday next week if required? Or, if you want me to setup some of my own TESTING here, I'll test it out on my TestList?
I should be able to give it to you tonight around or shortly after midnight PDT. All of my testing has resulted in success so far but I still recommend you try the new feature on a test installation. You can run dailymail manually with various cron-style followup settings to test the results.
Regards!
-
Hi DW,
'date' feigning to message codes so date codes are processed based on the previous day(s). This might be an important option for people with time-based messages.
That's really nice... 'projected message-codes'... I didn't know they were coming, but that's nice too, really nice.... having the 'feigned dates' pop in based on the 'catch-up' would be really, really cool... great job of programming on that one :shock:
This is sounding great, just a quick question... I have a further need with this, when you mention:
You want to send an email out not the very next Sunday after the last followup, but the Sunday 2 weeks from then. To do this, set up a Delay of 13 and a cron set for "Any Sunday" which is, as per the example above, * * 0
on your help page.... is it possible to put in a -13 delay to specify that I need this sent out, for example 13 days BEFORE the CRON set runs out... here's an example... FOLLOWUP1 needs to go out 2 weeks BEFORE 1 year elapses, FOLLOWUP2, 3, 4, etc.. then proceed on DELAYS... after FOLLOWUP1.
The real key here, is that FOLLOWUP1 can go out 2 weeks prior to the end of 1 year passing... is that possible with the way it is setup now? Can you allow negative values, when CRON is also set? It would really help, unless you can suggest a better way to do this with a CRON string alone (without me having to write my own code to calculate which month 12 months later would be, then subtract 2 weeks from that date)? I can't think of any other way to do it, can you?
I should be able to give it to you tonight around or shortly after midnight PDT. All of my testing has resulted in success so far but I still recommend you try the new feature on a test installation
Please take your time... it's not necessary that I have it tonight at midnight, as I don't work on weekends.... so, feel free to play around with this until Monday AM if you like... as far as testing goes, I hope that you've covered all scenarios, as I really want to start it rolling on my current list... are you suggesting I install an entire Other version of LMP and test on that? Or that I try to do this by running test CRON Followups on a test list within my current LMP?
-
Just curious, when do you expect to release the next version? I'm looking forward to some of the new features!
-
I'm going to put a few more features in and then will be releasing. It should be ready in the next week :lol: or two :D or maybe three. :)
Regards
-
Hi DW,
Ok, great, just a few questions on how this works... before I make changes, I have two questions:
1.) Subsequent Monthly CRONS will eventually not trigger
------------------------------------------------------------------
Quote:
Let's say that I want to hit the FIRST Weekday of the month, so I need Monday-Friday, then, I want the NEXT Followup to also hit the NEXT following FIRST Weekday (of the following month).... how would I do that, would the next followup Cron be the exact same as the Cron above?
Yes, that's right, but if the two crons are right next to each other you will need to put in a delay of a few days on the second one to get around the days following the first one.
Please explain this in detail, are you saying that 2 CRON schedulings that are in sequence MUST allow for a delay from the 1st to the 2nd? Otherwise they will both trigger and run in the same month? If the first CRON executes... you do not consider running the 2nd one until the next day... so... you're saying that FOLLOWUP2 in this scenario requires a delay of the number of days that may follow FOLLOWUP1 that could trigger FOLLOWUP2? So, for example, using November/December as example... FOLLOWUP1 is as follows:
1-3 * 1-5
This would trigger FOLLOWUP1 on Tuesday, Nov 1, 2005, the next CRON then must allow for a delay so that it does NOT hit on Nov 2-4, which would require something like:
1-3 * 1-5 DELAY: 4
Which would mean that FOLLOWUP2 will go out on Dec. 5th, (4 days delay from Thursday December 1st - which basically throws FOLLOWUP2 into the second week in December, instead of the first, is that right?
I think I see how this works, but am a bit confused by the logic (which can be fixed very easily), so it seems that you run the tasks as straight CRON jobs, and do not flag when one has been executed - and which month they run in?
You could solve this problem if you added a flag to your Followup table for each followup, the flag would be a simple MONTH number - since it appears that the only time this will be a problem is one month to the next (I don't think this problem could occur with other scheduling, ie. daily, weekly, day of week, etc, at least I couldn't think of a scenario)... So with the flag you set it to the MONTH that the FOLLOWUP runs in: in the example above you set it to 11 after FOLLOWUP1 executes... what this does is prevent FOLLOWUP2 from running in November, because you check the flag BEFORE running each followup, if it is the current month - and the CRON schedule for the followup is a MONTHLY (* in the 2nd CRON position), you ignore this followup... this would keep happening for each day in November that would trigger it, UNTIL you are in the month of December, since the monthly flag is less than the current month on December 1st, you can now trigger FOLLOWUP2. :)
Do you see how this would prevent the problems you speak of, and would help those out who do Monthly scheduling... I think it's going to be necessary, because if you don't allow this to run this way, subsequent followups are ALWAYS going to be have to be delayed further and further ahead of the previous FOLLOWUP, for example think of a situation where you have 10 FOLLOWUPS, all of them Monthly, each one would be triggered later and later in the month under your current logic, that would never work, since after about 2 or 3 followups, the 4th followup would NEVER get triggered properly.
2.) CRON followups with negative delays
-----------------------------------------------
You did not answer my previous question about allowing for a followup that is 2 weeks PRIOR TO a yearly schedule, for example:
The help page says:
Cron-style followups can also use the Delay value. Example: You want to send an email out not the very next Sunday after the last followup, but the Sunday 2 weeks from then. To do this, set up a Delay of 13 and a cron set for "Any Sunday" which is, as per the example above, * * 0
I'm asking that you allow for a negative delay here ONLY if the CRON schedule is being used, that would allow for the example I posted:
is it possible to put in a -13 delay to specify that I need this sent out, for example 13 days BEFORE the CRON set runs out... here's an example... FOLLOWUP1 needs to go out 2 weeks BEFORE 1 year elapses, FOLLOWUP2, 3, 4, etc.. then proceed on DELAYS... after FOLLOWUP1.
These two issues are going to present a problem, most likely for all users, can you please modify to fix these issues?
-
Which would mean that FOLLOWUP2 will go out on Dec. 5th, (4 days delay from Thursday December 1st - which basically throws FOLLOWUP2 into the second week in December, instead of the first, is that right?
I don't think so. The value of Delay is processed BEFORE the cron task, not after. The delay will decrease to 0 over the days and THEN the cron task will be processed when valid.
As for monthly scheduling, etc. I am thinking to provide a "repeat" option, either repeating it a few times or unlimited times. One might have unexpected results if they set a scheduled message to run on "any day" of a given month - the message might be sent EVERY day of that month. This can be avoided by specifying the particular day to send on.
I'm asking that you allow for a negative delay here ONLY if the CRON schedule is being used, that would allow for the example I posted:
Quote:
is it possible to put in a -13 delay to specify that I need this sent out, for example 13 days BEFORE the CRON set runs out... here's an example... FOLLOWUP1 needs to go out 2 weeks BEFORE 1 year elapses, FOLLOWUP2, 3, 4, etc.. then proceed on DELAYS... after FOLLOWUP1.
This might be difficult. Could you simply set your first delay to 352?
Regards
-
Hi DW,
I don't think so. The value of Delay is processed BEFORE the cron task, not after.
Got it, yeah, that makes a lot of difference, thanks :)
I think you should UPDATE your Cron Help Page to make this clearer, I didn't know if the Delay ran BEFORE or AFTER the Cron schedule.
As for monthly scheduling, etc. I am thinking to provide a "repeat" option, either repeating it a few times or unlimited times.
This won't help me much as all of my monthly mailings are different, I could see where it may help others though - nice feature...
This might be difficult. Could you simply set your first delay to 352?
Yes, sure this will work, I just thought it would be nicer to FORCE it into an exact yearly schedule and with leap years it will be off by one, but not bad, 1 day off every 4 years...
Also, knowing that the delay runs BEFORE the CRON helps, I can set the delay to 352, and then a CRON for a weekday everything will be fine...
Thanks again, setting it up now, I'm going to add myself into some lists, and I'll let you know if I see any problems..
Update - Questions - Followup
----------------------------------
Ok, here's a quick question, I've been setting up the CRONs on my Followups, but here's what I wonder: I have many lists already setup with DELAYED followups, I am changing those to CRON followups (with NO delay), can you tell me if the CountDOWN takes place as normal, since I see in the User Database that the Delay is currently reading 7, will that Count down and THEN start the CRONs? If it will, I think it's best if I can just SET ALL THE Delays to Zero for these lists right now, and let the CRONs run as if the list just started... can you send me a bit of SQL to reset all of the Delays to 0 in specific lists?
Formatting
------------
It would be nice to SEE the CRON on the Followup Screen, instead of 'Y', and having to Edit to see what the CRON setting is.
-
Brett,
I think you should UPDATE your Cron Help Page to make this clearer, I didn't know if the Delay ran BEFORE or AFTER the Cron schedule.
Sounds good - I have done this now. I might also update it further to include examples of areas of confusion we have mentioned in this post.
Yes, sure this will work, I just thought it would be nicer to FORCE it into an exact yearly schedule and with leap years it will be off by one, but not bad, 1 day off every 4 years...
That would be better, and a 25% failure rate is pretty bad... This could be solved with a new feature "send followup X days/months/years after signup/last followup". The feature could cause ListMail to simply process the # of days or years, taking leap years into account, and setting the delay #.
I have many lists already setup with DELAYED followups, I am changing those to CRON followups (with NO delay), can you tell me if the CountDOWN takes place as normal, since I see in the User Database that the Delay is currently reading 7, will that Count down and THEN start the CRONs? If it will, I think it's best if I can just SET ALL THE Delays to Zero for these lists right now, and let the CRONs run as if the list just started... can you send me a bit of SQL to reset all of the Delays to 0 in specific lists?
Countdown will take place as normal. If your users have a delay it will be decreased over the days to 0 and then the cron will be processed or standard followup will be sent.
The MySQL query you are looking for is (example for list 1):
UPDATE lm_users SET cdel = '0' where list = '1'
It would be nice to SEE the CRON on the Followup Screen, instead of 'Y', and having to Edit to see what the CRON setting is.
I thought about this, too. I would love to display it on that page. The only thing is the cron-style string can be up to 255 characters long and I can't fit that amount of information in the small space. Make it a textbox with a restricted size.. yes, of course. I'll do this but need to put the error checking in for the cron task there. No problem. 8)
Regards
-
Hi DW,
Ok, I've found what seems to be a problem with the NEW Cron scheduling...
I've added new CRON style scheduling to about 15-20 of my lists... everything seemed to be updating fine, DELAYs counting down, even AFTER I added the new CRONs... but.... here's the problem...
After I went into one of the LISTS and manually set the delay down to ZERO (using the MySQL query):
The MySQL query you are looking for is (example for list 1):
UPDATE lm_users SET cdel = '0' where list = '1'
... I just noticed this morning that the OTHER lists that I have setup with CRON style scheduling - the DELAY is NOT counting down... it has stayed at 5 (where it was when I used the MySQL query above) to set the other list down to zero...
Can you PLEASE HAVE A LOOK at this ASAP.... it's going to force me to do manual DELAY updates on ALL of my CRON style lists until you get this fixed!
This appears to be happening with ALL of my Lists, it seems like DailyMail is not even getting called?
FOLLOWUP
-----------
Ok, had a quick check, and DailyMail has not executed since October 27th, about 4 days ago... so I tried running the 'wget' call that I have in my local CRON which is below:
15 2 * * * /usr/bin/wget -O - http://mydomain.com/mail/dailymail.php?pw=xxxxx
I ran that from the command line, and I get this error:
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
[<=> ] 0 --.--K/s
Done!<br><br />
<b>Warning</b>: mysql_fetch_row(): supplied argument is not a valid MySQL result resource in <b
>/home3/mydomain/mydomain-www/mail/admin.php</b> on line <b>2499</b><br />
admin-45-You have an error in your SQL syntax. Check the manual that corresponds to your MySQL
[ <=> ] 369 319.48B/s
09:25:08 (319.46 B/s) - `-' saved [369]
There's an error on line 2499 in admin.php which is this line:
list($nbounce)=mysql_fetch_row(mysql_query("select nbounce from $ctable where 1",$link) or die('admin-44-'.mysql_error()));
I never had this problem previously? Have you updated something in admin.php that is causing this problem? Is there some other problem with MySQL?
FURTHER FOLLOWUP
-----------------------
Ok, I found/fixed the BUG at line 2499, it should read like this:
list($nbounce)=mysql_fetch_row(mysql_query("select nbounce from $ctable where 1",$link)) or die('admin-44-'.mysql_error());
But, when I fix that, I now get an error on the next mysql_query line, this one:$urows = mysql_query("select id,list,bounces from $utable where email like '$email'") or die('admin-45-'.mysql_error());
I get this error:Done!<br>admin-45-You have an error in your SQL syntax. Check the manual that corresponds to your [ <=> ] 183 87.23B/s
09:43:48 (87.22 B/s) - `-' saved [183]
So, it's bombing on this mysql query? Why, I don't know because it seems that it is IDENTICAL to what you had in the last version that runs just fine - apparently it does not like to 'die' here, because the rest of DAILYMAIL does not get executed.... so, it seems there's some problem with this mysql query? I'm going to switch it back to the old code WITHOUT the 'or die' and see if that works...
FURTHER FOLLOWUP
-----------------------
Ok, I removed all of the 'or dies' in the function bounce($email,$msg), now DAILYMAIL will run, BUT, I do NOT get my Delays updated, AND I do NOT get my Dailymail report, so something else has gone wrong.. it seems you have introduced a 'bug' DW in the code in admin.php that now prevents DAILYMAIL from running properly... this will have to be fixed... I cannot even RUN, or TEST ListMailPro until this is corrected.... I'm going to try to go BACK to the previous version and add in the CRON stuff... myself... I need this fixed ASAP as my CRONs will start firing off soon...
CORRECTION - FURTHER FOLLOWUP
------------------------------------------
On further checking, I noticed that DAILYMAIL was still processing in the send queue, though I did run it again (will that cause problems)... anyway, so it's been run twice now.. and it does appear that removing the 'or die' from the function bounce does FIX the problem.... so there's some problem with the MySQL in this line of code: $urows = mysql_query("select id,list,bounces from $utable where email like '$email'") or die('admin-45-'.mysql_error());
ADDITIONAL FOLLOWUP - CRONs (followups with delays) being triggered when they should NOT BE!
---------------------------------------------------------------------------------
Here's another problem I just found, I noticed in the tracking database I'm getting hits to a list that I have setup with CRON followups, BUT, they should NOT be getting them, there are only 2 followups, as indicated below:
FOLLOWUP1 CRON: 15-19 * 2-4 Delay: 0
FOLLOWUP2 CRON: * * 1,5 Delay: 2
As you see, followup1 should ONLY go out on the 15-19th of the month, and only if the day is a Tue-Thursday... followup2 ONLY goes out AFTER followup1, and ONLY if the day is a Monday or Friday, somehow, I notice now that in the list - FOLLOWUP2 has GONE OUT! Apparently FOLLOWUP1 has NOT, as I do NOT see any clicks on Tracked Links that were included in FOLLOWUP1 (so I assume that NO Followup1's went out). Anyway, this is troubling indeed, as I see that the SEQ # = 3 now for some of these users who have received FOLLOWUP2 - HELP - DW, is there some problem/reason why FOLLOWUP2 Triggered BEFORE FOLLOWUP1? Is there a problem with setting a CRON, then a CRON/DELAY, without having a FIRST (PLAIN DELAY) for the Followups?
Any ideas on what is going on here, DW??
-
Brett,
PHP doesn't seem to like the "or die()" statements nested. I noticed this when adding the statements to better track errors. I thought I had fixed them all but must have missed this one (also, a bracket was misplaced...). I didn't notice because I wasn't processing any bounces in my testing. :(
Change the line:
list($nbounce)=mysql_fetch_row(mysql_query("select nbounce from $ctable where 1",$link) or die('admin-44-'.mysql_error()));
To:
$brow = mysql_query("select nbounce from $ctable where 1",$link) or die('admin-44-'.mysql_error());
list($nbounce)=mysql_fetch_row($brow);
CRONs (followups with delays) being triggered when they should NOT BE!
...is there some problem/reason why FOLLOWUP2 Triggered BEFORE FOLLOWUP1?
I've done some testing but haven't recreated the error yet. I've tried, for example, setting up followup 1 for "tuesday" (* * 2) and followup 2 for "monday" (* * 1) to see if #2 would go out before #1 (it being Monday now), but nothing is sent, as expected. Also, with some debug output I see it only checking the cron entry for followup 1 to determine whether or not to send. I'll keep trying. Is there any chance at all some of your users were already on Seq #2? The MySQL statement I provided simply decreased delays to 0 - it was not intended to set all sequence numbers to 1.
Regards
-
Hi DW,
PHP doesn't seem to like the "or die()" statements nested.
What is the proper way to handle the 'or die()' then? Since this statement also fails:
$urows = mysql_query("select id,list,bounces from $utable where email like '$email'") or die('admin-45-'.mysql_error());
even though the one above worked just fine by fixing the parentheses?
Is there any chance at all some of your users were already on Seq #2? The MySQL statement I provided simply decreased delays to 0 - it was not intended to set all sequence numbers to 1.
It is possible.... but I'm not sure how, can you please provide the MySQL statement to set the SEQ # back to 1?
Thanks,
-
What is the proper way to handle the 'or die()' then? Since this statement also fails:
Are you sure it's all on one line? Do you get an error? I'm not having troubles with this one:
$urows = mysql_query("select id,list,bounces from $utable where email like '$email'") or die('admin-45-'.mysql_error());
To update all users to Seq #1 and Delay 0 this is the query (example for list 1):
update lm_users set cseq = '1', cdel = '0' where list = '1'
Is there any way you can test the new changes without using your live lists so-as to prevent undesired results with your active clients?
Regards
-
Hi,
HEY - BIG HINT... here's a problem, ran your update query, this one:
----------------------------------------------------------------------------
update lm_users set cseq = '1', cdel = '0' where list = '32'
and I got this:
Affected rows: 4447 (Query took 0.2094 sec)
BUT, looking at LMP it says:
Active Users 1-20 of 14335
ANY IDEAS????? As you can see, LMP reports 14,335 Active Users in this List (list 32), but the query only affect 4,447?
So, 4,447 were the users that had the WRONG delay, I'm starting to wonder, maybe these users were the ones that were added - in the last 4 days (when DAILYMAIL was NOT running properly)... that MUST be the problem, look into it.... this is what I wonder about, ADDING NEW users, to this list, is causing SOME of those users to MISS the Cron Schedule... test it out and see DW, I believe that could be it...
Are you sure it's all on one line? Do you get an error? I'm not having troubles with this one:
Yes, it's all one line, I reported the problem above:
Done!<br>admin-45-You have an error in your SQL syntax. Check the manual that corresponds to your [ <=> ] 183 87.23B/s
09:43:48 (87.22 B/s) - `-' saved [183]
Any ideas? This bombs, so I took out ALL of the 'or die' in the admin.php module after the first error (line 2499 and below).
Is there any way you can test the new changes without using your live lists so-as to prevent undesired results with your active clients?
No, it's the best way to do it, I've got email that must go out... so, 'let er rip'.... I'll keep watching to see if the sequences/delays get off... I'll let you know what I find.
-
Any ideas? This bombs, so I took out ALL of the 'or die' in the admin.php module after the first error (line 2499 and below).
Maybe something went wrong when removing the "or die" statements. I'm going to send you my admin.php via email. I have verified the fix for the "nbounce" line and successfully processed bounced messages via mailbox with it.
Regards
-
Hi DW,
Unfortunately, nothing went wrong when I replaced the 'or die', I'm an experienced programmer (20 years), I know how to make those replacements...
I received your update, but as I mentioned previously, this line does NOT execute (it bombs with a MySQL error), here's the line of code:$urows = mysql_query("select id,list,bounces from $utable where email like '$email'") or die('admin-45-'.mysql_error());
Here's the MySQL error straight from the command line:Done!<br>admin-45-You have an error in your SQL syntax. Check the manual that corresponds to your [ <=> ] 183 87.23B/s
09:43:48 (87.22 B/s) - `-' saved [183]
I have specifically listed this ERROR message above, DW, it seems that there is some problem with it executing, perhaps it has to do with the Version of MySQL (I'm running MySQL version 3.58 on this domain), maybe it's the older version of MySQL, anyway, they do NOT plan on upgrading on that server - so, perhaps you'll need to do away with the 'or die' calls for the next release.... since others may be running older versions of MySQL as well...
Let me know what you think?
Also, can you please answer my post at the end of this thread - I believe it's important for multi-list user Profile pages:
http://listmailpro.com/forum/index.php?topic=900.0
-
Brett,
I am running MySQL "3.23.58-1.73" in my testing.
The next step would be changing the code to find the error. It's possible that a malformed email address is coming through that needs to be addslashes()'d. Take the query and place it in a variable so it can be echo'd in order to see if there is a problem with it.
$cmd = "select id,list,bounces from $utable where email like '$email'";
echo "CMD=$cmd<br>";
$urows = mysql_query($cmd) or die('admin-45-'.mysql_error());
If the $cmd line reads as follows there should never be the possibility of a syntax error, unless $utable is somehow undefined.
$cmd = "select id,list,bounces from $utable where email like '".addslashes($email)."'";
I don't have time to work on the user profile script right now as I have a couple paid things to take care of and need to get the above issue sorted out and get back to work on the update.. I hope to eventually get the feature into ListMail.
Regards
-
Hi DW,
I'd like to test this, but I can't test it that way... I'm running in live mode, probably the best thing would be if you could, PLEASE provide a way to dump some of this info to a .log file (LOG FILE), it would help others too, who are trying to debug, that way you could just ask them for their log and you could determine what is going on...
should never be the possibility of a syntax error, unless $utable is somehow undefined.
I think this MAY be the problem, here's what I noticed... 1.) Bounced members from yesterday to today has NOT increased, even though Active members have, so I believe their should have been some bounces, 2.) I find in my bounced email box a group of addresses in there... I've saved that file, and tomorrow will look to see if they are removed... obviously if they are not removed tomorrow there IS A PROBLEM with the bounce tasks being handled properly...
so maybe $utable is not defined - as I mentioned above.... I can't keep running Dailymail to test it... please tell me the code to put in to write the info to a log file? OR - another option, when running DailyMail from LMP provide a checkbox (run in TEST MODE), which does NOT increment the Delay, and does NOT send messages?
That is the best way, right now, I can't just run dailymail over and over to test it, since it will BLOW my scheduling!
Further Update
----------------
Ok, I've confirmed that the BOUNCE handling seems to be working fine, so it seems that the $utable variable is fine... what's next for testing?
-
I apologize for the delay - I know these errors are hard on you. Did you try adding "addslashes()" around the $email variable as I mentioned? If you did, you may not have had a chance to test it. Before doing that I'd be curious to see the output of the query causing the error.
I have finished work on the new dailymail options. You can now disable individual parts of dailymail when running it manually. I have sent the new files to you by email. Code note: I had foolishly mixed the report with followup processing and had to move a few things around and add an extra cycle through the list loop to get the report working. I'm quite certain I've got it all sorted but please let me know if you notice any errors.
Let me know what you discover regarding the error during bounce processing - I'd love to get to the bottom of it. I think it could be caused by invalid addresses being allowed through. If that is the case the addslashes() suggestion should fix it and a call to the valid_email() function would guarantee input is as expected.
Have your cron-style followups been ok the last few days?
Regards
-
Hey Dean...
I'm going to put a few more features in and then will be releasing. It should be ready in the next week or two or maybe three.
I'm ready to take a look at the new version... Can't wait :D
-
Brett,
Have you had any problems with cron-style followups lately?
Don,
I am working away on the update and my new site @ lmhost2.com (http://lmhost2.com) (I changed the front page sales copy - let me know what you think). I've been a little distracted by a lot of support. I will put some emphasis on the ListMail update tonight and over the coming days - there are a lot of great suggestions to implement!
Regards
-
Dean,
The new site is looking good..."it's getting better all the time" (I just got home from seeing Paul McCartney)
Anyway...I did not want to bring this up (but I will) one comment about LMP...I love the program and can't wait for the new updates... I personally do not think it will be peoples first choice for an autoresponder or online newsletter etc mainly because there is no WYSIWYG html editor incorporated into the program.
I do not need this feature but I think the average user is "expecting" it in the program. Most of the other similar programs do have some sort of WYSIWYG html capabilities.
And one more thing, like I said before. I think it would be a good a idea to release smaller updates more often..."Less is more"..
I want LMP to be the number selling program of it's kind...I think it will benefit all of us :D ...Good luck, and I look forward to the next release.
Don
-
Hi DW,
Ok, I've installed two new .php files, Dailymail.php and Admin.php (for testing purposes only, to see what's causing the sql query problem).
One thing I noticed, I belive you should put this in Dailymail.php (line 23) - when running tests from the Command line? Of course in normal mode, they then need to be commented out - so PERHAPS, you should consider this - add a 'Testing' checkbox on Config Screen, if that's ON then all of these vars are set to 1, otherwise, they are set to zero:
$nomaint = 1;
$nosched = 1;
$nofups = 1;
$nobounce = 0;
right after this line: if($pw) $isadmin = checkpw($pw);
since I am running from the Command Line, I'm not sure those vars get set in the html code, so I added the code above, to ALLOW bounce processing, and turn off all other processing...
Adding the code you mentioned:$cmd = "select id,list,bounces from $utable where email like '".addslashes($email)."'";
echo "CMD=$cmd<br>";
$urows = mysql_query($cmd) or die('admin-45-'.mysql_error());
Did you try adding "addslashes()" around the $email variable as I mentioned? If you did, you may not have had a chance to test it. Before doing that I'd be curious to see the output of the query causing the error.
I did not try it WITHOUT the addslashes, but I can tell you adding the addslashes solved the problem, as the bounce processing went through the bounces as expected and completed it's execution as normal... so that fixed it! :) Please update your code accordingly, and add the Addslashes code in Admin.php (line 2489).
I have finished work on the new dailymail options. You can now disable individual parts of dailymail when running it manually. I have sent the new files to you by email... quite certain I've got it all sorted but please let me know if you notice any errors.
Yes, it works fine, thanks.... except see note above, I believe you should add the var setups at the beginning (for testing purposes only), and ALSO, you should consider writing to a .log file (for running from browser) - then write the .txt file out to browser, I don't see ANYTHING on the screen when running this from the browser, so generally I run from command line to see what's going on...
Have your cron-style followups been ok the last few days?
Yes, all has been fine with the followups, looks like you're good for the update.... can't wait to see what else you've added...
-
Brett,
Glad to hear you braved the new code. :)
Adding the addslashes solved the problem, as the bounce processing went through the bounces as expected and completed it's execution as normal... so that fixed it! :) Please update your code accordingly, and add the Addslashes code in Admin.php (line 2489).
I added the following to the top of bounce() function in admin.php to prevent unnecessary wasting of resources.
if(!valid_email($email)) return false;
addslashes() is still beneficial to defeat sql-injection if the valid_email() regexp is defeated.
That was a pretty big hole... :oops: I need to get an update out ASAP.
since I am running from the Command Line, I'm not sure those vars get set in the html code, so I added the code above, to ALLOW bounce processing, and turn off all other processing...
No, the vars don't get set when running from the command-line, so everything executes. Your addition is one way of working around it or you could have alternately set the variables via GET in the URL. ie. http://example.com/mail/dailymail.php?pw=mypass&nofups=1
ListMail automatically grabs all POST and GET variables regardless of the "Set Globals" PHP option. (top of admin.php) Note that POST overrides GET.
I will consider adding a "test mode" for all dailymail operations - good suggestion.
I don't see ANYTHING on the screen when running this from the browser, so generally I run from command line to see what's going on...
The variable that determines display is on this line of dailymail.php:
if($warn=='no') $outp = 1;
I don't appear to be clearing the $outp variable before or after this if() statement, so you could try adding &outp=1 to the URL.
Thanks a lot for your recent testimonial, by the way (where's your site link??)! I often feel at odds with you, especially when I make mistakes! Indeed, I carry a great deal of responsibility for a number of internet businesses' successful operation. As you can imagine, when things go wrong it can be a bit stressful... Let's just say having your post there means a lot to me. :) I also greatly value your input and help developing ListMail, and your patience when things get busy. Thanks for being such a great client. :D
I'm at work on more features for the update. 8)
Regards
-
Hi DW,
I added the following to the top of bounce() function in admin.php to prevent unnecessary wasting of resources.
I'm not sure that's the way to handle that! Think about it, since the bad email address MAY already be in the database, right? If the bad email address has a chance for slipping into the database to start with, it does need to be REMOVED from the database with the bounce code, so that it's not there anymore? I would prefer to lean toward the 'safe side' of things and make sure the code gets executed (by using addslashes) to ensure that AL BAD email addresses are removed from the database - this will guarantee that....
If there is any chance the bad email address may be in the database, then using the 'addslashes' code would make sure that it also gets removed from the database, but only if there is NO chance for the bad email address to make it into the database would the code you mention work you must know the answer to that. but please re-consider as I mentioned to guarantee that any and ALL bad email addresses are truly bounced and removed from the database.... the last thing I ever want is bad email addresses hanging around in the database... I would prefer that you leave OUT THE "valid_email" code, and instead just put the addslashes code in...
I will consider adding a "test mode" for all dailymail operations - good suggestion.
Thanks, that will solve many problems!
One thing, I forgot again to add my 'throttle code' into admin.php which has cost me many lost emails (and sales)... I suggest again that you consider adding that code, and adding a SWITCH for it in Config, enable Throttle, and include options for setting the number of emails to queue, and the throttle delay time... here's the code again:
if(($y % '100')==0 && $y <> sizeof($sendq)) sleep(25);
I place this at line 1712 in the current admin.php.
so you could try adding &outp=1 to the URL.
I don't see the output when running from the browser - so you'll need to think about how you can fix that, IF you want the user to see the output...
I'm at work on more features for the update.
I can't wait to see the new ones.... thanks again for your great work!
-
Brett,
Think about it, since the bad email address MAY already be in the database, right?
Actually no, there shouldn't be any. Since a very early version ListMail has done email format checking on all imports, additions, and edits. The only time this is ever an issue anymore is if a client is using a 3rd party script to add users to the ListMail database directly. It might be best to implement a feature that could be run every so often to verify email addresses. Even over long periods of time, however, without using a 3rd party script I don't think you will get any bad ones.
One thing, I forgot again to add my 'throttle code' into admin.php which has cost me many lost emails (and sales)... I suggest again that you consider adding that code, and adding a SWITCH for it in Config, enable Throttle, and include options for setting the number of emails to queue, and the throttle delay time... here's the code again:
I'll see if I can get this feature into the update - it's a fairly easy one. I'm not sure what you mean by setting the number of emails to queue, though - that could be more difficult.
Right now I'm diving into checkboxes on the User Database page as it's been far too long without them. With them I'm adding mass user operations. ie:
WITH [Selected] OR [All 123 Results] (spans multiple pages)
DO [Copy to List] -> [1: List Title] -> Next Followup[1][2][3]
OR [Move to List] -> [1: List Title] -> Next Followup[1][2][3]
OR [Remove]
OR [Remove From All]
OR [Create Selection]
OR [Set to Followup][1][2][3]
OR [Change Data] -> [Field] -> [New Value]
etc.
This should be fairly straightforward to implement and will be very useful to many, I'm sure. Can you think of any other operations that could be put in here? Do you think I should also add mass user operations for User Selections in light of the above feature(s)?
I will finish this and some changes to followup addition/renumbering I promised to Don before considering an official update.
UPDATE: After doing some thinking I probably shouldn't be working on the user checkboxes/mass user ops right now... While I did make some progress on it it is more time consuming than I had previously foreseen and way off track with some of the improvements I had planned on making ASAP. I will finish what I promised, add some other very important things like queue/dailymail protections and a resume script, add a few easier features and get the release out. If I manage to work quickly enough perhaps I will complete the above. If not, it'll come soon.
Unlimited custom fields may not make it as it is a difficult feature to implement, requiring many database / script changes, and I really want to get the new files out due to the bounce() security issue. I may also quickly implement some simpler requested features such as your Copy List feature which I just need to find room for. :)
After this update there will still be lots I want to get done quickly.
Regards
-
Next up is Don's request:
Being able to ADD followups BETWEEN existing followups. As I'm going to start on this tonight I thought I'd try to get some feedback.
I've been doing some thinking about this one as it's a high priority. It should be fairly quick to implement once I get it all planned out.
A) Users on a followup BEFORE the one added proceed as normal, receiving the new followup in turn.
B) Users on a followup AFTER the one added proceed as normal and never receive the new message.
Some issues that I have not completely figured out follow:
1) Adding a new followup and giving it a lower existing seq # could create a JavaScript confirmation pop-up warning of the automatic re-arranging of users. This is easy enough.
2) Changing an existing followup Seq # to a LOWER existing number should result in the same thing.
3) Changing an existing followup seq # to a HIGHER existing number should ALSO result in re-arrangement, with a gap left by the old moved # (automatically skipped).
4) This is a tough one. What if you change the list of an existing followup to another list that contains the same seq #. It should have the same warning and re-arranging... This will take some JavaScript storing EACH list # and ALL of its seq #s to be checked upon saving. This data would need to be loaded on both the followup editing / adding page and the main followup listing pages. For those with thousands of followups across dozens of lists the page would load a bit slower (not too much, considering the data is all integers), but it should work perfectly.
4a) Update: This could also be accomplished with an intermediary 'warning' page eliminating the bulk of JavaScript and giving you even more options!
5) If the target seq # does not already exist do NO re-arranging.
6) To take out the guesswork when you're on the message editing page and can't see your list of followups I could also change the "Add Followup" button to something like this:
Add followup.. (Select Box)[At beginning of sequence][After 1][After 2][After 3][etc][At End of Sequence] (Submit) [GO]
Maybe I do have it figured out. :) I'll stop talking and get to work... :D No responses necessary.
Regards
-
I have made progress on the new followup feature. At the same time I am attempting to eliminate 'gaps' and dupes in seq #s when possible.
NEW FEATURES (taken from new followup help page):
- On the Followups main page or when adding or editing a followup simply set the Seq # to an existing value, the one you want the message to be inserted before. When you click Save you will be presented a warning box informing you that your existing users will be automatically re-arranged upon insertion of the followup. When a followup is inserted all followups with a higher seq # increment by one along with their users. Users who were previously at the end of sequence are placed at the new end of sequence with a delay of 0.
- To delete a followup click "Delete" next to the followup on the main followup page. All followups with a higher seq # decrement by one along with their users. When deleting the last followup in sequence all users found to be at or above the 'end of sequence' are adjusted to be at the new lower value - therefore, they will be in queue to receive the next followup when or if you add it.
UPGRADE:
- remove followups with duplicate seq #s and email data to admin
- update all users on non-existent followup to next followup or eos (end of sequence)
- clear out existing gaps. simple loop through followups on each list incrementing a variable ($i) starting at 1. if $i does not match the next seq # the seq # is changed to $i, along with all users with that seq #. repeated until all followups are in order. all users at new eos or higher are placed at new eos
:?: Eliminating duplicate seq #s on upgrade is essential for the renumbering followups routine. Both are not really required but are more cosmetic. I could take them out in favour of simply updating all users at eos or higher to the proper eos. This might take some of the stress out of the upgrade on live installations. I have tested the features quite thoroughly and believe I have created and compensated for all scenarios, but since we can't/aren't eliminating gaps and dupes completely it doesn't seem necessary.
NOTES:
Due to the severity of some of these operations I would release a Beta version in the hopes that some knowledgeable ListMail clients would help with testing.
This will not completely eliminate the ability to create duplicate seq #s, which currently serve no purpose and may/will cause undesired results. You are able to create duplicate seq #s if you wish using the confirmation dialog. It will also not completely eliminate potential gaps, such as when moving followups (though, I will be eliminating this one) or renumbering them manually.
I am not currently regularly eliminating gaps unrelated to the current insert/delete operation. Should this be a regular process?
MORE FEATURES
I added a simple "Reset Users" button on the Followups page, but is it enough? What about a "Reset all users on [Followup X or EOS] to Seq [ ] Delay [ ]" feature? This should/will also be possible with the upcoming mass user operation features.
Let me know your thoughts
-
Hi DW,
Your work sounds very promising, I like how you've described it and believe keeping the addition direct entry of sequence numbers is fine...
Regarding this:
What about a "Reset all users on [Followup X or EOS] to Seq [ ] Delay [ ]" feature?
That would be GREAT! Is this something separate from LOOPING, or do you see this replacing the LOOPING feature? I could see that this would be used to set a sequence to LOOP after a specific followup, and that is VERY MUCH needed!
remove followups with duplicate seq #s and email data to admin
- update all users on non-existent followup to next followup or eos (end of sequence)
- clear out existing gaps. simple loop through followups on each list incrementing a variable ($i) starting at 1
This all sounds promising, but since I have little trouble with sequence numbers/followups it's not very helpful to me, and it's not likely that I will test this for you.... as you know I'm busy testing the new CRON scheduling features which are working brilliantly :D
Keep up the good work and I look forward to the updates coming more frequently, especially hoping for a 'COPY FOLLOWUP' option, or 'COPY GROUP OF FOLLOWUPS' from one list to another... MUCH NEEDED...
Background mailer (kick-start) and Split-Tested Mailings/Link Tracking are still also VERY IMPORTANT- see my original top 10 list posted on page 1 of this thread...
Really, though I believe you could solve ALL Of this by just posting the list in a Poll (you can do that in this forum?), and ask the users which are most important to them, that way you can get a feel for what is MOST important to MOST users - and this should be your priority, over anything else, even what others want you to do... please consider doing that...
-
Hi DW,
I've been using the CRON scheduled followups and they're working great, in fact, I LOVE THEM, but... there's still one issue that comes up, here it is:
I schedule followups in specific lists to run once a month, on the 1st, 2nd, 3rd, or 4th week of the month... I can do that easily by specifying the CRONs as follows:
FOLLOWUP LIST1: 1-5 * 2-4
FOLLOWUP LIST2: 8-12 * 2-4
FOLLOWUP LIST3: 15-19 * 2-4
FOLLOWUP LIST4: 22-26 * 2-4
As you can see these followups go out once a month for each of the corresponding lists, but only on Tuesday, Wednesday, Thursday, and only in the 1st week (for list1), 2nd week (for list2), etc... NOW, here is where my question comes in...
Using this setup, I can do fine with covering all of the Months, even those that start midweek, and I can still hit my 4 weeks every month... BUT, I'm still left some months with an extra week (week 5) where I could make good use of a scheduled followup - IF AND ONLY IF there is a WEEK 5 in that month, if there is not, I'd prefer that followup to be SKIPPED... so you can see the problem here, IF I schedule a 5th week as follows:
FOLLOWUP LIST5: 28-31 * 2-3
That followup will ONLY HIT if there is a 5th week (that is, a day from 28th to 31st that falls on Tuesday-Wednesday), that is fine and seems to limit that followup to ONLY those months with a 5th week (according to my definition), but HOW can I advance this FOLLOWUP if it is NOT hit during the month? If you're looking for an example see September 2006, there are 4 FULL WEEKS under my CRON Schedule, but the 5th week here is NOT available, since it comes too late in the month and is not a true 5th week.
I think what I'm after here is some type of WEEKLY CRON parameter (with a MONTHLY reset) that allows for ADVANCEMENT if it is NOT hit during that month? I know that this is NOT part of the standard CRON setup, but it would be interesting to think about adding to your CRON class DW? Unless you can think of something else, I can't see any way I can setup a followup to hit the 5th week - AND to also SKIP a month if it is not HIT?
What do you think DW?
Is there some way to force a followup reset - but it needs to be done automatically so I don't have to watch the calendar every month and see when it happens...
-
It would be quite easy to program and schedule a script to run once a month, before dailymail on the 1st, resetting your users to the first followup. Would this solve it?
Regards
-
Hi DW,
It would be quite easy to program and schedule a script to run once a month, before dailymail on the 1st, resetting your users to the first followup. Would this solve it?
I think I should have said 'skipped' and not reset.... read below...
I don't believe it will solve it, since some users would need to be reset to different followups, the FOLLOWUP they need to be reset to is the one IMMEDIATELY AFTER the FOLLOWUP that would NOT get triggered at the end of the month (on 4 week months).... as you see, this will not solve it..
Really what needs to happen is that FOLLOWUP needs to be skipped...
Have a look at a yearly calendar and you'll see what I mean... Most of the months have 5 weeks in them according to my schedule, only a few will NOT have 5 weeks, on those that do NOT have 5 weeks then this FOLLOWUP needs to be skipped, and the next following one go out INSTEAD, so that it skips that one on months that do NOT have 5 weeks in them (according to my schedule)....
I could post detailed info, but if you look at a calendar you'll see what I mean... this FOLLOWUP needs to be SKIPPED only on months that do NOT have 5 weeks in them, and the next followup goes out instead (the regular monthly followup).... let me know if you need further details... but the message above and a calendar should show you what I mean (especially September 2006)...
-
This won't necessarily be easy but should, of course, be possible.. ;)
It sounds like you don't want to reset because you have more emails in the sequence.
What about running a custom script before dailymail each day (or just the last few days of the month) that simply skips the message for users on that followup, if required. We could use the same cron-style function, and the PHP date() function, in this custom script to verify when to skip users. We could even check future dates if necessary... How do we determine which months to run it and which months not to? :? This is a bit confusing. :D
Regards
-
Let me know if you can figure out the logic behind the custom script you want, Brett. I'm racking my brain with the next update.. :)
I've been busy over the last 5 days or so working on LM.
Here's the latest [view Changes.txt (http://listmailpro.com/changes.php)] along with some notes:
- added storage of dailymail 'last ran' time. dailymail will not run via cron task until 23h55m from last ran.
This addition protects against runaway cron tasks that seem to 'refresh' the URL after a certain time. The last execution time can optionally be updated (though it might/probably will cause ListMail to skip your next dailymail!) when running dailymail manually.
Note: This feature might be detrimental when testing whether or not the cron task is configured correctly. For example, when testing 'wget' alternatives I like to run the cron task every minute. Perhaps a simple variable option, ie "&test=1" added to the cron command could generate a success message to allow for this. That, or an option could be placed within ListMail somewhere to put dailymail into and out of 'test' mode. This would need to be explained in the installation documentation. I suppose each method is equally useful. The first method has the added benefit that I don't need to use up any space in the ListMail interface on the Config page. While I could place the option on the manual Dailymail page it might be confusing for installation.
What do you think about my assumption of 23h55m?
- added queue protections to prevent duplicate emails when refreshing the send page or when resuming queues too soon
This is a really nice improvement that prevents errors (duplicate emails, mostly) and makes ListMail more user-friendly.
When you refresh a standard send page it says "This message has already been sent, did you hit refresh?". This is achieved by creating a "formid" on each "Send Email" page and storing it in the queue protect table (for 3 days). I took the caching off of that page so a new formid is always generated.
When you hit Resume on a completed mailing, it says "This mailing has been completed". This is done by storing the status of a queue which indicates it has completed (3 days).
When a mailing starts, a new entry is made in the new 'queue protect' table with the flag indicating the message is "queueing". That is, the users haven't been entered into the ListMail queue table yet, which is done before sending to the server. "Queueing" messages may not be resumed (a message will say "This message is in the process of queueing"), thereby preventing mailing if the ListMail queueing process is interrupted before it finishes.
As any mailing progresses it updates the 'queue protect' table with the current time every 20 seconds. You may not resume a queue that has responded in less than 60 seconds.
If you try to resume a message that has responded within 60 seconds, a page is shown. The page tells you how many seconds ago the queue last responded and provides another Resume button so you can try again. When the last response time is less than or equal to 22 seconds the page states that the queue 'appears to be running normally'. When the last response time is greater than 22 seconds the page states "seems to be failing". Once the 1 minute mark is reached you may resume a failed mailing.
I think this improvement will take a LOT of the headaches out of mail sending. It will also cut down on support requests as the sending process can now be ruled out in case of duplicates, etc.
Do you think I need options for how often the 'queue protect' table is updated and how long after a queue has stopped responding to allow a resume? Personally, I think 20 seconds and 1 minute is a perfect balance of speed and stability. :)
- added the ability to move or add followups into the middle of a live sequence, inserting them
- added 'smart' followup and user seq # renumbering when adding, inserting, and deleting followups
A very useful feature (inserting) and some cosmetic improvements. After spending an entire night on it I started over, sitting down with a pen and some paper to rewrite the logic. The result is safer, more straightforward and easy-to-test code. I have tested this quite thoroughly and am confident there will be no problems with it.
---
That's it for today. I believe the only other major feature I will implement before the update is an auto resume script. Here's what I envision:
Scheduled script, ie. "resume.php" runs every 5-15 minutes. The script checks the 'queue protect' table to see if any mailings have not responded in over 1 minute (or 1.5 - 2 minutes, just to be safer than with manual resumes). With this feature if a mailing stops responding for any reason (ie. server reboot, MySQL/web/mail service restarts or downtime) your mailing will resume shortly afterwards!
Regards :D
-
I was reading some old posts in this thread and found a solution I couldn't help but mention.
I could also use the ability to put "NOT" in user selection rules.
This is not as easy as it seems. Once I figured out the query I found it impossible to work into the standard user selection (partly due to the fact that I run an old copy of MySQL that doesn't support the query). I decided to keep everyone happy and more by creating a "Custom Query" option for User Selection. I believe this is only possible in MySQL 4 but the query (thanks Martin) to select active users who are not on another list will be:
SELECT *
FROM lm_users a
WHERE a.list = '2'
AND cnf = '1'
AND NOT
EXISTS (
SELECT *
FROM lm_users b
WHERE a.email = b.email
AND b.list = '3'
AND cnf = '1'
);
Try a custom query in PhpMyAdmin to see if this might work for you.
This feature will be available when the 1.85 update is released.
Regards
-
Hi DW,
Wow, all the changes you mention for upcoming release sound great - the auto-resume script is THE BEST - and yes, I would like to see that... as I still get times when I log in and find some of my emails did not go out... would love to see the auto-resume feature added...
That, or an option could be placed within ListMail somewhere to put dailymail into and out of 'test' mode
I like this idea too - having a 'TEST MODE' feature would be more useful than having to remember to add &test=1 to the task every time - I think this is the better option.
What do you think about my assumption of 23h55m?
Sounds fine to me.
Do you think I need options for how often the 'queue protect' table is updated and how long after a queue has stopped responding to allow a resume? Personally, I think 20 seconds and 1 minute is a perfect balance of speed and stability.
No, I don't think you need options, but then again, others might be on a slow server and it may affect them? To me 20 seconds/ 1 minute sounds just fine... you know more about server response times,etc than I do so go with the 20 secs/1 minute for now... and yes I do believe it will help you remove the duplicate sending problem...
Anyway, it seems this new version is going to have so many new features it's going to require lots of testing - have you given though to previous posts by others that you should be issuing releases with fewer time between them and letting people download the newer release more frequently - that way you don't have such a HUGE release?
Regarding the CRON style scheduling question I had (see previous detailed messages):
It sounds like you don't want to reset because you have more emails in the sequence.
Yes, that's right, I do NOT want to reset the sequence, BUT I do want to be able to send out email on the 5th week of the month, when there is an extra week - since I can use that for sending out an additional promotion.
What about running a custom script before dailymail each day (or just the last few days of the month) that simply skips the message for users on that followup, if required.
That sounds workable, though what I had in mind was that this would be BUILT in to CRON style scheduling - perhaps adding a Week of Month optional value (and perhaps a SKIP option) - that when set would look for followups meeting that requirement, if it is met the followup goes out - if it is not met the followup does NOT go out - and is skipped... if that is too confusing - which it may be for other users - then YES - your idea of a custom script sounds fine... when it runs, it would need to run on 28,29,30,31st of the month ONLY and force the FOLLOWUP to SKIP - ONLY if the followup is set (and is the TYPE OF FOLLOWUP I indicated) - do you see the problem here, since some sequences may NEVER have this type of followup setup, and some might...
the thing to keep in mind is this:
The cron code is: 28-31 * 2-3 That followup will ONLY be mailed out if there is a 5th week (that is, a day from 28th to 31st that falls on a Tuesday or Wednesday), that limits the followup to ONLY go out on those months with a 5th week (according to my definition) - a tue/wednesday only with a day of the month of 28, 29, 30 or 31.
We could use the same cron-style function, and the PHP date() function, in this custom script to verify when to skip users. We could even check future dates if necessary... How do we determine which months to run it and which months not to? This is a bit confusing.
Yes it is a bit confusing, for me too - what I want to do is fully laid out in previous posts in this thread - the followup ONLY needs to go out if there is a 5th Week of the month - for me that definition means that there is a Tuesday or Wednesday with a day of the month of 28, 29, 30, 31 as indicated above... BUT, keep in mind that I may NOT have this type of followup setup in ALL sequences, so - I don't know if it's possible, but it sounds like it may be... you'll need to think about whether it can be done for the situation indicated- ONLY on 28,29,30,31 look for a day of the week, if it is a Tuesday or a Wednesday than RUN THIS FOLLOWUP, otherwise SKIP IT... not sure how you can accomplish that? That is why I suggested adding this as an option to the CRON scheduling - by adding a WEEK OF MONTH parameter and a SKIP option...
Let me know if it's possible?
-
Brett,
having a 'TEST MODE' feature would be more useful than having to remember to add &test=1 to the task every time - I think this is the better option.
Sounds good. I could enable TEST MODE by default when installing and put "TEST MODE" on the dailymail report so that you know that nothing will happen if you do not toggle the option.
if that is too confusing - which it may be for other users - then YES - your idea of a custom script sounds fine... when it runs, it would need to run on 28,29,30,31st of the month ONLY and force the FOLLOWUP to SKIP - ONLY if the followup is set (and is the TYPE OF FOLLOWUP I indicated) - do you see the problem here, since some sequences may NEVER have this type of followup setup, and some might...
The cron-style feature alone could be difficult for some to grasp, so I think it could be confusing for others. Considering you're the only one who needs this (so far) I think a custom script would be the best way to do it.
The cron code is: 28-31 * 2-3
That followup will ONLY be mailed out if there is a 5th week (that is, a day from 28th to 31st that falls on a Tuesday or Wednesday), that limits the followup to ONLY go out on those months with a 5th week (according to my definition) - a tue/wednesday only with a day of the month of 28, 29, 30 or 31.
So the custom script would update all users, if necessary, set to receive any followup with that cron code. If the script is run just before dailymail on the same day (ie. at midnight, with dailymail running at 12:05am) then if today (the day the script is run) is the FIRST of the month, advance anyone with the cron timing above to the next followup. If this works as I envision it would be simple and effective. We could run the custom script once a month, on the first, before dailymail.
Update: Your "Copy List" request is now available in the v1.85 update.
Regards
-
I have released the latest files!
http://listmailpro.com/forum/index.php?topic=982.0
-
Hi DW,
I have released the latest files!
:D Congratulations! I know this has been a lot of work for you, and I'm sure others will REALLY appreciate the work you've put into this one, I know I've loved the CRON scheduling - brilliant, and works great - I highly recommend it...
With all of the updates including the AUTO-RESUME and the prevention of duplicate mailings, text-only features there will be a lot of testing/debugging going on, good job - I'd really like to see a 'TEST' mode - and I believe it would really help you out allowing others to set LMP into TEST mode, also, as I mentioned previously, a text LOG file would be beneficial for testing such as this - I hope you can consider this in the future...
I'd like to ask again that you think about adding the THROTTLE options into Configuration... you could include a THROTTLE section, with checkbox [] Throttle mail being sent, pause __ seconds for every ___ messages sent.
I added a simple "Reset Users" button on the Followups page, but is it enough? What about a "Reset all users on [Followup X or EOS] to Seq [ ] Delay [ ]" feature? This should/will also be possible with the upcoming mass user operation features.
No, a simple Reset Users is not enough - I don't see this 'Reset Users' feature, this would be best to include IN THE SEQUENCE, for example when adding a Followup - what we really need is a RECYCLE users option, for example at END of all FOLLOWUPS I would like to RECYCLE the user back to FOLLOWUP 2, not to the beginning of the list, but to a specific Followup in the sequence.
If the script is run just before dailymail on the same day (ie. at midnight, with dailymail running at 12:05am) then if today (the day the script is run) is the FIRST of the month, advance anyone with the cron timing above to the next followup. If this works as I envision it would be simple and effective. We could run the custom script once a month, on the first, before dailymail.
I see your point here, and yes, that would work, run a script every AM, I run Dailymail at 4:30AM, so this script would run just before it... so I believe you would need to look for 29th-1st of month - right? That way if the followup is called, ADVANCE to next one, is that right? I see now how this would be a solution. Can you provide me the php code for this script?
I think that in the future you could make your CRON scheduler even more powerful if you include other paramters such as I am describing (Week of Month), also Week of Quarter, or Week of Year etc... and also including a SKIP option to the CRON task, if this followup can NOT be reached - SKIP to the next...
Update: Your "Copy List" request is now available in the v1.85 update.
Now that I think of it, as you mentioned, the option to create 'SEQUENCES' of followups would be a better option, but I do appreciate the COPY LIST, that's great... and will let you get close to copying sequences.... I realize sequences may be harder to program... this is an alternative, thanks :)
-
This new feature -
dailymail will not run via cron task until 23h55m has passed
would really screw things up for me. I have multiple cron entries to send out the followups on different times for different days. I do this to make it seem more 'real', as if I'm sending out the mail by hand.
If this feature can be optional, then I won't have a problem with it.
-
Brett,
I see your point here, and yes, that would work, run a script every AM, I run Dailymail at 4:30AM, so this script would run just before it... so I believe you would need to look for 29th-1st of month - right? That way if the followup is called, ADVANCE to next one, is that right? I see now how this would be a solution. Can you provide me the php code for this script?
Something like this, set up to run once a month on the first, before dailymail, should do it:
<?php
if($pw<>'password') exit();
$path_to_lmp = './';
include($path_to_lmp.'config.php');
include($path_to_lmp.'admin.php');
// followup cron timings to check
$crons = array('28-31 * 2-3');
while(list($k,$v)=each($crons)){
// select all followups with this cron
$frows = mysql_query("select list,seq from $ftable where cron = '$v' order by seq;");
if(@mysql_num_rows($frows)>0){
while(list($l,$s)=mysql_fetch_row($frows)){
echo "cron $v found, seq $s list $l<br>";
// get next followup for this list
$frow = mysql_query("select seq,del from $ftable where seq > '$s' and list = '$l' order by seq limit 1;");
if(@mysql_num_rows($frow)>0){
list($ns,$nd)=mysql_fetch_row($frow);
echo "nextfup found = $ns $nd<br>";
} else {
// get list eos
$frow = mysql_query("select seq from $ftable where list = '".addslashes($l)."' order by seq desc limit 1;");
if(@mysql_num_rows($frow)>0){
list($ns)=mysql_fetch_row($frow);
$ns++;
} else {
$ns = '1';
}
$nd = '0';
echo "nextfup EOS = $ns $nd<br>";
}
// update users
mysql_query("update $utable set cseq = '$ns', cdel = '$nd' where list = '$l' and cseq = '$s';");
echo mysql_affected_rows() . " users updated on list $l seq $s to seq $ns del $nd<br>";
}
} else {
echo "no followups with cron $v<br>";
}
}
?>
Regards
-
Alan,
This new feature -
dailymail will not run via cron task until 23h55m has passed
would really screw things up for me. I have multiple cron entries to send out the followups on different times for different days. I do this to make it seem more 'real', as if I'm sending out the mail by hand.
If this feature can be optional, then I won't have a problem with it.
Thanks for contacting me. I'll make this optional ASAP and update this post. I do not recommend that you upgrade until then.
Regards
-
Hi DW,
Something like this, set up to run once a month on the first, before dailymail, should do it
Thanks a million :D
echo "no followups with cron $v<br>";
One thing I wonder about, you have echos here, since this is running from command line, there's no good way to capture the output unless I use the CRON command to capture the output of this script, something like this:
1 1 1 1 * /usr/bin/wget -O /dev/null -T 0 http://example.com/mail/myphpfile.php?pw=MyDailyMailPass 1> /home/myhome/temp/eowtemp.txt 2> /home/myhome/temp/eowtemp.txt
This example from your auto-resume CRON entry... BTW, I'm not familiar with the 1> and 2> are these significant, what is the difference in their use, is it ok to enter as above?
Thanks again, I appreciate it!
-
Brett,
You're very welcome - this is what I do. :D
I'm not familiar with the 1> and 2> are these significant, what is the difference in their use, is it ok to enter as above?
The 1> (or just >) and 2> are redirects for output of the program/script. I read the exact definition some time ago, I think the 2> is for additional error output at the programmers discretion. We probably only need 1> or > but adding 2> is more thorough and is sure to prevent all output. :)
You can use the paramaters 1> /dev/null 2> /dev/null to suppress all output as desired or remove them to send output to cron or the terminal. To have output sent to you by cron remove them and add the following line at the top of your crontab:
MAILTO=your@email.com
Note that the password var pw should be 'password'. You can also use the password as the first argument if running the script from the commandline.
Add the following line near the top of the script:
if($argv[1]) $pw = $argv[1];
Run the script like this:
php script.php password
or
1. type "which php" to find php, it's probably in /usr/bin
2. add the line #!/usr/bin/php to the top of the file containing the code, <?php will be the 2nd line
3. chmod +x the file
4. run it: ./script.php password
Hope that helps!
-
Hi DW,
You can also use the password as the first argument if running the script from the commandline.
I never knew that was possible! I didn't know you could RUN a .php file from command line like that, that's really cool!
Would it be better (more efficient AND stable) to setup the CRONs this way - run them direct from the CRON command line, instead of using WGET?
If this is better - I would prefer to run all of my .php files from the command line instead of using the WGET call... it also makes it easier for testing - though maybe not easier for you - since you'll need to have two versions (one command line, the other standard)?
Update: Your "Copy List" request is now available in the v1.85 update.
Back to this... I do see your Copy List option in the DEMO for v1.85, but I have to ask if you have given the COPY Followup (or series) more thought? I appreciate having the Copy List option, but really, when you think about it - the Copy FOLLOWUP, or Copy SERIES would be more useful, I can see a LOT of copy/pasting ahead - if you do not get this into a new version.... think about what you do most often with LMP, besides Mailing, you create/edit and copy FOLLOWUPS, having the ability to COPY these Followups from 1 list to another would be very helpful, but beyond that, having a way to create a Sequence, or Series of FOLLOWUPS that you can Name, and then copy the ENTIRE Sequence or Series would be even more useful.... I hope you can keep that in mind while working on upcoming updates?
-
Brett,
Would it be better (more efficient AND stable) to setup the CRONs this way - run them direct from the CRON command line, instead of using WGET?
I don't believe this is necessarily more stable or efficient because of the fact that running from the command-line may have different PHP settings. For instance, a script that times out at 30 seconds may not time out the same with wget. I use wget exclusively for this reason and can't guarantee results if running from the command-line.
I do see your Copy List option in the DEMO for v1.85, but I have to ask if you have given the COPY Followup (or series) more thought?
I promised this to a client just before the v1.85 release but held off because I knew everyone was anxious to get their hands on some new code. I have already finished it and it's really sweet. :) Check this out:
1. Normal state
(http://listmailpro.com/misc/1-normal.gif)
2. Save Set clicked
(http://listmailpro.com/misc/3-save.gif)
3. Load Set clicked
(http://listmailpro.com/misc/4-load.gif)
Oh yeah, and I also went ahead and improved the Reset Users option for good measure. ;)
4. Reset Users clicked
(http://listmailpro.com/misc/2-reset.gif)
If an option (Reset/Save/Load) not currently 'showing' it's sub-options is clicked, the other options automatically 'close' their sub-options.
I have a lot planned and just need to do it. I've realized that I haven't implemented as many features as I could/should have in the past and hope to make it up to you guys. :) I'll be going through the forum implementing old suggestions, etc. to not only get them out of the way and make ListMail a better program, but also to make my clients feel important and heard. :)
Regards
-
Hi DW,
I promised this to a client just before the v1.85 release but held off because I knew everyone was anxious to get their hands on some new code. I have already finished it and it's really sweet. Check this out:
Yeah, that is great, when can the rest of us get our hands on it? That will save me a TON of time!
One question about how it works, when you Save Set, and then Load it in another list, is it possible to Load it at ANY point in the current lists followups, or does it just load it at the end of the sequence? I realize this could be a pain to program... anyway, WOW, looks great...
Oh yeah, and I also went ahead and improved the Reset Users option for good measure.
That looks great too, but what about LOOPING, have you added that yet? I'd like to see a 'Loop User' function, or better yet,
Loop at end of Sequence Loop To Followup xx
Not sure how you'd handle Looping before end of sequence, but I don't need it, not sure others would either... anyway - YOU ARE AMAZING, keep up the great work... and please do let us know when/how we can get the next release with the FOLLOWUP Sets?
-
Yeah, that is great, when can the rest of us get our hands on it? That will save me a TON of time!
I provided it to my other client along with some custom SQL queries to create a field and a table. First you'd need to update to v1.85 then I can send you the updated files. Drop me an email if you want it. :)
One question about how it works, when you Save Set, and then Load it in another list, is it possible to Load it at ANY point in the current lists followups, or does it just load it at the end of the sequence?
Currently the feature simply REPLACES all followups on a given list with the ones from the saved set. (Yes, of course there's a pop-up confirmation) You make an interesting point, though. I could fairly easily provide an additional dropdown to choose * (replace all) or an existing seq # to insert after (what do we do with those followups equal to or ABOVE the inserted #? Move them up or delete them?)
That looks great too, but what about LOOPING, have you added that yet? I'd like to see a 'Loop User' function, or better yet,
Loop at end of Sequence Loop To Followup xx
I agree that this should be an option and I will be implementing it very soon. For the meantime, you can do the same thing like this:
http://listmailpro.com/forum/index.php?topic=232.0
Regards
-
Hi DW,
Regarding the 'save/load' set of followups and your quote:Currently the feature simply REPLACES all followups on a given list with the ones from the saved set. (Yes, of course there's a pop-up confirmation) You make an interesting point, though. I could fairly easily provide an additional dropdown to choose * (replace all) or an existing seq # to insert after (what do we do with those followups equal to or ABOVE the inserted #? Move them up or delete them?)
This will not work for me, I've got sets that need to be loaded BEGINNING at a certain Followup, I suggest you provide a way to do that, and then move ALL the other Followups DOWN - after the inserted Set of folllowups...
Is that possible, coming in the next code release?
-
Alan,
This new feature -
dailymail will not run via cron task until 23h55m has passed
would really screw things up for me. I have multiple cron entries to send out the followups on different times for different days. I do this to make it seem more 'real', as if I'm sending out the mail by hand.
If this feature can be optional, then I won't have a problem with it.
Thanks for contacting me. I'll make this optional ASAP and update this post. I do not recommend that you upgrade until then.
Regards
Hi,
Just checking to see if this has been made optional yet or not.
Thanks.