hMail: Resource Allocation Limitations

I've been running into some serious problems with my external message processing scripts. It appears that over time the server builds up processes that haven't exited. After a while so many processes build up that the server is unable to run any more and pop-up error message is generated for each additional attempt.


I was able to find some information (see references) that indicate that this is basically a desktop memory heap overflow. The server needs a discreet amount of memory for each object that is created on the desktop. In this case each command line window. But there is a limit to the number of objects that can be allocated (a limit on the total memory used by the desktop heap).

Every desktop object on the system has a desktop heap associated with it. The desktop object uses the heap to store menus, hooks, strings, and windows. The system allocates desktop heap from a system-wide 48-MB buffer. In addition to desktop heaps, printer, and font drivers also use this buffer. ⁽¹⁾

This is a problem for services as well. But to compound the problem any service which runs under the localsystem account with "Allow service to interact with the desktop" checked use up the active windows session. Services without this checked share memory in a default non-interactive session. Services run under a specific account have their own desktop heap allocation.

In essense, as processes are run by any particular "user" the memory allocation for that user is used up. If the processes are not cleaned up appropriately (e.g. terminated when complete) the allocation for that process is not released.

I've been able to mitigate the frequency of the problem by assigning the hMail service to a particular user. This is something I had intended on doing anyway, but I'll need to go back and check to make sure that the security and auditing settings are valid once all users have been created.

I had one thought as to the cause of the process build-up: could it be due to the remote server disconnecting when hMail takes too long to process the message? Basically the timeout being reached and the remote server disconnecting? I think this is a possibility because I had to set up my custom scripts to run while the SMTP session is still active. I'll have to run some tests to try and figure out if this is what's happening.

Whether or not this is the case, I'm thinking of switching to the Exec() method of the WScript object. That way I can terminate the object if it runs for too long. This will, however, require more care in setting up the scripts because I won't be able to just wait for the program to exit ... I'll have to actively monitor it to see when execution is complete. The Run() method waits for the program before continuing script execution while the Exec() method executes the program then continues on with the script.

References:
Searched google for "windows 2000 SharedSection"

1. PRB: User32.dll or Kernel32.dll fails to initialize
2. "Out of Memory" error message appears when you have a large number of programs running
3. Unexpected behavior occurs when you run many processes on a computer that is running SQL Server

Update 2006-08-20:
A quick test of the server indicates that the problem is not due to a premature disconnect. I would like to know why these processes aren't ending, but I think it's more important in the meantime to make sure they're killed if they take too long.

hMail: Anti-Spam/Anti-Virus Techniques

Switching to hMail
When I switched to hMail I had to make some changes to the way I check for bad mail on the server (right now sender sender address validity and viruses). One of the biggest problems with the built-in functionality of hMail, I find, is that the only option it uses when a positive result it returned (e.g. bad e-mail address or virus found) is that the message is dropped. There is no option using the built-in functionality to add a header or redirect to a new mailbox. This is not acceptable to me as I like to be able to confirm how well my bad-mail filteres are working.

The other problem is that hMail doesn't have built-in support for running external programs (beyond an external virus scanner; see above for why I don't use this). It does, however, include an event script that uses Microsoft scripting (VBScript, JScript, and maybe other WSH processors though I haven't confirmed that) to allow for flexibility in mail handling. The script can be used to run external programs (via the WScript object) and perform general message-handling (using the built-in hMail object model). With this interface I can use custom scripts to run additional processing tasks on any message that arrives at my server.

Modifications to my scripts
Using the event script I am able to call external scripts by creating an instance of the WScript object and using the Run() method to execute a command. I only had to make minimal changes to my scripts to enable them to be run in this way.

CheckAddr.pl was modified to add a new argument option of "hMail" which is pretty much the same as the Mercury argument except that instead of two additional arguments (the mail file and the result file) only one additional argument is needed (the mail file). Once the script is done running I use the return code to determine the value of a the X-Eclectic-AddrCheck header. This header is added to the message using the HeaderValue() method of the hMail message object.

Virus checking by F-Prot is still accomplished in the same way as before, calling a batch file that executes F-Prot. Once the batch file has exited the program then checks for a log file based on the name of the message file. The presence or absence of this log file indicates whether or not a virus was found and determines the value of the X-Eclectic-VirScan message header. If the log file does exist it indicates a virus was found and the file is parsed for the line indicating what virus was detected, which is then prepended to the subject of the message. The log file is then deleted.

I use the server rules to redirect any message marked as bad to the spam dump account.

Problem with the event model
I have one issue with the event model used in the event script ... the only viable event on which to process messages is OnAcceptMessage, which is run after the message is delivered but before the SMTP server disconnects. Any significant delay in processing the message (especially with my custom scripts called from the event script) could cause the SMTP session to time out.

I have to use this particular event handler because the server rules are run after a message is accepted (OnAcceptMessage) but before message delivery (OnDeliverMessage). Since I'm adding a header to the message based on the results of my custom scripts and then using server rules to determine whether or not to redirect those message to my spam dump I don't have any other choice at the moment. I'd much rather be able to run my custom scripts after the SMTP session has completely ended.

I'd like to see a new event added to future version of the server that occurs after the SMTP session but before any further processing by the server.

Mail Server Setup

For a while I was using Mercury/32 as my mail server. It's a great little program and very robust. But (based on what I've read on the support list) depending on the specific setup it seems it can be a little flaky with regard to stability. I've started to run into problems my self. But beyond that fact there are a few things about the program that I'm not too happy with. The biggest of those is the fact that M32 doesn't have an option to run as a service. Sure it can be addressed via various run-as-a-service programs ... but it's not quite the same and generally makes management a little more difficult.

So I recently switched to a new mail server, hMail Server. What brought it to my attention was it's database capabilities. I was actually looking for a program that stored the actual messages in a database. Sure it would be a lot of overhead, but there would be so much more flexibility for access and archiving ... depending on how the database was set up. hMail isn't quite what I was looking for, but it's integration with Windows made it pretty good. No need to get into the details, you can get that from the site. So I'll just leave it at the fact that I've been very happy so far. Plus, I've been able to implement just about all the functionality I was using in M32 ... making it a good drop-in replacement (well ... almost since I'm using IMAP).

M32 is supposed to be undergoing an overhaul right now. It'll be interesting to see how it progresses. I'll have to give it another look once the next major revision is released. If nothing else the native support for running as a service will be a most welcome change.

Argh! MTU strikes again!

Every time I set up a PC I run into networking problems. Invariably they're related to default MTU in Windows. Usually I never notice, until I need to access Microsoft.com whose site uses a lower packet size than the Windows default (thanks MS!) .

The whole thing is just very annoying and should'nt be such a pain. For some reason my laptop has no problems, but every desktop I've set up at home does.

My setup is a Windows 2000/XP multi-pc network on a linksys DSL/switch. I've gone with an MTU of 1454 to maximize compatibility. But it sure does suck that I'm not getting the most efficient throughput. Does anyone know how to deal with this without having to modify the registry of each PC set up?

What's it about?

I've been doing a lot of nothing in this blog. Partly because I haven't really spent much time thinking about what I want to do with it. Partly because I've significantly pared down the work I do at home.

I've been trying to ramp up my productivity at home, though, and I think this blog can help. I've been using my other blog to document what I've been doing at work. Specifically any problems I've encountered and solutions I've found. I think doing the same for this blog will be a good start.

So starting today I hope to start spending more time filling out the content here. Wish me luck.

more perl fun

Two fixes in two days for my mail reader. This is some kind of record!

The attachment downloader was having some problems when the filename included odd characters. Realized that a) I was not URL-encoding the filename in the attachment link and b) I was unencoding the filename in the script that sends the file (which is completely unnecessary).

Fixed and fixed.

regular expressions and other perl fun

So I have a home-grown, web-based, POP3 e-mail reader written in perl. Nothing fancy. Not really that great, but it works ... at least until I replace it with something more worthwhile like Horde IMP. Until the day comes that I replace my frankencode I do my best to keep it, at a bare minimum, functional. So I guess while I'm lazy enough to avoid implementing a better system I'm not too lazy to make the occasional fix and/or upgrade to my own creation. I guess I like the challenge more than anything else.

Anyway, the script recently began having some problems with the date parser. My original regular expression for pulling the date from the header was hitting a wall with a certain malformed value. You always think it's good enough to get by until something weird comes your way. My new regex date parsing code:

/(\d{1,2})\s(\w{3})\s(\d{4})\s(\d{1,2}):(\d{1,2}):(\d{1,2}).([+-]\d{4})?/i

Ugly, yes. Efficient, no. Functional ... well, at least until the next funky date comes along.

One of the effects of the update is that e-mail with an invalid date (or no date at all) are relegated to the bottom of the list. Where they went before I do not know.

I also noticed that the per-page message cache creation loop and display loop were both written incorrectly and thus some weirdness was happening with the inbox listing. (damnable zero-based indexes and what not).

These are not, by the way, the only known bugs. I have quite a bit of work to do to get the message parser up-to-snuff. Among the many problems the most in need of a little work are the few cases where the body of a message will not be displayed. I think it has something to do with HTML-based messages, but I haven't taken any time to fix this one just yet.

3 Months?!?

A blog sure sounds like a good idea when you've got all these thoughts running around in your head. Yet somehow I never seem to be able to find the time to devote to it. Oh well, maybe if I keep it up at this pace google won't kill my account.

Making my voice heard

Reported my first bug!

I found it in Gecko, the core rendering engine of Mozilla, Firefox, Thunderbird ...

Not a serious bug, but it's cool to be able to do something like that and follow the progress (if any).

Update 2004-12-13a: Nevermind. Got the smack-down big time. I need to do more research next time. :(

Update 2004-12-13b: Upon further review I realize how I misjudged the way gecko renders my code. It all goes back to margin collapsing. Probably one of the more difficult parts of CSS to get a handle on.

Nothing like being humbled by the masters to make you realize how much you still have to learn. I spend too much time assimilating just enough information to get the job done these days. Or perhaps I just attempt to assimilate too much information. I should spend a little more time delving into the more arcane aspects of some of the technologies I use. Time to look into classes or certification.

I don't think I'll be submitting a bug report anytime soon. There are plenty of people more capable than I working on that kind of stuff. I'll need to keep reminding myself of the old saying about the dangers of having just a little knowledge.

The syndicated Schneier

I just noticed something interesting about Bruce Schneier's security blog. Take a look at the following articles:

Aerial Surveillance to Detect Building Code Violations
License Plate "Guns" and Privacy

I guess when you're marketing security it helps to relate it to your audience. Don't get me wrong, I like Mr. Schneier's philosophy on security. I guess when you're in the business of marketing security (and I do think Mr. Schneier has a real interest in marketing security and not himself) you have to do just that.

Money making ideas in hindsight

From the 12.12 issue of Wired: The End of TV as We Know It

I've been toying around with the thought of TV over IP for years. No doubt I'm not the only one as evidenced by the fact that software has already been developed. But still I must wonder if I the things I think about sometimes are on the cutting edge of innovation, or a day late and a dollar short.

Well, definitely a dollar short. Plus I'm not much of a salesman and have little incentive (and a healthy fear of risk) to spend my days developing these ideas and marketing them to large companies. I need a wealthy patron and a good salesman by my side.

At any rate, I always felt that the cable companies were wasting an incredibly large pipe by fat-casting everything they have. A more individualized approach definitely makes more sense if you ask me.

Homeland Security

Tom Ridge resigns. Good. DHS needs a strong leader who can inspire his people. Mr. Ridge never struck me as being able to fill that role.

The irony of being President

I don't plan on spending all my time talking about politics ... but like so many others on the Internet I just can't help myself.

In order for the taxpayers of the United States to feel comfortable about supporting the United Nations, there has to be an open accounting.

Source: Bush Sidesteps Annan Resignation Issue, by Ken Guggenheim of the Associated Press

Which advocate of open government made the above statement? Ok, perhaps advocate isn't quite the correct term. I'm thinking more ... um ... opponent. If you said President Bush then you'd be correct. (Did you cheat and read the rest of the article?)

The Bush administration needs to have the same conversation with it's own staff and members of Congress. And I mean with all memebers of Congress, not just Republicans. The goverment should be accountable to it's constituents. That's not possible if activities of interest (with a few exceptions) aren't available to public scrutiny.

Did somebody say something about Republicans? According to the Washington Post, budget writers for the party have been trying to work out a method of reforming Social Security without causing a huge increase in the budget deficit. One method being put forward is to use accounting mechanisms so that the cost of the reform does not show up on the budget.

No doubt somebody with a shiny, new MBA thought this sounded like a good idea. The U.S. Government, however, is not a business and should not resort to accounting tricks to look like it's healthier than it truly is. I'd rather see a more accurate accounting (truth in accounting) within the goverment.

whoami

Eclectic adj. Selecting or employing individual elements from a variety of sources, systems, or styles.

Geek n. A person who is single-minded or accomplished in scientific or technical pursuits but is felt to be socially inept.

EclecticGeek n. Socially awkward individual who is single-minded in a variety of pursuits (or at least fancies himself that way) ... but spends entirely too much time on his computer.