We have a large website that sends out different kinds of emails to their users: newsletter, forgot password, random subscriptions created by users, etc.. The portal is pretty big and we need to send out thousands of emails per day.
The way it is organized now is that all emails are queued in database and then there is a cron script that reads DB every several minutes and sends out the batch of X emails. This script is essentially a for loop that goes through these X emails and for each of them establishes an SMTP connection and sends the emails. This however overloads the server pretty heavily, basically most of the time server is busy sending out these emails one by one.
I was wondering whether there is some way to send these queued emails to the SMTP server in batches. If this involves some specialized software, that's on problem, we can set up our own SMTP server (we'd prefer Linux/Unix based one). For instance something that would work is running a cron every day, convert that queue into an XML file and send that XML file to the SMTP server and then SMTP server would parse it and send out all these emails.
Assuming that your script is running on a Linux or Unix server: set up a local Postfix configured as a null client with connection cache enabled. Have your script submit its mails via pickup to the local Postfix.