#!/bin/bash

LOGDIR="/tmp"
PREFIX="sendmail"
NOW=$(date +%Y-%m-%dT%H.%M.%S)
CNT=1
PRIVATELOG="$LOGDIR/$PREFIX-$NOW.$CNT.log"
COMBINEDLOG="$LOGDIR/$PREFIX-combined.log"

# If privatelogs are being used...
if [ ! -z "$PRIVATELOG" ]; then
# ...make sure the filename is unique and create the file
while [ -f $PRIVATELOG ]; do
CNT=$(($CNT + 1))
PRIVATELOG="$LOGDIR/$PREFIX-$NOW.$CNT.log"
done

echo "$0 $*" > $PRIVATELOG
else
# ...otherwise swap filenames
PRIVATELOG=$COMBINEDLOG
COMBINEDLOG=''
fi

echo "[$NOW]" >> $PRIVATELOG
while read BUF
do
echo $BUF >> $PRIVATELOG
done

# Append privatelog to combinedlog when both logs are used
if [ ! -z "$COMBINEDLOG" ]; then
echo "[$NOW]" >> $COMBINEDLOG
cat $PRIVATELOG >> $COMBINEDLOG
fi

exit 0

When your application now sends mail, these things happen:

• No email is actually sent.
• The message gets appended to the file /tmp/sendmail-combined.log, on which you could set a ‘tail -f’ in order to see which emails would have been sent and what contet they would have.
• One new file (e.g. /tmp/sendmail-2011-02-08T08.02.48.1.log) gets written for every email sent. I personally only use the combined file.

Inspired by http://stackoverflow.com/questions/3710864/simulating-sendmail-with-dummy-script

In order to deploy bigger changes without bothering your visitors with strange behaviour during a data migration, updates and the like, you should use Apache2′s mod_rewrite. Just put the following lines in a .htaccess file in your webroot directory and all traffic (also deep links to subdirectories) gets diverted to the maintenance-page. The scond line sets an exception for your IP address, so you are the only visitor who is NOT redirected to the mainteance page:

RewriteEngine on
RewriteCond %{REMOTE_ADDR} !192\.168\.123\.101
RewriteRule !maintenance/index.html /maintenance/index.html [L,NC,R=301]

Do not forget to remove the .htaccess file after you have finished your work!

Make yourself a md5-hash using PHP ;) to test it:

marco@africa:~$ php -r 'echo md5("marco")."\n";'
f5888d0bb58d611107e11f7cbc41c97a


A daily running script – let’s call it the Importer – regularly exited randomly with a “Fatal error: Out of memory (allocated 12320768) (tried to allocate 2851436 bytes) in …” and I had to restart it manually nearly every morning. I had…

• …checked my application for memory wasting operations and loops and fixed them,
• …used ini_set(‘memory_limit’, ’64M’); at runtime, and
• …finally increased memory_limit = 64M in my php.ini.

But all this did not change the bahaviour of the Importer!

So I took a look at the PHP5 Changelog to find potentially fixed bugs in newer releases. Bug #39438 described exactly my problem. So a simple upgrade would help me. But it did not work with ‘apt-get upgrade’ or ‘apt-get install php5=5.2.8′ since the highest version in the apt source I used was the one that I already had: 5.2.0-8+etch13, issued in November 2006… (pretty ancient)

Finally it was this page that had the information we needed: an alternative apt source

deb http://packages.dotdeb.org etch all
deb-src http://packages.dotdeb.org etch all


After getting an impression whether dotdeb was a trustworthy source, we first tried it on our dev-system with ‘apt-get update; apt-get upgrade;’. At this point I was once more glad to have written so many UnitTests. They all passed and everything looked good.

Thanks Kim for your help!

At first I used fieldtype TEXT. If in this case somebody edited another field of such a record with phpMyAdmin and pushed the save button, the stored object in that record got currupted and the object could not be deserialized and instanciated anymore.

We now use the fieldtype BLOB instead, so phpMyAdmin does not give you the chance to edit the fieldcontent. And it works.

The 404-Page itself would then respond with its own 404 header and display a beautiful error page.

As I found out, this would NOT have the effect that I expected it to have, namely that the next Google bot crawling these URLs would kill the URL from the index since it ended up on a proper 404 page via the redirect. The headers in sequence with the above redirect look like this:

GET /pages/events.php?id=123 HTTP/1.1
HTTP/1.x 302 Found
GET /errors/404.php
HTTP/1.1 HTTP/1.x 404 Not Found


But wait, there is 302. To verify this for your own redirects, you can use the Firefox add-on Live HTTP Headers.

The problem here is the use of a default redirect, which uses a status code ’302 Found’. This equals the old ’307 Temporary Redirect’ and has the meaning for the bot, something hast just temporarily moved. In order to ‘tell’ the bot that this page should no longer be indexed and deleted, you must use a ’301 Moved Permanently’ and you do it like this:

header("HTTP/1.1 301 Moved Permanently");
header("Location: http://www.mycusweb.de/errors/404.php");
exit;


If you’re interested in the HTTP status codes, you can find the RFCs here.

After all of your redirects that should ‘tell’ bots to remove URLs from the search index use the 301 variant, you can then use the Google Remove Tool to speed things up and call the bot.

If you do not have screen yet, install it on your Debian box with: apt-get install screen

Commands:

• screen -S indexing – Create a screen with name ‘indexing’.
• screen -ls – Show available screens.
• screen -r indexing – Re-attach to the screen ‘indexing’.
• Strg-A, Strg-D – Detatch from a current screen (without terminating it).
• exit OR Strg-D – Exit from a current screen. This terminates the screen session.
  

In this case you can use the commandline-tool ‘at’ and schedule something like ‘at 1am tomorrow do xyz’.

If you do not have at yet, install it on your Debian box with: apt-get install at

There are 3 commands available:

• at <datetime> – Starts the scheduling-dialog for a specific date or/and time to execute certain commands.
• atq – Shows a list of already scheduled and pending jobs.
• atrm <job-id> – Deletes pending jobs you would like to remove from the job-queue.

Case: What do you have to do in order to set a sequence of commands as root at a specific time?

Let’s say it is Friday 15:00h (and the server-clock says that too) and you would like to fire the command

• echo “Good morning!” > hello.txt @ 08:00h tomorrow

You would do the following:

• Log in as root.
• Type ‘at 8.00′ <enter>.
• Since for today 08.00 is already in the past, at assumes that you mean tomorrow. But you can use all sorts of time- and date signatures, for example you could use ‘at 08:00 01.06.2008′.
• Now the prompt changes to at> and waits for your commands to be executed sequentially at the time you specified.
• Now you type your first command to be executet: ‘cd ~/stuff’ <enter> – to make sure where the next command is executed, since it writes a file.
• Now type your 2nd command: ‘echo “Hello 8.00″ > hello.txt’ <enter>
• The prompt shows another at>. If you have more commands add them. In our example we have only two. To finish we press STRG-D and you have your normal prompt back.
• Type the command ‘atq’ to see the pending jobs in the queue. Your job, you just added has a job-ID in the first column.
• To delete the job from the queue type ‘atrm <job-id>’.

Good timing!

• Created dir /mnt/terastation2
• Added the following line to /etc/fstab
  //TERASTATION2/SHARE /mnt/terastation2 smbfs username=marco,password=,uid=marco,gid=businex,dmask=707,fmask606
• And executed
  mount -a
  as root.

If you like to mount it only once in an while, you can also do it via the command:

• smbmount //TERASTATION2/share /mnt/terastation2 -o lfs
  You need to have smbmount installed.

Both should have the effect that you can access the Terastation on /mnt/terastation2. No more Windows clients involved for copying files around.

(1)

• Take a look at rsnapshot: http://www.howtoforge.de/rsync_incremental_snapshot_backups
• Install rsnapshot on your box following the above tutorial.
• Decide for a place for your snapshot-backups on the filesystem and edit /etc/rsnapshot.conf accordingly. I use /var/remotebackups.
• Take a look at the example zip-file (Download: remotebackups example for mySQL and files). Files should be copied to your backup-dir /var/remotebackups (except of /etc/rsnapshot.conf).

(2)

• Shellscripts in /var/remotebackups/_scripts retrieve mySQL-dumps etc. from remote servers and store it in the scripts’ own data/ directory.
• Those fetch-scripts can be triggered manually or all together via /var/remotebackups/_scripts/_execute_all_scripts.sh.
  Example: fetch-script.
• Scripts must be executed as root or with sudo.
• Access via scp or ssh to other machines pops up a password dialogue. To prevent this, setup passphraseless keys using the /<user>/.ssh/authorized_keys file on the remote machines and check it by executing something like this without being asked for a password:
  ssh <remote_user>@<remotemachine_or_ip_adr> “cd ~; pwd;”
  This should show you the path to the home directory of <remote_user>
  The howtoforge-page in the head of this file shows you how to do passphraseless authentication.

(3)

• The command “rsnapshot roundrobin” later triggers the roundrobin backup defined in /etc/rsnapshot.conf. All above data is then saved to a new slot in /var/remotebackups/roundrobin.0-9/….
  WARNING: Always execute data aggregation scripts to have a current version of data in each backup-dir prior to executing rsnapshot! Otherwise old data will be saved in a roundrobin slot. The easiest way to do all together is to just execute /var/remotebackups/_scripts/_execute_all_scripts.sh. I do it manually whenever I feel like needing a backup ;). You can easily automate backups using cron.
• In order to be able to save your snapshots to the 10 mentioned ’roundrobin’-slots of rsnapshot you need this line in your rsnapshot.conf
  interval<tab>roundrobin<tab>10
• I set a symbolic link from the actual conf-file /etc/rsnapshot.conf to have it all together in /var/remotebackups using this command
  ln -s /etc/rsnapshot.conf /var/remotebackups/

(4)

• To add a new backup do the following:
  (a) Create a dir below /var/remotebackups/_scripts like businex_mysql_blog_synergieraum including a fetch-script like fetch_dump.sh and a data-directory.
  (b) Edit the fetch_dump.sh-script with credentials so it copies whatever you would like to backup into the data-directory.
  (c) Add the reference to the new fetch-script in /var/remotebackups/_scripts/_execute_all_scripts.sh in order to have a central triggering script.
  (d) Add the new backup data-directory to your rsnapshot.conf.