Category Archives: Productivity

Migrate Email via IMAP using imapsync

This time I had to migrate not just a domain, but a whole bunch of existing imap-email accounts with it.

To do that, just create all email accounts on the new machine and have a list with all credentials ready. For the hard work I used a very good Perl tool – imapsync.

Very cool with imapsync is, that it actually syncs from the old server. So you do not have to deal with double email or lost mail. See also imapsync description here.

Receipe for Debian Linux:

Download imapsync from https://fedorahosted.org/released/imapsync/:

$ wget https://fedorahosted.org/released/imapsync/imapsync-1.525.tgz

Unzip and cdir into the unzipped directory.

Check your perl installation with:

$ perl -c imapsync

If you see a “Can’t locate Mail/IMAPClient.pm in @INC…” you need to install the IMAPClient:

# cpan Mail::IMAPClient

If you see “imapsync syntax OK”, you already have the required modules.

Migrate one emailbox:

$ ./imapsync --host1 imap.1und1.de --host2 imap.man.ticore.it --port1 993 --port2 993 --ssl1 --ssl2 --user1 testuser@migration-domain.xx --user2 new_username_on_target_system --password1 xxx --password2 yyy

If you get a “Can’t locate IO/Socket/SSL.pm in @INC…” error, this might fix the problem:

# apt-get install libio-socket-ssl-perl

Retry migration:

$ ./imapsync --host1 imap.1und1.de --host2 imap.man.ticore.it --port1 993 --port2 993 --ssl1 --ssl2 --user1 testuser@migration-domain.xx --user2 new_username_on_target_system --password1 xxx --password2 yyy

If output says “Info: host imap.man.ticore.it says it has NO CAPABILITY for AUTHENTICATE LOGIN” just add ‘–authmech2 PLAIN’

Retry migration:

$ ./imapsync --host1 imap.1und1.de --host2 imap.man.ticore.it --port1 993 --port2 993 --ssl1 --ssl2 --user1 testuser@migration-domain.xx --user2 new_username_on_target_system --password1 xxx --password2 yyy --authmech2 PLAIN

Now you have test-migrated one email account. If you have more, which is likely, takle a look at imapsync-1.525/examples. You find 2 important files:

  • file.txt – contains a list of accounts with the self-explanatory line-format user001_1;password001_1;user001_2;password001_2
  • sync_loop_unix.sh – mass-migrates the accounts from the list. Pleas note: You must set the two imap-hostnames manually in this file! Check the directions in the file. Use via
    $ ./sync_loop_unix.sh

Have fun syncing your IMAP stuff around!

Stuff I need to lookup every time

Set ignore to all files of a directory with subversion:
$ cd cache
$ svn propset svn:ignore '*' .
$ svn ci . -m 'Ignore set on cache dir.'

Show changed files between two revisions, overview
$ svn diff -r 300:HEAD --summarize

Show changed files between two revisions, for each revision:
$ svn log -v -r 300:304

See overall latest 20 commit-messages:
$ svn log -l 20

Branching and merging:
See: http://blog.evanweaver.com/2007/08/15/svn-branching-best-practices-in-practice/

Only grep in php source files, not jpgs, movies etc.:
$ grep -i 'whatever' `find . -name '*.php' -print`

Add all new files in a large filestructure to subversion, like after an update of vendors in Symfony2
svn st | grep "^?" | awk "{print $2}" | xargs svn add $1

Remove all deleted files from a large filestructure from subversion, like after a vendors update in Symfony2
svn st | grep '^!' | awk '{print $2}' | xargs svn delete --force

Setup external libraries in subversion:
svn mkdir ZFVersions;
svn add ZFVersions;
svn ci ZFVersions -m 'Added dir for all versions.';
cd ZFVersions;
svn mkdir 1.11;
svn add 1.11;
svn ci 1.11 -m 'Added version subdir.'
cd 1.11;
svn propset svn:externals 'Zend http://framework.zend.com/svn/framework/standard/tags/release-1.11.0/library/Zend' .; # This will checkout in another dir Zend into your dir 1.11. You need this since autoloading is using paths like this require_once(Zend/Feed/Rss.class.php)!!
svn commit -m 'Set external.';
svn up .; # Loads external lib.

Correct date problems in a mysql database – 2020 instead of 2011 in YYYY-MM-DD dates:
UPDATE accounts SET member_startdate = CONCAT('2010', '-', MONTH(member_startdate), '-', DAYOFMONTH(member_startdate)) WHERE YEAR(member_startdate) > 2011;

What is it you guys always look up?

Feedbackloops in Software Development

Since feedback is a form of communication, excessive feedback will have the effect like an explosion in communication paths and will eventually produce more noise than signal.

Any feedback loop in the system must help facilitate the management of change and efficiently adapt requirements along the the way. If this is not the case for a feedbackloop, folks will try to avoid it.

As developers we also experience a decrease in quality and value of feedback if it is not immediate enough and/or requires too much effort from a team member to receive it. So most of the feedbackloops are built right into modern software development. Take a look at the following list of areas where a software development team or team member receives feedback in different forms:

  • Your IDE checks syntax and indicates errors before you hit ‘save’.
  • A release demo for your customer generates prose user feedback.
  • Commit hooks refuse your changes.
  • Deployment on development and staging systems works or doesn’t.
  • The Compiler compiles or refuses to compile and returns errors.
  • Daily standup meetings serve as publishing institution of individual statuses, schedules and problems. Team members get updated, help each other out or problems get escalated up the command chain.
  • Pair programming for direct feedback from colleagues on development skills, technical decisions, used tools etc.
  • Sprint reviews reflect development processes, collect team feedback and improve them.
  • Test driven development repeatedly asserts requirements and APIs of existing and future components and their interdependencies.
  • The continuous integration server runs all test suites, generates documentation, runs the build, code sniffer etc. and informs the whole development team if something goes wrong.

I hope this article serves you as a short reminder for where and how feedback can create value in your software projects. And remember: Software is written by humans and feedback is the opposite of thought-reading, which often leads to unnecessary friction and pain.

Interesting Links:

Useful Linux Commands 04/2009

I had a list of files from a large file structure as a result from a maintenance script run with lines like this:

/home/web/.../sources/.../2008/12/25/4f1feabbd76f79ecab150bdee3f6ae4d.xml
/home/web/.../sources/.../2008/12/25/e506e433a2d87f0275c7641da59bbf7f.xml
/home/web/.../sources/.../2008/12/28/901c4f081645b986e9b1377d3f586b8e.xml
/home/web/.../sources/.../2008/12/28/6bec4d4bbcf8f596c40694210d220a3b.xml
/home/web/.../sources/.../2008/12/24/477c535d6111605c8f6020a959f32fde.xml
/home/web/.../sources/.../2008/12/24/9f253a96fc26d8f6d9e61b8f1bdb3453.xml

Each line represented a document path to a file which was supposed to be removed from the filesystem. You can do that with the following simple oneliner:

for LINE in $( cat ../log/my_empty_files.txt ) ; do rm $LINE ; done

You can try it with ‘echo’ instead of ‘rm’ first to see if it would work:

for LINE in $( cat ../log/my_empty_files.txt ) ; do echo " # $LINE" ; done

Bulk Image Resize using Conditional Width

I am currently working on a project in which we have lots of images from an old CMS waiting to be migrated into a new layout. Of course there are restrictions so it should not happen that certain image types exceed a certain max. width.

OK, we have many many images… So I took a closer look at ImageMagick (also take a look at the usage examples). And I have to say: Awsome!

You can install ImageMagick on Ubuntu or Debian with a simple
# apt-get install imagemagick

In combination with a bit conditional scripting I came up with the following solution:

Console doing bulk resize.

Console doing bulk resize.

I wanted to have a shell script that, given a directory containing all our images, checks the width of each image and resizes it if it exceeded a certain width. Simple, but powerful.

Usage:

$ ./resize_image_dir.sh ../../brand_logos

And you are done with thousands of images in a minute. Do not forget to make a backup if designers change the desired width later…

You can download the shell scripts with example images ready to test:
!resize_conditional_images_bulk2

Etherpad

EtherPad is a free webbsased editor which lets people remotely and simultaneously edit the same text document. Just click ‘Create new Pad‘ and you see the URL which you can send to your remote friend(s).

etherpad_screen

Use it for group development on code snippets, catch realtime meeting minutes, brainstorming ideas and many things more…

Thanks AppJet for this nice tool!

Hippie Completion in Eclipse

I was using a factory like this

$factory = new Object_Factory();
$curr_object = $factory->create('brand');

to generate instances of objects I needed and wondered why Eclipse could not auto-complete e.g. getters and setters in my code for me later on for the $curr_object. My productivity and my motivation to type all that stuff were seriously suffering…

So I found a very nice solution: Hippie Completion!

Instead of listing possible completion solutions like with Strg-Space the

key-combo “Alt-/” (/ on the number block!)

just completes what you started typing. You can also press Alt-/ as many times you like to page through all terms you could have meant like with the Strg-r on the console.

In case you try and wonder why it is not working like expected right away: Precondition for the hippie completion in Eclipse is, you typed the term to be completed once before!

Why didn’t anyone tell me this cool one before!