I have just received a new book: “Apprenticeship Patterns“: Skimming through the first pages I noticed the words in the 3rd column extending from the points in the 2nd column and thought, ‘Hmm, I have heard about these 2nd column points’…

The following table popped up in my mind, which I would like to share with you:

Software development practices:

Traditional <2000	Agile Manifesto 2001	Craftsman Manifesto 2010
processes and tools	Individuals and interactions	but also a community of professionals
comprehensive documentation	Working software	but also well-crafted software
contract negotiation	Customer collaboration	but also productive partnerships
following a plan	Responding to change	but also steadily adding value

Any ideas for a 4th column? Looking forward to actually reading the book…

We have just had the case of having ftp access to a site on a shared hosting LAMP webspace but needed also the database and mainly the database scheme to setup our own development-system for the app. This is actually very straight forward using the following snippet uploaded and executed on the webserver:

<?php
exec( 'mysqldump -h dbXXX.1und1.de -u dboYYY -pZZZ dbYYY > dump.sql', $ret );
var_dump($ret);

This is what you do:

• Find the configuration with the db-credentials in the source of the application and replace
  • dbXXX.1und1.de with the hostname,
  • dboYYY with the username,
  • ZZZ with the password (notice there is NO blank between the option -p and the password!),
  • dbYYY with the database you would like to backup.
• Save the snippet as file with extension .php and copy it via ftp to the webspace.
• Find the URL to execute the script.
• If it worked, there should be a new file dump.sql in the same directory, download it, it contains the db-schema and all data.
• Delete both files to wipe your traces.

If it does not work try something like “exec(’ls -la’);”. If you see the contents of the directory you can use the exec-function and the problem is something else.

Happy hacking!

Recursively remove all .svn directories from a working copy:

find . -name .svn -exec rm -rf {} \;

Recursively remove all ._xyz-files (OSX meta file info) from your WebDav-Drive, set via hourly cron:

find /var/data/ -name ._* -exec rm {} \;

More nice find examples...

This is just a link to a very good tutorial on how to make your own SVN post-commit hook using a PHP script. It sends the following information via email:

• Committer name
• Commit message
• List of files edited
• Diff of changes made

http://techchorus.net/writing-php-script-send-svn-commit-changeset-email-notification

I recently played the ballpoint game for the first time and was impressed…

• … how clearly it demonstrates the different phases of a team going through the ’storming, norming and performing’-phases for everyone - even for management spectators.
• … how the iteration-rhythm helps to keep focus and increase the team performance to a very high and predictable level.
• … how easy it felt to deliver good results once the team started to perform after the 2nd and 3rd iteration.
• … how much fun it can be, to be part of and work in a cool software develoment team.

You can find various explanations on how the ballpoint game is organized and played.

Here is a nice video that illustrates what happens through the iterations: http://www.youtube.com/watch?v=d4-El7gJuZE

Thanks to Stefan and René for the inspiration!

It is not brand new but still a very interesting talk: Robert Lefkowitz on open source software development methodologies with lots of nice anecdotes.

Watch the 21min video: http://oscon.blip.tv/file/1108217/
Or listen to the audio: http://itc.conversationsnetwork.org/shows/detail3995.html

I just hacked together the following quick and dirty PHP-script to use the tidy-extension from the command line. Maybe somebody else needs something like this somewhere. Check the comment for details:

<?php

//
// (a) Save this as tidy.php
// (b) Call it from commandline like this: $ php tidy.php trashed.html > tidy.html
//     to tidy the file trashed.html to a new file tidy.html.
//

// Installation and configuration see http://php.net/tidy
$config = array('indent'=> true, 'output-xhtml' => true, 'wrap'=> 200);

$tidy = new tidy;
$tidy->parseString(file_get_contents($_SERVER['argv'][1]), $config, 'utf8');
$tidy->cleanRepair();

echo $tidy;

Since one of my coolest (private) projects www.lsis.com somewhat lacks the expected amount of traffic, I would like to emphasize the core feature in this blobpost - just to make you aware of it or encourage you to tell your friends or have ideas to apply the concept to other areas.

In short LSIS - Life Science Information Service - on a daily basis crawls a list of essential news sources in the field of life sciences and indexes this content after applying a series of quality measures. You can traditionally serach from our search frontend OR save your searches once you are comfortable with the results. You can then give your ‘channel‘ an name and decide if you would like to get new articles matching your saved query pushed onto your email stack or subscribe to it via RSS-link and process it with all your other industry feeds in the feed reader of your choice. All this is completely for free and anonymous!

You can also browse a series of existing channels, which are preset saved searches of current life-science buzz-topics and subscribe to as many as you like via email or RSS.

Let’s say you are interested in

• New Drug Applications
• The company Merck Serono
• The Granulocyte-Colony Stimulating Factor
• Botox / Botulinum Toxin Type A

and whatever else you might want to monitor just save yourself a query and start being alerted on a daily basis - make your colleagues wonder what cool up-to-date industry guru you are.

Here is how you install PEAR on Mac OSX:

sudo mkdir /usr/local/temp;
sudo chmod -R 777 /usr/local/temp;
sudo mkdir /usr/local/share/pear;
sudo chmod -R 777 /usr/local/share/pear;
curl http://pear.php.net/go-pear | sudo php;

Set /usr/local as path prefix and install.

You should then be able to use pear, e.g.: pear help.

In order to install modules you also need a temp dir:

sudo mkdir /usr/local/temp;
sudo chmod 0777 /usr/local/temp/;

You can now install PHPUnit:

sudo pear channel-discover pear.phpunit.de;
sudo pear install phpunit/PHPUnit;

Have Fun!

Every old PHP4 application faces this question: Does it also run under a newer PHP5 version? Hopefully you have migrated or deprecated all your legacy stuff already… or at least have customers that understand the value of a proper relaunch and are also willing to pay for it.

To make some customer fossils run on a PHP5 host, we did the following dirty hacks:

• Set “AllowOverride All” in your Apache config in order to enable .htaccess files for runtime configuration.
• Copy your legacy app on your new php5 host.
• Place a .htaccess file in the directory containing something like this:

  php_flag register_globals on
  php_flag register_long_arrays on
  php_flag zend.ze1_compatibility_mode on
  php_flag short_open_tag on
  php_value auto_prepend_file "/home/web/my_legacy/_prepend.php"

  More info on configuration changes in php and list of ini-directives.

• Place a file called _prepend.php in the same directory and set path info accordingly:

  <?php
  // See also .htaccess file in this dir with more directives set for this site
  
  // Php4ify error reporting
  error_reporting(E_ERROR | E_WARNING | E_PARSE);
  
  // Add required include paths to some distant libs
  $path_arr = array();
  $path_arr[] = '/home/web/blah/libs/legacy';
  $path_arr[] = '/home/web/blah/libs/whatever';
  
  $path_str = implode(PATH_SEPARATOR, $path_arr);
  set_include_path( get_include_path() . PATH_SEPARATOR . $path_str);

This is indeed very very dirty and should only be considered as last-resort to make some legacy app runnable under php5 without having to do crazy search-and-replace orgies on old stuff.

As a development team we always run into situations where we have trouble setting up a proper development environment for each of the team members to get going or add new staff on the go. It annoyed me every time since it causes a lot of unnecessary communication and friction.

I often heard of virtualization but never actually played seriously with it. The idea is:

If we could have a virtual machine for every project that contains an equivalent environment like the production system, everybody working on it…

• … could just rely on his development environment by just starting the VM without having to set up anything half-baked themselves.
• … could use his favourite working environment OS, IDE and tools on which they are most comfortable and thus happy and productive.
• … could work on their own checked out working copy using version control.
• … could immedately see what they built refreshing the local browser or starting Unittests on the VM via ssh to check their dev increments.

We used http://www.virtualbox.org. A good starting point to get to know VirtualBox better and learn how to start your first virtual machine: https://help.ubuntu.com/community/VirtualBox

Our target was to be able to startup the development VM as guest system on any developers development machine being the host system, open a browser on the host (!) and call for example http://develop/ to see the webroot of the VM. Additionally we set up samba and ssh on the VM in order to have the webserver’s webroot on the VM available via the filesystem. In order to do that you need to…

• …start your VM with networking set to ‘Host interface’ instead of the default NAT. This is explained in detail on this page (sorry German) http://www.nwlab.net/tutorials/virtualbox/virtual-networking.html - for me it was tricky to get the guest machine available on the host and have internet access at the same time.
• …edit the hosts file (on OSX ’sudo nano /private/etc/hosts’ and reboot) on the development machines and add something like the following line: ‘192.168.56.101 develop’. To find out the IP enter ’sudo ifconfig’ (OSX/Linux) on your host system after you have started the VM. You will see aditional adapters set by virtualbox and the IP address.
• …configure /etc/samba/smb.conf on the VM, restart samba and connect (e.g. smb://develop/webroot). We check out a working copy of the applicaton under development directly onto the webroot and create a new PHP-Project there in our IDE. Update and commit directly from there.

---

If you search the web for ready-made VMs you find mostly VMware images. You can not run them directly in VirtualBox and need to convert them.

Under Linux you can convert the VMware image (.vmdk) into a VirtualBox image (.vdi) like this:

sudo apt-get install virtualbox-ose virtualbox-ose-guest-utils;
sudo apt-get install qemu;

qemu-img convert xxx.vmdk xxx.bin;
VBoxManage convertdd xxx.bin xxx.vdi

Install the required packages with apt-get install once. VBoxManage is part of VirtualBox. The last two lines do the conversion.

---

We ended up creating a fresh install from a Debian 5 netinstall iso. The iso-file can be mounted as CDROM on the creation of the new VM with VirtualBox. Receipes for setting up the appropriate LAMP environment with apt-get install can be found on the web. You only have to do it once. Save the state of your VM afterwards.

---

There are ways to generate a virtual machine from a physical server. Use Google to find receipes. I used http://www.partimage.org on a Debian Etch system with the live CDROM from http://www.sysresccd.org. This requires that you are able to umount your filesystem or rather boot into the live cd on the production/staging machine in order to generate the partimage.

You mount an external drive over the network or a usb harddrive. The partition (e.g. /dev/sda1) you would like to backup must be umounted. From the live cd you can see your partitions, including attached usb drives, with the ‘fdisk -l‘ command. Just mount the target (e.g. /mnt/usbdrive) and start partimage from the commandline. Dialogues guide you through the image creation.

---

In case you wonder what is meant by the ‘Host key’ to enter or leave a running VM with your mouse… it is the right (!) Strg-Button on your Keyboard.

---

I just installed Ubuntu Server on a VM from my MacBook. To have a usable keyboard once you logged on to the new VM, you must do the following in order to have a keymap including the pipe symbol, braces etc.:

• sudo apt-get install console-data; #to install the keymaps
• sudo loadkeys mac-macbook-de; #to set the keymap for German MacBook

Once you have done that, you can use your right Command-Key as ‘Alt-Gr-Key’ like on a PC keyboard. The pipe symbol is then typable with ‘Alt-Gr + >’, Braces and Brackets are typable via ‘Alt-Gr + 6,7,8,9′.

---

This is how you copy a virtual machine using VirtualBox tools:

$ VBoxManage clonevdi /Users/marco/MyMachine.vdi /Users/marco/MyMachine_copy.vdi
$ VBoxManage internalcommands setvdiuuid /Users/marco/MyMachine_copy.vdi

From an old post: To start out with XML-Schema this might be of interest to you:

XSD-Tutorial: http://www.liquid-technologies.com/XsdTutorial_01.aspx

Free graphical Tool: http://www.liquid-technologies.com/LiquidXMLStudio.aspx

[2009-06-27] Update: A very cool feature is the generation of a Schema, based on example-XML files you give Liquid XML Studio! I discovered this when I built a Schema that would not validate against my desired XML structure and I could not figure out why. I generated the XSD like this:

• Open one of your XML examples with Liquid XML Studio.
• Select from the menu “Tools / Infer XSD Schema”.
• You are asked for more examples. I did it with just one XML file.
• And bingo: You have your XSD.

As I did a diff of my handcraftet version of the XSD and the generated one, it revealed the reason for not validating nicely for me ;).

Cool tool, you can use with the 30-day trial licence.

I have just finished the book ‘Managing Humans: Biting and Humorous Tales of a Software Engineering Manager’ by Michael Lopp and I would like to recap the idea behind the 2 identified types of ploblem solvers on a development team. These are their characteristics:

• Incrementalist: They are driven by constantly making small forward increments. They are aware of available resources and the landscape in which they operate at any time. Since they know that there is no final solution, they are good brainstormers to come up with quick solutions. They love discussions and drive progress.

• Completionist: They need time to figure out the plan to analyse and solve a problem before they start moving in a direction. They apply a strategic vision to integrate their solution into the greater picture.  If a Completionist is quiet, is does not mean he has nothing to say. It is just unlikely for him to talk about something without a fully formed plan. After having thought all through, the Completionist knows exactly what to do.  It is the architect type of guy striving for a perfect longtime solution.

In the end both types like to get stuff done. The difference is just how they get there and that is exactly the point around which both regularly argue with one another.

If you think about your team and who shows tendencies towards one of the two types, how can this insight make your team communication and problem solving habits more effective? As a team lead you definitely need both types and it is your responsibility to engage both in a healthy discussion.

This is a simple example-snippet of a form with a dynamic add-row feature, done with jQuery. You can also submit the form and see the results. The form re-displays all entered values. You can extend it to max. 10 rows. Take a look at the source, there you will find all you need to know:

• Demo: dyn_form_rows.php
• Source: dyn_form_rows.phps

I love jQuery!

I am a fan of good receipes and checklists and just discovered the podcast “The Managing Software Development (MSD) Show” by James Edgell, which I highly recommend for all folks being responsible for IT-people and in the end for what they produce.

The value of your software development resources (people) consists of two things: their technical knowledge and their behaviour. The first can be developed by training and gained experience, the second is harder to change. And since everything is about bahaviour in people management, we also focus on it in terms of performance measurement.

James recommends the following:

• Identify important abstract behaviours (see following list).
• Measure the performance of every desired behaviour for each of your directs at regularly scheduled checkpoints. Scoring is from 1-5, where 1 is least demonstrated and 5 is most demonstrated. Add score corrections for plusses and minusses that are not covered by the behaviours in the list, e.g. for a rare talent or special industry knowledge no one else has.
• Set annual goals to improve on some of them. Goals should be ’smart’ (s=specific, m=measurable, a=attainable, t=timebound).
• Sum up the scores and order them by score. You now have a handy helper to make decisions regarding: bonus payment, promotion, layoffs.
  Divide the list in 4 sections from top to bottom:
  • 10% - your excellent people.
  • 20% - exceed expectations.
  • 60% - meet expectations.
  • 10% - need to improve.

Here are the behaviours:

• Strategic planning: Consideration of future needs, vision of the future.
• Maintained industry awareness: Latest trends, not only technical, understands customer business, evolution and lifecycles.
• Innovation: Brings in new ideas, continuous brainstorming, brings vitality to the organization.
• Builds and sustains relationships: Inside the team, the department, also builds those relationships actively outside the department and organization.
• Communicates effectively: Oral, written documentation and email, accurate to-the-point information or distraction.
• Leads and develops a team: Gets things done and drives the team, also motivates to do unpopular tasks.
• Enthusiasm: Does his things with high energy and enthusiasm.
• Assertiveness: Challenges the organization and the team to get the best possible outcome, does not settle with what is already there.
• Decisiveness: Stands by team decisions, solves conflicts quickly, does not undermine made decisions once they have been made.
• Clear and focussed thinking: Concentrates on what is relevant to make progress working on issues, is result oriented.
• Planning and organizing: How efective they organize their tasks and the team’s tasks.
• Productivity: Are they always on time, do they produce work of good quality, are they busy to get things done effectively.
• Customer focus: Mentality towards the customer.
• Integrity: Honesty, ethical thinking, are they trying to do the right things.

Thanks James, please give us more of that!

Today I listened to an interesting podcast “John Gruber & Merlin Mann’s Blogging Panel at SxSW” in which a famous line by Walt Disney was quoted:

“We don’t make movies to make money, we make money to make more movies.”

This indeed makes a huge difference!

A nice gadget: www.jetlev-flyer.com

Once in a while every bigger website is relaunched.

In order to deploy bigger changes without bothering your visitors with strange behaviour during a data migration, updates and the like, you should use Apache2’s mod_rewrite. Just put the following lines in a .htaccess file in your webroot directory and all traffic (also deep links to subdirectories) gets diverted to the maintenance-page. The scond line sets an exception for your IP address, so you are the only visitor who is NOT redirected to the mainteance page:

RewriteEngine on
RewriteCond %{REMOTE_ADDR} !192\.168\.123\.101
RewriteRule !maintenance/index.html /maintenance/index.html [L,NC,R=301]

Do not forget to remove the .htaccess file after you have finished your work!

I had a list of files from a large file structure as a result from a maintenance script run with lines like this:

/home/web/.../sources/.../2008/12/25/4f1feabbd76f79ecab150bdee3f6ae4d.xml
/home/web/.../sources/.../2008/12/25/e506e433a2d87f0275c7641da59bbf7f.xml
/home/web/.../sources/.../2008/12/28/901c4f081645b986e9b1377d3f586b8e.xml
/home/web/.../sources/.../2008/12/28/6bec4d4bbcf8f596c40694210d220a3b.xml
/home/web/.../sources/.../2008/12/24/477c535d6111605c8f6020a959f32fde.xml
/home/web/.../sources/.../2008/12/24/9f253a96fc26d8f6d9e61b8f1bdb3453.xml

Each line represented a document path to a file which was supposed to be removed from the filesystem. You can do that with the following simple oneliner:

for LINE in $( cat ../log/my_empty_files.txt ) ; do rm $LINE ; done

You can try it with ‘echo’ instead of ‘rm’ first to see if it would work:

for LINE in $( cat ../log/my_empty_files.txt ) ; do echo " # $LINE" ; done

I am currently working on a project in which we have lots of images from an old CMS waiting to be migrated into a new layout. Of course there are restrictions so it should not happen that certain image types exceed a certain max. width.

OK, we have many many images… So I took a closer look at ImageMagick (also take a look at the usage examples). And I have to say: Awsome!

You can install ImageMagick on Ubuntu or Debian with a simple
# apt-get install imagemagick

In combination with a bit conditional scripting I came up with the following solution:

I wanted to have a shell script that, given a directory containing all our images, checks the width of each image and resizes it if it exceeded a certain width. Simple, but powerful.

Usage:

$ ./resize_image_dir.sh ../../brand_logos

And you are done with thousands of images in a minute. Do not forget to make a backup if designers change the desired width later…

You can download the shell scripts with example images ready to test:
!resize_conditional_images_bulk2