Recursively remove all .svn directories from a working copy:

find . -name .svn -exec rm -rf {} \;

Recursively remove all ._xyz-files (OSX meta file info) from your WebDav-Drive, set via hourly cron:

find /var/data/ -name "._*" -exec rm {} \;

Do not forget to set your path ;).

As a development team we always run into situations where we have trouble setting up a proper development environment for each of the team members to get going or add new staff on the go. It annoyed me every time since it causes a lot of unnecessary communication and friction.

I often heard of virtualization but never actually played seriously with it. The idea is:

If we could have a virtual machine for every project that contains an equivalent environment like the production system, everybody working on it…

• … could just rely on his development environment by just starting the VM without having to set up anything half-baked themselves.
• … could use his favourite working environment OS, IDE and tools on which they are most comfortable and thus happy and productive.
• … could work on their own checked out working copy using version control.
• … could immedately see what they built refreshing the local browser or starting Unittests on the VM via ssh to check their dev increments.

We used http://www.virtualbox.org. A good starting point to get to know VirtualBox better and learn how to start your first virtual machine: https://help.ubuntu.com/community/VirtualBox

Our target was to be able to startup the development VM as guest system on any developers development machine being the host system, open a browser on the host (!) and call for example http://develop/ to see the webroot of the VM. Additionally we set up samba and ssh on the VM in order to have the webserver’s webroot on the VM available via the filesystem. In order to do that you need to…

• …start your VM with networking set to ‘Host interface’ instead of the default NAT. This is explained in detail on this page (sorry German) http://www.nwlab.net/tutorials/virtualbox/virtual-networking.html - for me it was tricky to get the guest machine available on the host and have internet access at the same time.
• …edit the hosts file (on OSX ’sudo nano /private/etc/hosts’ and reboot) on the development machines and add something like the following line: ‘192.168.56.101 develop’. To find out the IP enter ’sudo ifconfig’ (OSX/Linux) on your host system after you have started the VM. You will see aditional adapters set by virtualbox and the IP address.
• …configure /etc/samba/smb.conf on the VM, restart samba and connect (e.g. smb://develop/webroot). We check out a working copy of the applicaton under development directly onto the webroot and create a new PHP-Project there in our IDE. Update and commit directly from there.

---

If you search the web for ready-made VMs you find mostly VMware images. You can not run them directly in VirtualBox and need to convert them.

Under Linux you can convert the VMware image (.vmdk) into a VirtualBox image (.vdi) like this:

sudo apt-get install virtualbox-ose virtualbox-ose-guest-utils;
sudo apt-get install qemu;

qemu-img convert xxx.vmdk xxx.bin;
VBoxManage convertdd xxx.bin xxx.vdi

Install the required packages with apt-get install once. VBoxManage is part of VirtualBox. The last two lines do the conversion.

---

We ended up creating a fresh install from a Debian 5 netinstall iso. The iso-file can be mounted as CDROM on the creation of the new VM with VirtualBox. Receipes for setting up the appropriate LAMP environment with apt-get install can be found on the web. You only have to do it once. Save the state of your VM afterwards.

---

There are ways to generate a virtual machine from a physical server. Use Google to find receipes. I used http://www.partimage.org on a Debian Etch system with the live CDROM from http://www.sysresccd.org. This requires that you are able to umount your filesystem or rather boot into the live cd on the production/staging machine in order to generate the partimage.

You mount an external drive over the network or a usb harddrive. The partition (e.g. /dev/sda1) you would like to backup must be umounted. From the live cd you can see your partitions, including attached usb drives, with the ‘fdisk -l‘ command. Just mount the target (e.g. /mnt/usbdrive) and start partimage from the commandline. Dialogues guide you through the image creation.

---

In case you wonder what is meant by the ‘Host key’ to enter or leave a running VM with your mouse… it is the right (!) Strg-Button on your Keyboard.

---

I just installed Ubuntu Server on a VM from my MacBook. To have a usable keyboard once you logged on to the new VM, you must do the following in order to have a keymap including the pipe symbol, braces etc.:

• sudo apt-get install console-data; #to install the keymaps
• sudo loadkeys mac-macbook-de; #to set the keymap for German MacBook

Once you have done that, you can use your right Command-Key as ‘Alt-Gr-Key’ like on a PC keyboard. The pipe symbol is then typable with ‘Alt-Gr + >’, Braces and Brackets are typable via ‘Alt-Gr + 6,7,8,9′.

---

This is how you copy a virtual machine using VirtualBox tools:

$ VBoxManage clonevdi /Users/marco/MyMachine.vdi /Users/marco/MyMachine_copy.vdi
$ VBoxManage internalcommands setvdiuuid /Users/marco/MyMachine_copy.vdi

I had a list of files from a large file structure as a result from a maintenance script run with lines like this:

/home/web/.../sources/.../2008/12/25/4f1feabbd76f79ecab150bdee3f6ae4d.xml
/home/web/.../sources/.../2008/12/25/e506e433a2d87f0275c7641da59bbf7f.xml
/home/web/.../sources/.../2008/12/28/901c4f081645b986e9b1377d3f586b8e.xml
/home/web/.../sources/.../2008/12/28/6bec4d4bbcf8f596c40694210d220a3b.xml
/home/web/.../sources/.../2008/12/24/477c535d6111605c8f6020a959f32fde.xml
/home/web/.../sources/.../2008/12/24/9f253a96fc26d8f6d9e61b8f1bdb3453.xml

Each line represented a document path to a file which was supposed to be removed from the filesystem. You can do that with the following simple oneliner:

for LINE in $( cat ../log/my_empty_files.txt ) ; do rm $LINE ; done

You can try it with ‘echo’ instead of ‘rm’ first to see if it would work:

for LINE in $( cat ../log/my_empty_files.txt ) ; do echo " # $LINE" ; done

I am currently working on a project in which we have lots of images from an old CMS waiting to be migrated into a new layout. Of course there are restrictions so it should not happen that certain image types exceed a certain max. width.

OK, we have many many images… So I took a closer look at ImageMagick (also take a look at the usage examples). And I have to say: Awsome!

You can install ImageMagick on Ubuntu or Debian with a simple
# apt-get install imagemagick

In combination with a bit conditional scripting I came up with the following solution:

I wanted to have a shell script that, given a directory containing all our images, checks the width of each image and resizes it if it exceeded a certain width. Simple, but powerful.

Usage:

$ ./resize_image_dir.sh ../../brand_logos

And you are done with thousands of images in a minute. Do not forget to make a backup if designers change the desired width later…

You can download the shell scripts with example images ready to test:
!resize_conditional_images_bulk2

I was having trouble with a server running Debian 4.0 (etch). Using the standard sources in the /etc/apt/sources.list the supported PHP5 version was 5.2.0-8+etch13 which contained a very annoying bug for my application.

A daily running script - let’s call it the Importer - regularly exited randomly with a “Fatal error: Out of memory (allocated 12320768) (tried to allocate 2851436 bytes) in …” and I had to restart it manually nearly every morning. I had…

• …checked my application for memory wasting operations and loops and fixed them,
• …used ini_set(’memory_limit’, ‘64M’); at runtime, and
• …finally increased memory_limit = 64M in my php.ini.

But all this did not change the bahaviour of the Importer!

So I took a look at the PHP5 Changelog to find potentially fixed bugs in newer releases. Bug #39438 described exactly my problem. So a simple upgrade would help me. But it did not work with ‘apt-get upgrade’ or ‘apt-get install php5=5.2.8′ since the highest version in the apt source I used was the one that I already had: 5.2.0-8+etch13, issued in November 2006… (pretty ancient)

Finally it was this page that had the information we needed: an alternative apt source

deb http://packages.dotdeb.org etch all
deb-src http://packages.dotdeb.org etch all


After getting an impression whether dotdeb was a trustworthy source, we first tried it on our dev-system with ‘apt-get update; apt-get upgrade;’. At this point I was once more glad to have written so many UnitTests. They all passed and everything looked good.

Thanks Kim for your help!

I use a Subversion repository which sits on a Samba share in our LAN. Most of the times I used it from Windows machines where I simply assigned a drive letter (N:\) to the filesystem of the network share (//TERASTATION2/share) hosting the repository. A SVN checkout including URL looked like this:

• svn co file://N:/_repos/projectname/trunk .

Now I have my new Ubuntu/Linux notebook and asked myself how I could use the repository from there too. I tried using smb://terastation2/share/_repos/myproject/trunk - but this did not work in combination with file://.

The solution looks like this:

• Create a dir to mount the share to: $ sudo mkdir /mnt/terastation2;
• Mount the filesystem: $ sudo smbmount //TERASTATION2/share /mnt/terastation2 -o lfs;
  (You can create a launcher to do that for you in the future)
• Then you can do your desired operations: $ svn co file:///mnt/terastation2/_repos/projectname/trunk .;

I have an o2 surf stick to provide me with connectivity on the go via UMTS. It worked fine so far under Windows and it worked also my MacBook. Now I wanted to use it also from my new Ubuntu (Intrepid Ibex) laptop. The first thing I tried was to start it with wine. But this did not work out… The solution was brought to me through the following links:

• http://umtsmon.sourceforge.net/downloads.shtml
• http://www.trueten.de/archives/4587-Mit-dem-eee900-und-ubuntu-via-UMTS-ins-Internet.html=holprige
• http://www.draisberghof.de/usb_modeswitch/

The receipe of my setup was:

• install dependent libraries
• $ sudo apt-get -f install gcom libusb-dev
• download usb_modeswitch sources
• $ tar xjvf usb_modeswitch-0.9.5.tar.bz2
• $ cd usb_modeswitch-0.9.5/
• $ ./compile.sh
• copy the built binary into my bin dir
• $ sudo cp usb_modeswitch /usr/bin/
• copy the default config to /etc and uncomment the right device
• $ sudo cp usb_modeswitch.conf /etc/usb_modeswitch.conf
• I guessed and uncommented the area with the many ‘Huawei’ device entries
• now comes what you will have every time you want connect
• $ sudo usb_modeswitch
• $ sudo usbmon &
• starts the small GUI application, enter you PIN and select ‘connect’

I have just installed Ubuntu Intrepid Ibex on my laptop and had trouble getting a WiFi-connection to work. Since channels 12 and 13 are blocked on some WLAN apapters in the US, you have to unlock them following these instructions:

• http://wiki.ubuntuusers.de/Intrepid_Ibex#WLAN-auf-Kanaelen-12-und-13 (in German)

Thanks for the help Torsten!

gt5 is a handy shellscript that extends the linux command du (disk usage). It can be run from the console or remotely over ssh. Nothing to be installed. It uses GNU tools like head, sed, awk and lynx. It generates html which is then passed to lynx to be displayed.

You can install it via: sudo apt-get install gt5

You will also find screen shots at: http://gt5.sourceforge.net

The advanced packaging tool has a built in cow. A nice piece of programmer humor.

(1) Finds files in generated documentation, containing <span class=”field”>webservice:</span> and writes a file containing a clickable list of links to those pages:
~/sites/html/phpdoc_all_global$ find -type f -print0 | xargs -0 grep -li ‘<span class=”field”>webservice:</span>’ | while read in; do echo “<a href=\”$in\”>$in</a>”; done > clickable_list_page.html

(2) A for-loop on the commandline. The * equals all files and dirs in current directory (try “echo *”). Of course you can replace the ‘echo $dir’ with whatever command you like using $dir as loop-variable.:
for dir in *; do echo $dir; done

(Thanks Alexander)

(3) A simlpe piped grep, which displays the files (-l) containing the string “<FormularObjectGenerator”. And since I was searching recureively (-r) in a svn working copy, I did not want to see all the double filepaths containing “.svn” I filtered by excluding (-v) this pattern from the output.

grep -lr “<FormularObjectGenerator” ./ | grep -v “\.svn”

Inspired by the book “The Productive Programmer” I was paying attention more and more on how I actually get things done and how I could better exploit existing shortcuts or faster paths.

Please do not ask why, but I had the following problem: One my PHP-Apps had scattered syntax errors. The question was, how I could locate them in the application. The first attempt was to let Zend Studio analyse the project… This was very slow and contained a lot of information which I actually did not want to see.

So I came up with the following solution:

find -name ‘*.php’ | xargs -i php -l {} | grep ‘Parser error:’

There are 3 parts piped together:

1. find: Lists all .php files in your project.
2. xargs: Executes one command for each of the lines coming from standard input. The -i flag replaces one line from STDIN with the {}. The php-executalbe has a flag -l (=lowercase L), which lets php check for correct syntax only.
3. grep: Filters the output (lots of output like ‘No syntax errors detected in ./includes/show.inc.php’) to the actual information I liked to see: The files which had syntax errors.

From this experience I learned the following:

• Pay attention to flags of commandline tools, even if they do not appear useful at first sight - like the ‘php -l script.php’. They might be very useful when combined via pipe with other commands.
• I understood one more time the power of the commandline and will more often visit man pages of commands.

Here is a tool which generates loads of random passwords: pwgen.

On Debian you can install it with ‘apt-get install pwgen’.

On the command line if you close a console with a running job, you kill the job. This is different with the tool ’screen’, where you can attach and detach from a ’screen’ without terminating it. You can even start a job in a screen on another machine, detatch, travel somewhere else and re-attach to it on another machine.

If you do not have screen yet, install it on your Debian box with: apt-get install screen

Commands:

• screen -S indexing - Create a screen with name ‘indexing’.
• screen -ls - Show available screens.
• screen -r indexing - Re-attach to the screen ‘indexing’.
• Strg-A, Strg-D - Detatch from a current screen (without terminating it).
• exit OR Strg-D - Exit from a current screen. This terminates the screen session.
  

Case: I had many .sql-files from a backup containing one table-dump each which should be loaded into another MySql-DB.

Simply using phpMyAdmin would not have worked with regard to the size of the tables. So I was thinking about a way to do it on the commandline under Linux. A foreach-solution would have caused me to enter the MySql password for each file… which was what I wanted to avoid since the DB contained 70 tables.

Solution: I concatenated all .sql files into one and imported it into MySql in one go. Supposed your present working directory is the directory containing all .sql-files do the following:

cat *.sql >> ../complete_dump.sql;
cd ..;
mysql my_database -h mysql_host -u myuser -p < complete_dump.sql;
password: ****

And your’re done!

The tool “ots” simply is an open source tool for summarizing texts content.

You can install it on your Linux box with: “apt-get install libots0 libots-dev”

http://libots.sourceforge.net - Here you will also find the academic publications around that topic.

Example usage: cat myarticle.txt | ots –ratio 5

This command would summarize the article, giving the 5% of the article which is most important and return the summary.

This blog-post (http://blog.wired.com/business/2008/05/microsoft-cans.html) would be summarized by the above command to:

“Microsoft said today it plans to phase out its book scanning effort, an initiative launched less than three years ago.” - Nice!

Since I am doing more and more stuff on the commandline, I noticed that sometimes I just wait for some task to finish to do a next step in a sequence to accomplish a certain goal. What if a running task would take an estimated 3-8 hours and it is Friday afternoon?

In this case you can use the commandline-tool ‘at’ and schedule something like ‘at 1am tomorrow do xyz’.

If you do not have at yet, install it on your Debian box with: apt-get install at

There are 3 commands available:

• at <datetime> - Starts the scheduling-dialog for a specific date or/and time to execute certain commands.
• atq - Shows a list of already scheduled and pending jobs.
• atrm <job-id> - Deletes pending jobs you would like to remove from the job-queue.

Case: What do you have to do in order to set a sequence of commands as root at a specific time?

Let’s say it is Friday 15:00h (and the server-clock says that too) and you would like to fire the command

• echo “Good morning!” > hello.txt @ 08:00h tomorrow

You would do the following:

• Log in as root.
• Type ‘at 8.00′ <enter>.
• Since for today 08.00 is already in the past, at assumes that you mean tomorrow. But you can use all sorts of time- and date signatures, for example you could use ‘at 08:00 01.06.2008′.
• Now the prompt changes to at> and waits for your commands to be executed sequentially at the time you specified.
• Now you type your first command to be executet: ‘cd ~/stuff’ <enter> - to make sure where the next command is executed, since it writes a file.
• Now type your 2nd command: ‘echo “Hello 8.00″ > hello.txt’ <enter>
• The prompt shows another at>. If you have more commands add them. In our example we have only two. To finish we press STRG-D and you have your normal prompt back.
• Type the command ‘atq’ to see the pending jobs in the queue. Your job, you just added has a job-ID in the first column.
• To delete the job from the queue type ‘atrm <job-id>’.

Good timing!

Some days ago I saw some colleagues using more than >2 computers with one keyboard and one mouse, dragging the mouse across multiple screens with different operating systems and using the clipboard accross them. Wow!

This is done with a tool named Synergy. Here is my setup with Windows XP and Mac OSX:

I did the following to make it work:

• I installed Synergy on my windows box (the 2 screens on the left) .
• Started Synergy and started it as the ’server’.
• I configured the server with screens named like the IP-Addresses of the machines I wanted to use and defined which machine with its screens are on the left and which ones are on the right. This defines where the mouse cursor transfers to the next screen. I also used alias names for the screens.
• Started Synergy on the server (Windows).
• Then I downloaded Synergy for Mac OSX.
• My MacBook is also attached to a 2nd Mac-screen (no. 3 in the picture). In this setup this one is optional. I had it lying around, so I used it.
• The configuration has to be saved manually in the file synergy.conf. I used some aliases in order to avoid trouble.
• Now I started the Synergy-client on the MacBook with “/Users/Shared/synergy-1.3.1/synergyc -f mmm” from the command line.
• Problems with the connection are shown on the server and on the client and can easily be fixed with the documentation on the synergy website.

You will find all you need to know on http://synergy2.sourceforge.net

A good tutorial: http://www.elsniwiki.de/index.php/Main/SynergyKonfigurieren

grep -R “<rec_lang>de<\/rec_lang>” /var/repository/000/023/*
Show all files that contain a the string <rec_lang>de</rec_lang>

find . -name ‘*.php’ | xargs perl -pi -e ’s/this->type/this->beanType/g’
Find the string ‘this->type’ and replace it with ‘this->beanType’ in all *.php-files in and under the current working directory. Thanks to Darren and Reto!

Here is a list of interesting things you can do on a linux command line. I will continue this list as I come across new ones:

find /var/files/000 -name *.xml | xargs rm -f
Delete all XML-files in all subdirectories of /var/files/000

ls -laR | wc -l
Count all files under the current directory. Be patient on large dir-structures.