Monthly Archives: June 2007

Software development mechanics (1)

I have often wondered why it is so difficult for non-developers (management, sales people, customers, inexperienced developers etc.) to understand the mechanics of software development, its life-cycle and the related elements like for example the importance of

  • code-base quality,
  • refactoring,
  • versioning,
  • unit testing,
  • agile development practices,
  • requirements management,
  • project-wide standards,
  • creating a productive, open and creative atmosphere around development,
  • permanent curiosity for new tech-stuff and the freedom to play with it,
  • and many more…

Some individuals of those groups don’t even at least try to find out more about it. Obviously they have not realized yet that when building complex websites, maintaining and extending them for many years they are already developing software – and that this is different from building a toaster once and selling it. So, I decided to post some related articles, books, podcasts etc. whenever I get across them which convey the nature of software development in understandable words.

I hope this selection motivates people to read/listen an maybe even understand one day and thus reduces unnecessary friction.

Here are the first links:

Firefox Plugin Gnosis: Semantic Webservice

Info taken from the plugin download page:

“ClearForest Gnosis is the cutting edge of real time semantic processing for the web. By evaluating the pages you read – as you read them – Gnosis immediately locates key information such as people, organizations, companies, products and geographies hidden within the text.”

Gnosis Screenshot

“By simply hovering over any of the identified topics, you can immediately locate relevant news, blog entries, maps, company information and Wikipedia entries.”

Goto: Firefox-Plugin Gnosis

Organize remote backups with rsnapshot

If you have some customer projects running on webservers (must be accessible to you via ssh) and you would like to regularly backup critical data like mySQL databases or filestructures this post might be useful for you. All you need is a linux box connected to the internet.

(1)

(2)

  • Shellscripts in /var/remotebackups/_scripts retrieve mySQL-dumps etc. from remote servers and store it in the scripts’ own data/ directory.
  • Those fetch-scripts can be triggered manually or all together via /var/remotebackups/_scripts/_execute_all_scripts.sh.
    Example: fetch-script.
  • Scripts must be executed as root or with sudo.
  • Access via scp or ssh to other machines pops up a password dialogue. To prevent this, setup passphraseless keys using the /<user>/.ssh/authorized_keys file on the remote machines and check it by executing something like this without being asked for a password:
    ssh <remote_user>@<remotemachine_or_ip_adr> “cd ~; pwd;”
    This should show you the path to the home directory of <remote_user>
    The howtoforge-page in the head of this file shows you how to do passphraseless authentication.

(3)

  • The command “rsnapshot roundrobin” later triggers the roundrobin backup defined in /etc/rsnapshot.conf. All above data is then saved to a new slot in /var/remotebackups/roundrobin.0-9/….
    WARNING: Always execute data aggregation scripts to have a current version of data in each backup-dir prior to executing rsnapshot! Otherwise old data will be saved in a roundrobin slot. The easiest way to do all together is to just execute /var/remotebackups/_scripts/_execute_all_scripts.sh. I do it manually whenever I feel like needing a backup ;). You can easily automate backups using cron.
  • In order to be able to save your snapshots to the 10 mentioned ’roundrobin’-slots of rsnapshot you need this line in your rsnapshot.conf
    interval<tab>roundrobin<tab>10
  • I set a symbolic link from the actual conf-file /etc/rsnapshot.conf to have it all together in /var/remotebackups using this command
    ln -s /etc/rsnapshot.conf /var/remotebackups/

(4)

  • To add a new backup do the following:
    (a) Create a dir below /var/remotebackups/_scripts like businex_mysql_blog_synergieraum including a fetch-script like fetch_dump.sh and a data-directory.
    (b) Edit the fetch_dump.sh-script with credentials so it copies whatever you would like to backup into the data-directory.
    (c) Add the reference to the new fetch-script in /var/remotebackups/_scripts/_execute_all_scripts.sh in order to have a central triggering script.
    (d) Add the new backup data-directory to your rsnapshot.conf.