Feedbackloops in Software Development

Productivity, Software development February 28th, 2010

Since feedback is a form of communication, excessive feedback will have the effect like an explosion in communication paths and will eventually produce more noise than signal.

Any feedback loop in the system must help facilitate the management of change and efficiently adapt requirements along the the way. If this is not the case for a feedbackloop, folks will try to avoid it.

As developers we also experience a decrease in quality and value of feedback if it is not immediate enough and/or requires too much effort from a team member to receive it. So most of the feedbackloops are built right into modern software development. Take a look at the following list of areas where a software development team or team member receives feedback in different forms:

  • Your IDE checks syntax and indicates errors before you hit ’save’.
  • A release demo for your customer generates prose user feedback.
  • Commit hooks refuse your changes.
  • Deployment on development and staging systems works or doesn’t.
  • The Compiler compiles or refuses to compile and returns errors.
  • Daily standup meetings serve as publishing institution of individual statuses, schedules and problems. Team members get updated, help each other out or problems get escalated up the command chain.
  • Pair programming for direct feedback from colleagues on development skills, technical decisions, used tools etc.
  • Sprint reviews reflect development processes, collect team feedback and improve them.
  • Test driven development repeatedly asserts requirements and APIs of existing and future components and their interdependencies.
  • The continuous integration server runs all test suites, generates documentation, runs the build, code sniffer etc. and informs the whole development team if something goes wrong.

I hope this article serves you as a short reminder for where and how feedback can create value in your software projects. And remember: Software is written by humans and feedback is the opposite of thought-reading, which often leads to unnecessary friction and pain.

Interesting Links:

Software Craftsmanship Manifesto

Books, Software development January 22nd, 2010

I have just received a new book: “Apprenticeship Patterns“: Skimming through the first pages I noticed the words in the 3rd column extending from the points in the 2nd column and thought, ‘Hmm, I have heard about these 2nd column points’…

The following table popped up in my mind, which I would like to share with you:

Software development practices:

Amazon

Traditional
<2000
Agile Manifesto
2001
Craftsman Manifesto
2010
processes and tools Individuals and interactions but also a community of professionals
comprehensive documentation Working software but also well-crafted software
contract negotiation Customer collaboration but also productive partnerships
following a plan Responding to change but also steadily adding value

Any ideas for a 4th column? Looking forward to actually reading the book…

Add a PHP Post-Commit-Hook to your SVN

Snippets, Software development, Tools, Tutorials November 10th, 2009

This is just a link to a very good tutorial on how to make your own SVN post-commit hook using a PHP script. It sends the following information via email:

  • Committer name
  • Commit message
  • List of files edited
  • Diff of changes made

http://techchorus.net/writing-php-script-send-svn-commit-changeset-email-notification

The Ballpoint Game - A Scrum Simulation

Softskills, Software development October 29th, 2009

I recently played the ballpoint game for the first time and was impressed…

  • … how clearly it demonstrates the different phases of a team going through the ’storming, norming and performing’-phases for everyone - even for management spectators.
  • … how the iteration-rhythm helps to keep focus and increase the team performance to a very high and predictable level.
  • … how easy it felt to deliver good results once the team started to perform after the 2nd and 3rd iteration.
  • … how much fun it can be, to be part of and work in a cool software develoment team.

You can find various explanations on how the ballpoint game is organized and played.

Here is a nice video that illustrates what happens through the iterations: http://www.youtube.com/watch?v=d4-El7gJuZE

Thanks to Stefan and René for the inspiration!

Robert Lefkowitz, Exceptional Software Explained: Embrace Error

Software development, Videos October 29th, 2009

It is not brand new but still a very interesting talk: Robert Lefkowitz on open source software development methodologies with lots of nice anecdotes.

Watch the 21min video: http://oscon.blip.tv/file/1108217/
Or listen to the audio: http://itc.conversationsnetwork.org/shows/detail3995.html

Consistent Development Environments using VirtualMachines

Linux, Software development, Tools July 30th, 2009

As a development team we always run into situations where we have trouble setting up a proper development environment for each of the team members to get going or add new staff on the go. It annoyed me every time since it causes a lot of unnecessary communication and friction.

I often heard of virtualization but never actually played seriously with it. The idea is:

If we could have a virtual machine for every project that contains an equivalent environment like the production system, everybody working on it…

  • … could just rely on his development environment by just starting the VM without having to set up anything half-baked themselves.
  • … could use his favourite working environment OS, IDE and tools on which they are most comfortable and thus happy and productive.
  • … could work on their own checked out working copy using version control.
  • … could immedately see what they built refreshing the local browser or starting Unittests on the VM via ssh to check their dev increments.

We used http://www.virtualbox.org. A good starting point to get to know VirtualBox better and learn how to start your first virtual machine: https://help.ubuntu.com/community/VirtualBox

Our target was to be able to startup the development VM as guest system on any developers development machine being the host system, open a browser on the host (!) and call for example http://develop/ to see the webroot of the VM. Additionally we set up samba and ssh on the VM in order to have the webserver’s webroot on the VM available via the filesystem. In order to do that you need to…

  • …start your VM with networking set to ‘Host interface’ instead of the default NAT. This is explained in detail on this page (sorry German) http://www.nwlab.net/tutorials/virtualbox/virtual-networking.html - for me it was tricky to get the guest machine available on the host and have internet access at the same time.
  • …edit the hosts file (on OSX ’sudo nano /private/etc/hosts’ and reboot) on the development machines and add something like the following line: ‘192.168.56.101 develop’. To find out the IP enter ’sudo ifconfig’ (OSX/Linux) on your host system after you have started the VM. You will see aditional adapters set by virtualbox and the IP address.
  • …configure /etc/samba/smb.conf on the VM, restart samba and connect (e.g. smb://develop/webroot). We check out a working copy of the applicaton under development directly onto the webroot and create a new PHP-Project there in our IDE. Update and commit directly from there.

If you search the web for ready-made VMs you find mostly VMware images. You can not run them directly in VirtualBox and need to convert them.

Under Linux you can convert the VMware image (.vmdk) into a VirtualBox image (.vdi) like this:

sudo apt-get install virtualbox-ose virtualbox-ose-guest-utils;
sudo apt-get install qemu;

qemu-img convert xxx.vmdk xxx.bin;
VBoxManage convertdd xxx.bin xxx.vdi

Install the required packages with apt-get install once. VBoxManage is part of VirtualBox. The last two lines do the conversion.


We ended up creating a fresh install from a Debian 5 netinstall iso. The iso-file can be mounted as CDROM on the creation of the new VM with VirtualBox. Receipes for setting up the appropriate LAMP environment with apt-get install can be found on the web. You only have to do it once. Save the state of your VM afterwards.


There are ways to generate a virtual machine from a physical server. Use Google to find receipes. I used http://www.partimage.org on a Debian Etch system with the live CDROM from http://www.sysresccd.org. This requires that you are able to umount your filesystem or rather boot into the live cd on the production/staging machine in order to generate the partimage.

You mount an external drive over the network or a usb harddrive. The partition (e.g. /dev/sda1) you would like to backup must be umounted. From the live cd you can see your partitions, including attached usb drives, with the ‘fdisk -l‘ command. Just mount the target (e.g. /mnt/usbdrive) and start partimage from the commandline. Dialogues guide you through the image creation.


In case you wonder what is meant by the ‘Host key’ to enter or leave a running VM with your mouse… it is the right (!) Strg-Button on your Keyboard.


I just installed Ubuntu Server on a VM from my MacBook. To have a usable keyboard once you logged on to the new VM, you must do the following in order to have a keymap including the pipe symbol, braces etc.:

  • sudo apt-get install console-data; #to install the keymaps
  • sudo loadkeys mac-macbook-de; #to set the keymap for German MacBook

Once you have done that, you can use your right Command-Key as ‘Alt-Gr-Key’ like on a PC keyboard. The pipe symbol is then typable with ‘Alt-Gr + >’, Braces and Brackets are typable via ‘Alt-Gr + 6,7,8,9′.


This is how you copy a virtual machine using VirtualBox tools:

$ VBoxManage clonevdi /Users/marco/MyMachine.vdi /Users/marco/MyMachine_copy.vdi
$ VBoxManage internalcommands setvdiuuid /Users/marco/MyMachine_copy.vdi

Free XSD Editor - Generate XSDs from XML

Software development, Tools, Tutorials June 28th, 2009

From an old post: To start out with XML-Schema this might be of interest to you:

XSD-Tutorial: http://www.liquid-technologies.com/XsdTutorial_01.aspx

Free graphical Tool: http://www.liquid-technologies.com/LiquidXMLStudio.aspx

[2009-06-27] Update: A very cool feature is the generation of a Schema, based on example-XML files you give Liquid XML Studio! I discovered this when I built a Schema that would not validate against my desired XML structure and I could not figure out why. I generated the XSD like this:

  • Open one of your XML examples with Liquid XML Studio.
  • Select from the menu “Tools / Infer XSD Schema”.
  • You are asked for more examples. I did it with just one XML file.
  • And bingo: You have your XSD.

As I did a diff of my handcraftet version of the XSD and the generated one, it revealed the reason for not validating nicely for me ;).

Cool tool, you can use with the 30-day trial licence.

Generalizing Specialists, Agile Heros

Industry Terms, Software development March 25th, 2009

Inspired by the Podcast ‘Tips and Advice - Manifesto for Agile Software Development‘ in which Bob Payne and George Dinwiddie discuss the points made in the Agile Manifesto and talk about agile environments and mechanics. If you would like to discover the power of agile teams you should definitely check this one out!

They also mention the term ‘Generalizing Specialist’. Scott W. Ambler wrote about this kind of developer in his essay ‘Generalizing Specialists: Improving Your IT Career Skills‘. This is a kind of developer who is constantly learning and playing with adjacent topics and thus widens his knowledge over the years of his worklife. The effects and advantages for the team, the project owner and the developer are depicted nicely in this essay.

Frameworks…

Articles, Software development, Tools March 13th, 2009

Find out interesting thoughts and experiences about the selection and usage of a frameworks for your projects in this set of slides ‘Living with Frameworks‘ by Stuart Herbert, Technical Manager at www.gradwell.com.

You will learn about:

  • How frameworks save you time (=money) and ensure quality but can also waste resources if applied in another way the framework was intended to be used.
  • The importance of the framework guru role.
  • That a chosen framework and architecture should be strategically introduced (top-down).
  • Introduction and proper use of a new framework has a steep learning curve.
  • Legacy code and the parallel maintenance hassle.
  • Refactor early, refactor often, perform regular code reviews with the framework guru.
  • Frameworks will not fix bad practice (specification, quality, no training).
  • Your framework should fit your overall development plan and practice.
  • Training your staff in your framework is a means of building your team.
  • Headcount on projects has increased, means more teamwork, organisation around needs to mature too.
  • Importance of upgrades and backward compatibility.

Planning Poker for better Estimates

Articles, Software development, Tools January 11th, 2009

There are many essays and articles on ‘Planning Poker’ out there. So I do not need to repeat the principles here. By checking out the recommended links (and folloups and others from your own research) you need to understand the following:

  • What a story point of a project looks like and how you generate those story points in advance. Remember, your estimation should include design, coding, testing and delivery. In Planning Poker it is your goal to come up with estimates to those those story points with your team.
  • The process of how one story point is estimated using the Planning Poker card deck and how to narrow down your team members’ estimates through discussion of the ‘played’ estimates.
  • The background of the card deck used and why there are such odd numers in it.
  • The effect all this will have to your project and the team.

The important thing is not the single estimate as such, but rather the open discussion among the the team members in advance, the mix of different perspectives and the collection of confidence in the team’s estimates. If the team is not able to get confident for an estimate you better recheck the user story in question and clarify it - possibly again with the customer. The more confident the team in one estimate is (=no big spread in single estimates of the team members), the less unclear ‘assumptions’ have been made in the estimate itself. This, in the end, reduces the overall risk (friction and interruptions) during the actual project and most importantly makes your customer happy.

Recommended articles and links:

Software Development Myths

Software development November 28th, 2008

There is no doubt: The environment in which software is designed, produced and maintained has fundamental impact on what comes out below the bottom line for all players playing in this game. I have learnt that there is little in software development, you can do intuitively right the first time - even if you have a background from another production discipline. Surprisingly even people working in a software development environment sometimes just don’t have the required understanding nor are they open to it.

To make these points of un-openness transparent, a great way is to talk about common myths around software development. Other people have done this before, so all I deliver here is a list of links that I can recommend for reading and discussion among your team:

Now check yourself: How much cynicism did you feel reading things like ‘More developers mean faster development’ or ‘Every design decision is documented before it is implemented’ or ‘You dont need developers when designing software’ or ‘The answer to every challenge we face in the software industry lies in doing more work in shorter times’ etc?

If you did not feel any cynicism at all or very little, well… it is time to leave the cave.

Browsershots

Software development, Tools October 27th, 2008

Browsershots loads a given URL, using different browsers on different OSses and generates a screenshot for each one. You can also specify window size, JavaScript-ing and Flash version. Simply enter your URL and wait until your ‘job’ has been processed. Finally you will see a screen like the one on the left.

http://browsershots.org

Video: Software Craftsmenship

Software development, Videos October 15th, 2008

The video (82min) shows Scott Dillman’s talk about ‘Software Craftsmenship’ at Agile 2008. He explains a series of fundamentals around software development in organizations and being a software developer, best practices, code quality, passing on knowledge, culture, metrics and many many more. Watch it!

Goto: http://www.infoq.com/presentations/Craftsmanship-Scott-Dillman

Podcast: OpenSource Insights

Inspiration, Podcasts, Software development October 10th, 2008

I would like to recommend an inspiring podcast I listened to this morning while driving to work. Simon Phipps, chief open source officer at Sun Microsystems offers a very encouraging view on open source, the stages in open source development, mindset and business modells.

Goto episode 39 of FLOSS Weekly: http://twit.tv/floss39

Survey 2008: ‘State of Agile Development’

Articles, Software development September 1st, 2008

VersionOne have published their 3rd Annual ‘State of Agile Development’ survey: Results are available on this page or as pdf.

Overview of topics covered:

  • State of agile methods.
  • Mix of participanting companies.
  • Reasons for adopting agile methods.
  • Concerns against it.
  • Practices applied.
  • Reasons for failures.
  • Where agile creates value.
  • What tools are used.

Make Your Projects Transparent

Links, Software development August 12th, 2008

As projects and teams I am working on get more interwoven and organizational issues and management processes are gaining more and more traction on the 3 project dimensions time, budget and quality, I was thinking about what I would do as a manager in charge to let steam off the many issues we were facing and become productive again.

A low hanging fruit I identified, was the overall uncertainty about the many projects and tasks running at the same time with nebulous requirements and developers working in parallel on many things without clear priorities, which resulted in an ad-hoc development and fixing-style of work. This created an atmosphere in which it was nearly impossible to plan - a vicious-circle.

As a consequence I was thinking about how we could …

  1. … make the ongoing project-related processes more transparent to all participants (Chicken and Pigs),
  2. … display project progress, status, expected outcome and deadlines, and
  3. … visualize the current consumption of available ressources and the team’s mood.

During my research I came accross the term ‘Big Visual Charts’. Once you enter the arena of visualizing processes, you will also come across closely related topics like Kanban (Japanese for card or sign) and Lean Production.

Summarizing points:

  • Hand-made drawings on a wall draw more attention than everything electronically distributed or printed stuff. They are easy to maintain and unique.
  • Capture what is important for success, spotted problems, especially recurring ones. Capture good things and bad things equally.
  • Do not be fingerpointing in your visualizations, just display facts to trigger goal oriented behaviour. Your goal is always to become more effective through behaviour adaption.
  • As time goes by, your charts evolve and display certain patterns which work as trigger for people to come together and find solutions as a team.
  • A central project dashboard helps the team and the project stakeholders to understand the bigger picture, what they are currently doing, what comes next and what they want to focus on. It helps the team to self-organize and prioritize tasks.

Recommended links:


Examples and inspiration:


One nice example of a project planning board is this one from the ‘Agile in Action’ site. See ** link from the above list for explanations.
Nice to see the Southpark icons for each team member ;).


This one focusses on the planned stories (green) plus incoming change requests (yellow) and defects (red) to be fixed. It becomes obvious if there where fewer changes and less defects, the team could concentrate more on the production of story points. See * link from above list for explanations.

A burndown chart in Scrum showing the remaining stories in the project backlog. It also visualizes the progress towards a deadline. See **** link from the above list for explanations.
There are many more chartable things:
  • Show team velocity, the number of story points finished or how many ‘ideal man-days’ your team spent actually in production per iteration. This knowledge will also help you plan more accurately in the future.
  • Defects reported and defects fixed.
  • Unittests and actually working unittests, or code coverage of your test suites. Discover how better code coverage and more tests affect reported defects and leave more time to be productive in other areas in the future or reduce overtime.
  • Hours spent pair-programming.
  • Track unplanned interruptions by adding a red sticker on a calendar or the story card you are currently working on.
  • Team code ownership, number of people contributed to certain modules or average contributors.
  • Last date the boss ordered pizza for the team.

Happy charting!

The Chicken and Pig Analogy

Industry Terms, Software development August 12th, 2008

If you engage in agile development methods, sooner or later you will come accross the ‘Chicken and Pig’ analogy. It’s purpose is to depict the different levels of commitment of project stakeholders. It summs up like this:

A chicken and a pig decide to start a restaurant. The chicken suggests to serve bacon and eggs. The pig responds, “No thanks. For you, bacon and eggs is simply involvement. For me, it is total commitment!”

This translates into: Chicken, while still project stakeholders, are only losely committed to the project and are not accountable for deliverables. And Pigs are the ones held accountable for deliverables (developers, architects, designers).

Now, how can you translate this controverse picture into being more productive with your team and still love your work without getting fried or frying colleagues?

Stages of Innovation

Softskills, Software development July 17th, 2008

The five stages of innovation:

1. People deny that the innovation is required.
2. People deny that the innovation is effective.
3. People deny that the innovation is important.
4. People deny that the innovation will justify the effort required to adopt it.
5. People accept and adopt the innovation, enjoy its benefits, attribute it to people other than the innovator, and deny the existence of stages 1 to 4.

Taken from http://sydfish.wordpress.com/2008/03/17/the-five-stages-of-innovation/

Developer Mantras

Software development July 16th, 2008

A nice list of developer mantras:

http://www.pragprog.com/the-pragmatic-programmer/extracts/tips

Excerpt:

  • Don’t Repeat Yourself
  • Eliminate Effects Between Unrelated Things
  • Don’t Assume It - Prove It
  • Design with Contracts
  • Finish What You Start
  • Refactor Early, Refactor Often
  • Test Your Software, or Your Users Will
  • Don’t Think Outside the Box - Find the Box
  • Coding Ain’t Done ‘Til All the Tests Run
  • There Are No Final Decisions
  • Fix the Problem, Not the Blame
  • Crash Early
  • Minimize Coupling Between Modules
  • Don’t Program by Coincidence
  • Start When You’re Ready
  • Find Bugs Once
  • and many more …

If you want to find out more, check out the books at the bottom of the mentioned URL.

Apply, Inspect and Adapt

Softskills, Software development July 4th, 2008

Inspired by a talk of Joseph Pelrine

http://www.infoq.com/presentations/Agile-Adoption-Joseph-Pelrine

about why agile methods sometimes fail in organizations, I became aware of the following point:

The cycle to successful progress in agile teams is NOT Inspect > Adapt but rather Apply > Inspect > Apadpt.

Agile teams operate in complex environments (systems). Often there are no direct connections between causes and effects observable about things that are going on during project sprints. The only chance is to address issues in retrospect with a bit more distance and the picture how things evolved in the underlying system. You can not do a useful insprection and then do adaptions without really having tried (=applied) things out in your system’s environment.

Some other points from Joseph Pelrine’s session on why agile sometimes fails:

  • Not understanding the Apply > Inspect > Adapt cycle.
  • If you are doing agile by the book (only Inspect > Adapt), you are not doing agile.
  • Agile techniques are scary to many people. They can not deal with what comes up and tend to get crushed.
  • People not willing or able to step outside of their comfort-zones.
  • Problems in communication and conflicts.
  • Use of the wrong tools in wrong places just for the sake of using them.