An Introduction to the International Software Benchmarking Standards Group

This is a guest blog by Harold van Heeringen, ISBSG president

In this blog, I would like to tell you about the International Software Benchmarking Standards Group, the ISBSG, or as some people say: Icebags. Not completely accurate, but much easier to say than ISBSG.

Icebags is an international not-for-profit organization with the mission to improve decision making in the industry by collecting data of completed software projects and maintenance & support. The members are international software metrics associations, like for instance IFPUG, Nesma, FiSMA and GUFPI-Isma. Also, a couple of companies are gold partners, like Galorath. Galorath supports this organization and provides the Icebags data to customers by offering a separate SEER-SEM database with the ISBSG data. This gives users a second opinion on any estimate created in SEER.

Icebags is a small group of people who altruistically try to collect as much data as possible, simply because we feel this benefits the IT industry. Our lifeblood is data. Our project database contains over 7500 completed projects and the M&S database over 1100 applications. Unfortunately, it’s becoming more and more challenging to get people to submit data. To be able to use industry data for estimation and benchmarking, the size must have been measured in a standard way. ISBSG data is therefore mainly based on function points, as this is still the only international standard to measure the size of functionality in an objective, repeatable and verifiable way. This is a huge issue, especially now the industry is transforming from traditional development methods to agile development methods in such a fast pace.

It seems that especially the agile community has stepped away from the use of functional size measurement, instead using story points for sprint estimation and planning. While it’s true that story points have their value, especially in a team context, it’s clear that metrics based on story points can’t be used to estimate projects, releases or sprints for other teams or organizations. As most agile teams will consider function point analysis to be of waste, as it does not help them with to improve the product at hand, functional size measurement is usually not done any longer. Only organizations with a mandatory process in place to measure performance of completed sprints, releases and projects and use this for estimation and benchmarking still do function point analysis. Unfortunately, very few organizations have that level of maturity. Hence, the challenge for Icebags. As the industry adopts agile software development, and functional size measurement gets less and less popular and considered waste by the teams, how can you collect meaningful data from the industry?

Recently, I went to the ISBSG headquarters in Australia and met with some the staff and discussed the possibilities to face this challenge. As I support these people and share their ideals to collect data, I would like to ask everybody who has data and who can share this data, to submit to the ISBSG.

All submissions are anonymized and can’t be tracked down to the submitting organization or individual. For every submission, you’ll receive a free benchmark report that compares your project to a few relevant peer groups in the industry. They take data in almost every format, easiest submission for just a few projects is probably the online form.

So, if you care about the IT industry, like those at Galorath, please support Icebags and submit data!