Estimating Costs of Open Source Software

3.3.2.1             Open Source Software

Open source software is a great advantage when building or enhancing software systems.  Open source provides functionality without all the development work.  Open source can be components such as a graphics library or an entire application. The strict definition of open source software says the source code is free.  By this definition, many items people refer to as open source are not.  For example, Google Docs is free but it is not open source.  Those items are generally classified as Commercial Off the Shelf (COTS). Since in many cases the world uses these terms interchangeable this document will address open source first, then some other derivations often considered open source.

 

Open Source Cost WBS and Data Dictionary.  Regardless of the open source software utilized, there are common elements of cost summarized here as open source work breakdown structure items, and defined below.

Open Source WBS Elements
X.1 Open Source Systems Engineering Labor

X.2 Open Source Development

X.2.1 Familiarization Labor

X.2.2 Development Labor

X.3 Open Source maintenance labor

X.4 Open Source software licenses

 

Definitions:

X.1 Open Source Systems Engineering Labor: Contractor or internal systems engineering to select and prove an open source option. Estimating systems selection & approval costs involves the work required to choose the package to be used from alternatives and verifying its applicability to the mission.

X.2 Open Source Development

X.2.1 Familiarization Labor: Labor to understand the OSS component sufficient to use it

X.2.2 Development/Deployment Labor: Labor to refine user requirements, configure, add tables, and test the open source solution.

X.3 Open Source maintenance labor: Labor to maintain & support the open source solutions including testing and deploying updates and adapting to new requirements

X.4 Open Source software licenses: the cost of procuring software licenses

Note1 Labor to build glue code which links the OSS to the developed software is outside the open source software estimate.  Rather, it is part of the normal development labor estimate. Additionally, if the open source package offers a plug-in type capability for developing expanded functionality that effort is not included. Neither does it include software development to modify the open source code itself.  Modification of open source code should be avoided when possible and normally falls under traditional software development

 

Note2 OSS obsolescence rework goes into maintenance labor – or restart from the beginning as a new development effort.  In both cases it begins with Systems Engineering effort to choose products.

 

3.3.2.1.1          Spectrum of Types of Open Source Software

Open source software comes in several main varieties.  Various open software providers have unique offerings, making a complete covering of all possible combinations impossible.  The spectrum of common types includes:

Open Source Open Use: This software is acquired in source code format, often with no warranty.  Acquirers are responsible for compiling, testing and other effort.  Compiled versions may or may not be available for download as well.

Open Source Black Box Use: Non Developmental Item.  Software cannot be changed by acquirers.  Changes may come from the open source community but this is out of the acquirers’ control.

Open Source Black Box Use from Vendor: Non Developmental Item with Vendor Support.  Software cannot be changed by acquirer but there is a vendor that supports the products as with other proprietary software.

There are two additional types that might be called open source that will not be covered here.

Open Source Open Use Developmental: This can be estimated as standard development effort. The organization utilizes open source software, but then the development organization makes changes.  Once changes are made the software becomes reused developmental software since support is lost from the open source community, thus increasing the organization’s maintenance responsibility.  Some updates from the open source community may be applicable but the acquiring organization must determine that and add and test those enhancements manually. NOTE THIS BECOMES INTERNAL SOFTWARE FOR MAINTENANCE ONCE CHANGED.  Assumptions that the program’s changes will be accepted by the open source community and distributed back is high risk and not recommended for costing purposes.

Open Source Open Use, Component: These are areas where software has been submitted as open source but the applications may not be complete or have more general use.  These can be estimated the same as open source, open use.

Where not to estimate open source or only estimate license costs:  Open Source Black Box Use Items such LINUX Operating System, Subversion Source Control system - unless the organization plans to modify them.

Operating system upgrades can be a significant source of testing effort that is generally costed as part of software maintenance for operational systems rather than being considered open source.  Upgrades are often controlled during development so they occur when appropriate.  Their test effort is often captured as part of development effort. Deployment is normally part of IT costs and may be included with some cloud systems.  License costs still need to be considered.

Swapping open source components in existing systems:  For existing systems changing parts to open source may cost more than keeping the proprietary version intact because these will require redesign, reimplementation, and retest of the existing system with open source.

 

3.3.2.1.2          Cost Estimating Methodology by Spectrum Type

These differing methods of acquiring open source software each have acquisition labor costs as discussed above.  Table 1 below summarizes the key cost driver elements that can be used to estimate the total acquisition cost.  Detailed discussion of the individual labor cost estimation, by spectrum type, follows.

 

  X.1 

Systems Engineering

X.2 

Development

X.3 

Maintenance

X.4 

Additional Costs

Open Use Compute Effective Size, Functionality or SLOC, or use Systems Engineering model Use Effective Size Cost Model with Use Total or Effective Size Licensing Cost
Black Box Use Compute Effective Size, Functionality or SLOC Similar to Open Source Open Use Same as Open Use Licensing Cost
Black Box Use from Vendor Compute Effective Size, Functionality or SLOC Various, good approach is function points Same as Open Use Licensing Cost plus

Support

Open Use Developmental Compute Total, Effective, New Size Estimate as Development Same as Open Use May have licensing cost

Table 1 - Summarized Open Source Estimating Process

3.3.2.1.2.1       Estimating Open Source Open Use

Open source open use can be a great cost advantage since the software may already be proven, supported and widely used.  Figure 1 shows the types of labor cost elements discussed above in the time-flow they typically occur.

 Figure 1 - How to Estimate:  Open Source Open Use

3.3.2.1.2.1       X.1      Costing Open Source Open Use Systems Engineering: Estimate Systems Selection / Approval Cost

If the package has already been approved for use and its applicability is obvious this step’s estimation cost approaches zero.  For example, if LINUX is approved as a current standard there is no need to estimate such labor.

Labor may be estimated by consideration of such cost driver parameters as:

  • Estimated number of unique functions to be understood, learned, tested (convert to 4 IFPUG* function points per function if desired)
  • Data Tables** Referenced (groupings of data that might be used as outputs, inquiries, or internal functions - can convert to 7 function points per group if desired)
  • Data tables configured (generally, can convert to 10 function points each)
  • Component selection completeness (the degree of effort already accomplished to select an open source product)
  • Experience with component (how familiar the acquirer is with the open source product under consideration)
  • Amount of Glue code required to prove the application (while some open source products can be configured to interact with developed software, others must have custom code developed to “glue” the product to the larger application)
  • Test Level required (degree of rigor required e.g. operational system vs. simulation model vs. trainer)

* The International Function Point Users Group (IFPUG) is a US-based worldwide organization of Function point analysis (https://en.wikipedia.org/wiki/Function_point_analysis) metric software users. IFPUG owns Function Point Analysis (FPA) as defined in ISO standard 20296:2009 which specifies the definitions, rules and steps for applying the IFPUG's functional size measurement (FSM) method.

** User recognizable group of logically related data or control information either maintained or referenced by the application

Open Source Labor via Sizing From Metrics Database:

Alternately the scope (Effective Size) may be estimated by finding a size from one of the open source metrics databases such as OpenHub.net or coverity.com, or obtain the open source code directly and run the University of Southern California Code Counter to determine the size of the open source software in Logical source lines of code (SLOC).   Caution: when obtaining size from outside sources verify it is logical size, not physical or some other size definition or costs will be significantly overstated.  If it is another size definition, logical size can usually be approximated from it.  Given SLOC, use of a reuse table can estimate the Systems Engineering effort needed which can be translated into costs by running a cost model or using a standard dollars per line of code.

Open Source Labor via Readily Available Systems Engineering Model:

Using a systems engineering model one can estimate the Open Source Systems Engineering labor:  There are several ways to do this.  One way is to use parameters for a Systems Engineering Cost model.

Common cost driver parameters include such considerations as:

  • Number of Easy/Nominal/Difficult Adopted or Managed Requirements (indicators of the amount of functionality to be utilized in the open source package under consideration, and the degree of testing needed)
  • Number of Interfaces to external software packages
  • Purchase Agents’ Understanding of the EGS Architecture and Requirements as it relates to the OSS under consideration (degree of understanding the development software and therefore its use of the OSS package)
  • Migration Complexity (the extent to which the legacy system functionality might be migrated to the OSS package)

 

3.3.2.1.2.1       X.2      Estimating Open Source Open Use Development

Estimate effective size:

Find Total Size - either find a size from one of the open source metrics databases such as OpenHub.net or coverity.com or obtain the open source code and run the University of Southern California Code Counter to determine the size of the open source software in Logical source lines of code (SLOC) considering items in table 2.   Caution: when obtaining size from outside sources verify it is logical size, not physical or some other size definition.  If it is another size definition logical size can usually be approximated from it.

Logical Lines of Code Definition
Statement Type Disposition Include Disposition Exclude
Executable Include
Declarations Include
Complier Directives Include
Comments Exclude
Blank Lines Exclude

Table 2 – Logical Lines of Code Definition

Estimate Rework Factor – after calculating the total size, use a rework table to compute the multiplication factor for effective amount of effort required.  Table 3 below provides an example of items of consideration when deriving effort associated with reused code.  Common elements are the redesign effort (how much reverse engineering knowledge must be gained of the OSS package functionality), reimplementation (coding required), and retest (how thoroughly the package must be tested in the target application to ensure compatibility with the parent application functionality).

Rework Factor - generalized process with OSS considerations noted
Step 1: Set Reverse Engineering & Redesign Factors
There should be no redesign effort for OSS
Estimate the degree of functional understanding needed
Step 2: Set Reimplementation (recoding) Factors
Unless modifying the OSS this should be zero
Step 3: Set Retest Factors
Consider test plans/procedures/drivers/reports effort
Determine regression testing level required
Step 4: Integrate the effort from steps 1-3
Generates a cumulative rework multiplier factor

Table 3 - Rework Table

Generate Effective Size - knowing these effort drivers, one can generate a cumulative rework multiplier factor and apply it to the known source code size and yield an effective effort size.  This effective size can then be translated to cost by rule of thumb conversion factors or with greater fidelity via software cost models.

3.3.2.1.2.1       X.3 Maintenance Effort / Cost

Use a cost model or dollars per line of total or effective code to estimate cost depending on the load provided by the maintenance contractor – depending on whether you want to estimate the effort to maintain the interface to the OS code (and upgrades) or the entire code base (if you have modified the OS code).

3.3.2.1.2.1       X.4. Estimate Licensing Costs

Open source software licensing costs may be free but are not necessarily free.  Depending on the type of license there may be constraints on the usage of the products.  Estimating license costs should include vendor quotes for open source software with vendor support.  Open source open use items licensing costs, if any, should be available on the internet.

3.3.2.1.2.2       Open Source Black Box Use

Open source, black box use is open source software where it is accessed and used as if it was proprietary software, that is organizations can use it, have visibility into the software’s functionality, can configure it, but cannot or do not change it. This is estimated the same as any other off the shelf software with the primary difference to open source open use being that SLOC sizing is commonly not available, thus requiring a different methodology to yield size.  This difference is noted in green in Figure 2

Figure 2 - How to Estimate:  Open Source Black Box Use

3.3.2.1.2.2       X.1      Black Box Use Systems Engineering: Estimate Systems Selection / Approval Cost

Same as Open Source Open Use (3.3.2.1.2.1   X.1).

3.3.2.1.2.2       X.2      Black Box Use Development - Perform Sizing Without Source Sizing

There are several functional ways to size open source software without access to the source code.

1) Search the open source metrics databases and determine if there is a reported size.  If so it should be close enough to use the same sizing as Open Source Open Use.

2) If analogy is not available estimate the size by counting the number of functions that must be learned and used, and estimate effort the same as Open Source Systems Engineering effort.

3) As discussed earlier (paragraph 3.3.2.1.2.1  X.1), count the number of IFPUG functional sizing function points of the portions of the OSS software that will be learned and used using the IFPUG standard and compute effective size by computing the rework factor and multiplying by function points.  Table 4, below, provides common cost drivers items that must be considered in the product under consideration, along with brief descriptions of the items and common function point conversion factors.

Table 4 - Function Point Consideration Items

3.3.2.1.2.2       X.3      Maintenance – same as Open Source Open Use (3.3.2.1.2.1  X.3).

3.3.2.1.2.2       X.4.     Estimate Licensing Costs - same as Open Source Open Use (3.3.2.1.2.1  X.4).

3.3.2.1.2.3       Open Source Software Black Box Use From a Vendor (Non Developmental Item - NDI)

Use the same approach as Open Source Open Use.  In this case however, the source code will not be available so use the functional methods described above in paragraph 3.3.2.1.2.2  X.2 and table 4.

3.3.2.1.2.4       Open Source Software Open Use Developmental

If you start with OSS code and modify it, you typically then have full responsibility for all aspects of that code.  Estimate as for standard development code plus OSS familiarization (see table 3), noting that the OSS provider may still charge licensing costs.

Galorath's SEER-SEM provides all the functionality to estimate both the labor costs of the open source software and the costs of understanding / reverse engineering it.