How to Perform a Benchmark Analysis for a System Design

Benchmarking & Gap Analysis in the Concept Phase

DEFINITION

Benchmarking is the process of comparing the current project, methods, or processes with the best practices and using this information to drive improvement. The Gap Analysis naturally flows from the Benchmarking Analysis. Once we understand what is expected of the product in the industry, we must then compare that with current capabilities, and this becomes the Gap Analysis.

SITUATION

If your customers don't give you specific requirements for reliability and quality, and if the specifications for your industry don't either, you must determine these requirements for yourself. Benchmarking is the best method for doing this. Once you have determined your requirements, you need to determine how achievable these requirements are by performing a Gap Analysis. The larger the Gap, the more effort will be required to meet your requirements.

OBJECTIVES

The objective of Benchmarking is to set appropriate reliability and quality metrics for your product based on similar products in your industry.

The objective of a Gap Analysis is to measure the gap between where you currently are and where you want to be (or where you have set your reliability goals at, often based on Benchmarking results).

VALUE TO YOUR ORGANIZATION

Benchmarking against other companies can give you valuable information because this helps define your reliability and quality metrics.

The gap analysis becomes the basis for identifying implementation actions and priorities. Knowing the size of the gap for each particular reliability and quality metric is valuable because it will then tell you how much resources you will need to dedicate to meeting these metrics.

RELIABILITY INTEGRATION

An example of Reliability Integration during Benchmarking is as follows:

Using Benchmarking and the Gap Analysis to Help Write the Reliability Program and Integration Plan

The Reliability Program and Integration Plan must describe the reliability goals of the project and what reliability tools are required to achieve these goals. The Benchmarking results and the Gap Analysis results are two of the best sources of information in establishing this. A good benchmarking exercise can reveal not only reliability results of other products in a similar industry, but the tools used to achieve these results. And the gap analysis will show how far you are away from achieving your goals.

METHODOLOGY

In Performance Benchmarking, we focus on elements of technical quality and reliability as well as features.

In Process Benchmarking, we focus on discrete work processes such as in-house manufacturing vs. contract manufacturer model.

For both Performance and Process Benchmarking, the first step is to develop a set of reliability and quality metrics for your product. Mean Time Between Failure(MTBF), Availability, Annualized Failure Rate (AFR) and Dead on Arrival (DOA) rate are very common.

Next, we must decide what levels to set for each metric. For example we may set our Reliability Metrics at 5% AFR and 1% DOA. The best method for determining these levels is to benchmark the metrics against other companies.

When using Benchmarking to help set our goals, we must decide with whom to benchmark. This may not always be your direct competitors – it may be non-competitors in the same industry or related industry. And the benchmarking data from these companies may come from a variety of sources – web sites, web searches, technical papers, trade magazines, and the like.

Next, we gather the information, analyze and summarize for each metric and set our goals appropriately.

Finally, we measure the gap for each of these metrics by comparing the goals to our current performance level. This can then be fed directly into the Reliability Program and Integration Plan so that we can specify reliability tools and tasks that will be most effective at reducing/eliminating the gap.

CASE STUDIES/OPTIONS

The following case studies and options provide example approaches. We shall tailor our approach to meet your specific situation.

1. Performance Benchmarking

   A Computer Peripheral company was competing in reliability and marketing asked us to help with setting up Performance Benchmarking goals, focusing on elements of technical quality and reliability as well as features. For that, we developed a list of companies in their industry as well as a list of metrics to benchmark against. Most of the information came from web searches but some of it came from trade magazines and research reports.

2. Benchmarking used to Define the Reliability Program Plan

   A Medical company was having issues with reliability and needed a level of improvement for their next generation product. For this situation, we performed Performance and Process Benchmarking. Then we performed an Assessment of Capabilities which led to a Gap Analysis. This then fed in as goals into their Reliability Program Plan. The plan then defined what tools and processes needed to change in order to meet these goals.

How to Perform a Benchmark Analysis for a System Design

Source: https://opsalacarte.com/consulting/concept/benchmarking-gap-analysis/

Share this post