Instrument tolerances: manufacturer vs. process

Janice Bennett-Livingston
Industrial Measurements
Life Science

This week's blog comes from Vaisala's Calibration and Repair Service department:

Using the instrument maker’s tolerance, there is often a higher risk of an “Out of Tolerance” appearing on a calibration certificate; this costs money. Consider the following scenario: You have just received back from calibration the set of temperature sensors used to monitor the warehouse. Most of the instruments have been shown to be within manufacturer published tolerances. A few of the instruments, though, are listed as out of tolerance. Now the fun starts! 


The out-of-tolerance results need to be fully documented, including a full investigation into the use history of the instruments since their last calibration. All of this is rightfully required by your Quality Management system to ensure that no product has been negatively affected by the larger than expected errors in the readings. The investigation shows that the instrument manufacturer’s tolerances for the instrument are tighter than the tolerance required to monitor the warehouse. All of the out-of- tolerance points from the calibration certificates show that the readings of the instruments are within the tolerances for the warehouse monitoring system; therefore, no products have been negatively affected. 

This is a common situation that many of us find ourselves dealing with, a difference between manufacturer tolerances and process tolerances. In many cases, we have selected our process monitoring instruments based on more than just the measurement tolerances of an individual instrument.

We may have chosen an instrument with higher accuracy for a number of reasons, including: 

  • Compatibility with existing monitoring systems
  • Lower risk of an out-of-tolerance reading affecting products
  • Better overall value
  • The manufacturer was on the Approved Vendor

Whatever the reason, the manufacturer-specified tolerances are often significantly tighter than our process requires. This leaves us in a situation where we have a higher risk of an “Out of Tolerance” appearing on a calibration certificate than if the instrument being used had a tolerance matching our process tolerance. This in practice is costing extra money either due to an increase in calibration costs, a shortening of calibration intervals, or an increase in investigations of out-of-tolerance conditions.

There is a potential solution which will help contain costs and will also not increase the risk of negatively affecting product quality. When sending instruments out for calibration, have the calibration laboratory use the process tolerance when evaluating for in- or out-of-tolerance. The process tolerance is typically the more relevant limit rather than the manufacturer specification.

The process tolerance was established not by looking at what instruments are capable of performing, but by looking at the requirements of the process. It is this limit that tells us when the product may be affected and is, therefore, a more relevant tolerance to ensure that the measurement instrument is meeting.

Most calibration laboratories default to using manufacturer specifications when performing this evaluation. The calibration laboratory does not know what the process limits are for an instrument unless we tell them; the only information that is readily available at the calibration laboratory is the manufacturer specification. When selecting a calibration vendor, ask if they have the capability to use customer specified acceptance limits.

In most cases, this should not pose a significant problem. When sending the instruments in for calibration, including the instructions for the values to use for the process limits. Using process limits instead of manufacturer specifications provides a low-risk and low-cost mechanism for reducing the cost and effort associated with out-of-tolerance events. It provides a more relevant analysis of the calibration results and will limit the need to reduce calibration intervals in the case of equipment not meeting manufacturer specifications.


Learn more about Vaisala calibration services

Find your local representative



Janice Bennett-Livingston

Marketing Manager

In addition to editing the Vaisala Life Science blog, Janice Bennett-Livingston is the Global Life Science Marketing Manager for Vaisala's Industrial Measurements business area.

Pre-Vaisala writing credits include a monthly column called "Research Watch" for Canada's award-winning magazine alive, as well as articles in Canadian Living and other periodicals. Other past work: copywriting for DDB Canada, technical writing at Business Objects, and communications specialist for the British Columbia Child & Family Research Institute.


Supakorn Surakulananta

Jul 25, 2018
To whom it may concern,

I have a Vaisala 2 items, HMT120/130, M170. It's time to calibrate this instrument. I'm not Metrologist and not expert in a calibration. I'm a beginner in this.

Please let me know how to set the maximum permissible error (MPE) of an instrument. I'm not sure it may be called "Tolerance of an instrument".

Janice Bennett-Livingston

Jul 25, 2018
Hi Supakorn,

Thank you for reading our Blog! I have forwarded your question to our Application Engineer and here is the reply:

Justin writes:

The HMT120 and HMT130 have a +/-1.5 %RH accuracy spec with a stability of +/-2 %RH over two years. Drift in the readings that exceed these levels would be considered unacceptable, or ‘out of tolerance’. This level can be verified with a calibrated reference such as your HM70(with MI70 Indicator), keeping in mind the accuracy of that unit as well. Calibration procedures are published in the User Guides for each instrument, which can be downloaded for free on our website. Please search by product name: I.E. HM70, MI70, HMT120/130.The frequency of calibration is user dependent, and does not have a strict interval. For example, the HMT120 and HMT130 have a +/-1.5 %RH accuracy with a maximum drift of +/-2 %RH for two years.

If your process requires an instrument that has an accuracy of +/-5 %RH, then you would be able to confidently employ a 2-3 year calibration interval. Should you choose to maintain the maximum potential accuracy of your instrument, an annual calibration interval is recommended.

Please keep in mind that the probes for the HMT120 and HMT130 can be interchanged, and so having a spare probe for each instrument would allow the unit to remain in operation while your other probe is being calibrated/adjusted.

You can find some information here

Thank you for contacting us!

Best regards,

Justin Walsh | Application Sales Engineer

Add new comment