What is the procedure 1?
In the procedure 1, also called type 1 study, an examiner measures the same calibrated reference part several times under the same conditions. Repeat measurements result in repeat scattering as sample standard deviation s and the systematic deviation as difference between mean and reference value.
Both are related to the tolerance of the characteristic, which is to be tested later with this measuring medium. Thus, a measuring medium is not capable of itself, but for a certain tolerance. Operator, part and interaction influences are not examined by the procedure 1; for this there are further studies.
Formulas for Cg and Cgk
The app expects a tolerance percentage of 20 percent: Cg sets 20 percent of the characteristic tolerance T in relation to the six-fold repeat scatter. Cgk deducts the amount of the systematic deviation from 10 percent of the tolerance and divides through the triple repeat scatter. If the mean is exactly on the reference value, Cg and Cgk are the same.
Company and customer guidelines sometimes use different tolerances, factors or minimum numbers of measurements. The customer's specification or its own test guidelines are decisive; the app does not provide a limit value.
Cg = 0.2 × T ÷ (6 × s) Cgk = (0.1 × T − |x̄ − x_m|) ÷ (3 × s)
Sample calculation with the default settings of the app
Five repeat measurements give 9.999; 10.001; 10.000; 9.999 and 10.001 mm on a reference part with 10.000 mm. The mean is 10.000 mm, the systematic deviation is therefore zero. The sample standard deviation is 0.001 mm.
With a characteristic tolerance of 0.1 mm, Cg = 0.2 × 0.1 ÷ (6 × 0.001) = 3.333 and Cgk = (0.01 − 0) ÷ (3 × 0.001) = 3.333. The resolution of 0.001 mm corresponds to 1 percent of the tolerance. Five values are sufficient here only to show the calculation path; for a proof of ability, guidelines require significantly more repetitions.
Delimit resolution, measurement uncertainty and process capability
The app also shows what proportion of the tolerance is the smallest display step. A measuring device whose resolution proves a large part of the tolerance cannot make the scattering of the process visible, even if Cg looks computationally good.
The measurement uncertainty, on the other hand, describes just how exactly a single measurement result is, and summarizes several contributions in a budget. The process capability Cp and Cpk evaluates the scattering of the manufactured parts, not that of the measuring medium. All three characteristic values answer different questions and do not replace each other.
Sources and classification
- NIST/SEMATECH e-Handbook: Gauge R & R studies
Repeatability, comparability and biases of measuring devices in the production environment. The formulas for Cg and Cgk with a 20 percent tolerance share are those of the app calculator.
- CNC Calculator App: Measuring medium capability type 1
Formulas, presets and instructions of the computer from which the calculation example originates.
Questions and answers
What is the difference between Cg and Cgk?
Cg evaluates only the repeat scattering of the measuring medium in relation to the tolerance. Cgk also takes into account how far the mean of the measurements differs from the calibrated reference value. Without systematic deviation both are the same.
Which Cg-value is sufficient?
This determines the customer's specification or the own test guideline, together with tolerance percentage and number of measurements. The app calculates Cg and Cgk, but does not set a limit value.
Why is 20 percent of tolerance expected?
This is the convention used by the app's computer for the procedure 1: The measuring device should consume only a small part of the tolerance. Other guidelines use different proportions; then the results are not directly comparable.
Does the procedure 1 replace a complete measurement system analysis?
No. It looks at a reference part and an operator. Influences of different examiners, parts and their interactions are only examined by a further study.


