Dynamic Calibration and
Validation Testing
The world leader in serving science
Why is Dynamic Calibration Required
Factory calibration calibrates each detector so it is free of interelement correlations and determines the factors in the A3 model
that enhances the performance of Thermo Gamma-Metrics coal
analyzers.
But the factory can not exactly match the field conditions;
therefore fine tuning of the calibration is required in the field.
Sources used in the field are different from what was used in the factory
Coal stream is continuous through the unit in the field while it is a finite
length (equal to the calibration standard length) in the factory.
The range of ash and sulfur are narrower in the field therefore the analyzer
can be tuned to be more accurate over the narrower range.
The analyzer must be calibrated to the reference sample system.
Precision and Accuracy and Catch 22
Neither
Precise
nor
Accurate
Precise
but not
Accurat
e
Bias Adjustment
Needed
Precise
and
Accurate
The Catch 22 is we dont know what the location of the bulls eye is. We are all
measuring it with a number of systems each with their variances. Typically an
analyzer customer thinks his sample system and lab have no error and the
analysis results from the sample system/lab is the bulls eye for an analyzer.
The objective
During the field calibration and validation process, the objective
is to calibrate the analyzer to match the results of the
customers reference system, usually his sample system and
lab.
The analyzer measures the quality of all the coal that passes
through it.
The lab analyzes a very small subset of the material on the belt
to determine the quality of a large lot (usually one hour) of coal
flow.
It is critical that the sample analyzed in the lab is representative
of the coal on the belt seen by the analyzer.
It is critical that the lab prepare and analyze samples with precise
adherence to standard procedures,. with well maintained
equipment and well trained people.
Sample collection
Collection of each gross sample is controlled by ASTM
Standards D2234 (Collection of a Gross Sample of Coal) and
D2013 (Preparing Coal Samples for Analysis) in the USA.
D2234 covers collection of a gross sample without any crushing.
D2013 covers the reduction and division of the gross sample
collected according to D2234 with crushing.
To calibrate an analyzer generally requires 30 gross samples
covering a range of coal quality.
At least 20 gross samples are required to validate the analyzer
calibration.
Minute-by-minute sulfur and ash data for a unit train
loading.
LoadingfromHighSulfurPile
DozerPushingfrom
HighAshPile
D2234defineshowmanyincrementsare
requiredtoaccuratelyrepresentthequalityof
thistrain.
The highlights of ASTM D 2234 collection of a gross
sample.
Number of increment samples required to make up a single gross
sample?
For coal with consistent quality (clean coal from a single seam) 15 increments
up to a 1000 ton lot size.
For inconsistent coal (raw coal, partial wash coal, any blend coal) 35
increments up to a 1000 ton lot size.
For lot sizes greater than 1000 tons, the number of increments required is;
N 2 N1
total lot size (tons )
1000 tons
N1=minimumincrementsshownabove
N2=numberofincrementsrequiredforlotsize
7
Example of number of increments required
Loading a unit train
Blended Coal dozed from different stacking tube piles; N 1 = 35
The unit train capacity is 10,000 tons
Therefore
10,000
N 2 35
110 .68 111
1000
111 individual increments are needed to represent the coal on the belt.
Impact of number of increments on the repeatability and how
well the sample represents the total lot.
Asthesegraphsshow,itiscriticalthatenoughincrementsaretakentoaccurately
[Link],therandom
errorinthesamplecollectionprocesswillnegativelyimpactthecalibrationofthe
analyzer.
9
The highlights of ASTM D 2234 collection of a gross
sample, continued.
When a sample increment is taken the sampling device must have an
opening at least 2-1/2 to 3 times the top size of the coal, but never
smaller than 1.25-inches. This applies to stop belt sampling, falling
stream cutters or sweep samplers. Falling material must strike the
sample device at right angles to the opening.
In addition the cutter speed is specified (18inches/second) and the
minimum amount of sample that has to be taken, based on the material
top size is also specified.
Top size
5/8 inch
16-mm
2 inch
50-mm
6 inch
150-mm
2-lbs
1-kg
6-lbs
3- kg
15-lbs
7-kg
Minimum increment weight
10
The highlights of ASTM D 2234 collection of a gross
sample, continued.
If a very large sample increment is collected in the primary
sample, it is often required that the primary sample be reduced in
size with secondary sampling before a crushing step is
undertaken.
At least six secondary increments are required for each primary
increment.
All the rules concerning minimum increment weight, cutter speed
and cutter opening apply to this secondary sampler.
Once the sample has passed through a crusher, ASTM D 2013
defines the sampling procedures.
11
ASTM D 2013 Preparing coal samples for analysis
D 2013 covers both the manual and the mechanical reduction
and devision of samples collected according to D 2234.
Applies to sample reduction after crushing in the sample system
Applies to the lab manual reduction of the sample collected by a
sample system
At least 60 increments are required from the sample taken by D
2234. Good practice recommends that at least one tertiary cut
be taken for every primary cut if this is a larger number than 60.
The minimum sample size required is determined by how fine the
coal is crushed and whether the coal is 100% clean coal from
one seam or is it a variable coal quality (raw coal or blends).
12
ASTM D 2013 Preparing coal samples for analysis,
continued
PreparationofLaboratorySamples
Crush to 95% passing
through sieve
Divide to a minimum weight of, Grams
Group A
Group B
No.4 (4.75 mm)
2000
4000
No. 8 (2.36 mm)
500
1000
No. 20 (850 m)
250
500
No. 60 (250 m)
50
50
Tableshowstheminimumweightofthefinalsampledepending
onthematerialsizefollowingcrushing.
GroupA=coalsthathavebeencleanedinallsizes
GroupB=Allothercoalsincludingunknown
13
ASTM D 2013 Preparing coal samples for analysis,
continued
Good practice in a sample system
Cutter opening after crushing 1- inch minimum opening
Maximum cutter speed 18 inches/sec.
Final sample weight collected by the sample system about 40
pounds.
ASTM D 2013 manual sample reduction procedures must be
rigorously followed in the lab sample preparation process to
insure the sample is not biased or contaminated in the process.
Errors made in sample collection or preparation will invalidate a
sample.
14
Dynamic Calibration of the Analyzer
Dynamic calibration of the analyzer involves calibrating the analyzer so
it matches the reference system (the sample system and the lab).
The Moisture, Sulfur, Ash Oxides and the Ash are calibrated by plotting
the analyzer results against the reference system.
15
Dynamic Calibration of the Analyzer
As the X-Y plot of the sulfur calibration data showed, PGNAA
responses are linear; therefore, the field calibration of the
analyzer will be the equation of the linear regression fitting the
analyzer raw results to the reference system results.
Before the linear regression is performed, the data should be
screened for outliers (data that does not match the trend seen
with the rest of the data).
Outliers need to be addressed before the analyzer is calibrated.
16
Handling Outliers in the Data
Errors can happen in the collection of samples, problems in the lab, improperly
recording values, etc. therefore it is not unexpected to see outliers in a data set.
If an outlier is seen, an effort needs to be made to determine is there an
attributable reason. In the data below the sample weight for all samples except
the outlier were 40-pounds 4. The sample weight of the outlier was 8-pounds.
Something must have happened to the sample system. Therefore this point was
discarded and not used in the calibration.
Outlierdatapointdiscarded
fromthedatasetbecauseit
wasdeterminedthecollected
sampleweightwastoosmall
toberepresentativeofthe
tonssampled.
17
Dynamic calibration
Thedynamiccalibrationiscompletedbyapplyingthegain(inthiscase0.8108)and
offset(+0.0642)[Link]
moisturetoo,completingthedynamiccalibrationoftheanalyzer.
18
Validation Test
Once the analyzer has been dynamically calibrated the validation test
can be performed.
The validation test ideally consists of the collection of 30 samples,
collected and analyzed in exactly the same manner as the dynamic
calibration samples.
As the normal guarantees are only for ash, sulfur and often moisture, the
ash oxides only need to be run if the customer is specifically interested in
them.
The Root Mean Square Deviation (RMSD) of the analyzer to the lab
data is calculated.
n
RMSD
analyzer
lab
i
i
i 1
n=numberofdatasets
19
RMSD example
Analyzer Ash
Lab Ash
Difference
Difference2
8.25
8.40
-0.15
0.0225
7.88
8.20
-0.32
0.1024
9.01
8.73
0.28
0.0784
8.65
8.82
-0.17
0.0289
7.50
7.22
0.28
0.0784
8.02
7.71
0.31
0.0961
Sum of differences
squared =
Number of samples =
RMSD =
20
0.4067
0.26
Acceptance of Analyzer Performance
The analytical precision of the analyzer is guaranteed by Thermo.
RMSD measures the combined error of the analyzer and the sample
system/lab reference system.
If the assumption is made that the analyzer and the sample system/lab
have approximately the same precision (an assumption that has been
proven to be often true). Then the precision of the analyzer can be
estimated as:
RMSD
2
21
Acceptance continued
If the measured RMSD of the analyzer is equal to or less than
the precision guarantee, the analyzer has passed the
acceptance test.
If the RMSD is larger than the guarantee, the analyzer precision
may still satisfy the guarantee.
If the lab and analyzer precision assumed to be equal, an RMSD of
0.707 would indicate an analyzer precision of 0.50.
If the precision of the analyzer is in question, then ASTM D 6543
Evaluation of Measurements Made by On-Line Coal Analyzers,
details the procedure to conduct a Grubbs Estimator
measurement of the precision of the analyzer.
22
Grubbs Estimator
Requires dual ASTM sampling of the coal stream at two different
locations.
Requires the collection of 60 samples from each sample location
Requires the samples to be processed completely by independent labs.
The result is an estimate of the precision of the three analysis systems,
the coal analyzer, sample collection/lab one and sample collection/lab
two.
The Grubbs estimate of the precision of the analyzer is what is
compared to the guarantee performance.
Thermos guarantee is a one sigma guarantee (68% of the results will
fall within plus or minus the guarantee value), and sukbject to statistical
significance testing.
23
Precision of Sample Collection
How well does the sample collected represent the true quality of
the lot?
What is the precision of the sample collected by the sample
system?
For the example in slide 9 the measured ash had a precision of 0.60 and the sulfur
precision was 0.036 at the ASTM recommended increment.
This number will be less for coal streams with less variability and more if the variability is
greater.
The sample collected by the sample system goes back to the lab
and is air dried, crushed and manually reduced to 500 grams that
goes into the lab.
The lab then runs very small subsets of the prepared sample in a
number of instruments.
24
Reproducibility Lab to Lab
Repeatability is the ability of a single lab to duplicate the same
results when repeating the same analysis of the same sample.
Reproducibility is how well the results from two different labs
should compare for the same sample.
As the coal analyzer is one lab we should look at the
reproducibility of results on analysis system to another 95% of
the time (2 sigma).
D 3173 Moisture Reproducibility - 0.3% for coals with less than 5%
moisture and 0.5% for coals with more than 5% moisture.
D 3174 Ash Reproducibility 0.32% on a dry basis
D 4239 Sulfur Reproducibility 0.02 + 0.09(average sulfur between
labs)
25
