How We Use ICP-OES Results Of Unknown Accuracy And Precision

ABOUT THIS ARTICLE

This article was authored by @Dan_P, @taricha and @Rick Mathew. We collaborated in the development of the experimental methods, the cost of the project and the analysis of the data. The project was started in August of 2021 with the purpose of understanding the quality of the data received from ICP-OES measurements. Our intent is to bring some clarity to the reefing community in order to help evaluate, interpret and respond to this data. What follows are the outcomes of these efforts.

INTRODUCTION & BACKGROUND

If you have been a visitor to the R2R Forums for any length of time, you have most likely run into a post about ICP (Inductively Coupled Plasma). If you do a search on the Forum for “ICP” you will find 20 some pages of posts. Here is a link https://www.reef2reef.com/search/3220407/?q=ICP&o=relevance. When you scanned through these you noticed a number of them had a recurring theme. “I have this report with all of these numbers…Now what?” Are these values real? Can this be right? What should I do?

These Reefers are reacting to the results of their ICP test that is telling them that one or more of their “parameters” are off. They then need to make a decision as to what to do. This very fact is what places a high priority on the quality of the measurement, both the precision and the accuracy. Measurement precision and accuracy statements are hard to come by from the ICP vendors. A statement of measurement uncertainty by element would be very helpful to the Reefing Community. As a matter of fact it has been said that a measurement without knowledge of its uncertainty is of little value (Walter Lewin). Best practices for reporting measurement calls for providing this information .(4) Vendors that do provide such information do so based on samples that are preserved and stored under laboratory conditions. These are not hobby water samples shipped across the globe and stored before testing. These are the samples we used in our experiment to better describe real world performance of ICP-OES measurements. We have seen Limit of detection (LoD) data from a number of vendors; however this does not address the questions of precision and accuracy. This was the motivation for our project. @Dan_P , @taricha and I set out on a quest to take a closer look at these primary questions:

1)
Are these measurements real?

2)
What is the Repeatability (precision) and does it vary by element?

3)
What is the Variability between vendors and does it vary by element

4)
What can we say about accuracy?

Very little published information exists that can directly answer these questions or help us evaluate vendors. For example, in 2016 Ross and Maupin (https://reefs.com/magazine/skeptical-reefkeeping-12/ ) sent certified seawater samples in triplicate to Triton to evaluate the ICP service. Our analysis of the data gives a reason to wonder about the quality of data we receive. The analysis (SEE CHART 1) compares the average result for reported element concentrations to the certified standard amounts. A relative accuracy of 100% indicates perfect agreement. A higher or lower amount indicates less than accurate results. The second analysis is the relative standard deviation (RSD) of the results which is the standard deviation divided by the average result. For this statistic, the smaller the number the more precise or replicable were the results. The elements are arranged according to increasing concentrations. Although the RSDs are small, indicating a high repeatability across the elements for samples measured back to back to back, the relative accuracy suggests unreliable measurement below 10ppb Ba. This represents half the elements tested. Because this data only pertains to a single vendor we cannot apply it as a general tool, but only a reference point as to what we might expect from the ICP-OES measurement protocol.

1651926570931.png

CHART 1
While the Ross and Maupin data raises questions about the reliability of data for low concentration elements, Fritz Aquatics released a multi-vendor, multi-time study (https://fritzaquatics.com/assets/files/uploads/ICP_TES.pdf) that indicates variability in even the highest concentration elements. Fritz prepared a single master sample of their salt mix and sent it to multiple vendors twice, about a month apart.

Chart 1A and 1B below show this variation in the 6 highest concentration elements: Ca, Cl, K, Mg, Na, and S. Chart 1A (left) shows that the choice of vendor resulted in a roughly ±5% difference of higher or lower element measurements (relative to the average of all vendor measurements) and this difference was consistent across the 6 elements. Chart 1B (right) shows that the vendors also disagreed about the relative increase or decrease in the element concentrations of this single sample during the storage time with one vendor reporting a consistent 5% increase, and another a >10% decrease - again, largely consistent across the 6 elements.

1651926686569.png
1651926701654.png

-------------------------------------------CHART 1A------------------------------------------------------------CHART 1B

If vendor selection and when they analyze the sample introduces variations like those indicated above to the measurement of a single sample, then it’s worthwhile to try to further understand the variations that appear in the measurement process and what we can actually conclude from our ICP test results.

A series of experiments were conducted both individually and collaboratively to seek to answer these questions and evaluate vendors. Although this was not an exhaustive experimental effort and there are admittedly some gaps in the study, we feel that we have gained valuable insight into the quality of data received from ICP vendors which we would like to share. We also would like to share an approach for dealing with ICP data that is not provided with accuracy or precision information.

EXPERIMENTAL SETUP AND METHOD

Sampling:

A single large Master sample of either aquarium water or Instant Ocean was taken in each case. Some of the samples were spiked with trace elements, a list of which is below. In the case of the dilution study the samples were diluted with RODI water by a specified percent to represent different levels of salinity (15%, 30% and 40% dilutions). The large samples were then divided into the separate samples required by the vendor. Each vendor got samples from the same exact large sample. In some cases the vendor received multiple samples from the same sample. See Illustration below.

1651926787950.png



Some experiments had 4 sample sets, some had 3, some had 2 and some had 1. They were sent to multiple vendors up to a maximum of 4 individual vendors. In all but one experiment all of the samples were shipped to the vendors at the same time. In total there were 14 individual Master Samples taken and separated into 34 individual sample sets. Additionally some of the data sets were taken from work that was done by others. (3)

Analysis Procedure:


1651926898507.png


*Some samples were held back in cold storage and shipped later to evaluate variability over time.

THE FINDINGS

In order to not get bogged down in a massive amount of data, we are going to present the Findings first. The charts and graphs included here are a result of our findings. If you are interested in the gory details we would be glad to provide them to you.


We provide two evaluation procedures that we believe will be useful to judge whether the ICP data you receive is good enough to meet your needs. The first is a vendor evaluation based on our analysis of our entire database of ICP results and does not require knowledge of statistics. The second method is a standard procedure for evaluating analytical methods. It can be used to evaluate vendors based on our database, or if you are comfortable with numbers, can be used to evaluate your data when you collect replicate measurements.

Vendor Evaluation Tool: Is It Real? (Tool # 1)

The first tool is a close look at the reality of a reported measurement. The focus here is on the trace elements not the majors. Being as we did not have Certified Reference Standards, it is difficult to establish accuracy. However, demonstrating failure is easier. If identical samples of water sent to the same vendor generate results of Zero and 1.5ppb for an element, then you can conclude they are not both accurate reflections of your water and thus a reported measurement at the 1ppb level isn't necessarily real. You can cover a lot of ground evaluating vendor performance by identifying failure to measure similar samples similarly. For example, ICP-Analysis (ICP-A) reported Rb at 800ppb and zero from the same water, so that's failure at 1ppb, 10ppb, 100ppb, and maybe 1000ppb


Using this thinking along with information gleaned from our dilution studies, the vendor evaluation chart below was created.

1651926993975.png

CHART 2
1651927093541.png

CHART 3

1651927154173.png


CHART 2 & 3 KEY

ICP-A = ICP Analysis FM= Fauna Marin

To use the chart, select the vendor, find the element and the ppb range it is being reported in and note the color code. Read the description based on the color code. Where the color is black, there was insufficient information to evaluate the vendor and element. If the color is red then there is most likely no need to go any further because the measurement is a value that is not consistently distinguished from zero. By going through the trace elements in your ICP report using this chart you can decide whether the measurement of an unwanted (or desired) trace element is actually large enough to be confident of its level in your water.

Another data set, when paired with the above charts, helps illustrate the amount and frequency of trace element measurements reported by vendors that are not repeatably distinguished from zero in hobby samples. In the below data, 4 replicates were sent to each vendor. As you can see in charts 3A-D, all vendors reported some non-zero trace elements that were not distinguished from zero across all 4 replicates. It is also apparent that some vendor reports include more of these possible-zero elements and with higher variation than other vendors do.

1651927292764.png

----------------------------------------------CHART 3A---------------------------------------------------------------------------CHART 3B


1651927365398.png

------------------------------------------------CHART 3C------------------------------------------------------------------------- CHART 3D

Vendor Evaluation- Gage Variation (Precision) (Tool # 2)

There are two components that are important in any measurement system (gage reading): There is the accuracy, which describes how close the measurement is to the true value, and the precision, which describes how close the measurement is to a previous measurement of the same sample. This is often referred to as the repeatability.

If we assume an instrument produces accurate results but is highly variable (low reproducibility) then the probability that a single measurement is the true value is low. This would mean that the confidence in any single measured value would be low.

The requirement for the level of precision is defined by its purpose of use. For example, measuring the length of a race course that is 10 miles long, a precision of .25 miles could be fine, but if the course is only 1 mile this level of precision would be inadequate.


Bringing this into our reef keeping world, some elements have a precision in parts per million (PPM) while others will be parts per billion (PPB) which is 1000 times smaller. The general practice in the hobby is not to control the level of an element to a specific value, but a range. An example would be calcium 405ppm to 510ppm or a control range of 105ppm. If our measurement gage had a gage variation of 100ppm this would be of little value to us because the precision is almost as large as our control range.

FINDING THE GAGE VARIATION OF A MEASUREMENT SYSTEM

One of the ways to find this variation is to use a method called Gage R&R. It is a scientific study that falls under Measurement System Analysis (MSA). It differs from Relative Standard Deviation (RSD) in that it compares the variability to the element’s required control range. We will not go into any details of this method. In its simplest form it is measuring the same sample multiple times with the same measurement system and doing statistical analysis on the data and producing a value that we will call the Gage Variation. Using this number and knowing individual control range of the elements we can evaluate the relative performance of different measurement systems. In our case the measurement system is different vendors.

EVALUATING THE RESULTS

It is generally accepted practice to use the following scale to evaluate the performance of the measurement system:

High Precision Measurement

The Gage Variation is less than 10% of the control range

Moderate Precision Measurement

The Gage Variation is between 10%-30% of the control range

Low Precision Measurement

The Gage Variation is greater than 30% of the control range.

Using this scale with our example of a calcium control range of 105,


High Precision < 10% of 105 = 10.5 ppm

Moderate Precision < 30% of 105 = 31.5 ppm

Low Precision Measurement > 30% of 105 = > 31.5 ppm

These would represent our Target Precision Values for Calcium using a control range of 105ppm. If the control range were to be expanded or reduced then the target values would increase or decrease accordingly.

To check your understanding of this approach, look at the bars for calcium in CHART 5 below. The bars with the black stripe represent the high and moderate precision level for our 105 ppm control range. The other bars represent the vendor precision in our experiments. You can tell which ones deliver high precision measurements, and which ones deliver moderate precision results. The moderate precision result is not “bad” but it is not useful for a control range of 105. If your control range is larger, this vendor would then be more likely to satisfy your measurement needs.

By using the experimental methods we described above we sent multiple sets of the same sample to multiple vendors. We collected the data and generated the Gage Variation for the elements by vendor. Using generally recommended control ranges for the individual elements and the Gage Variation we generated graphic comparisons like the ones below to help us discern the confidence in the values reported for each element by vendor. Chart 4 represents the control limits used in this experiment. These were taken from vendors report sheets. Although there could be much debate about the “exact” figures the basic principle holds true. The performance of a measurement system can be evaluated by its Gage variation relative to the required control range.

1651927533108.png

CHART 4

1651927640652.png

CHART 5
1651927690630.png

CHART 6
1651927754312.png

CHART 7
1651927819883.png

CHART 8
1651927899228.png

CHART 9
1651927944966.png

CHART 10
1651928026642.png

CHART 11

THE PROCEDURE

The Vendor Evaluation Procedure”

Below is the procedure we developed to evaluate ICP results before we conclude there is an issue or finding. Tool #1 is our quick evaluation of vendor capability to measure an element’s concentration based on our experimental results. Use this tool to pare down the list of element measurements for further investigation and to eliminate false emergencies. The next tool is the Gage variation. If you don’t have multiple measurements of the same sample, use the results we generated from our pool of ICP data to evaluate whether a vendor has in the past produced measurements with enough precision to satisfy your need.

1651928114837.png

FIG 1

EXAMPLES

EXAMPLE 1---Major Element Data (PPM) (Na, Mg, S, Ca, K)

The Situation

The ICP report indicates that one or more of these elements are out of range by 30% or more. Your home test indicates it is not.

The Data

Vendor Evaluation Is it Real?


Because these are major elements Tool # 1 does not apply.

Vendor Evaluation Gage precision:

Locate your element and vendor. Note the precision value in relationship to your desired precision (high or moderate).

The Assessment

Focus your attention on the precision value of the vendor for that element. This will tell you if the measurement value is reliable, i.e. would the vendor be able to reproduce this value if they re-measured the sample? If the precision value is equal to or less than the target value selected (high or moderate), then you can have a degree of confidence that the measurement is of sufficient precision for the elements control range. However if the precision value is greater than the target value then the precision would not be sufficient for measurement for the elements control range at the level of precision required (high or moderate). Using CHART 5 and 7 above for:

Calcium (Ca) 3 of the 4 vendors were able to deliver high precision whereas one was able to deliver only moderate precision.

Potassium (K) all of the vendors can deliver better than moderate precision and close to high precision values.

For Magnesium (Mg) none of the vendors delivered high precision

IT IS IMPORTANT TO REMEMBER WE ARE ONLY LOOKING AT PRECISION NOT ACCURACY! It is possible that these measurements are very precise but precisely wrong...Low accuracy high precision (See Fig 2). We have very little data as it relates to the accuracy of different vendors. We can gain some insight on the accuracy from the Ross-Maupin article mentioned above, as it relates to a single vendor using the ICP-OES. However this does not answer the question of individual vendor’s accuracy. That being said if the Gage variability is high meaning low precision, even if the vendor is accurate the chances of any single measurement being the true value is low. This is especially true if the target (represented by the control range) is small.




1651928331235.png

FIG 2
Possible Courses of Action:

For Calcium (Ca)
---Based on the above assessment I would tend to believe the measurement value from 3 of the 4 vendors. I would assume that my measurement was incorrect and work to validate and improve my test procedure (1) before making any adjustments. The measurement from the vendor represents the state of your system several days ago and things will be different so I would not make the adjustment based on the vendor measurement, but dial it in using my validated test method.


For Potassium (K) --- Based on the above assessment I would tend to believe the measurement value from the vendors. I would assume that my measurement was incorrect and work to validate and improve my test procedure (1) before making any adjustments. The measurement from the vendor represents the state of your system several days ago and things will be different so I would not make the adjustment based on the vendor measurement, but dial it in using my validated test method.

For Magnesium (Mg) --- Based on the above assessment I would be skeptical of any of the values reported by the vendors based on a control range of 140 ppm. The gage variability for all of the vendors for this element are all in the moderate range and approaching the top of the range with exception of one vendor. I would assume my test measurement to be equally valid and would make any adjustments according to my testing results. In addition I would validate and improve my test method.

EXAMPLE 2---PHOSPHOROUS (PPB)


The Situation

Your home measurement is 35ppb but the vendor measurement is 13 ppb.

The Data

Vendor Evaluation Is it Real?


With the exception of one vendor the reported value will most likely be a real value if the level is 10ppm or above. If it falls in this range, the next question is how reliable is the measurement value.


Vendor Evaluation Gage precision:

Chart 9 indicates that none of the vendors were able to provide a high precision measurement for phosphorous. This would mean that the reported value even if the vendor can accurately measure phosphorous, has a low probability of being the true value and depending on the vendor could have± 8-88 ppb variability. This would not be adequate for making any system changes based on the above control limits of 6-23 ppb because the variability is too high.


Possible Courses of Action:

Based on the above assessment I would be skeptical of the reported value. Again it is important to remember that the measurement results are on a water sample taken several days earlier. Your system will not be the same so I would make no adjustments based on the reported value. I would instead work to validate my own test procedure (1) and determine its variability and base my actions on this data.


Special Note on Phosphorous

Phosphorous belongs to a group of elements that can be depleted over time because of their bioactivity. This group also included Silica, Iron, and Iodine. This group of “possible-zero” elements is of interest to hobbyists who often wish to manage them towards some non-zero target. It’s also the most challenging group for measurement by ICP for the same reason: these elements are often near zero, and biological processes can have a large appetite for their consumption. This process continues when a sample is stored and or shipped. This would mean the initial sample measurement could be different then the measurement of the same sample several days later.

We have done some extensive experiments that demonstrate this principle. You can read about it here (2)


So these are just two examples of how one might use the results of this set of experiments to help you to interpret your ICP test results. The procedure described in Fig 1 can be used with any of the elements. There was not sufficient data to provide a complete set of Gage Variation Charts but most of the elements we would be concerned with are included and the Vendor Evaluation Charts (2 & 3) as well as Chart 1 can help guide the interpretation.

This is for sure not a comprehensive study. There were only 4 vendors and limited number of samples, but none the less it provides a general picture of ICP-OES capabilities and a way of qualitatively looking at the results of an ICP report to help decide a course of action.

This study produced a great deal more data information that is not included in this document. This includes the raw data used in the study, an explanation of each of the approaches to the analysis. We have attached some of the documents in case you desire to look into the further. If you would like additional information just PM the authors (@Dan_P, @taricha and @Rick Mathew)


SUMMING IT UP

If we take a step back from the details of this work and have a look at some general conclusion what might they be?



1) ICP-OES as currently practiced can provide sufficient levels of precision for some elements to help effectively manage a reef system.

2) There are some elements that ICP-OES as currently practiced cannot provide sufficient levels of precision in order to be confident in a single measurement to allow effective management of a reef system.

3) The variability of the measurement of Magnesium is unexpectedly high for all of the vendors evaluated.

4) The single vendor accuracy assessment conducted by Ross and Maupin indicate the ICP-OES as practiced by this single vendor produce results that for some elements are sufficient for use in managing a reef system and some elements are not.

5) Precision performance varies by element, in a manner largely dependent on element concentration regardless of vendor. Each group: major, trace, or in- between has elements that are better- or worse-measured than other elements in the same group.

6) A significant portion of the measured values of trace elements appear to not be “real” - that is, the reported values are not well distinguished from zero by a vendor doing multiple analyses on the same samples. This significant portion can be the majority of measured trace elements for some ICP vendor reports.


7) There is variability between vendors and it varies by element. No two vendors will generate the same results from the same samples. Comparison of vendor results to an expected standard will show that even the vendors that do the “best” will have misses on some elements that are “worse” than other vendors.


REFERENCES AND LINKS

(1) These 4 articles can help validate and improve your home testing methods

https://www.reef2reef.com/ams/getting-it-right-make-testing-count-by-using-a-quality-system-approach.741/

https://www.reef2reef.com/ams/part-2-getting-it-right-make-testing-count-by-using-a-quality-system-approach-colorimetric-visual-testing.744/

https://www.reef2reef.com/ams/part-3-getting-it-right-colorimetric-visual-testing-methods-what-we-see.747/

https://www.reef2reef.com/ams/part-4-getting-it-right-colorimetric-instrumental-testing-methods-digital.748/

(2) Storage effect on ICP test results link

https://www.reef2reef.com/ams/sample-storage-and-its-impact-on-measurement-results-part-3.800/

(3) Additional Data Sets used from work by others.

Ross & Maupin https://reefs.com/magazine/skeptical-reefkeeping-12/

Ross---Data provided for single vendor multiple samples different shipment dates

Fritz Aquatics https://fritzaquatics.com/assets/files/uploads/ICP_TES.pdf




(4) Best Practices in Reporting Measurements

https://www.sigmamagic.com/blogs/measurement-uncertainty/

https://www.isobudgets.com/how-to-report-uncertainty-in-measurement/

https://www.youtube.com/watch?v=6htJHmPq0Os
 
Last edited by a moderator:

blaxsun

10K Club member
Review score
+0 /0 /-0
View Badges
Joined
Dec 15, 2020
Messages
12,416
Reaction score
12,555
Review score
+0 /0 /-0
Location
The Abyss
Excellent article! A lot of interesting results and conclusions here. I've often wondered if the results for certain elements could be influenced by shipping times and your research and analysis seems to indicate that this is in fact the case. The magnesium results were particularly surprising...
 
OP
Rick Mathew

Rick Mathew

Valuable Member
Review score
+0 /0 /-0
View Badges
Joined
Mar 21, 2016
Messages
1,253
Reaction score
4,240
Review score
+0 /0 /-0
Location
North Central Florida
Excellent article! A lot of interesting results and conclusions here. I've often wondered if the results for certain elements could be influenced by shipping times and your research and analysis seems to indicate that this is in fact the case. The magnesium results were particularly surprising...
We thank you!....Yes the Magnesium results are not what we expected, but then again we were not sure what to expect :thinking-face:...I had seen hints of this in some previous work but this results seems to clarify that there is more variability than would be expected.
 

Laith

Well-Known Member
Review score
+0 /0 /-0
View Badges
Joined
May 16, 2017
Messages
849
Reaction score
1,558
Review score
+0 /0 /-0
Location
Nyon, Switzerland
Excellent article, thanks for the time and effort that you have put into it!

However, the overall feeling that I get from it (I need to read it again in detail and without distractions) is... I'm not sure actually :thinking-face:.

Once again, I need to really go into the results as presented in more detail but my takeaway from it is that at the moment there is a significant amount of variance in the ICP testing which leads me to question whether it is worth the expense (I send samples for ICP testing every three months, always to the same company).

Or let me put it another way: from what I have read, I would not want my blood sample sent for analysis to any of these vendors :oops:.
 
OP
Rick Mathew

Rick Mathew

Valuable Member
Review score
+0 /0 /-0
View Badges
Joined
Mar 21, 2016
Messages
1,253
Reaction score
4,240
Review score
+0 /0 /-0
Location
North Central Florida
Excellent article, thanks for the time and effort that you have put into it!

However, the overall feeling that I get from it (I need to read it again in detail and without distractions) is... I'm not sure actually :thinking-face:.

Once again, I need to really go into the results as presented in more detail but my takeaway from it is that at the moment there is a significant amount of variance in the ICP testing which leads me to question whether it is worth the expense (I send samples for ICP testing every three months, always to the same company).

Or let me put it another way: from what I have read, I would not want my blood sample sent for analysis to any of these vendors :oops:.
Blood Sample !! Yikes!! No way :eek:...The uncertainty of the measurements due to variation in the overall measurement protocol is a valid takeaway.
 

Lasse

7500 Club Member
Review score
+0 /0 /-0
View Badges
Joined
Mar 20, 2016
Messages
9,528
Reaction score
27,145
Review score
+0 /0 /-0
Location
Källarliden 14 D Bohus, Sweden
Again it is important to remember that the measurement results are on a water sample taken several days earlier

I always take own measurements from the same water sample as I send in and at the same time. That´s the only way if I want to compare my own results with the result from the ICP vendor. I have taken 22 ICP test from the same Vendor (Triton) since I start my aquarium. It means that when I evaluate my results nowadays - I can look at the history and get a better understanding if I should act or not. The last 2 years have I also use another vendor in 7 occasion because they measure other parameters too. This vendor is not one the ones tested here - its OCEAMO in Austria. They measure salinity too and adjust wanted value to the actual salinity. I have compared the results and both are useful tools in my valuations - together with all historical data I have. OCEAMO have also now a new method (ICP-MS) with higher sensitivity (and more elements) - but also a higher price tag (nearly the double).

Sincerely Lasse
 

taricha

2500 Club Member
Review score
+0 /0 /-0
View Badges
Joined
May 22, 2016
Messages
4,142
Reaction score
6,299
Review score
+0 /0 /-0
However, the overall feeling that I get from it (I need to read it again in detail and without distractions) is... I'm not sure actually :thinking-face:.
"hmmm..." is a perfectly appropriate reaction.
ICP vendor data is good enough for some of the conclusions that hobbyists make, and not nearly good enough for other things hobbyists want to do with the data.
 

Laith

Well-Known Member
Review score
+0 /0 /-0
View Badges
Joined
May 16, 2017
Messages
849
Reaction score
1,558
Review score
+0 /0 /-0
Location
Nyon, Switzerland
Blood Sample !! Yikes!! No way :eek:...The uncertainty of the measurements due to variation in the overall measurement protocol is a valid takeaway.

I've done some scientific experimenting in the past so I am at a bit of a loss to understand why this would be happening, at least to the extent that I'm seeing from the article. There will always be variances but this much?

The protocol for every ICP test would logically be exactly the same, there is no reason for any variance in the protocol from company to company.

The only reasons I can think of that there would be such variance:

- Shortcuts in protocol to save time and process as many samples as possible at the lowest cost.
- Lack of frequent enough calibration of the equipment used, again to save time and process as many samples as possible at the lowest cost.
- A complete misunderstanding of the effects of shipment time and conditions of the samples from the time the sample is taken to the time the test is made.

I wonder how many people would pay an extra premium on the cost of the tests but in return get back an article such as yours that shows a lot more confidence in the results you get back? I know I would...

Keep in mind that I am by no means an expert in ICP testing (far from it, never done it!), so my conjectures can be completely wrong!
 

taricha

2500 Club Member
Review score
+0 /0 /-0
View Badges
Joined
May 22, 2016
Messages
4,142
Reaction score
6,299
Review score
+0 /0 /-0
I always take own measurements from the same water sample as I send in and at the same time. That´s the only way if I want to compare my own results with the result from the ICP vendor. I have taken 22 ICP test from the same Vendor (Triton) since I start my aquarium. It means that when I evaluate my results nowadays - I can look at the history and get a better understanding if I should act or not.
Lasse, you have a more sophisticated approach to data than most.
Trends in ICP results paired with trends in simultaneous hobby measurements gives much more needed context to take action.
 
OP
Rick Mathew

Rick Mathew

Valuable Member
Review score
+0 /0 /-0
View Badges
Joined
Mar 21, 2016
Messages
1,253
Reaction score
4,240
Review score
+0 /0 /-0
Location
North Central Florida
I always take own measurements from the same water sample as I send in and at the same time. That´s the only way if I want to compare my own results with the result from the ICP vendor. I have taken 22 ICP test from the same Vendor (Triton) since I start my aquarium. It means that when I evaluate my results nowadays - I can look at the history and get a better understanding if I should act or not. The last 2 years have I also use another vendor in 7 occasion because they measure other parameters too. This vendor is not one the ones tested here - its OCEAMO in Austria. They measure salinity too and adjust wanted value to the actual salinity. I have compared the results and both are useful tools in my valuations - together with all historical data I have. OCEAMO have also now a new method (ICP-MS) with higher sensitivity (and more elements) - but also a higher price tag (nearly the double).

Sincerely Lasse
Lasse: I do the same...It gives that historical look back and give at least some perspective on the measured values. I am also interested in having a look at the ICP-MS..Thanks for your reply

Rick
 
OP
Rick Mathew

Rick Mathew

Valuable Member
Review score
+0 /0 /-0
View Badges
Joined
Mar 21, 2016
Messages
1,253
Reaction score
4,240
Review score
+0 /0 /-0
Location
North Central Florida
I've done some scientific experimenting in the past so I am at a bit of a loss to understand why this would be happening, at least to the extent that I'm seeing from the article. There will always be variances but this much?

The protocol for every ICP test would logically be exactly the same, there is no reason for any variance in the protocol from company to company.

The only reasons I can think of that there would be such variance:

- Shortcuts in protocol to save time and process as many samples as possible at the lowest cost.
- Lack of frequent enough calibration of the equipment used, again to save time and process as many samples as possible at the lowest cost.
- A complete misunderstanding of the effects of shipment time and conditions of the samples from the time the sample is taken to the time the test is made.

I wonder how many people would pay an extra premium on the cost of the tests but in return get back an article such as yours that shows a lot more confidence in the results you get back? I know I would...

Keep in mind that I am by no means an expert in ICP testing (far from it, never done it!), so my conjectures can be completely wrong!
Good observation!...As you note there is variation in every measurement...The question is always how much and how does the magnitude of the variation impact the usefulness of the data... I think your list of possible sources of variation is right on...One or all of these could explain such large variation. As we have pointed out in previous postings, water sampling and testing is no trivial process.

The individual vendors would need to conduct a measurement analysis using their protocols and from that could identify the sources and magnitude of their variation. This could then be reported as a measurement error or uncertainty for each element. This would help the receiver of the data to understand the quality of the measurement that is being reported---This is the Best Practice for reporting measurement results...(you can find references to this in footnote 4 of this article)

If you have a look at this post by @Christoph you can see the idea of reporting the measurement with an uncertainty value https://www.reef2reef.com/threads/i...seawater-certified-reference-material.891657/



Thanks for the reply

Rick
 
Last edited:
OP
Rick Mathew

Rick Mathew

Valuable Member
Review score
+0 /0 /-0
View Badges
Joined
Mar 21, 2016
Messages
1,253
Reaction score
4,240
Review score
+0 /0 /-0
Location
North Central Florida
I'm fortunate to have testing facility 15 minutes from where I work... grab a sample in the morning and have results about an hour later.
Nice!...You could conduct a similar experiment removing the Transportation and Storage variable...That would be very interesting to see.

Thanks for your reply
 

Lasse

7500 Club Member
Review score
+0 /0 /-0
View Badges
Joined
Mar 20, 2016
Messages
9,528
Reaction score
27,145
Review score
+0 /0 /-0
Location
Källarliden 14 D Bohus, Sweden
I have run this aquarium for 6 years. It has been nearly 5 years without any WC and adding different compounds into the aquarium. I would never dare to do it this way without having access to ICP tests. I killed my old aquarium probably because of overdosing potassium - had no access to ICP test - got it just as the same time - everything crash. I took 2 analyzes showing potassium over 600 ppm. During the same time - my PO4 level was around 2.4 (2.4). I did know the high PO4 figures and before the age of ICP test I would have been told that it was the high PO4 that killed my fish. But - I swear - my corals have never looks as god as they did at that time.

Regular ICP test is for me a game changer according to maintain a mixed reef without WC

Sincerely Lasse
 

Laith

Well-Known Member
Review score
+0 /0 /-0
View Badges
Joined
May 16, 2017
Messages
849
Reaction score
1,558
Review score
+0 /0 /-0
Location
Nyon, Switzerland
I have run this aquarium for 6 years. It has been nearly 5 years without any WC and adding different compounds into the aquarium. I would never dare to do it this way without having access to ICP tests. I killed my old aquarium probably because of overdosing potassium - had no access to ICP test - got it just as the same time - everything crash. I took 2 analyzes showing potassium over 600 ppm. During the same time - my PO4 level was around 2.4 (2.4). I did know the high PO4 figures and before the age of ICP test I would have been told that it was the high PO4 that killed my fish. But - I swear - my corals have never looks as god as they did at that time.

Regular ICP test is for me a game changer according to maintain a mixed reef without WC

Sincerely Lasse
I agree with you Lasse: ICP testing for me is worth it, otherwise I wouldn't bother:

- it tests many elements that I cannot test myself.
- If a result seems odd it at least tells me that I may need to look closer at the result and the possible reasons behind it. Usually I send a second sample in to verify.
- As I always do my three month ICP testing with the same company, I assume that the company uses the same protocol for their testing and the same calibration procedures. Therefore any variation from one test to the next may indicate something.

The above is why I do an ICP test every three months (both on my tank water and RO/DI water).

I also test all the elements that I can test myself on the day (or the day before) I send my sample in. I have not seen any alarming differences between my tests and theirs. But for all the other elements I am very careful with jumping to frantic fixes if I see an odd result.

A good example is that my ICP tests were consistently showing high salinity (38ppt) while my Hanna electronic refractometer (as well as my Red Sea refractometer) was showing 35ppt. I just assumed that it was due to evaporation of the sample by the time it reached the lab. But I started wondering about it and purchased a floating hydrometer from Tropic Marin and it showed 38ppt.

That is why I do an ICP test every three months. But I am not confident enough in the results to use them to dose every possible trace element that shows low on the test. And that is the main reason that I still do water changes...
 

Dennis Cartier

Valuable Member
Review score
+0 /0 /-0
View Badges
Joined
Aug 25, 2016
Messages
1,619
Reaction score
2,022
Review score
+0 /0 /-0
Location
Brampton, Ontario
Hmm, so based on your results, it looks like some vendors have some challenges with detecting traces accurately and repeatably. Yet, all these vendors provide guidelines on corrections that are suggested to be made (either using their proprietary additives or generally available additives).

Based on what you found though, making corrections based on the ICP results seems a bit of a gamble or shall we say a leap of faith.
 
OP
Rick Mathew

Rick Mathew

Valuable Member
Review score
+0 /0 /-0
View Badges
Joined
Mar 21, 2016
Messages
1,253
Reaction score
4,240
Review score
+0 /0 /-0
Location
North Central Florida
Hmm, so based on your results, it looks like some vendors have some challenges with detecting traces accurately and repeatably. Yet, all these vendors provide guidelines on corrections that are suggested to be made (either using their proprietary additives or generally available additives).

Based on what you found though, making corrections based on the ICP results seems a bit of a gamble or shall we say a leap of faith.
Yes Dennis your observation is a good one. As @Laith commented above using ICP-OES results to make addition or dosage adjustments could be problematic. The other issue that I struggle with is this. From the time I take my sample till I get my results could be several days...Our tanks are living organisms and things change over time...Including water changes...dosing etc. So making a change on "old" data does seem a bit risky to me. I think @Laith & @Lasse strategies are good ones for the use of ICP-OES results.

Thanks for the reply

Rick
 
Last edited:

Dan_P

2500 Club Member
Review score
+0 /0 /-0
View Badges
Joined
Sep 21, 2018
Messages
3,310
Reaction score
3,752
Review score
+0 /0 /-0
I have run this aquarium for 6 years. It has been nearly 5 years without any WC and adding different compounds into the aquarium. I would never dare to do it this way without having access to ICP tests. I killed my old aquarium probably because of overdosing potassium - had no access to ICP test - got it just as the same time - everything crash. I took 2 analyzes showing potassium over 600 ppm. During the same time - my PO4 level was around 2.4 (2.4). I did know the high PO4 figures and before the age of ICP test I would have been told that it was the high PO4 that killed my fish. But - I swear - my corals have never looks as god as they did at that time.

Regular ICP test is for me a game changer according to maintain a mixed reef without WC

Sincerely Lasse
Lasse, what trace elements are you monitoring? Have noticed any of the elements depleting faster than others?
 

How much of your saltwater reef tank is automated?

  • Nothing is automated

    Votes: 53 12.9%
  • 25% or so

    Votes: 129 31.3%
  • 50% or so

    Votes: 98 23.8%
  • 75% or so

    Votes: 87 21.1%
  • Close to 100%

    Votes: 43 10.4%
  • Totally Automated (we need proof)

    Votes: 2 0.5%
Biota Marine Life
Top