ICP Tests - 2 labs comparative analysis

[Ali-F]

Hello everyone,

I would like to share my recent experience with the ICP tests and discuss them with the Reefing community to hear your opinions. First, let me explain my long-standing approach, which I am generally very satisfied with. I don’t do home tests except for the no3 and po4. I rely on the ICP tests for several reasons, including errors that can occur with measurements, mixing, storage, and validity issues with testing reagents... etc. That's why I find it more convenient to send a sample to the lab that will thoroughly examine it and provide me with precise results and recommendations that are easy to follow.

Recently, I started feeling that the results from the lab were not logical, so I decided to send three water samples to three different labs for a comparative analysis. I selected labs with a good reputation in the reefing community, including the lab I've been dealing with for years. Indeed, I took three samples at the same time and from the same spot the same way I usually follow. I then shipped the samples to the respective labs. Unfortunately, due to my mistake in providing the address to one of the labs, one of the samples was lost. This led to testing two samples in different labs.

The results came out unexpectedly, and it's not possible to definitively say that either of the labs made an error. The results showed similarities in some elements and significant variations in others. This has made me reconsider the regularity of sending samples for testing. I incur financial costs for the precise results, but the unpredictable outcomes have made me question the continued periodic submission of samples for ICP tests.

I am attaching the results from both labs along with a summary file I created for comparison.

Feel free to comment and share your opinions.

 

[maged88]

Hello everyone,

I would like to share my recent experience with the ICP tests and discuss them with the Reefing community to hear your opinions. First, let me explain my long-standing approach, which I am generally very satisfied with. I don’t do home tests except for the no3 and po4. I rely on the ICP tests for several reasons, including errors that can occur with measurements, mixing, storage, and validity issues with testing reagents... etc. That's why I find it more convenient to send a sample to the lab that will thoroughly examine it and provide me with precise results and recommendations that are easy to follow.

Recently, I started feeling that the results from the lab were not logical, so I decided to send three water samples to three different labs for a comparative analysis. I selected labs with a good reputation in the reefing community, including the lab I've been dealing with for years. Indeed, I took three samples at the same time and from the same spot the same way I usually follow. I then shipped the samples to the respective labs. Unfortunately, due to my mistake in providing the address to one of the labs, one of the samples was lost. This led to testing two samples in different labs.

The results came out unexpectedly, and it's not possible to definitively say that either of the labs made an error. The results showed similarities in some elements and significant variations in others. This has made me reconsider the regularity of sending samples for testing. I incur financial costs for the precise results, but the unpredictable outcomes have made me question the continued periodic submission of samples for ICP tests.

I am attaching the results from both labs along with a summary file I created for comparison.

Feel free to comment and share your opinions.

Hi Ali,
As I faced almost the same situation as yours , I decided to do ICP test only when I visually notice something wrong in the tank, having my KH auto tested using KHCarer, testing Po4 and No3 2 times a week , and relying on dosing a comprehensive elements using RKS system 4 supplements.
So I believe that even if the home test readings may not be as accurate as Lab ones, I found that stability in numbers is the key rather than those numbers are high or low

 

[rtparty]

Couple quick notes:

ICP testing (especially from those 2 labs) is far from accurate or precise.
Some labs are better than others but even that is a crap shoot. ATI and Oceamo are the only 2 I would ever send a sample to and even then I doubt they would come close to matching each other.

Check this thread out:

HOBBY GRADE TEST KITS CAN OUTPERFORM ICP MEASUREMENTS…REALLY??

INTRODUCTION BACKGROUND In May of 2022 we (@Dan_P, @taricha and I) posted an article that looked at the quality of the data reported by ICP-OES measurements from several vendors. Our goal was to help bring some clarity to the community to help evaluate and interpret the results of an ICP test...

www.reef2reef.com

Trusting ICP testing (again speaking generally about ICP testing) is not advised

 

[Ali-F]

Hi Ali,
As I faced almost the same situation as yours , I decided to do ICP test only when I visually notice something wrong in the tank, having my KH auto tested using KHCarer, testing Po4 and No3 2 times a week , and relying on dosing a comprehensive elements using RKS system 4 supplements.
So I believe that even if the home test readings may not be as accurate as Lab ones, I found that stability in numbers is the key rather than those numbers are high or low

Thanks Maged for the reply. I agree with you that stability in numbers is very important.

The thing with the ICP test results is that they cover a wider range of elements than home test kits which 1) helps to discover any unwanted elements before anything goes wrong and 2) shows depleted minor and major elements. Those usually help me take the best course of action that benefits the corals.

But looking at the other thread (https://www.reef2reef.com/threads/h...n-outperform-icp-measurements…really.1007101/) After a quick look, the story has taken a different turn. I don't want to rush into a decision. I need to carefully read what's written multiple times to fully grasp the situation before deciding on my next steps.

 

[Ali-F]

Couple quick notes:

ICP testing (especially from those 2 labs) is far from accurate or precise.
Some labs are better than others but even that is a crap shoot. ATI and Oceamo are the only 2 I would ever send a sample to and even then I doubt they would come close to matching each other.

Check this thread out:

HOBBY GRADE TEST KITS CAN OUTPERFORM ICP MEASUREMENTS…REALLY??

INTRODUCTION BACKGROUND In May of 2022 we (@Dan_P, @taricha and I) posted an article that looked at the quality of the data reported by ICP-OES measurements from several vendors. Our goal was to help bring some clarity to the community to help evaluate and interpret the results of an ICP test...

www.reef2reef.com

Trusting ICP testing (again speaking generally about ICP testing) is not advised

Thank you for your reply and for sharing the link. The article seems incredibly informative, and I'm looking forward to reading it in its entirety. However, I must admit, I've reached a point where I'm questioning things. Take salt companies, for instance, they conduct tests on every batch as a measure of quality assurance, some companies even declaring it on the packaging. But now, who can guarantee the accuracy of these claims? What about calibration solutions? Is it appropriate to use them? Do we use them to correct what is wrong or disrupt what is right?

Never mind all this hallucination. I don't want to complicate things further. I will stick to the basics and simplicity..... stability over accuracy.

 

[Reefahholic]

Yes, please DO NOT send ICP’s or replenish any low elements. SUPER INACCURATE!

Big No No!

 

[rtparty]

Yes, please DO NOT send ICP’s or replenish any low elements. SUPER INACCURATE!

Big No No!

Based on the two tests above, what would you have them replenish and why? Which one of those tests is accurate or reliable enough to go off of? Which is right and which is wrong? Or are both wrong?

 

[Reefahholic]

Based on the two tests above, what would you have them replenish and why? Which one of those tests is accurate or reliable enough to go off of? Which is right and which is wrong? Or are both wrong?

As far as reliability and trustworthiness you already know we prefer OCEAMO ICP-MS. ;-)

Honestly, I can’t see either of those downloads because my phone is full.

 

[rtparty]

As far as reliability and trustworthiness you already know we prefer OCEAMO ICP-MS. ;-)

Honestly, I can’t see either of those downloads because my phone is full.

Wasn’t the question.

I’d look at the data before passing judgement

 

[Ordovician_Reef]

Hello everyone,

I would like to share my recent experience with the ICP tests and discuss them with the Reefing community to hear your opinions. First, let me explain my long-standing approach, which I am generally very satisfied with. I don’t do home tests except for the no3 and po4. I rely on the ICP tests for several reasons, including errors that can occur with measurements, mixing, storage, and validity issues with testing reagents... etc. That's why I find it more convenient to send a sample to the lab that will thoroughly examine it and provide me with precise results and recommendations that are easy to follow.

Recently, I started feeling that the results from the lab were not logical, so I decided to send three water samples to three different labs for a comparative analysis. I selected labs with a good reputation in the reefing community, including the lab I've been dealing with for years. Indeed, I took three samples at the same time and from the same spot the same way I usually follow. I then shipped the samples to the respective labs. Unfortunately, due to my mistake in providing the address to one of the labs, one of the samples was lost. This led to testing two samples in different labs.

The results came out unexpectedly, and it's not possible to definitively say that either of the labs made an error. The results showed similarities in some elements and significant variations in others. This has made me reconsider the regularity of sending samples for testing. I incur financial costs for the precise results, but the unpredictable outcomes have made me question the continued periodic submission of samples for ICP tests.

I am attaching the results from both labs along with a summary file I created for comparison.

Feel free to comment and share your opinions.

For my MS and PhD work I spent 100s of hours doing ICPMS, OES and many other analytical methodologies working with marine carbonates. I would NEVER trust a commercial lab running ICP for $40 a test. Are they accurate, likely they are most of the time, to a reasonable degree. But you never KNOW they are right and worse yet, a lot of the information they might give you, you really do not know how to use (nobody does).

Now if I have a problem that I think might be linked to elemental imbalances, I would send three samples to three different labs. And would act on them ONLY if it were for am element that we have at least a pretty good idea of what role it plays in the system.

Just my two cents worth

 

[DaJMasta]

ICP testing (especially from those 2 labs) is far from accurate or precise.

Trusting ICP testing (again speaking generally about ICP testing) is not advised

Bold claims when the best we've otherwise got is household test kits.

If you want to test, ICP is, by far, the most accurate and precise testing we have access to. Just because it's a hard problem to solve and we want it to just give an absolute measurement with no error bars doesn't mean it's infallible.


I think your comparison paints a worse picture than the data represents, and it could use a lot of extra data and considerations to be more than an experiential test (which you're not claiming it is, I understand that.)

A big improvement, in my mind, would be reformatting the graphs. List a scale on the Y axis, list the units being used, and don't start the Y axis away from zero. The actual difference between tests is much better quantified when they are compared at full scale or relative to each other by percentage of reading/significant digits of difference/etc. By zooming in on the top, you make the difference visually appear significant even when it isn't, and it's a misleading way to present it, even if unintentional.

The harder part of a good analysis of systems would be to get some information at claimed accuracy specifications for their given technology. I don't really expect these companies to list it, helpful though it would be, but if you could at least find out the technology used in their system (OES vs. Mass Spectrometer) or the company their system is from, you may be able to get an idea at theoretical optimum performance values for the system. The actual performance will differ based on the rigor of their rechecking of standards and maintaining environmental conditions and system maintenance and cleanliness, but knowing the technology should at least get you started. A simple explanation from a manufacturer of both is here: https://www.thermofisher.com/us/en/...on-icp-oes-icp-ms-trace-element-analysis.html and the short of it is that a mass spectrometer ICP test is going to going to be able to detect lower concentrations in a sample and will be better able to differentiate very close looking elemental signatures in the measured spectrum. That means if you have an OES test claiming sub-ppb measurements and a mass spec test claiming something else, I would immediately be suspect of the OES test for even providing a value which should be well into their system's noise level.

There's also a capability-of-the-technology note that can explain your lack of certain elemental results (and perhaps raise an eyebrow as to the technique used for ones claiming a number for it). When specifically talking about a mass spectrometer based test:

ICP-MS can measure virtually every naturally occurring element plus many non-natural “radiogenic” isotopes such as technetium, neptunium, plutonium, and americium. The only elements that ICP-MS can’t measure are H and He (which are below the mass range of the mass spectrometer), Ar, N, and O (which are present at high level from the plasma and air), and F and Ne (which can’t be ionized in an argon plasma). Of these “impossible” elements, even F can be analyzed indirectly using a triple quadrupole ICP-MS.

From here: https://www.agilent.com/en/product/...pectrometry-icp-ms/what-is-icp-ms-icp-ms-faqs

Then if you really want to try and put a manufacturer/system through its paces, of course as you said more data points for a single sample (both measurement companies and recommended values) should give you a better representation of what's there, but repeated measurements of the same sample would really be a better tell of overall consistency. If you can take a single water sample, split it into several tests, and then submit the tests at different times (maybe a day or two apart, it depends on how frequently they run new batches of tests), you would have a much better picture of the variation that may be present in a company's technique and handling. If you take several samples and submit them all at once, should they run in the same batch, they will appear closer because of the same calibration state (maybe this is also useful data, though, as a noise level), but submitting the same sample for processing in different batches means the machine doing the work will have been recalibrated at least once in between, and the variation you see may be larger (and better representative of the variation you can expect on a given single sample) if the drift between calibrations is a significant factor.

Now I'm not going to make claims that any one company is 'doing it right' or is more reliable than another, and there are things accurately measured that aren't going to well represent the problem in the tank (ICP is by definition an elemental analysis, and we do not add elements to our tank, we add compounds comprised of elements, and different compounds of the same elements ICP is completely blind to), but if you want to measure the component elements of your tank water, there is not a better option than ICP on the commercial market and other composition analysis techniques (XRF, Raman, etc.) really don't offer substantial precision/accuracy advantages for testing a water sample.

 

[rtparty]

Bold claims when the best we've otherwise got is household test kits.

If you want to test, ICP is, by far, the most accurate and precise testing we have access to. Just because it's a hard problem to solve and we want it to just give an absolute measurement with no error bars doesn't mean it's infallible.


I think your comparison paints a worse picture than the data represents, and it could use a lot of extra data and considerations to be more than an experiential test (which you're not claiming it is, I understand that.)

A big improvement, in my mind, would be reformatting the graphs. List a scale on the Y axis, list the units being used, and don't start the Y axis away from zero. The actual difference between tests is much better quantified when they are compared at full scale or relative to each other by percentage of reading/significant digits of difference/etc. By zooming in on the top, you make the difference visually appear significant even when it isn't, and it's a misleading way to present it, even if unintentional.

The harder part of a good analysis of systems would be to get some information at claimed accuracy specifications for their given technology. I don't really expect these companies to list it, helpful though it would be, but if you could at least find out the technology used in their system (OES vs. Mass Spectrometer) or the company their system is from, you may be able to get an idea at theoretical optimum performance values for the system. The actual performance will differ based on the rigor of their rechecking of standards and maintaining environmental conditions and system maintenance and cleanliness, but knowing the technology should at least get you started. A simple explanation from a manufacturer of both is here: https://www.thermofisher.com/us/en/...on-icp-oes-icp-ms-trace-element-analysis.html and the short of it is that a mass spectrometer ICP test is going to going to be able to detect lower concentrations in a sample and will be better able to differentiate very close looking elemental signatures in the measured spectrum. That means if you have an OES test claiming sub-ppb measurements and a mass spec test claiming something else, I would immediately be suspect of the OES test for even providing a value which should be well into their system's noise level.

There's also a capability-of-the-technology note that can explain your lack of certain elemental results (and perhaps raise an eyebrow as to the technique used for ones claiming a number for it). When specifically talking about a mass spectrometer based test:

From here: https://www.agilent.com/en/product/...pectrometry-icp-ms/what-is-icp-ms-icp-ms-faqs

Then if you really want to try and put a manufacturer/system through its paces, of course as you said more data points for a single sample (both measurement companies and recommended values) should give you a better representation of what's there, but repeated measurements of the same sample would really be a better tell of overall consistency. If you can take a single water sample, split it into several tests, and then submit the tests at different times (maybe a day or two apart, it depends on how frequently they run new batches of tests), you would have a much better picture of the variation that may be present in a company's technique and handling. If you take several samples and submit them all at once, should they run in the same batch, they will appear closer because of the same calibration state (maybe this is also useful data, though, as a noise level), but submitting the same sample for processing in different batches means the machine doing the work will have been recalibrated at least once in between, and the variation you see may be larger (and better representative of the variation you can expect on a given single sample) if the drift between calibrations is a significant factor.

Now I'm not going to make claims that any one company is 'doing it right' or is more reliable than another, and there are things accurately measured that aren't going to well represent the problem in the tank (ICP is by definition an elemental analysis, and we do not add elements to our tank, we add compounds comprised of elements, and different compounds of the same elements ICP is completely blind to), but if you want to measure the component elements of your tank water, there is not a better option than ICP on the commercial market and other composition analysis techniques (XRF, Raman, etc.) really don't offer substantial precision/accuracy advantages for testing a water sample.

Yeah, every study and experiment done testing the hobby ICP labs paints a picture far from what you just said.

The first couple and last couple pages of the following thread are a good read

HOBBY GRADE TEST KITS CAN OUTPERFORM ICP MEASUREMENTS…REALLY??

INTRODUCTION BACKGROUND In May of 2022 we (@Dan_P, @taricha and I) posted an article that looked at the quality of the data reported by ICP-OES measurements from several vendors. Our goal was to help bring some clarity to the community to help evaluate and interpret the results of an ICP test...

www.reef2reef.com

 

[gbroadbridge]

Hello everyone,

I would like to share my recent experience with the ICP tests and discuss them with the Reefing community to hear your opinions. First, let me explain my long-standing approach, which I am generally very satisfied with. I don’t do home tests except for the no3 and po4. I rely on the ICP tests for several reasons, including errors that can occur with measurements, mixing, storage, and validity issues with testing reagents... etc. That's why I find it more convenient to send a sample to the lab that will thoroughly examine it and provide me with precise results and recommendations that are easy to follow.

Recently, I started feeling that the results from the lab were not logical, so I decided to send three water samples to three different labs for a comparative analysis. I selected labs with a good reputation in the reefing community, including the lab I've been dealing with for years. Indeed, I took three samples at the same time and from the same spot the same way I usually follow. I then shipped the samples to the respective labs. Unfortunately, due to my mistake in providing the address to one of the labs, one of the samples was lost. This led to testing two samples in different labs.

The results came out unexpectedly, and it's not possible to definitively say that either of the labs made an error. The results showed similarities in some elements and significant variations in others. This has made me reconsider the regularity of sending samples for testing. I incur financial costs for the precise results, but the unpredictable outcomes have made me question the continued periodic submission of samples for ICP tests.

I am attaching the results from both labs along with a summary file I created for comparison.

Feel free to comment and share your opinions.

I stopped doing ICP as I found that the results sometimes were obviously suspicious.

I figured that rather than spending $60 a test (that's what they cost here), I'd be better off having two reliable tests for main elements and assuming that traces would be replenished as required by a small water change.

That method has been working fine here for the last 12 months with no trace element deficiencies causing any problems being seen.

 

[Reefahholic]

Wasn’t the question.

I’d look at the data before passing judgement

You’re right. Those are definitely not very common. I have never seen any of their data! I should not be passing judgment here.

 

[KStatefan]

I know it is a calculated figure, but why would you ever give a customer a figure with this many insignificant places?

PO4
Phosphates
61.240283999999996 ppb

 

[ingchr1]

I know it is a calculated figure, but why would you ever give a customer a figure with this many insignificant places?

PO4
Phosphates
61.240283999999996 ppb

Same with many of the other results, but certainly not to that extent.

When reporting in ppb, is anything other than a whole number necessary?

When reporting ppm, are three decimal places necessary? Values in the hundreds, thousands and tens of thousands of ppm probably don't need any decimal places.

Maybe not just necessary, but is reporting to these levels of decimal places even accurate?

The fact that something like that made it into a report should give the consumer great pause about that lab.

 

[gbroadbridge]

Same with many of the other results, but certainly not to that extent.

When reporting in ppb, is anything other than a whole number necessary?

When reporting ppm, are three decimal places necessary? Values in the hundreds, thousands and tens of thousands of ppm probably don't need any decimal places.

Maybe not just necessary, but is reporting to these levels of decimal places even accurate?

The fact that something like that made it into a report should give the consumer great pause about that lab.

It just misleads consumers into the false belief that the test is more accurate than competitors.

 

[Ordovician_Reef]

I know it is a calculated figure, but why would you ever give a customer a figure with this many insignificant places?

PO4
Phosphates
61.240283999999996 ppb

That is 61, at ppb everything else if make believe.

 

[exnisstech]

I don't trust them. A while back I sent two samples from two tanks just for the heck of it. The test included RODI testing so I also sent two samples of RODI from the same container. The RODI results came back drastically different even tho they were the same water. I figure if they can't test RODI accurately I'm not about to start changing things in my tanks based on ICP results.
Like many other things in this hobby businesses have figured out reefers are willing to spend foolish amounts of money on things we didn't even know we needed until they became available to us. Unless someone is on the shine (I am not) they are a waste of money.
The only time I would ever consider one again is if I'm having unexplained losses and it would be a last resort. This is JMO which we are all entitled to

 

[jason2459]

If you want to test, ICP is, by far, the most accurate and precise testing we have access to.

Sending in some vials to someone for discount shotgun ICP testing OES or MS is perfectly fine and easy to do. Just don't fall for it being very accurate or precise.