HOBBY GRADE TEST KITS CAN OUTPERFORM ICP MEASUREMENTS…REALLY??

[Charlie the Reefer]

I’m not sure if you saw this so I will post it again.

This was a guy that recently switch from ATI-OES to OCEAMO-MS.

As you can see below, the trace metals were there, but too low for the OES sensitivity. It’s interesting to see this kind of data basically back to back. I know this guy did nothing different than what he normally does, and I’ve seen hundreds like this. Not only that, but I literally watch all their values go from very low or depleted and into our target range. It’s exciting. We only dreamed about this kind of data 10 years ago.

Here’s another new guy. 1st analysis is ATI-OES and the last 9 are OCEAMO-MS. His consumption took off and it becomes a chase for a bit, but you can clearly see he’s making some nice progress by #10. Others do it much faster, but he logged his results nicely.

Hello everyone,

This is my data and I’m happy to elaborate more if anyone is curious. And to reiterate, I am a humble newcomer to this hobby and do not have much to show for (YET!)

For my own part, I don’t have a doubt in my mind that what we dose in moonshiners corresponds to the ICPs we get back, after doing this for near a whole year now. You double or triple dose cobalt, nickel etc and you WILL see it in subsequent ICP. I would indeed presume it’s a fluke / testing error if I only saw one or two elements increase or become detectable after upping dosage. But it’s a bit more convincing when you see the 15 elements you upped dosage on move on the next ICP. And you really do see what your tank regularly “consumes” (if that is even the right word), not unlike alk/CA

I’m not here to argue against the original study that was done, just to provide my own anecdote. I agree with the philosophy that trends are far more important than precise measurements.

I also want to thank everyone contributing in this thread, including the individuals who attempted the study and those fervently debating this fascinating topic.

We should all keep in mind we are collaborating to improve best practices for everyone!

Again if anyone wants to ask me specific questions about that 10 ICPs data I’m happy to provide.

Charlie

 

[Sisterlimonpot]

just a fun side-discussion (even if it seems to dominate the thread).

The side discussion has definitely dominated.

My take away from the study is that home testing (with all their flaws) aren't any worse than icp. And the biggest reservation for me is that when I get a questionable result from my home test, I can simply run the test again. When an icp spits out a questionable reading, there's nothing to do but send in another sample, which is impossible because time had elapsed and the sampling is completely different.

The idea that icp was touted to not make those errors have seemed to be proven false and the results are only as good as the person running the machine.

 

[MnFish1]

We found that if you have a trusted kit and are comfortable using it, it's at least equal to or better than ICP for those things. And there's no reason to throw out your own test results because an ICP report disagrees.

If one measures element xxxx once/week on a home test - and it's always 8, there is no way to tell if your ICP is right or wrong if it comes back at 10 or 12. You could be doing your home test consistently incorrectly, and the ICP value might be correct. So - I would say that in an individual case like you're describing above - if there is a discrepancy between 2 tests, you should get a third test done to determine which test is correct. Looking at the data - there were at least a couple areas where results seemed widely disparate in the replicate samples done by the 'testers'.

Additionally, my guess is that the 3 of you (as you said at some point) - may be far better than the average 'tester'. So - applying your results to a broader population may be premature?

I appreciate the work put into this - and I guess it shows what was expected - that home tests are pretty close to most ICP testing - if the tests are done correctly.

I'm curious as to what the results would have been if you just took seawater (from a tank) - divided it up - and sent it to the ICP companies - as compared to spiking the sample - as this COULD also have caused some differences (not saying it did).

 

[Thales]

Mixing testing methodologies will make you insane, so I say pick one and stick with it (and be realistic about the numbers because there is so much room for error and variability) I like ICP for spot checks every once and a while to check for anything obviously out of whack because I don't see the need to tweak all the time and don't like having to wait for results, YMMV and good on the people that like that kind of thing.

 

[Rick Mathew]

I'm curious as to what the results would have been if you just took seawater (from a tank) - divided it up - and sent it to the ICP companies - as compared to spiking the sample - as this COULD also have caused some differences (not saying it did).

Very interesting question...Here is my take. It most likely would cause differences because we have added other variables..such as the bio-activity in our tanks...how it would turn our would be a speculation, but it might be an interesting experiment for the future...Thanks for your curiosity

 

[Rick Mathew]

I think this is what they’re looking for, but Christoph has already answered these questions. Nothing will ever be enough.

Actually in its simplest form this is what would be helpful (see below)...If the reports we received would indicate the level of expected error for the measured value, but if I understand Christoph's response to @Dan_P s question (What is preventing Oceamo from stating the accuracy and precision for each element in their ICP report?") the way his system and methodology is will not allow this

Christoph's Response

"Thank you Dan! Unfortunately you will most likely not like my answer to your question:

Asking for accuracy and precision for every individual element in each individual run is very easy - but to deliver this data in a meaningful way is unfortunately almost impossible." This does not mean they are not accurate or precise it only means we don't know

 

[KStatefan]

Actually in its simplest form this is what would be helpful (see below)...If the reports we received would indicate the level of expected error for the measured value, but if I understand Christoph's response to Dan's question (What is preventing Oceamo from stating the accuracy and precision for each element in their ICP report?") the way his system and methodology is will not allow this

Christoph's Response

"Thank you Dan! Unfortunately you will most likely not like my answer to your question:

Asking for accuracy and precision for every individual element in each individual run is very easy - but to deliver this data in a meaningful way is unfortunately almost impossible." This does not mean they are not accurate or precise it only means we don't know

But without some type of certification you still have to have faith in the company that those are correct. which is where we are now.

 

[danimal1211]

Perhaps it would be benificial to discuss what “laboratory practices” and “protocols” were used by the hobbiest testers to achieve their results. I would think enlightening the community on how to more accurately utilize their at home test kits would be very beneficial.

 

[MnFish1]

Very interesting question...Here is my take. It most likely would cause differences because we have added other variables..such as the bio-activity in our tanks...how it would turn our would be a speculation, but it might be an interesting experiment for the future...Thanks for your curiosity

Thanks - that what I was getting at. I think this is something a couple other posters were getting at as well. It would be my hypothesis that it would not cause differences. And - it would be 'real-world'.

 

[MnFish1]

Mixing testing methodologies will make you insane, so I say pick one and stick with it (and be realistic about the numbers because there is so much room for error and variability) I like ICP for spot checks every once and a while to check for anything obviously out of whack because I don't see the need to tweak all the time and don't like having to wait for results, YMMV and good on the people that like that kind of thing.

Curious - which methods do you use to minimize variation/error? (Besides using the same test - and monitoring trends). I was going to start a new thread which would be heresy - suggesting that in fact - unless you see a problem - you probably don't need any testing - certainly not the same amount as touted by the manufacturers of the test kits. (I grant this takes a certain amount of experience to judge).

 

[Rick Mathew]

Perhaps it would be benificial to discuss what “laboratory practices” and “protocols” were used by the hobbiest testers to achieve their results. I would think enlightening the community on how to more accurately utilize their at home test kits would be very beneficial.

Excellent Point! I can only speak for myself. I essentially follow the principles and practices you can find in the 4 articles I posted in 2019... you can read them here

Part 1 https://www.reef2reef.com/ams/getti...count-by-using-a-quality-system-approach.741/

Part 2 https://www.reef2reef.com/ams/part-...tem-approach-colorimetric-visual-testing.744/

Part 3 https://www.reef2reef.com/ams/part-...etric-visual-testing-methods-what-we-see.747/

Part 4 https://www.reef2reef.com/ams/part-...ric-instrumental-testing-methods-digital.748/.

Since these were published I have made some refinements, which I should probably add as a Part 5. but for the most part these describe how I approach my testing. For me the most important aspect is identifying where potential variability exists (sources of error) and work to eliminate them.

Happy to answer any questions..

 

[Thales]

Curious - which methods do you use to minimize variation/error? (Besides using the same test - and monitoring trends). I was going to start a new thread which would be heresy - suggesting that in fact - unless you see a problem - you probably don't need any testing - certainly not the same amount as touted by the manufacturers of the test kits. (I grant this takes a certain amount of experience to judge).

Except for the trident and other apex related stuff, I haven't tested anything since March, and October before that.

 

[MnFish1]

Except for the trident and other apex related stuff, I haven't tested anything since March, and October before that.

Curious - and if so I agree with you - you have not been doing hobbyist tests since March>

 

[Rick Mathew]

But without some type of certification you still have to have faith in the company that those are correct. which is where we are now.

Yes this is correct...and I would add even with a certification! Trust is built over time and with verification I think it was Ronald Reagan that said "Trust but Verify"

 

[Reefahholic]

Since I like to make a fool of myself,
1) How are these labs measuring the non ICP values?
Salinity
Kh
NO2
NO3
PO4 Orthophpsthate…..

Look at this picture below:

We found that if you have a trusted kit and are comfortable using it, it's at least equal to or better than ICP for those things. And there's no reason to throw out your own test results because an ICP report disagrees.

Did you guys actually calculate the margin of error % difference? I’m just curious what it came out to be with OCEAMO alone.

When an icp spits out a questionable reading, there's nothing to do but send in another sample, which is impossible because time had elapsed and the sampling is completely different.

Even if some time has elapsed it affects each element differently. Some would be close to what they were and others may be pretty far off. Depends on the system and element. For example, if you added 40 ceramic frag plugs in the sump, and you notice your Vanadium level is now 17, and you’re questioning the result and why it shot up, just know that the Vanadium level will not go anywhere quickly until you remove the 40 ceramic frag plugs. Yes, I’m using myself in this example. Lol

Very interesting question...Here is my take. It most likely would cause differences because we have added other variables..such as the bio-activity in our tanks...how it would turn our would be a speculation, but it might be an interesting experiment for the future...Thanks for your curiosity

Here’s what Christoph told me initially. Iron is usually well detected by ICP-MS, and also ICP-OES isn’t too bad with iron. However, iron is prone to precipitation or biogenic accumulation, both factors could remove it from a solution. In NSW or tank water, we have a lot of organics floating around that can act as a stabilizing agent for many trace elements including iron. Precipitation is much less likely. You guys were using artificial (freshly made) seawater, that lacks those stabilizing organics. Furthermore, you added the spike concentration using a standard that’s aimed to be used in acidic solution only, to make it stable, basically containing bare iron ions. His hypothesis was that the experimental setup was not suitable, and the iron quickly precipitated on the container walls, due to the lack of organics and non-acidic environment.

But without some type of certification you still have to have faith in the company that those are correct. which is where we are now.

Even with a certification. Nothing is perfect. Even the most elite labs will make mistakes
and have some errors. I’ve seen it many times.

 

[Reefahholic]

Yes this is correct...and I would add even with a certification!

100% look above this post.

 

[Rick Mathew]

Even with a certification. Nothing is perfect. Even the most elite labs will make mistakes
and have some errors. I’ve seen it many times.

Absolutely!!

 

[Rick Mathew]

. His hypothesis was that the experimental setup was not suitable, and the iron quickly precipitated on the container walls, due to the lack of organics and non-acidic environment.

Should be easy enough to test this Hypothesis don't you think?

 

[Reefahholic]

At the end of this discussion I think we all learned something. I hope to see some of you guys ICP testing soon.

Should be easy enough to test this Hypothesis don't you think?

I think so. Maybe send one like you previously did it, and another with NSW. See how much variability there is with iron in both samples.

 

[MnFish1]

Should be easy enough to test this Hypothesis don't you think?

Actually - IMHO - hypotheses that seem 'easy enough to test' are often extremely difficult - unfortunately like the hypothesis you're trying to 'prove' here.