The real Black Box light study

[theKoolAidMan]

Well, I sure didn't mean to cause a ruckus. Apologies. I was only saying that the Seneye isn't that bad of a tool to use if done consistently across all units tested. Even BRStv noted how accurate it was compared to some of the other more expensive units. So just comes down to how the test is done, etc.

And yes - I don't think anyone is going to change from any brand but the exercise is fun never the less and may shed some light on stuff. In any case hopefully the people doing it are having some fun.

You didn't at all. Listen, if we all want to play scientist here...then you go the whole hog. We need to be able to have these open and frank discussions. There's a reason real scientific papers are peer reviewed.

 

[dantimdad]

Well, I sure didn't mean to cause a ruckus. Apologies. I was only saying that the Seneye isn't that bad of a tool to use if done consistently across all units tested. Even BRStv noted how accurate it was compared to some of the other more expensive units. So just comes down to how the test is done, etc.

And yes - I don't think anyone is going to change from any brand but the exercise is fun never the less and may shed some light on stuff. In any case hopefully the people doing it are having some fun.

I agree. Having used a Seneye personally. I found it easy to use and easy to position to get the same results.

Since college I have always measured stuff three times to get more accurate results. So, when I used the PAR meter (thanks @Brew12) I found it very consistent.

Although I must admit, I was disappointed in the light I built at the time. :D

 

[theKoolAidMan]

By the same token, aren't you wanting data to say you made a wise decision buying kessil?

Haha, if I bought Kessil I probably would! I do not, and never have owned any Kessil light. I've run black boxes and I've run AI, both the Hydra 26 and AI Prime.

 

[dantimdad]

Sorry, I assumed (wrongly I admit) you were specifically defending that brand.

I do apologize.

 

[theKoolAidMan]

Sorry, I assumed (wrongly I admit) you were specifically defending that brand.

I do apologize.

No problem. And please don't take my responses as being hostile in any way. I am appreciative of the OP doing these tests, but just because he's taken the initiative and fronting the time and costs doesn't mean that we shouldn't be critical of the results and methodology where warranted. This should be about getting GOOD data, not just any data. Far too many best practices in this hobby have been built around shoddy data or a small kernel of science wrapped up in layers of anecdotes. Let's be better than that!

 

[Manose]

I will take pictures today of how we hold the seneye on the reef plug in order to get consistent data and if improvements can be made lets discuss.
I think a hobbyist data can be very valuable or as valuable as any real lab data run by a scientific firm, this test is about consistency and results obtained are used for a broad range of data.

I just now question what exactly is the point of this series? Is it to test how black boxes actually perform vs how name brand lights actually perform? Or is it for black box owners to get some data, no matter how inaccurate to point to in order to convince themselves they made a wise purchasing decision? My suspicion is that it has become the latter.

That is not the case at all...
Many black box owners know how well or not well their lights work in the real world application of reef keeping. This is a hobbyist study using a device that was praised by the BRS crew and was inexpensive. The accuracy may be a little off but it will be off for all lights and give us a consistent number throughout.
I welcome all the help I can get as well as some good constructive criticism, but to say this will be a biased test and put the black box in a far superior favor then please save those comments for somewhere else please.
I started this test as to maybe give something back to this wonderful community here at R2R so maybe we can learn something from it together as a community.

 

[Ariel V Rosa]

I like the direction this thread is taking, peer review is important and so is a discussion on the methods and devices used for data collection. I hope no one really uses this thread as a tool to bash brands or become keyboard heroes not willing to do the work them selves. The O.P. is putting lots of time an effort into this and that has to carry a lot of weight.

Hopefully further contributions are more constructive than inflammatory, its easy for us to get on a high perch over the internet and use language or fling oppinions we would not dare say in a setting with actual people sitting across from us.

 

[Manose]

I like the direction this thread is taking, peer review is important and so is a discussion on the methods and devices used for data collection. I hope no one really uses this thread as a tool to bash brands or become keyboard heroes not willing to do the work them selves. The O.P. is putting lots of time an effort into this and that has to carry a lot of weight.

Hopefully further contributions are more constructive than inflammatory, its easy for us to get on a high perch over the internet and use language or fling oppinions we would not dare say in a setting with actual people sitting across from us.

I will be honest and say that I am somewhat discouraged right now, but I started this and I will finish it with all the gathered knowledge here.

 

[Porpoise Hork]

In regards to the PAR results with the Kessil, is there any modification needed to the readings the SenEYE is giving? I ask this because with the Apogee units you have to multiply the sensor results by 1.32 to get the correct PAR reading. I wonder if something similar needs to be done here.

 

[Manose]

In regards to the PAR results with the Kessil, is there any modification needed to the readings the SenEYE is giving? I ask this because with the Apogee units you have to multiply the sensor results by 1.32 to get the correct PAR reading. I wonder if something similar needs to be done here.

That is a great question and I would love to know the answer.

 

[Johniejumbo]

Please don’t be discouraged! I’ve been following this for a while now and am really enjoying this project. It’s been my favorite thread! Keep up the good work. :)

 

[theKoolAidMan]

I will take pictures today of how we hold the seneye on the reef plug in order to get consistent data and if improvements can be made lets discuss.
I think a hobbyist data can be very valuable or as valuable as any real lab data run by a scientific firm, this test is about consistency and results obtained are used for a broad range of data.


That is not the case at all...
Many black box owners know how well or not well their lights work in the real world application of reef keeping. This is a hobbyist study using a device that was praised by the BRS crew and was inexpensive. The accuracy may be a little off but it will be off for all lights and give us a consistent number throughout.
I welcome all the help I can get as well as some good constructive criticism, but to say this will be a biased test and put the black box in a far superior favor then please save those comments for somewhere else please.
I started this test as to maybe give something back to this wonderful community here at R2R so maybe we can learn something from it together as a community.

My suggestion that there were ulterior motives in the tests was unfounded and disrespectful. Please accept my apology. I honestly do believe you're doing this for the right reasons and should not have said that.

 

[theKoolAidMan]

I will be honest and say that I am somewhat discouraged right now, but I started this and I will finish it with all the gathered knowledge here.

Please accept my apology if my comments about the motives for the test are what have discouraged you. That was uncalled for on my part.

 

[Manose]

Please accept my apology if my comments about the motives for the test are what have discouraged you. That was uncalled for on my part.

Do not worry as lets work together as to make this a really good study.

 

[theKoolAidMan]

Do not worry as lets work together as to make this a really good study.

Thank you for being gracious. I fear that my off the cuff quip made in bad judgement will cloud some good points numerous posters have been bringing up about the Kessil results.

 

[Manose]

I am here now as I stayed in a hotel as to avoid driving home late at night. I have the Noobsyche light hooked up and waiting to get some more readings. I will take pics of how we use the wand and hook the meter to the frag plugs so we can keep it the same level each time.

 

[naterealbig]

You didn't at all. Listen, if we all want to play scientist here...then you go the whole hog. We need to be able to have these open and frank discussions. There's a reason real scientific papers are peer reviewed.

I agree 100% @theKoolAidMan. If there is a large margin of error already, then it will push to the resulting data sets for all tested lights - there is no way around it. Let me play devil's advocate here though: what if BRS test results are incorrect?. There will always be a margin of error that is exacerbated (or extrapolated) by using different test processes and equipment; the question is, what margin are we willing to accept?.

To what seems to be the original point of this thread: accuracy and precision is a must. Without accuracy, sure, the lights can be tested in relation to one-another, but this does not answer the original question - what is the efficacy of the Chinese Black Boxes on our tanks. How they relate to one another tells us nothing without an accurate baseline.

Meh to the accusations regarding BRS test results surrounding financial gain. To those touting this, have you actually watched the videos in their entirety?. In many videos, Ryan bluntly suggests that several of the products they sell (lighting, GFO, etc) are not necessary, or are only necessary under certain circumstances. He also routinely challenges vendors and OEMs of the products BRS sells.

In the Kessil video he does admit that they tested the new light differently than the old. The difference was the spectrum they used in the latest test was comparable to what we would see in your average reef tank. So, the results of this test are much closer to what we would see when the lights are actually being used. This data is much more valuable to me, as I would postulate that 0.0% of reef keepers use all LED channels at 100%.

 

[Manose]

Thank you for being gracious. I fear that my off the cuff quip made in bad judgement will cloud some good points numerous posters have been bringing up about the Kessil results.

No way as Kessil 360x is outstanding in my opinion as I noticed the huge area this light covers and covers very well. I am going test area on the light later on with a 48in x 24in frag tank.

 

[theKoolAidMan]

I agree 100% @theKoolAidMan. If there is a large margin of error already, then it will push to the resulting data sets for all tested lights - there is no way around it. Let me play devil's advocate here though: what if BRS test results are incorrect?. There will always be a margin of error that is exacerbated (or extrapolated) by using different test processes and equipment; the question is, what margin are we willing to accept?.

To what seems to be the original point of this thread: accuracy and precision is a must. Without accuracy, sure, the lights can be tested in relation to one-another, but this does not answer the original question - what is the efficacy of the Chinese Black Boxes on our tanks. How they relate to one another tells us nothing without an accurate baseline.

Meh to the accusations regarding BRS test results surrounding financial gain. To those touting this, have you actually watched the videos in their entirety?. In many videos, Ryan bluntly suggests that several of the products they sell (lighting, GFO, etc) are not necessary, or are only necessary under certain circumstances. He also routinely challenges vendors and OEMs of the products BRS sells.

In the Kessil video he does admit that they tested the new light differently than the old. The difference was the spectrum they used in the latest test was comparable to what we would see in your average reef tank. So, the results of this test are much closer to what we would see when the lights are actually being used. This data is much more valuable to me, as I would postulate that 0.0% of reef keepers use all LED channels at 100%.

I would also add that Ryan has openly stated on camera on multiple occasions that they do not carry certain brands or certain products from certain brands due to poor reviews or them themselves not finding the product to be effective. He has said in the past that if a product starts getting poor reviews, they look into it, and if they find those reviews to be justified, they pull the product and refuse to sell it.

 

[Manose]

I agree 100% @theKoolAidMan. If there is a large margin of error already, then it will push to the resulting data sets for all tested lights - there is no way around it. Let me play devil's advocate here though: what if BRS test results are incorrect?. There will always be a margin of error that is exacerbated (or extrapolated) by using different test processes and equipment; the question is, what margin are we willing to accept?.

To what seems to be the original point of this thread: accuracy and precision is a must. Without accuracy, sure, the lights can be tested in relation to one-another, but this does not answer the original question - what is the efficacy of the Chinese Black Boxes on our tanks. How they relate to one another tells us nothing without an accurate baseline.

Meh to the accusations regarding BRS test results surrounding financial gain. To those touting this, have you actually watched the videos in their entirety?. In many videos, Ryan bluntly suggests that several of the products they sell (lighting, GFO, etc) are not necessary, or are only necessary under certain circumstances. He also routinely challenges vendors and OEMs of the products BRS sells.

In the Kessil video he does admit that they tested the new light differently than the old. The difference was the spectrum they used in the latest test was comparable to what we would see in your average reef tank. So, the results of this test are much closer to what we would see when the lights are actually being used. This data is much more valuable to me, as I would postulate that 0.0% of reef keepers use all uld findLED channels at 100%.

This is not about BRS as this is about what we can do to make this study better. I will say that BRS did inadvertently create a bad stereotype when they chose the cheapest made Chinese light they could find to do their study with. We have all seen the comments people make when you say you have a "Black Box" light. "OMG dude your house is going to burn down" NOT all lights made in China are crap and we need to use this study to see what is good or maybe good with more advanced testing. To do this we need to use some lights that WE KNOW will perform such as the Kessil 360x,Hydra 26,and Radion xr 15pro gen 4 to get some numbers to go by as to formulate which of the BB lights might be worth a better look.
BRS has done some amazing things for this hobby and I support them 1000% and will always appreciate what they do for us.