Polo reef disaster . . . OMG

[areefer01]

I am questioning whats been said, because its not adding up. So far, what weve been told is that 2 independent valves spontaneously malfunctioned. That doesnt add up. I dont understand why questioning it is so wrong?

It doesn't matter if it doesn't add up to us, it is not our investigation. They will release what they do which may, or may not, answer questions.

On the other hand you do know that investigations take time, right?

Also, for the record, I didnt accuse you, or anyone else of calling me a villian. I simply said (and still will argue), that villianizing asking "why" is vapid. Im not aiming that at you, or anyone else specifically. No offense intended. Im just saying calling foul for asking why isn't where it's at.

I am the one who asked the question to you. It is perfectly fine to direct it to me. I'm not offended. Yes, you can ask. I am zeroed in on 'doesn't add up' verbiage. Hindsight being 20/20 I maybe should have kept my thought to myself.

Any your simplifying it a bit much, from what can gather. Two independent valves simultaneously spontaneously failed. And closed loop weas connected to sand filter that was connect to RODI reservoir that overflowed. Anyway, Im just saying Im curious to see what comes of it.

I am not simplifying anything - I am not in any position to speculate what happened nor do I care to. No one has evidence to say something isn't adding up is my point. They are still assessing. Give them time.

 

[jmaneyapanda]

It doesn't matter if it doesn't add up to us, it is not our investigation. They will release what they do which may, or may not, answer questions.

On the other hand you do know that investigations take time, right?



I am the one who asked the question to you. It is perfectly fine to direct it to me. I'm not offended. Yes, you can ask. I am zeroed in on 'doesn't add up' verbiage. Hindsight being 20/20 I maybe should have kept my thought to myself.



I am not simplifying anything - I am not in any position to speculate what happened nor do I care to. No one has evidence to say something isn't adding up is my point. They are still assessing. Give them time.

?? If it doesnt matter, why bother posting it? Why bother making multiple videos? It, in fact, DOES matter, because people can learn from it. Whether it be check your redundancies, or make sure your wifi works, or even Dont plumb your closed loop to an open container near an unmonitored drain, we ALL can learn. Where did I say I need answer now? And Time isnt permitted? I said I look forward to teh tarnparency. Thats it.

Youre not the only one who has stated that (unless youve said it twice to me). You can imply my "it doesnt add up" as being nefarious, but its bluntly not. Its saying that what we see at gross appearance in not easily comprehendable. Again, two independent valves catastrophically failed at the same time? Or, even if not at the same time, then unnoticed on debatably the most staffed aquarium around? It. Doesnt. Add. Up. That statemnt doesnt blame, accuse, or denegrate anyone. It simply says "this is unexplained thus far".

Again, how am I not giving anyone and everyone time? I find your statements in this post to be conflicting. You say "It doesnt matter", and then "Give them time". I clearly understand that it doesnt matter TO YOU. But this is a reefing forum, talking about reef tanks, to help other reefers be better. And a post was made publicly by a tank owner about a failure, and it was posted here. Why is that different, better or worse, than any other reefer saying "I had a catastrophe", and the forum asking, why? And trying to figure it out. To be better reefers.

 

[areefer01]

Again, how am I not giving anyone and everyone time? I find your statements in this post to be conflicting. You say "It doesnt matter", and then "Give them time". I clearly understand that it doesnt matter TO YOU. But this is a reefing forum, talking about reef tanks, to help other reefers be better. And a post was made publicly by a tank owner about a failure, and it was posted here. Why is that different, better or worse, than any other reefer saying "I had a catastrophe", and the forum asking, why? And trying to figure it out. To be better reefers.

There is no conflict. I said I am not in any position to speculate what happened nor do I care to. You said it doesn't add up - that is implying evidence is missing or suspicious. I won't include lying. Does it not?

We are at an impasse and mudding up the thread. You are focused on the end result to help others and did not imply further. I'll bow out.

 

[jmaneyapanda]

There is no conflict. I said I am not in any position to speculate what happened nor do I care to. You said it doesn't add up - that is implying evidence is missing or suspicious. I won't include lying. Does it not?

We are at an impasse and mudding up the thread. You are focused on the end result to help others and did not imply further. I'll bow out.

Evidence IS missing!! From Andrews statements, as well as yours! They are still investigating, remember? What weve been told thus far DOES NOT ADD UP. I dont understand why youre implying a nefarious rationale about that. It's pretty clear both you and Andrew feel the same.

Honestly, it feels like you are just concerned Im attacking Andrew for hiding things. Ive been very careful with my words to not do so. There is nothing Ive said that is an attack on him or his staff. Just a statement of {sic} "thats too weird to have just happened".

Mudding up the thread? LOL what? Were talking about the exact thing the threads about. But OK. Thanks for discussing. No harm no foul, and no offense taken, nor given.

 

[JayM]

I am questioning whats been said, because its not adding up. So far, what weve been told is that 2 independent valves spontaneously malfunctioned. That doesnt add up. I dont understand why questioning it is so wrong? Is Andrew above questions? Criticisms? Could everyoen else here learn from it? Could not being transparent preserve his prestige and fame? Could not being transparent protect someones reputation? All these things might be true or not. Andrew publicly posting and illuminating this issue doesnt prohibit these questions from being asked. I *assume* you're implying Im being negative by asking and by stating the thinsg said dont add up. Believe me, Im not. I could say far worse things, which I wont. All Im saying is, if this is going to be discussed AT ALL, which it is, then we shoudl really discuss it. If you think its not proper, and that we should all just give our performative "I wish the best" and sing kumbayah, we'll disagree. Because then this forum has lost its educational value.

Also, for the record, I didnt accuse you, or anyone else of calling me a villian. I simply said (and still will argue), that villianizing asking "why" is vapid. Im not aiming that at you, or anyone else specifically. No offense intended. Im just saying calling foul for asking why isn't where it's at.



He, himself, even said they are still looking into it. Any your simplifying it a bit much, from what can gather. Two independent valves simultaneously spontaneously failed. And closed loop weas connected to sand filter that was connect to RODI reservoir that overflowed. Anyway, Im just saying Im curious to see what comes of it.

Where did he say that "that 2 independent valves spontaneously (and simultaneously) malfunctioned"?

The first valve isolates the second from the system. The first could have failed some time ago and not been noticed because of the second redundant valve. Then the second valve then fails creating havoc.

And maybe they did fail together, maybe it's another issue altogether. We don't know because he's still trying to figure it out definitively, and has stated such.

It's his tank and his money. He doesn't owe anyone any answers, nor any transparency. I'm sure his prestige and fame aren't factoring in to any of what's going on at the moment. Appreciate the info he is providing and just accept it as such.

 

[IceNein]

Where did he say that "that 2 independent valves spontaneously (and simultaneously) malfunctioned"?

The first valve isolates the second from the system. The first could have failed some time ago and not been noticed because of the second redundant valve. Then the second valve then fails creating havoc.

And maybe they did fail together, maybe it's another issue altogether. We don't know because he's still trying to figure it out definitively, and has stated such.

It's his tank and his money. He doesn't owe anyone any answers, nor any transparency. I'm sure his prestige and fame aren't factoring in to any of what's going on at the moment. Appreciate the info he is providing and just accept it as such.

Another possibility is that the manual valve was out of position, so the automatic valve failing alone caused the problem. That's the problem with complex systems, it's easy to accidentally have a valve out of alignment when there's redundancies.

In the Navy we had pre-start up valve lineups where people throughout the steam plant went around and manually verified that all the valves were in the correct position. We had those exactly because it's possible that someone was doing maintenance and made a mistake and forgot to put a redundant valve back into the correct position.

Navy ships have operating procedures that are created after failures to prevent them in the future. There is no operating manual for a complex privately operated reef tank. Everyone feels like they "have control" until circumstances force them to realize they do not.

I haven't created an operating manual for my tank, which is obviously a lot less complex. But how many pieces of equipment would I add before I decided that I need a consolidated operating manual with start up, shut down, water change, and various maintenance procedures? At what point would I start laminating check lists?

My gut feeling is that I would add systems little by little, feeling like I was in control, until some disaster happened. Then I would modify my procedures. How many of us have done tank maintenance, turned off equipment, and then only realized 12 hours later that we had forgotten to turn something back on when we restored the system?

This is all to say that people who are getting so in the weeds and criticizing them for their failure don't really have the ability to self reflect onto how much of their criticism applies to them?

When was the last time you preventatively replaced a gasket in your system? When was the last time you repacked a valve?

 

[BZOFIQ]

This just makes my point stronger about not using WiFi around any life support devices.

No Ethernet - NO GO!

 

[cba191]

Sure he can rebuy coral. But this is something that breaks your spirit. I don’t care how rich someone is or how tough someone acts. This hurts. He is connected to his tank, just as we are with ours.

His wife isn’t asking for coral donations to save money. She wants to cheer him up and to show him he’s got a community of people that care and that want to support him through a hard time. People that want to give back to someone that’s given so much. There is a difference between a coral bought and a coral gifted. Only one comes with meaning. Andrew needs to heal and rebuild just like his reef does.

I would be honored to give a part of my reef, to help rebuild his. Polo Reef + Team, I will be in touch. When you are stable and the sands have settled, I have a homegrown 50+ head Duncan colony I’ll send your way

I think it would be kind of cool knowing that I contributed something to the coolest tank ever!

 

[jmaneyapanda]

Another possibility is that the manual valve was out of position, so the automatic valve failing alone caused the problem. That's the problem with complex systems, it's easy to accidentally have a valve out of alignment when there's redundancies.

In the Navy we had pre-start up valve lineups where people throughout the steam plant went around and manually verified that all the valves were in the correct position. We had those exactly because it's possible that someone was doing maintenance and made a mistake and forgot to put a redundant valve back into the correct position.

Navy ships have operating procedures that are created after failures to prevent them in the future. There is no operating manual for a complex privately operated reef tank. Everyone feels like they "have control" until circumstances force them to realize they do not.

I haven't created an operating manual for my tank, which is obviously a lot less complex. But how many pieces of equipment would I add before I decided that I need a consolidated operating manual with start up, shut down, water change, and various maintenance procedures? At what point would I start laminating check lists?

My gut feeling is that I would add systems little by little, feeling like I was in control, until some disaster happened. Then I would modify my procedures. How many of us have done tank maintenance, turned off equipment, and then only realized 12 hours later that we had forgotten to turn something back on when we restored the system?

This is all to say that people who are getting so in the weeds and criticizing them for their failure don't really have the ability to self reflect onto how much of their criticism applies to them?

When was the last time you preventatively replaced a gasket in your system? When was the last time you repacked a valve?

Yes I dont disagree. However....This aquarium was staffed by multiple people constantly. This type of oversight would be a problem in and of itself.

I had a reef tank in the early 2000's that had six MP40s and 4 battery backups (this was before mp60s existed and the battery backups were made by Icecap and not Ecotech themselves). I had occasionally tested the batteries to make sure theyd work, but got lazy about it. Sure enough, on Christmas day, I lost power, five of the six MP40s were in stall off the batteries, and lost a pair of Conspic Angels, an trio of Meredithi angels, A wrought Irton butterfly, and interruptus angel, a declivis butterfly, A pair of crosshatch triggers (that were 4", at that!), amongst other fish. I learned very quickly that safeguards only safe guard when they work. And learned a terribly painful lesson that day. I was mad at ecotech for the non functioning of the equipment, but ultimately blamed myslef for not testing as much.

That story just goes to suggest that if people are hired for operational care and upkeep, and a manual valve that keeps the aquarium from a RODI reservoir and open drain are left unchecked and unmonitored to where it could been open all along, some heads should roll.

 

[jmaneyapanda]

Where did he say that "that 2 independent valves spontaneously (and simultaneously) malfunctioned"?

The first valve isolates the second from the system. The first could have failed some time ago and not been noticed because of the second redundant valve. Then the second valve then fails creating havoc.

And maybe they did fail together, maybe it's another issue altogether. We don't know because he's still trying to figure it out definitively, and has stated such.

It's his tank and his money. He doesn't owe anyone any answers, nor any transparency. I'm sure his prestige and fame aren't factoring in to any of what's going on at the moment. Appreciate the info he is providing and just accept it as such.

This is true. But then there is even greater negiligence to allow a failed valve to just go unnoticed and unmonitored ina line that connect the RODI reservoir (and an unmonitored drain) to the closed loop fo the tank.

But look, if he doesnt want to have any transparency, which youre right, he doesnt owe anyone, then dont post now MULTIPLE videos about it. His wife shouldnt post videos soliciting donations. Just dont involve us. He chsoe to do that. In moderate detail, promising more detail. Why is so wrong to be curious to see what he's promising to show us?

Why is every white knighting him so much? Im not attacking him nor his staff. Im just curious the detail to hsi HIGHLY publicized reef failure (the publicity at his own hands).

 

[Jimbo327]

He's got a lot going on. Lots of equipments, filters, pumps, etc. Most likely, they were using automated valves (that can be opened/closed via control by their software) instead of mechanical check valves. Under normal operation, it would stay open, and when there is a problem...it would close. If that automation fails or the valve doesn't close on signal...then it'll back siphon down with gravity and down the drain if there is open loop (not closed loop). It doesn't take long to drain 17k gal water. A 6" pipe can gravity drain 17k gal water is less than an hour if fully open.

Again, this is all speculation. Andrew has much better automation and equipment than most of us. He'll figure it out soon, build redundacy and be back in business. **** happens, it's reefing, all part of the hobby.

 

[fish_collector]

I could do without the bickering, just sayin.

Andrew's tank suffered an "event" that he and his team are investigating, when they determine what went wrong they will correct it. Hopefully for all of us who are interested in what went wrong, he will publish a video and show us what they found. Until then this is all just speculation from the little bit of info he has divulged to us. His description of 2 valves in series that connect to the RODI tank may be grossly oversimplified, it was obvious someone off camera was assisting him with the information he was sharing. It is likely there is a lot more to it than he has told us. His wifi that quit working on the same night as his tank crash, well then there's that too. Give them some space and I'm sure we'll get some answers soon enough.

 

[mfinn]

I could do without the bickering, just sayin.

Andrew's tank suffered an "event" that he and his team are investigating, when they determine what went wrong they will correct it. Hopefully for all of us who are interested in what went wrong, he will publish a video and show us what they found. Until then this is all just speculation from the little bit of info he has divulged to us. His description of 2 valves in series that connect to the RODI tank may be grossly oversimplified, it was obvious someone off camera was assisting him with the information he was sharing. It is likely there is a lot more to it than he has told us. His wifi that quit working on the same night as his tank crash, well then there's that too. Give them some space and I'm sure we'll get some answers soon enough.

I agree.

 

[Reefing Madness]

Not doing this guys, how bout we all take a breath here......Thats not how we treat our members!! You don't have to like how its being layed down, but we are not here to argue about it

 

[ReefED!]

Such awful news

I know how I feel when a prized fish or two gets sick and dies - or some corals die

Can’t imagine that experienced amplified like this

 

[jmaneyapanda]

.

 

[littlebeard]

Full story.

 

[fish_collector]

I was impressed that his team kept their cool and that each team member knew the system well enough to be able to make decisions quickly.

 

[SaltyBlue]

Was sad to see this. Andrew was on the ReefBeef where Sanjay had just lost his system and I’m sure he took a lot away from that - such as Sanjay saying he wanted to observe how the system recovers vs a hard reboot.

The team had just done a good job responding to a leak and electrical drama / generator issues in the days prior.

Didn’t see it mentioned if his skynet controller had a heartbeat monitor but it’s something I’m sure they’ll look in to.