Disaster recovery/ rebuild thread.

[Ann Enomy]

I just wanted to Start a thread on My re-build recovery efforts as I have had some great advise and would like to share the process as it is happening. Hopefully others can learn from my mistakes and avoid some of the heartache I have suffered.

4/21/23 19:22 GFCI tripped from faulty cooling fan. Ecotech Battery backup did not work. Apex did not notify me of power failure. Tank is at my office. Arrived at work 4/22/23 at 8:00 to almost total loss. Tank temp was 68˚. I removed all the bodies and only had anemones, Leather corals and sinularia left alive in the tank. I want to unpack this a bit.

The faulty fan was this one: cooling fan It was only 2 years old. This was mounted over my sump and was not wet but did have a bit of salt creep on it. Once removed from the tank I plugged it into a normal outlet and was shocked. Dumb move on my part. If you isolate something that is tripping a GFCI, Don't assume its the outlets fault!

Ecotech battery backup failure: I purchased in August of 2019 and set it and forgot it. Please Please Please test the health of this battery! The process is simply unplugging the MP40 controller for power and unplugging the battery backup from power. The display on the controller should tell you how much power is left in the battery. I will add pictures of this and My battery rebuild in later posts.

Apex Failure: 100% user error! I had an Apex classic that was hardwired in to our server that was also on a battery backup as well as our router but then in 2020 I updated to the new stile apex to add some additional modules and we also around that time moved to cloud based server so no more hardwire anything. I copy/pasted most of the programing so before a (if Ebb86 off than on) statement would alert me of any power failure, it no longer worked. Never knew about heartbeat. Big oops on my part. There are so many resources on APEX but I was cocky as long term control freak. Lesson learned.

More to come:

 

[Stephen Glover]

Definately some tough lessons to learn. The important thing is to come back better than before. Thanks for sharing.

 

[Gumbies R Us]

Sorry that had to happen to you! Best of luck on the rebuild!

 

[Ann Enomy]

Days 1-5: Continued Death and destruction.

So the fish died overnight but the corals looked as though they would make it. This is a tank pic from 2 days later.

At day 3 I started to notice some areas of necrosis on the sinularia colonies:

By day 4 those became much larger areas and I made the decision to toss it. as I was removing them I felt a few tips that were still solid and am trying to save some frags. If I had to do it over again I would have fragged out above the necrosis right away as it was apparent (when removing it) the dying tissue was killing more tissue.

 

[Ann Enomy]

Recovery effort:
Everyday I have been doing 20% WC's. Moving all the rocks around. Using a powerhead to blow off all the rocks and the ceramic media/chambers in the sump and testing water quality .

Interesting water parameters. Up until April 29th I was reading well within normal nitrates and phosphates. Water testing on May 1st showed 1.3 ppm phosphate an 10.2 nitrates. May 2nd is now showing 2.2 ppm phosphate and 18.8 nitrates. My guess is that there was an ammonia spike that I do not have a test kit for. Phosphates are probably from all the biomass that I am stirring up.

Here is the tank on 5/1/23:

Already looking better but not ready for fish yet. Still need to repair battery and continue with the post RIP cleaning until numbers are normal.

 

[Ann Enomy]

Starting to get very impatient. 20% water changes daily are still happening and still vacuuming and using powerhead to blow off rock. Currently nitrates ate at 26.0 and Phosphate at 2.1. Things are getting worse rather than better. GHA starting to take hold. At least the nems and leathers still look good and with my rock moving I saw my peppermint shrimp, and 3 emerald crabs!

 

[Ann Enomy]

Rebuilt the battery backup today.

this is the battery I used: 12 volt 20 amp hour battery

process was simple, drill out rivets with 3/32 drill, replace battery, use some #6 3/8” pan head screws and your done.

Whole process cost me $40ish makes me feel stupid for paying $200.

will be replacing this battery at least every other year now. Setting up reminder now.

does anyone “exercise “ there batteries by turning power off to backup and mp40’s for a hour or 2 a night? Just a thought.

 

[Fishy Jacket]

Rebuilt the battery backup today.

this is the battery I used: 12 volt 20 amp hour battery

process was simple, drill out rivets with 3/32 drill, replace battery, use some #6 3/8” pan head screws and your done.

Whole process cost me $40ish makes me feel stupid for paying $200.

will be replacing this battery at least every other year now. Setting up reminder now.

does anyone “exercise “ there batteries by turning power off to backup and mp40’s for a hour or 2 a night? Just a thought.

So I guess you are sayin' the metal box, the little bit of wire, and the logo were not worth $160?!
Joking aside, sorry for your loss and thanks for sharing the situation and steps you are going through to recover. Very informative read and a good reminder to us all to check, check, and check again. Both on systems and assumptions.

 

[Ann Enomy]

So I guess you are sayin' the metal box, the little bit of wire, and the logo were not worth $160?!
Joking aside, sorry for your loss and thanks for sharing the situation and steps you are going through to recover. Very informative read and a good reminder to us all to check, check, and check again. Both on systems and assumptions.

For that price I would have gotten a deep cycle marine battery that could run the whole thing for a week! but I have a shiny metal box, might as well use.

 

[Ann Enomy]

When vacuuming and blowing off rocks during the water changes I am no longer kicking much of anything up so I thought now is a good time to rework the aquascape. I tried to create more overhangs and swim throughs. I used a 1/2 masonary drill bit and Ceramic rods and epoxy to hold together. Unfortunately you can see I'm starting to get a bit of GHA.

 

[Ann Enomy]

Did my testing today. Nitrates are down to 10.6ppm and Phosphates are down to 0.6. I think I'm ready for fish!

Question: I am thinking about introducing the Clownfish harem first as my LFS has just got in a shipment of about 200 juveniles that are about 1/2 in each. Figured smallest fish first. Do you think these are the best bet? Also want to eventually add cardinal fish, chromis and firefish as other "peaceful" fish and am worried that clowns will turn into bullies.

 

[Ann Enomy]

I got clowns! 12 little guys. Already hosting. Mood improving

Hope the video works. never inserted one before

 

[Ann Enomy]

I have a general question. I'm kind of going nuts with redundancy and alarms now but after mopping up a water spill when the handle on my five gallon bucket broke during a water change it made think "what is the life span of a typical fish tank?"

My tank is an "all glass" 120 gal tank that I drilled on the back for a shadow overflow. It has been in operation for about 10 years and I do have MP40s on it but never have done a "standing wave".

Does anyone just replace there tank in precaution or is it always a a reaction to a failure?

I know often it's start with a 40gal and 10 years and 4 tanks later end with a 300gal but I'm happy where I am at 120gal.