reef-pi :: An opensource reef tank controller based on Raspberry Pi.

[Ranjib]

Over my last 25 years of having my Reef tank (or tanks), literally everything has failed a minimum of once over time. Some items like heaters and pumps have failed many more than one time. ATO floats have failed. I even had a tank fail once (which ensued in a flood). When you add yet another layer and introduce automation to a tank, you expect that its going to fail at some point.

My belt and suspenders system is like this;

1. My Reef-Pi sends me an email twice per day to just say Hi. I have this setup in the scheduler. That way I know things are running.
2. The most critical item in any tank is temperature. For this reason, I have a backup heater and stand alone thermostat with sensor in the tank. It is set to 75 Degrees. Tank is set to 78 and typically swings -1 to +.5, so in theory, the back up heater never comes on. (this did save my *** last year when my home heating plant went down; Two heaters were enough to keep the tank warm).
3. I have a full home automaton system. Via the API, my system pings the Reef-Pi every half hour. If it gets no response, Both my wife and I get a text and the home automation panels come up with an alert.

So like I said above, the more you add, the higher risk of failure. As good as Reef-Pi is, you are still relying on software and a home built system on a project board. Until this becomes embedded software on a physical circuit board, failures will happen and at a higher level.

I would encourage everyone to do a risk analysis and add in safety features like I have done. And look at the bright side - You never really knew what was going on with your tank when not at home. Reef-Pi has brought the view of your tank to your cell phone 24/7. That is a huge jump from where we all were prior.

Now that’s what I call a reliable operations. Thank you so much for sharing your strategy.
I have thought of a summary system, where reef-pi will send a text message at periodic intervals with summary of tank parameters

 

[pickupman66]

Over my last 25 years of having my Reef tank (or tanks), literally everything has failed a minimum of once over time. Some items like heaters and pumps have failed many more than one time. ATO floats have failed. I even had a tank fail once (which ensued in a flood). When you add yet another layer and introduce automation to a tank, you expect that its going to fail at some point.

My belt and suspenders system is like this;

1. My Reef-Pi sends me an email twice per day to just say Hi. I have this setup in the scheduler. That way I know things are running.
2. The most critical item in any tank is temperature. For this reason, I have a backup heater and stand alone thermostat with sensor in the tank. It is set to 75 Degrees. Tank is set to 78 and typically swings -1 to +.5, so in theory, the back up heater never comes on. (this did save my *** last year when my home heating plant went down; Two heaters were enough to keep the tank warm).
3. I have a full home automaton system. Via the API, my system pings the Reef-Pi every half hour. If it gets no response, Both my wife and I get a text and the home automation panels come up with an alert.

So like I said above, the more you add, the higher risk of failure. As good as Reef-Pi is, you are still relying on software and a home built system on a project board. Until this becomes embedded software on a physical circuit board, failures will happen and at a higher level.

I would encourage everyone to do a risk analysis and add in safety features like I have done. And look at the bright side - You never really knew what was going on with your tank when not at home. Reef-Pi has brought the view of your tank to your cell phone 24/7. That is a huge jump from where we all were prior.

Thanks for the information. I have been reefing for about 18 years and have had controllers for the last 8. started with a DA reefkeeper elite. I then went to the Archon as a beta tester. it was a very solid unit overall and i saw the potential, but the company didnt follow through and they closed. I went to reef-pi after a failure of it. I still have it but had turned of the notification of temp for some reason.

My favorite temp control was the medusa I used to have. that thing was solid and never failed me. i just wanted internets.

I have since re-added the Archon temp notification here and will setup a schedule of the reefpi to say hello twice a day. that is a great idea. Can you elaborate how you have done that?

 

[theatrus]

Now that’s what I call a reliable operations. Thank you so much for sharing your strategy.
I have thought of a summary system, where reef-pi will send a text message at periodic intervals with summary of tank parameters

This would be fantastic. Bonus points if it can rasterize the dashboard charts (= dependencies )

 

[bishoptf]

This would be fantastic. Bonus points if it can rasterize the dashboard charts (= dependencies )

I was thinking about something but on my simple level of a cron job (how often is up to you) tailing the last 50 to 100 lines from syslog and sending that via email, reef-pi is pretty chatty so should be a pretty good summary, maybe make it 200 lines..what I was thinking for health check in, quick and dirty but useful.

 

[Zekth]

I think it’s ok to submit suggestions as issues , we (the dev team) can triage them weekly and remove duplicates, bucket them etc. thoughts @Zekth ?

Issues is easier to track for everyone. Ranjib is the one who monitors the most this thread but me and michael are most of the time checking github.

 

[b4tn]

So all this talk about failures. I looked at my dashboard and my historical CPU usage had been reset along with historical ATO and heater usage. My dosers missed the normal dose at 11:05 and dosed at 11:22 instead. I looked at aadafruit IO and noticed a dip in my lights at 10:44 and also noticed my CPU usage dropped to zero around 10:17. So I went and checked the logs and something strange happened. I am not home so it may have been a power outage and all appears to be running on the screen lol I will know more when I get home tonight. But I looked through the logs and found some strange stuff. Looks like at 10:17 reef-pi rebooted for some reason? I am not sure about the TLS handshake error that coincides with my lighting dip in adafruit IO but I am assuming maybe a communication error with IO? I dont recognize the IP address but I x'd out the last two octets just in case.

Feb 12 10:43:57 myReef reef-pi[308]: 2019/02/12 10:43:57 http: TLS handshake error from 143.85.X.X:56111: EOF

Feb 12 10:43:57 myReef reef-pi[308]: 2019/02/12 10:43:57 http: TLS handshake error from 143.85.X.X:52124: EOF

Feb 12 10:43:58 myReef reef-pi[308]: 2019/02/12 10:43:58 http: TLS handshake error from 143.85.X.X:59768: EOF

Feb 12 10:43:58 myReef reef-pi[308]: 2019/02/12 10:43:58 http: TLS handshake error from 143.85.X.X:52107: EOF


Feb 12 10:17:03 myReef fake-hwclock[84]: Tue Feb 12 15:17:01 UTC 2019

Feb 12 10:17:03 myReef systemd[1]: Started Create Static Device Nodes in /dev.

Feb 12 10:17:03 myReef systemd[1]: Starting udev Kernel Device Manager...

Feb 12 10:17:03 myReef systemd-fsck[118]: e2fsck 1.43.4 (31-Jan-2017)

Feb 12 10:17:03 myReef systemd-fsck[118]: rootfs: clean, 129237/1926848 files, 1373258/7779456 blocks

Feb 12 10:17:03 myReef systemd[1]: Started File System Check on Root Device.

Feb 12 10:17:03 myReef systemd[1]: Starting Remount Root and Kernel File Systems...

Feb 12 10:17:03 myReef systemd[1]: Started udev Kernel Device Manager.

Feb 12 10:17:03 myReef systemd[1]: Started Remount Root and Kernel File Systems.

Feb 12 10:17:03 myReef systemd[1]: Starting Load/Save Random Seed...

Feb 12 10:17:03 myReef systemd[1]: Starting Flush Journal to Persistent Storage...

Feb 12 10:17:03 myReef systemd[1]: Starting udev Coldplug all Devices...

Feb 12 10:17:03 myReef systemd[1]: Started Load/Save Random Seed.

Feb 12 10:17:03 myReef systemd[1]: Started Flush Journal to Persistent Storage.

Feb 12 10:17:03 myReef systemd[1]: Started Set the console keyboard layout.

Feb 12 10:17:03 myReef kernel: [ 0.000000] Booting Linux on physical CPU 0x0

 

[pickupman66]

I have thought of a summary system, where reef-pi will send a text message at periodic intervals with summary of tank parameters

that would be pretty cool. be able to setup the # of seconds between the notification. list box of adds to which in or out and the state at the time of the interval. Much like that Filter from Finding Nemo.

the time is 3pm. Tank temperature is 77.5 ° heater is off. ATO is on.

 

[daveCB]

Thanks for the information. I have been reefing for about 18 years and have had controllers for the last 8. started with a DA reefkeeper elite. I then went to the Archon as a beta tester. it was a very solid unit overall and i saw the potential, but the company didnt follow through and they closed. I went to reef-pi after a failure of it. I still have it but had turned of the notification of temp for some reason.

My favorite temp control was the medusa I used to have. that thing was solid and never failed me. i just wanted internets.

I have since re-added the Archon temp notification here and will setup a schedule of the reefpi to say hello twice a day. that is a great idea. Can you elaborate how you have done that?

I just did a simple reminder timer that fires every 12 hours. If I could figure out the timer formatting, I probably could set it up better.

 

[daveCB]

Ranjib - lets talk 3.0 - How are you planing on implementing the ability to connect to other Pi's on your network? Via macros? Will you them have the ability to combine I/O's from other Pi's onto the main dashboard?

My builds will end up something like this; Working on Lighting now.

Reef-Pi - Power (mostly pumps, heat and cool), Wavemaker, ATO, Temperature including heating and cooling control, General messaging
Light-Pi - Lights including main lighting, refug lighting and moon lighting
Dose-Pi - PH and dosing

I would love to get control of all of this all under one controller/dashboard. I have other ideas that I will post over at github.

 

[bishoptf]

Ranjib - lets talk 3.0 - How are you planing on implementing the ability to connect to other Pi's on your network? Via macros? Will you them have the ability to combine I/O's from other Pi's onto the main dashboard?

My builds will end up something like this; Working on Lighting now.

Reef-Pi - Power (mostly pumps, heat and cool), Wavemaker, ATO, Temperature including heating and cooling control, General messaging
Light-Pi - Lights including main lighting, refug lighting and moon lighting
Dose-Pi - PH and dosing

I would love to get control of all of this all under one controller/dashboard. I have other ideas that I will post over at github.

Basically a centralized configuration and management system, aka a single pane of glass. When done right it's really nice, I will also say troubleshooting *cough* can be quite fun. Basically the common configuration bits would only have to be done once and shared among all the minions.

Would make for a modular like approach.

 

[KevinDodge]

Feedbacks like these are most welcome. Theres no bad word or disrespectful elements in it, and thats all we worry about. Calling out Glitches and possible improvements are always a good thing and useful for other members , thank you for sharing your feedback.

The pwm feature in dosing pump allows us to control the speed of dosing pumps. Currently we recommend (the guide) dc peristaltic pumps, but we'll move to stepper motors at some point, which requires pwm as well. Hence its written with pwm support in mind. If you are using something that does not require pwm (for example BRS dosing pump) then just a normal timer should do the job.
The forward/reverse thing is bit hacky as it possible due to the l293d chip we have used in the guide. Now, though this is handy, its not a common feature across all dosing system. Which means enforcing this feature in doser module will make it hard for many other dosing system to work with reef-pi. It is possible to just alter the l293d circuit and have jumpers to change direction (and user have to change them physically to alter direction), and keep the software simple, without direction control .
You will need the pca9685 IC (light controller) for more than two pwm channels, its bit complicated than the dosing guide but should not be very different. Again, if you dont need pwm/speed control just use the timer feature. On this note, we are allowing sub-second durations in timers in next version of reef-pi.

I am the one that wrote the small change for the sub-second duration change .

The pwm is also useful for peristaltic pumps, especially for nano tanks where more precise dosing maybe required, I've hooked up a PCA 9685 and a pair of l293d modules, from this I can run 16 dosers if needed with high precision, I don't think any DIY or commercial unit can do that.

The cheap l293d modules I think are the way to go, less cost and less effort than building a proto board.

Reef pi is definitely the easiest and most flexible DIY controller that you can build and the best part is it's constant evolution, a big thanks to @Ranjib

I have been using the l293d breakout boards and have found them to be very effective. I am using it to control a cheap dc peristaltic pump. My worry is building the PCA9658 PWM board to control more than 2 pumps. It looks like a lot of work, though I will probably end up doing it for lighting control.

From my testing of dosing pumps I found that adjusting the power level was inconsistent. If I changed the power from 100% to 90% this didn't not result in a 10% output decrease. In a past project I have written I have found that I can get a very linear output decrease by using sub second durations (which is why I added the feature). As a side note, if you were to move to using a stepper motor then the PWM power adjustment should be very consistent.


The heart burn I am having over using "timer" instead of "dosing pump" is simply that what I am doing is "dosing" just without PWM.

If I were to redesign the dosing pump feature I would
1. Add an optional "equipment" to control the pump being on and off. This would allow for directional control of a pump.
2. Make the PWM jack optional
(Note, that there must be at least an equipment or a jack)
3. Instead of dosing in "seconds" I would change it to dose in milli-liters
4. Store calibration results in the Dosing Pump object to make dosing in milli-liters possible.
4A. Calibration results would be stored as part of the object. (20 seconds at 50% power produced 10ml)
4B. Calibration results would be required to enable the dosing pump.
4C. When dosing, the system would use calibration results to determine the number of seconds to run the pump in order to dose the specified amount of liquid.

This makes it much easier to determine how much you are actually dosing, rather than having to do the calculation in your head. This also makes it so when you calibrate your pumps again in the future you aren't changing how much you are dosing, instead the system figures out how to adjust it based on the calibration change.

Thoughts?

 

[Bigtrout]

I am the one that wrote the small change for the sub-second duration change .



I have been using the l293d breakout boards and have found them to be very effective. I am using it to control a cheap dc peristaltic pump. My worry is building the PCA9658 PWM board to control more than 2 pumps. It looks like a lot of work, though I will probably end up doing it for lighting control.

From my testing of dosing pumps I found that adjusting the power level was inconsistent. If I changed the power from 100% to 90% this didn't not result in a 10% output decrease. In a past project I have written I have found that I can get a very linear output decrease by using sub second durations (which is why I added the feature). As a side note, if you were to move to using a stepper motor then the PWM power adjustment should be very consistent.


The heart burn I am having over using "timer" instead of "dosing pump" is simply that what I am doing is "dosing" just without PWM.

If I were to redesign the dosing pump feature I would
1. Add an optional "equipment" to control the pump being on and off. This would allow for directional control of a pump.
2. Make the PWM jack optional
(Note, that there must be at least an equipment or a jack)
3. Instead of dosing in "seconds" I would change it to dose in milli-liters
4. Store calibration results in the Dosing Pump object to make dosing in milli-liters possible.
4A. Calibration results would be stored as part of the object. (20 seconds at 50% power produced 10ml)
4B. Calibration results would be required to enable the dosing pump.
4C. When dosing, the system would use calibration results to determine the number of seconds to run the pump in order to dose the specified amount of liquid.

This makes it much easier to determine how much you are actually dosing, rather than having to do the calculation in your head. This also makes it so when you calibrate your pumps again in the future you aren't changing how much you are dosing, instead the system figures out how to adjust it based on the calibration change.

Thoughts?

Part of your doser troubles with PWM may be that there are pwm bugs in reef pi for both the pca9685 and the raspberry pi pwm, namely:
1. The pca9685 frequency does not change if you change the setting box in configuration screen. Mine is stuck at 1500hz which may be too fast for your application. Its too fast for my mosfet modules im using for my light pwm, and may not be correct for your pumps
2. The raspberry pi pwm frequency runs inverse to the setting box, and duty cycles are incorrect.

There are fixes being worked on

 

[theatrus]

So all this talk about failures. I looked at my dashboard and my historical CPU usage had been reset along with historical ATO and heater usage. My dosers missed the normal dose at 11:05 and dosed at 11:22 instead. I looked at aadafruit IO and noticed a dip in my lights at 10:44 and also noticed my CPU usage dropped to zero around 10:17. So I went and checked the logs and something strange happened. I am not home so it may have been a power outage and all appears to be running on the screen lol I will know more when I get home tonight. But I looked through the logs and found some strange stuff. Looks like at 10:17 reef-pi rebooted for some reason? I am not sure about the TLS handshake error that coincides with my lighting dip in adafruit IO but I am assuming maybe a communication error with IO? I dont recognize the IP address but I x'd out the last two octets just in case.

Feb 12 10:43:57 myReef reef-pi[308]: 2019/02/12 10:43:57 http: TLS handshake error from 143.85.X.X:56111: EOF

Feb 12 10:43:57 myReef reef-pi[308]: 2019/02/12 10:43:57 http: TLS handshake error from 143.85.X.X:52124: EOF

Feb 12 10:43:58 myReef reef-pi[308]: 2019/02/12 10:43:58 http: TLS handshake error from 143.85.X.X:59768: EOF

Feb 12 10:43:58 myReef reef-pi[308]: 2019/02/12 10:43:58 http: TLS handshake error from 143.85.X.X:52107: EOF


Feb 12 10:17:03 myReef fake-hwclock[84]: Tue Feb 12 15:17:01 UTC 2019

Feb 12 10:17:03 myReef systemd[1]: Started Create Static Device Nodes in /dev.

Feb 12 10:17:03 myReef systemd[1]: Starting udev Kernel Device Manager...

Feb 12 10:17:03 myReef systemd-fsck[118]: e2fsck 1.43.4 (31-Jan-2017)

Feb 12 10:17:03 myReef systemd-fsck[118]: rootfs: clean, 129237/1926848 files, 1373258/7779456 blocks

Feb 12 10:17:03 myReef systemd[1]: Started File System Check on Root Device.

Feb 12 10:17:03 myReef systemd[1]: Starting Remount Root and Kernel File Systems...

Feb 12 10:17:03 myReef systemd[1]: Started udev Kernel Device Manager.

Feb 12 10:17:03 myReef systemd[1]: Started Remount Root and Kernel File Systems.

Feb 12 10:17:03 myReef systemd[1]: Starting Load/Save Random Seed...

Feb 12 10:17:03 myReef systemd[1]: Starting Flush Journal to Persistent Storage...

Feb 12 10:17:03 myReef systemd[1]: Starting udev Coldplug all Devices...

Feb 12 10:17:03 myReef systemd[1]: Started Load/Save Random Seed.

Feb 12 10:17:03 myReef systemd[1]: Started Flush Journal to Persistent Storage.

Feb 12 10:17:03 myReef systemd[1]: Started Set the console keyboard layout.

Feb 12 10:17:03 myReef kernel: [ 0.000000] Booting Linux on physical CPU 0x0

Is your Pi exposed to the public internet, or at least the port 443? Could just be the normal internet scans you see on the internet. Interestingly, the /16 here is owned by USAISC :spookie:

The point after this is a reboot, even though it looks like time went backwards at that point?

 

[b4tn]

Is your Pi exposed to the public internet, or at least the port 443? Could just be the normal internet scans you see on the internet. Interestingly, the /16 here is owned by USAISC :spookie:

The point after this is a reboot, even though it looks like time went backwards at that point?

Very strange, I do have port forwarding enabled for 443 so I can access anywhere from my phone. I am not sure why a reboot happened, we are having ice and snow storms although I dont think bad enough to hit power. I only took a short snip of the logs I may have to look a little closer when I get home.

 

[bigcheese]

Basically a centralized configuration and management system, aka a single pane of glass. When done right it's really nice, I will also say troubleshooting *cough* can be quite fun. Basically the common configuration bits would only have to be done once and shared among all the minions.

Would make for a modular like approach.

+1 for this...
I'm working on a build for a power bar right now that solves a lot of challenges, but I can foresee a similar modular approach across the board.
My build is to add upon Ranjib's power build (props for ripping apart that Prime strip to get the 5V PSU and surge circuitry), but I'm using 2 Arduinos* to do the actual power relay control, interfacing with Reef-pi as an i2c device. I could see the same for a doser module, a sensor cluster module, etc. Let the RPi do the high-level stuff and leave the realtime down and dirty to modular boxes.

*Why Arduinos? First, <5 sec boot time. Return power to the critical components faster after a hard power failure. Why 2? High availability. One primary, one secondary, sharing state. If the primary fails, the secondary takes over. Reef-pi would be able to write state changes to ports and store persistent (power-on) states to the 'duino, and use it as a power sensor.

 

[bigcheese]

Edit: double post.

 

[Bigtrout]

+1 for this...
I'm working on a build for a power bar right now that solves a lot of challenges, but I can foresee a similar modular approach across the board.
My build is to add upon Ranjib's power build (props for ripping apart that Prime strip to get the 5V PSU and surge circuitry), but I'm using 2 Arduinos* to do the actual power relay control, interfacing with Reef-pi as an i2c device. I could see the same for a doser module, a sensor cluster module, etc. Let the RPi do the high-level stuff and leave the realtime down and dirty to modular boxes.

*Why Arduinos? First, <5 sec boot time. Return power to the critical components faster after a hard power failure. Why 2? High availability. One primary, one secondary, sharing state. If the primary fails, the secondary takes over. Reef-pi would be able to write state changes to ports and store persistent (power-on) states to the 'duino, and use it as a power sensor.

Esp32 modules would work here as well!

 

[bishoptf]

Very strange, I do have port forwarding enabled for 443 so I can access anywhere from my phone. I am not sure why a reboot happened, we are having ice and snow storms although I dont think bad enough to hit power. I only took a short snip of the logs I may have to look a little closer when I get home.

Ymmv but I would never port forward to my internal Network even over 443, that's just me I'm an infosec person so just see way to many bad things going on but that's just my .02.

 

[b4tn]

Ymmv but I would never port forward to my internal Network even over 443, that's just me I'm an infosec person so just see way to many bad things going on but that's just my .02.

I dumbed it down a little. I have a DMZ setup with a second router. Still not the best solution but its what I had on hand.

 

[bigcheese]

Esp32 modules would work here as well!

I am all over that Adafruit Feather M0!

 

[theatrus]

I am all over that Adafruit Feather M0!

Much prefer wired everything. My original LEDBrick lights still run BLE and... yeah, if you don't need it, don't use wireless :-D