• For a Limited Time the R2R Partner Membership is NOW OPEN! Get some cool swag and chances to win part of over $20,000 in prizes! Click here for more details

reef-pi :: An opensource reef tank controller based on Raspberry Pi.

Des Westcott

Well-Known Member
View Badges
Joined
May 29, 2018
Messages
564
Reaction score
933
Location
Durban - South Africa
I'm getting strange "Temp sensor failure" Alerts for all 3 of my temps since last night. They seem to have dropped out at roughly the same time. Been on 3.5 for 5 days now.

Seems like all 3 sensors are just unavailable. My Temp setups don't have any Sensors listed anymore and I can't find any to edit existing ones or set up new.
Reloading and Rebooting reef-pi didn't help anything, But I have now done a hard power cycle and it seems to have made things work again.....
 
Corals.com

theatrus

Valuable Member
View Badges
Joined
Mar 26, 2016
Messages
1,315
Reaction score
2,541
Location
Sacramento, CA area
Reloading and Rebooting reef-pi didn't help anything, But I have now done a hard power cycle and it seems to have made things work again.....
I suspect one of the sensor probes "latched up" and held the bus, so none of the remaining sensors could respond. Killing the power fixed it of course.

You could use a Raspberry Pi GPIO pin to control the power to the sensors so you can at least soft restart it when it does fail. Normally, the Maxim sensors shouldn't do this (they can only hold the bus for short bursts).
 
OP
Ranjib

Ranjib

7500 Club Member
View Badges
Joined
Apr 16, 2016
Messages
8,270
Reaction score
14,724
Location
Palo Alto, Bay Area

robsworld78

Active Member
View Badges
Joined
Feb 14, 2020
Messages
218
Reaction score
473
Location
Edmonton, Canada
Reloading and Rebooting reef-pi didn't help anything, But I have now done a hard power cycle and it seems to have made things work again.....
I never liked a single bus for multiple sensors especially as there's lots of junk, anything can happen. A few months ago I looked into multiple buses and to my surprise it's really easy to setup, just need to change the config.txt file. Then you can connect one sensor to one pin, if one goes down the others won't. The beauty is all sensors show up in reef-pi from as many buses as you want, I've tested 3.

 

DeeEmm

New Member
View Badges
Joined
Sep 20, 2020
Messages
23
Reaction score
26
Location
RADelaide
https://raspberrypihq.com/use-a-push-button-with-raspberry-pi-gpio/#:~:text=Connecting the Raspberry Pi's general,pin 1 using a resistor. does this help? you can also search for how to connect float switches to pi. or check the design of dfrobot's photoelectric sensor breakout board.
It's more of a software question than a how to connect the wiring question.

- If the software uses the internal pull UP resistor, I will need to ground the input to give it a state change
- If the software uses the internal pull DOWN resistor, I will need to raise the input to vcc (with a current limiting resistor) to give it a state change
- If no internal pullups are set in software than I can use an external pull up or pull down resistor and the above cases will apply, (except that the on and off signal will only be correct in one case.)

BTW, I'm not using any hats / shields breakout boards, I want to connect direct to the GPIO

The physically connecting part is easy.

But what Reef Pi does with the result is unclear.

From your example I assume that you are using the internal pull-down? Is that correct?

/DM
 

Des Westcott

Well-Known Member
View Badges
Joined
May 29, 2018
Messages
564
Reaction score
933
Location
Durban - South Africa
I suspect one of the sensor probes "latched up" and held the bus, so none of the remaining sensors could respond. Killing the power fixed it of course.

You could use a Raspberry Pi GPIO pin to control the power to the sensors so you can at least soft restart it when it does fail. Normally, the Maxim sensors shouldn't do this (they can only hold the bus for short bursts).
Well it happened again that I lost all my temp sensors. Luckily there was someone at home and I asked them to do a "Widows reset" - turn everything off and then back on again and it seems to have corrected itself again. I'm seeing feeds back in AIO. When it happened last night, the heaters were off, so my temp dropped about 2 degrees C, this time it looks like the heaters were on and the temp rose by nearly 1.5 degrees C, despite getting alerts that he temp was out of range (0 deg C)

So my question is :- Is this likely a hardware issue or software? What steps can I take to prevent it happening again? Replace sensors? Create a backup SD? Do a clean install and keep as a spare?

I've had no problems with this for a year, now it's done this twice in less than 24 hours.
 

DeeEmm

New Member
View Badges
Joined
Sep 20, 2020
Messages
23
Reaction score
26
Location
RADelaide
Suspect that the water sealing crimp on a sensor has given out. Replace would be my call.

Sounds likely if a DS18B20 sensor is being used as the bus controller is integrated, if it gets wet then the whole bus will go down.

It surprises me that there is no watchdog / bus check implemented. It should be possible to monitor the bus status before transmitting / receiving to detect bus issues. Some automatic recovery may then be possible with an auto reboot. Even if bus status is not available to the Pi, it should be easy to set up a simple change monitor function. the DS18B20 has a 12 bit resolution I would expect that the value would be different every time you read it.

Other solutions might be to allow for more than one input parameter into the ATO function, a second identical redundant I2C network could then be used. A failure in one network would then not result in a loss of control. (if sensor1 < temp OR sensor2 < temp then heater ON)

OR use of a hardware watchdog such as a CAT1161 timer or even something like the 'Super-Watchdog'.


I think that some form of system integrity checking is critical for any control system, especially one where it is possible to end up with a situation like Des has with temp increases due to a failed temp sensor / control bus.

/DM
 

theatrus

Valuable Member
View Badges
Joined
Mar 26, 2016
Messages
1,315
Reaction score
2,541
Location
Sacramento, CA area
I think that some form of system integrity checking is critical for any control system, especially one where it is possible to end up with a situation like Des has with temp increases due to a failed temp sensor / control bus.
Strongly agree. At the minimum, we should look at using multiple 1-wire buses if the Linux kernel driver supports it, which prevents one failed sensor dominating the bus. That would be simple for most people to build into their setups. I haven't dug into the kernel driver to see if it supports that, or how special the devicetree support in the RPi kernel is (which the RPi kernel is many levels of special :))
 
Top Shelf Aquatics
OP
Ranjib

Ranjib

7500 Club Member
View Badges
Joined
Apr 16, 2016
Messages
8,270
Reaction score
14,724
Location
Palo Alto, Bay Area
Strongly agree. At the minimum, we should look at using multiple 1-wire buses if the Linux kernel driver supports it, which prevents one failed sensor dominating the bus. That would be simple for most people to build into their setups. I haven't dug into the kernel driver to see if it supports that, or how special the devicetree support in the RPi kernel is (which the RPi kernel is many levels of special :))
kernel supports multiple 1 wire gpio pins, if thats what you are asking,
 
OP
Ranjib

Ranjib

7500 Club Member
View Badges
Joined
Apr 16, 2016
Messages
8,270
Reaction score
14,724
Location
Palo Alto, Bay Area
It is :) Probably the best recommendation for reliability would be enabling multiple pins?
I dont know.. i have to think about this, specifically the failure conditions. I hear your suggestions.. but i dont know yet if thats all we have to consider. More research on my end i guess :)
 

robsworld78

Active Member
View Badges
Joined
Feb 14, 2020
Messages
218
Reaction score
473
Location
Edmonton, Canada
It is :) Probably the best recommendation for reliability would be enabling multiple pins?
I've setup 3 buses on Pi 3 and 4 with no issues, was too easy.

 

DeeEmm

New Member
View Badges
Joined
Sep 20, 2020
Messages
23
Reaction score
26
Location
RADelaide
I like the idea of multiple busses along with additional ATO logic, Whilst this does mean that the users has to create the additional redundant network, it opens up the possibilities of more complex control schemas.

At the moment we only have

IF THIS THEN THAT

What we really need is

IF THIS AND / OR THIS THEN THAT

Then just set up two temp probes, each on a different bus. Problem solved.

It also means that the system degrades nicely. i.e. there is no loss of control of the system (i.e. the heater) in the event of a single temp probe failure or single bus issue. This is basically how a safety PLC works.

You can use the same control logic to implement redundant control for any connected device, not just heaters and temp probes. E.g. redundant float control for ATO, redundant Ph control, in fact redundant control for any device or control schema.

Plus there's the added bonus of more complex control schemas that would be available with the additional ' AND / OR THIS ' logic.
 

stefanm

Active Member
View Badges
Joined
Oct 11, 2017
Messages
267
Reaction score
346
I thought this might be helpful if you're facing any issues with reef pi.

I had a few things that wouldn't work as they should, no issues with the hardware. I was in the verge of wiping the sdcard and reinstalling, the pi has been running for for around 20 months, I felt the software had been corrupted.

So instead of removing the card and doing it all again, I decided to try updating the raspberry pi os, it worked a treat and no need to reconfigure anything.

I guess that the power cuts we have had which the battery backup has run out (around 10 times) had corrupted the os.
 

Bigtrout

Valuable Member
View Badges
Joined
Dec 16, 2018
Messages
1,159
Reaction score
2,777
I thought this might be helpful if you're facing any issues with reef pi.

I had a few things that wouldn't work as they should, no issues with the hardware. I was in the verge of wiping the sdcard and reinstalling, the pi has been running for for around 20 months, I felt the software had been corrupted.

So instead of removing the card and doing it all again, I decided to try updating the raspberry pi os, it worked a treat and no need to reconfigure anything.

I guess that the power cuts we have had which the battery backup has run out (around 10 times) had corrupted the os.
Take a look at my build thread somewhere I have instructions for adding a Real Time Clock board. This helps with corruption issues from power outages/internet outages. Sometimes raspian gets wonky when file timestamps dont match. Without a RTC if raspian comes up without internet, it wont be able to get the correct time.

Since adding the RTC i have not had an issue, going on 1.5 years now and we get power outages and internet outages regularly in the woods of NW Pennsylvania.
 

Have you ever used a spoon, hose and rubber band together to remove algae?

  • YES

    Votes: 0 0.0%
  • NO

    Votes: 3 25.0%
  • What the heck? (see the thread)

    Votes: 9 75.0%

Online statistics

Members online
1,793
Guests online
3,758
Total visitors
5,551
JASON FOX SIGNATURE CORALS
Top