Help with esp32 sketch

[Wolfw28]

For completeness, here is the cabling. I have set the circuit to 3.3V and connected 3.3V to it. Pull-up is done with one 3.4k resistor on each SDA and SCL.
SDA and SCL are connected to the Default pins for I2C: Nr 21 and 22

Can not wait to take it for a test drive

 

[Sral]

Can not wait to take it for a test drive

Yeah, I can't wait to make it work as well XD

I'll probably test the pH circuit with my Raspberry Pi and ReefPi, just to be sure that the circuit itself works correctly.

 

[Sral]

Buja !

@robsworld78 Please notice: the pH circuit's onboard LED was a good idea ... when I supply 3.3V it doesn't light, but when I supply 5V (while also changing the jumper back to 5V) it does. So the circuit seems to have had no power. (although I checked and there were 3.28 V on the spare 3.3V header).
Also: the I2C "error" noted above also returns "0" (meaning no error) when I do not even connect the pH circuit -.-

Now I get this

And 6.986 looks surprisingly like a neutral pH :D
(e.g. 0V on the input, since I shorted it with a 10k resistor)

Cleaning up the code I now get:

Reef-Pi also reads beautifully:

 

[Sral]

BTW, the ESP32 supports I2C clock stretching, so I can even increase the clock speed to 100kHz, probably even beyond if we want to:

@Ranjib Any idea how we want to implement this in ReefPi, since we now have several kinds of Analog Inputs ?
My current thinking is to "simply" define DS18B20_COUNT, FLOW_METER_COUNT, (maybe PH_COUNT for extendability) and handle them as you did before, reroute Analog Inputs ID 0 to DS18B20_COUNT to the DS18B20 instance, etc.

That way we do not even need to change the ReefPi driver and can handle everything on the settings side in the ESP32 code.

I'll write something up and try to offer this to you on your GitHub repository, probably as a separate branch or so. I'm not that experienced in git yet, so I'm open to suggestions :grinning-face-with-sweat:

 

[Sral]

@Ranjib I created a pull request on GitHub.
Sadly I just noticed that the pull request is targeted at "main". Maybe we should test this further in a separate branch before merging.

Here is a representation of one of the "problems" that I mentioned:

I get a supposed query on the oneWirePin 4. This corresponds to an AnalogInput ID of 0 or 1, but I have no such querry set on ReefPi:

Maybe it is a communication error and if we then run "int id = String(bindings.get("id")).toInt();" it has nothing to work with and returns a 0. Anyways, none of my set AnalogInputs actually shows a value of -127, only my pH has a dip to 0.0, so it seems fine for now.

 

[andieasy]

Hi @Sral , what os do use to run reef pi?
Don't you have a drop down menu bug like mine?

 

[Sral]

Hi @Sral , what os do use to run reef pi?
Don't you have a drop down menu bug like mine?

Sry, I'm using a dedicated Raspberry Pi Zero for my ReefPi and no Linux PC lying around to test the ESP32 properly in this regard.

Writing this I was just thinking that you could probably set these things using the http API without the Webinterface ...
that could circumvent your problem. You can probably use the same "curl" command that @Ranjib used in his ESP32 Readme in the Ubuntu console, as far as I have understood:

First, set a cockie with your ReefPi login credentials, as described in the Troubleshooting guide:

curl -X POST -c cookie.txt -d '{"user":"reef-pi", "password":"reef-pi"}' http://reef-pi.local/auth/signin

Then you can use this cookie to view all outlets:

curl -b cookie.txt http://reef-pi.local/api/outlets

Then you could use this cookie to create an outlet, although I'm not quite sure of the exact entries:

curl -b cookie.txt  -k -H "Accept: application/json" -H "Content-Type:application/json" \
-X PUT -u $USER_NAME:$PASSWD \
-d '{\
"driver":"$ESP32_DRIVER_NAME",\
"equipment":"",\
"id":$NUMBER_OF_CURRENT_OUTLETS,\
"name":"$OUTLET_NAME",\
"pin":$PIN_NUMBER,\
"reverse":false}' "http://reefpi.local/api/outlets"

You would of yourse need to insert some of your specific settings, like the reefPi webaddress, if it is different from reefpi.local, the name of your ESP32-driver that you created in ReefPi, the desired Pin and Outlet-name, the ID of the outlet, although I'm not sure if that is realy just itterative.
@Ranjib is that code correct ?

 

[robsworld78]

Buja !

@robsworld78 Please notice: the pH circuit's onboard LED was a good idea ... when I supply 3.3V it doesn't light, but when I supply 5V (while also changing the jumper back to 5V) it does. So the circuit seems to have had no power. (although I checked and there were 3.28 V on the spare 3.3V header).
Also: the I2C "error" noted above also returns "0" (meaning no error) when I do not even connect the pH circuit -.-

Now I get this

And 6.986 looks surprisingly like a neutral pH :D
(e.g. 0V on the input, since I shorted it with a 10k resistor)

Cleaning up the code I now get:

Reef-Pi also reads beautifully:

Sorry it took me so long to reply, I've been really busy lately, it comes in spurts. Glad you got it figured out and it's best to run it at 5v as 3.3v wouldn't be isolated. Not sure what I was thinking but I've since removed that, it was only on a batch of boards. Would have been nice to see 7.0, we'll blame the 0.014 on the resistor accuracy lol. :) I've never tried that, did you put the 10k across the center pin and shielding of the BNC?

Nice to know the ESP32 supports clock stretching, unless a device requires 100kHz I recommend staying at 10kHz especially if wires are being used as there's less chance it locks ups or you get an error.

 

[Sral]

@Ranjib

Sorry it took me so long to reply, I've been really busy lately, it comes in spurts. Glad you got it figured out and it's best to run it at 5v as 3.3v wouldn't be isolated. Not sure what I was thinking but I've since removed that, it was only on a batch of boards. Would have been nice to see 7.0, we'll blame the 0.014 on the resistor accuracy lol. :) I've never tried that, did you put the 10k across the center pin and shielding of the BNC?

Nice to know the ESP32 supports clock stretching, unless a device requires 100kHz I recommend staying at 10kHz especially if wires are being used as there's less chance it locks ups or you get an error.

Exactly, I simply made the resistor’s leads a bit wider than the distance between center and shield, stuck one end in the center and let the other press against the shield with its elastic force.

The deviation is as you mentioned probably dependent on the offset‘s accuracy and therefore the offset resistors’ accuracies. That should however be completely eliminated with proper calibration :D

I was even thinking to use this as a calibration for pH 7.0, but I’m not sure if that’s also dependent on the contact voltage on the probe. If the probe is well designed this should hold true, but that’s a big IF

 

[robsworld78]

@Ranjib

Exactly, I simply made the resistor’s leads a bit wider than the distance between center and shield, stuck one end in the center and let the other press against the shield with its elastic force.

The deviation is as you mentioned probably dependent on the offset‘s accuracy and therefore the offset resistors’ accuracies. That should however be completely eliminated with proper calibration :D

I was even thinking to use this as a calibration for pH 7.0, but I’m not sure if that’s also dependent on the contact voltage on the probe. If the probe is well designed this should hold true, but that’s a big IF

Thanks, I'll give that a go. Any idea what resistor value to use for 4.0 and 10.0? Now I'm curious if those hold true as well.

 

[Sral]

Thanks, I'll give that a go. Any idea what resistor value to use for 4.0 and 10.0? Now I'm curious if those hold true as well.

Sadly that won't work this way.
Applying a resistor (in fact, any resistor that's far smaller than the OP-Amp input resistance) only ensures that you have close to 0 V on the input plus/minus the contact voltage between the center gold plated pin and the tin (or so ?) plated shield. That means the measured voltage is more or less equal to your offset voltage, e.g. represents a pH probe that reads 0.0mV (+/- contact voltage), which should theoretically happen at exactly 7.0 pH.

pH values of 4.0 and 10.0 on the other hand would create voltages over the pH probe, which depend on the quality and age of the probe, as well as other factors like temperature, I guess. You could apply an isolated, calibrated and exact +/-210mV (or whatever voltage is the theoretically correct one for pH 4 and 10) on the input, but that doesn't mean that the probe you will connect will show these exact values at pH 4.0 and 10.0 as well. That's why you probably have to calibrate every time you connect a different probe or after so many months of ageing. Although it is worth a try, would be interesting how close this gets and how long this stays true.

 

[Ranjib]

@Ranjib I created a pull request on GitHub.
Sadly I just noticed that the pull request is targeted at "main". Maybe we should test this further in a separate branch before merging.

Here is a representation of one of the "problems" that I mentioned:

I get a supposed query on the oneWirePin 4. This corresponds to an AnalogInput ID of 0 or 1, but I have no such querry set on ReefPi:

Maybe it is a communication error and if we then run "int id = String(bindings.get("id")).toInt();" it has nothing to work with and returns a 0. Anyways, none of my set AnalogInputs actually shows a value of -127, only my pH has a dip to 0.0, so it seems fine for now.

I’ll take a look

 

[robsworld78]

Sadly that won't work this way.
Applying a resistor (in fact, any resistor that's far smaller than the OP-Amp input resistance) only ensures that you have close to 0 V on the input plus/minus the contact voltage between the center gold plated pin and the tin (or so ?) plated shield. That means the measured voltage is more or less equal to your offset voltage, e.g. represents a pH probe that reads 0.0mV (+/- contact voltage), which should theoretically happen at exactly 7.0 pH.

pH values of 4.0 and 10.0 on the other hand would create voltages over the pH probe, which depend on the quality and age of the probe, as well as other factors like temperature, I guess. You could apply an isolated, calibrated and exact +/-210mV (or whatever voltage is the theoretically correct one for pH 4 and 10) on the input, but that doesn't mean that the probe you will connect will show these exact values at pH 4.0 and 10.0 as well. That's why you probably have to calibrate every time you connect a different probe or after so many months of ageing. Although it is worth a try, would be interesting how close this gets and how long this stays true.

Ah I see, I was just curious. It's 59.16 mV per pH so 10.0 is 177.48mV and 4.0 is -177.48mV. Those values are based on 25c but for what we measure temp doesn't affect it all that much, I was surprised how little. It matters if you were reading 4.0 at 5c.

 

[cmitcham]

hey guys, is this something we want in the esp32 code?

 

[Sral]

hey guys, is this something we want in the esp32 code?

Nice idea, not sure if that’s the best way, but maybe one could also add a whole settings page to this without too much trouble.

 

[cmitcham]

i think it can do other settings as well. just watching a youtube now :)

 

[Sral]

i think it can do other settings as well. just watching a youtube now :)

Any insights so far ? I haven't had time apart from my doser and LED modules to look into this myself.

Found another nice circuit to compensate the AnalogInput inaccuracies somewhat easily:

I was already thinking about something like this. My only concern is that this will likely be a bit nonlinear by itself, e.g. if the input voltage itself is 0 then, depending on the device, the offset will be slightly different than if the input voltage is the full 3.3V. Although this might actually be less of an issue than I think, since the Author mentions an accuracy of 0.2% over the whole range after calibration.

I'll test this and get back to you. For the inexperienced user this is certainly easier than an OP-Amp in my oppinion.

 

[Ranjib]

Any insights so far ? I haven't had time apart from my doser and LED modules to look into this myself.

Found another nice circuit to compensate the AnalogInput inaccuracies somewhat easily:

I was already thinking about something like this. My only concern is that this will likely be a bit nonlinear by itself, e.g. if the input voltage itself is 0 then, depending on the device, the offset will be slightly different than if the input voltage is the full 3.3V. Although this might actually be less of an issue than I think, since the Author mentions an accuracy of 0.2% over the whole range after calibration.

I'll test this and get back to you. For the inexperienced user this is certainly easier than an OP-Amp in my oppinion.

If it’s a fixed shape non linear curve, we can transform it to a linear form with a suitable transformation function, that should not be a big deal. I can help with that.

 

[Sral]

@theatrus
Since you seem to have some knowledge about stack issues on these platforms (I'm grateful for every scrap here ^^): I have read that Arduino Strings can cause heap fragmentation when rapidly calling string concatenations (e.g. "+=").
If you look at the decoded backtraces I have posted, it always seems to occur on WebRequests and including some "std_function". Maybe the latter has something to do with string handling ?
Could it be that by parsing the http strings "incorrectly" the heap works up fragmentation, causing this issue ?

If one looks at the library file "ESPAsyncWebServer/src/WebRequest.cpp" at line 365, or searches for "_temp +=" you will see that there are quite a lot of these concatenations going on and the definition "_temp = String();"

Also @Ranjib: On my ESP32 system I had quite a high request rate, e.g. something like every 10s on 5 channels, e.g. pH, floweter, AnalogInput and 2 lights. With this I had infrequent crashes, but at a very consistent rate of 12 per hour. If that hypothesis of heap fragmentation is correct one should get a change in crash rate, depending on how often ReefPi does requests, right ?

 

[theatrus]

@theatrus
Since you seem to have some knowledge about stack issues on these platforms (I'm grateful for every scrap here ^^): I have read that Arduino Strings can cause heap fragmentation when rapidly calling string concatenations (e.g. "+=").
If you look at the decoded backtraces I have posted, it always seems to occur on WebRequests and including some "std_function". Maybe the latter has something to do with string handling ?
Could it be that by parsing the http strings "incorrectly" the heap works up fragmentation, causing this issue ?

If one looks at the library file "ESPAsyncWebServer/src/WebRequest.cpp" at line 365, or searches for "_temp +=" you will see that there are quite a lot of these concatenations going on and the definition "_temp = String();"

Also @Ranjib: On my ESP32 system I had quite a high request rate, e.g. something like every 10s on 5 channels, e.g. pH, floweter, AnalogInput and 2 lights. With this I had infrequent crashes, but at a very consistent rate of 12 per hour. If that hypothesis of heap fragmentation is correct one should get a change in crash rate, depending on how often ReefPi does requests, right ?

On any embedded device, repeated allocations and frees are obviously a bad idea. `operator+=` and `.append` on std::string are synonymous, so they only cause allocations at 50% marks.

If you control the std::string you're providing, its a good idea to `.reserve` a reasonable buffer size for the operation you'll be doing. Of course, you should also be checking it repeatedly for being over a threshold or its a classic DoS situation.