Grafana + Apex

Would you be interested in learning more about the setup of Grafana+Apex?

  • Nah, too much trouble

    Votes: 0 0.0%

  • Total voters
    13

MikeTheNewbie

Active Member
View Badges
Joined
Aug 16, 2020
Messages
361
Reaction score
246
Location
Michigan
Rating - 0%
0   0   0
Hello fellow reefers, I recently came across Grafana, an open source tool that helps create really useful dashboards for time series data. I gave it a try and I'm really pleased with the results.
I'd say it might be more of an advanced topic since it needs some understanding of software but, ironically, I think it benefits new reefers like me the most.
It has helped me better understand what is going up in my tank (build thread here), visualize and understand trends, I spotted and fixed a problem with my MP40 configuration, and I'm still trying to figure out the relation between CO2, lights, and pH in my tank.

Here are some of the charts I use the most
General.png

The top left chart shows the temperature of my Display Tank compared with my basement and the rest of the house (the tank is in the basement).
The top right chart shows the tank temperature in blue and the power drawn by the heaters (I use 2 x BRS 300W Titanium Heaters)
The bottom left chart is an attempt to make sense of pH and CO2. I added the ORP by mistake and decided to leave it there to see if I could spot any correlation. You can see the daily pH swing of almost 0.2 and you can also see that the CO2 levels go up during the day when I'm at the basement. Unfortunately, or fortunately, I haven't seen much correlation between pH and CO2 in my case. Most of the swing is tied to the lighting as far as I can see.
Lastly, on the bottom right, I was trying to spot correlations between the room humidity and my salinity but my Neptune Auto Top-of Kit seems to be doing a good job and maintaining salinity in a +/- 0.2 range. I trust the Auto Top Off more than the accuracy of the salinity/conductivity probe to be honest :p

Testing.png

So this is the new feature I added to my dashboard. I do manual water tests using Hanna checkers and RedSea tritation test kits. I used to save the info in a spread sheet but last week I started loading the data directly to the same database that stores the info from Apex (InfluxDB). The pH chart shows the sad truth about my pH. The pH measurement in the first group came from Apex and is inaccurate. The one in this group is the one that I have done manually with my trusted Hanna checker. The Alkalinity increase is because I started dosing sodium carbonate hoping it will help with pH. I haven't paid attention to the high magnesium.
Below is the web form I'm using to manually load the results but I'm planning to do some automation once I get my hands on an automatic tester.
LogForm.png


Flow.png

Finally, the measurements of my 2 MP40s. It is based on power drawn from the Neptune Energy Bar. The example shows how for some reason when I was using Anti-sync with the master MP40 running Lagoon at 50% the anti-sync MP40 was running very slowly (see the highlight on the left chart with the Left MP40 drawing 14.1W and the right one drawing 6.12W). Now I just set them independently. You can see that on the later readings. The drop on 6/20 was a power outage :S

So, again, just trying to gauge interest to see if its worth to spend time documenting the setup so others can follow.
Please answer the survey. Would you be interested in learning more about the setup of Grafana+Apex?
 

laverda

2500 Club Member
View Badges
Joined
Jun 6, 2008
Messages
2,893
Reaction score
2,165
Location
Anaheim
Rating - 0%
0   0   0
My first thought was no, as I do not trust my Apex any more. Plus I never heard of Graham before. I didn't know if it was another new piece of equipment or possibly a graphing tool. Lol
After reading you post it seems it could be really useful. For one I want to be able to see more than a week data at a time. Like a month, 6 months or even a year. 2 when selecting some items to graph together I have found Apex useless.
 
OP
OP
M

MikeTheNewbie

Active Member
View Badges
Joined
Aug 16, 2020
Messages
361
Reaction score
246
Location
Michigan
Rating - 0%
0   0   0
Yeah, I should have given more context. Ypu're right, Grafana is a software tool. It uses an external database (InfluxDB in my case) and a service (Telegraf) to pull data from Apex or other sensors. I currently have Apex + Awair (to monitor CO2) + SmartThings sensors for temperature and humidity.
Apex has memory space limitations so it stores measurements every 10 minutes. With Grafana/InfluxDB you choose the resolution. I have it set to every 10 seconds and you can trim/summarize old data if you start running out of space.
The software is free open source but it needs some hardware to run. I use a NAS (Network Drive) but it can be deployed in a Raspberry Pi or an old computer.
I also have some odd readings from Apex (salinity and pH). The salinity is working better after Neptune replaced it and I'm actively looking for a solution for my pH probe. I think its best to have multiple independent ways to test the same parameters and compare them. In this case I have my Apex probes and the readings from the Hanna Checkers.
You just gave me an idea to plot both sources of PH and both Sources of salinity to see the differences over time and notice when one starts to drift.
 

robsworld78

Well-Known Member
View Badges
Joined
Feb 14, 2020
Messages
952
Reaction score
1,281
Location
Edmonton, Canada
Rating - 0%
0   0   0
I must say I assumed Apex had a charting system like this, can you not overlay parameters and view long term with it? I figured that was a big reason to get an Apex so you could see all the fine details and know exactly what's going on. Maybe Neptune needs to hire you @MikeTheNewbie. :)
 
OP
OP
M

MikeTheNewbie

Active Member
View Badges
Joined
Aug 16, 2020
Messages
361
Reaction score
246
Location
Michigan
Rating - 0%
0   0   0
Apex does have good charting features, they let you compare different parameters. Its charting functionality is limited though. I'm sure they had to strike a balance between easy of use and features. Plus they are also constrained by memory space. The benefit of exporting your metrics to an external database is that you have a lot more options to customize and create new chart. You can also mix metrics from Apex and other devices and one of the most important for me is to keep the metrics long term. Hard drive space or solid state drive space is way cheaper than storing data in the memory module used by Apex. So you can keep years of data and increase the sample rate. Mine is capturing every 10 seconds compared to every 10 minutes in Apex. And I plan to keep the data for the life of the tank (hopefully 10+ years) compared to the 3 months Apex is currently storing my data for.
In a nutshell I think Apex functionality is good and easy to use. The approach I'm following is more complicated but has a ton of advantages.
 
OP
OP
M

MikeTheNewbie

Active Member
View Badges
Joined
Aug 16, 2020
Messages
361
Reaction score
246
Location
Michigan
Rating - 0%
0   0   0
@DopamineKata I use telegraf.
@BZOFIQ I already ordered a Raspberry PI 4 and got an SSD. As soon as the Pi arrives I'll start playing with it and document the tutorial. I didn't want to share my notes from my current setup because I installed it in a Synology NAS and some parts were already there. With the Raspberry I'll document everything from scratch.
 

BZOFIQ

2500 Club Member
View Badges
Joined
Jul 31, 2014
Messages
4,690
Reaction score
3,990
Location
NYC
Rating - 100%
9   0   0
@DopamineKata I use telegraf.
@BZOFIQ I already ordered a Raspberry PI 4 and got an SSD. As soon as the Pi arrives I'll start playing with it and document the tutorial. I didn't want to share my notes from my current setup because I installed it in a Synology NAS and some parts were already there. With the Raspberry I'll document everything from scratch.

I'd actually prefer a Synology setup but am not complaining either way.
 

Betex

Well-Known Member
View Badges
Joined
Mar 20, 2020
Messages
532
Reaction score
440
Rating - 0%
0   0   0
@MikeTheNewbie How did you extract the data from apex just exporting? I love grafana and never thought about using it on my reef but this seems great to give it a try.
 
OP
OP
M

MikeTheNewbie

Active Member
View Badges
Joined
Aug 16, 2020
Messages
361
Reaction score
246
Location
Michigan
Rating - 0%
0   0   0
@MikeTheNewbie How did you extract the data from apex just exporting? I love grafana and never thought about using it on my reef but this seems great to give it a try.
I use telegraf to query Apex and push the data to Influx.
You can get the current data from Apex at: http://<YourApexIP>/cgi-bin/status.xml
There is also a telegraf plugin for Apex that makes things less complicated.
In case you are using something else like nodered, or just to get a history directly from Apex you can also use http://<YourApexIP>/cgi-bin/datalog.xml?sdate=<YYMMDD>&days=<DD> where YYMMDD are the end date and the DD is the amount of days before that date you want to query. Depending on your needs, the log might be too coarse/infrequent so I prefer to just query the status.xml at a 20s interval
 

DopamineKata

Beltalowda
View Badges
Joined
Nov 5, 2021
Messages
222
Reaction score
336
Location
Columbus
Rating - 0%
0   0   0
@DopamineKata I use telegraf.
@BZOFIQ I already ordered a Raspberry PI 4 and got an SSD. As soon as the Pi arrives I'll start playing with it and document the tutorial. I didn't want to share my notes from my current setup because I installed it in a Synology NAS and some parts were already there. With the Raspberry I'll document everything from scratch.

You know right after I asked that I figured it out. I'm planning on putting APKs together for the bits and bobs so I can run them on my Asustor NAS.
 

DopamineKata

Beltalowda
View Badges
Joined
Nov 5, 2021
Messages
222
Reaction score
336
Location
Columbus
Rating - 0%
0   0   0
I use docker containers so the Raspberry PI steps are going to be very similar or easier with Synology :)

Oddly enough that's what I'm using on my Asustor. I got everything up and running by hand last night over SSH but I want to package it all up in APKs so that has clean startup / shutdown scripts.
 

flourishofmediocrity

Active Member
View Badges
Joined
Nov 25, 2020
Messages
263
Reaction score
316
Location
Snohomish
Rating - 0%
0   0   0
I do something similar for my cable modem, but I use Nagios to get the data and RRD to store it. I know there is a company (forget the name now) that is working on a cheap hardware interface for probes and then is working on a cloud based software controller, kind of like a seneye. It seems like a less expensive probe array with cloud monitoring would be welcomed in the hobby if it was easy to verify the calibrations.
 

BZOFIQ

2500 Club Member
View Badges
Joined
Jul 31, 2014
Messages
4,690
Reaction score
3,990
Location
NYC
Rating - 100%
9   0   0
@DopamineKata I use telegraf.
@BZOFIQ I already ordered a Raspberry PI 4 and got an SSD. As soon as the Pi arrives I'll start playing with it and document the tutorial. I didn't want to share my notes from my current setup because I installed it in a Synology NAS and some parts were already there. With the Raspberry I'll document everything from scratch.


Mike did you ever have a chance to do a step-by-step?
 
OP
OP
M

MikeTheNewbie

Active Member
View Badges
Joined
Aug 16, 2020
Messages
361
Reaction score
246
Location
Michigan
Rating - 0%
0   0   0
Oh man! You just reminded me that I paid for that raspberry pi 4 and it never arrived!
I'll try to buy it somewhere else to set it up and do the write up.
 

Bubbles, bubbles, and more bubbles: Do you keep bubble-like corals in your reef?

  • I currently have bubble-like corals in my reef.

    Votes: 16 34.8%
  • I don’t currently have bubble-like corals in my reef, but I have in the past.

    Votes: 6 13.0%
  • I don’t currently have bubble-like corals in my reef, but I plan to in the future.

    Votes: 15 32.6%
  • I don’t currently have bubble-like corals in my reef and have no plans to in the future.

    Votes: 8 17.4%
  • Other.

    Votes: 1 2.2%
Back
Top