Pumphouse Project - Logic to water pump control and IoT

[jclaudii]

Long time absence from here so forgive me for not getting on in so long!

Project description:

• monitor a well pump cycle rate (state is on or off and time it was on pumping and using energy)
• set thresholds for an alarm if state is on for too long or cycle rate frequency (ex. turns on/off 15 times in under 60 seconds or runs for more than 10 min)
• relay this data to an IoT or webserver
  • I have thingspeak free account but I think it is too slow (15 sec).
  • I have a windows 10 desktop that is on most of the time, send data to this and use it as a local webserver/DB/Datalogger
  • I also have a raspberry pi in a box somewhere that I could figure out how to use again if need be
• I would like to know the follow data:
  • is pump state on or off
  • pump on time per cycle
  • max pump on time
  • min pump on time
  • cycle rate per min
  • cycle rate per hour
• Bonus data:
  • Outside Temp
  • pumpHouse Temp (working with ds18b20)
  • phototransistor for light source in pumphouse
  • relay to control light source based on temperature
  • water pressure sensor(transducer)
  • current monitoring would be the cats meow
  • pumpMotor SSR to disconnect power to prevent pump damage.

I have a version of code wrote on an esp8266 platform called a Ledunia ( **broken link removed** ) I e-mailed them a few weeks ago and they seem to have taken down their main product page This sketch is mainly to provide data logging and hopefully alerting via IoT of an alarm condition when thresholds are breached. This will hopefully alert me to long running pump times (kids left water hose on while filling up animal water buckets), Pump is just running for the hell of it and using electricity (this could be an indication of a faulty pump or pressure control), or pump is cycling on and off too quickly which will lead to a worn out pump and wear out the pressure control contact switch. Overall I am trying to figure out if my pump is on when it should not be and wasting electricity; as I seem to have some high energy use in our household. The well pump is set to turn off at about 55PSI and turn on at about 30PSI. I have a bladder pressure tank and a pressure flow regulating valve set to allow the pump to continuously run when the pressure tank is full and there is still flow happening to try and curb any on/off cycling.

Eitherway, it is roughly a esp8266 with 32mb of memory that is in the shape of a USB dongle. I basically used a button press example but I can not for the life of me figure out how to implement an easy counting solution. Meaning that the pump state change happens too fast for it to record the change to thingspeak. The only values that are coming out of the program that record somewhere to thingspeak is the cycle rate per min and cyclerate per hour. I do have data that feeds to thingspeak, but unsure on how to record pump cycles when my IoT resolution is every 15 sec, which means if I see pump state is <ON> and in turns on and off 5 times in 15 seconds I have no way to know that on the thingspeak page. How should I record the pump "on time" over time so I can see how it behaves for various uses(example kids watering animals, shower, laundry, dishwaser, etc).

What are some of your ideas on implementing such a solution? Should I set up a local database on a desktop to record the data stream too and if so what DB should I use? If I set it up locally how do I display the data on a that same machine so I can see it in graph form similar to how do on thingspeak? Should I change by counter to data gathering every 20 seconds and only send data every 20 seconds to thingspeak. Then reset the counter bucket. Should implement some type of array to store the data in on the esp8266 and then average the values to send out to Thingspeak? If I do a local DB I shoud be able to record data every second correct?

Thank you for looking over this project and giving your feedback. I appreciate your time!

I am avoiding using delay by using millis() as much as possible. I am also grabbing time via the NTPclient.

I basically did the the following for counting cycle rate per min, below is written in Arduino IDE:
Link to Thingspeak public channel:

http://thingspeak.com/channels/440869

void pumpTimer() {
  static byte sw1;  //used for recording the pumpOn time only once and resets when pump goes off
  pumpState=digitalRead(PUMP);
  if(pumpState==HIGH) {  //pump is on
    //Serial.println ("PRESSED");
    if (sw1==0){
      pumpOn = currentMillis; //records time pump turned on
      sw1=1;  //bypass this on next loop through until pump turns off
    }
  }
  else {  //pump is off
    //Serial.println ("NOT ON");
    if (sw1==1){
    pumpOff = currentMillis;  //capture time pump turned off
    pumpTime = pumpOff - pumpOn;  //calculate pump runtime
    sw1=0; //reset check bit
    cycleCount++;  //sketc lifetime counter
    cyclesPerMinTemp++; //buket that is reset every 60 sec
    cyclesPerHrTemp++;  //bucket that is reset every 1 hour
    if(pumpTime > maxCycle) {
      maxCycle=pumpTime;
    }
    //set min and max cycle times
    if(pumpTime < minCycle){
      minCycle=pumpTime;
    }
    }
    //if(mTimer==0) {
    //  cycleMTime=currentMillis;
    //  mTimer=1;
    //}
  }
  //logic for minute and hour cycle counts
  if (currentMillis - cycleMTime > 60000) {
    //mTimer=0;
    cyclesPerMin = cyclesPerMinTemp;  //set bucket from temp bucket
    cyclesPerMinTemp = 0;  //reset temp bucket
    cycleMTime=currentMillis;  //record current time
      if (currentMillis - cycleHTime > 360000) {
    //mTimer=0;
    cyclesPerHr = cyclesPerHrTemp;
    cyclesPerHrTemp = 0;
    cycleHTime=currentMillis;
  }
  }
}

 

[dr pepper]

Interesting project.
I've built a few things with the Esp8266.
I'd probably just use the Esp, the chip can serve a webpage from its flash using spiffs, this page could contain a table with all the data you want on it, and you dont need the extra computer on burning up power.
There are platforms you can use, such as blynk and thingspeak, however I like to build my projects that do not depend on other services, but the code is a bit tougher to write.
To write webpages you need to know Html, Javascript & maybe Jquery reasonably well, then if you want things like graphs you can use google charts.
How are you getting on with Ntptime, I'm having trouble with my current project, the code often on power up doesnt get the time at all.

 

[jclaudii]

The Ntptime function is as follows and seems to always capture the time correctly and puts it on my serial out. I have not determined how to send that time to thingspeak, but I really don't need too as it puts timestamp on the data when it is sent.

I included both
#include <NTPClient.h>
#include <WiFiUdp.h>

//time handling for ntpclient
const long utcOffsetInSeconds = -21600;
char daysOfTheWeek[7][12] = {"Sun", "Mon", "Tue", "Wed", "Thu", "Fri", "Sat"};
// Define NTP Client to get time
WiFiUDP ntpUDP;
NTPClient timeClient(ntpUDP, "pool.ntp.org", utcOffsetInSeconds);

void setup() {
  Serial.begin(115200);
  connectWifi();
  DS18B20.begin();
  timeClient.begin();
  pinMode(PUMP, INPUT);

void ntpTimeDate ()
{
   timeClient.update();
  Serial.print(daysOfTheWeek[timeClient.getDay()]);
  Serial.print(", ");
  Serial.print(timeClient.getHours());
  Serial.print(":");
  Serial.print(timeClient.getMinutes());
  Serial.print(":");
  Serial.println(timeClient.getSeconds());
  //Serial.println(timeClient.getFormattedTime());
 
}

I then used ntpTimeDate as a function and can call upon it anywhere I want. I think this method will work fairly well as it will allow me to put a time stamp with any data I record like you mentioned. I thought about storing the time/date local on the ESP and just letting it keep up with the time and only capture new time on void setup and like once every hour or so to save it from having to get the time from the internet everytime. It would especially help with errors, as I have no provision in my code for a time grab fail.

The 15 second limitation is causing me some issues on thingspeak. I may end up having to see if 20 second averages, 20 second cycle counts and current run time updated every 15 sec (but I am not sure how to record or display this data set) will give me useful data to use.

I did not know if the ESP had enough umph to server up this data as webserver and capture data every second. I was wanting local connectivity as I have more control of the data.

 

[dr pepper]

I'm using ntptimelib.
Playing with it now, got it a little more stable.
Told it to use port 8888, and the get routine exits if it doesnt get the time in 1 minute.
It takes a while to grab the time on boot, but seems to get it first time every time thereafter.
Udp as a protocol is not that reliable, I tried to use it to for comms over the net, looses quite a lot of packets.
The time taken for some of the calls in your code will vary quite a bit.

If your using thingspeak you dont need the time, have a look in the ide's basics examples, look at blink without delay, you can call routines at set time intervals without using delay, and without hanging up the chip, I use this a lot.
15 seconds is too soon in my experience to open & close a page, you need 25 really, if you want to send data at quicker intervals use something like ajax, however you might not be able to do that with thingspeak, I dont know it that well, like I say I dont want dependants.

Currently I'm serving 4 webpages with my project, and I'm using 30.5 kbytes of data aquisition.
Everything is in the chip, you can get esp's that have 4mB of flash(Ok not the same chip as the processor, but literally under the hood).
Theres now an esp32, I htink you can program it with the same code as the esp8266, it has a lot more features.

 

[jclaudii]

Dr pepper,

The Ledunia chip I am using has 32mb of memory on it, but I don't have an option to address it in the arduino compiler, so I just went with whatever was the largest memory bucket for esp8266.

I have removed almost any mention of delay in the actual code, but I have not looked at the accompanying .h files

I may try running a webserver on the little chip and see what happens.

A question to ponder: I am using 3.3v to button then to dataPin and to ground via a 10kohm resistor. This resistor gets quite hot when the button is held for 1 min or more. Would there be a better alternative resistor that would not get as hot? I usually just go with what is suggested in an example I find online and try that.

 

[Nigel Goodwin]

A question to ponder: I am using 3.3v to button then to dataPin and to ground via a 10kohm resistor. This resistor gets quite hot when the button is held for 1 min or more. Would there be a better alternative resistor that would not get as hot? I usually just go with what is suggested in an example I find online and try that.

The more usual way is to have the resistor to 3.3V (pull-up) and the button to ground - but it doesn't affect the operation of the circuit (basically it's 'safer' as you're not running power to switches).

As for the resistor getting hot - if it is, then you're NOT using a 10K resistor - you're using one MUCH smaller (at a guess I'd say you mis-read the colour codes - VERY easy on 1% resistors - and you're actually using 100 ohm instead). Dissipation in a 10K at 3.3V is only 1mW, so no way will it get warm.

 

[dr pepper]

I bet Nige is right, I wonder if you've mistaken 100r for 10k, I do stuff like that and worse now & again.
I'll look that chip up, sounds like its maybe a esp32.
My webpage is only 3k, but it draws 2 nice looking charts that are full screen, and the second page is about the same, only it has 4 guages.
My memory usage is 3k for the chart data table, and about 6 k for the webpages, the program only uses about 30% of the program memory, you can have a professional looking pages served by a little server with only a few K of memory, I use references so the browser gets its data from google etc for the charts & jquery stuff.
Some pins in the Esp I think have their own pull up resistors, you just turn them on, then you dont even need an outboard one.

 

[Nigel Goodwin]

I bet Nige is right, I wonder if you've mistaken 100r for 10k, I do stuff like that and worse now & again.

I have a great tendency to check them on one of my Chinese kit component testers - just to make sure

However, as we're doing small scale production, the resistors are usually in bags anyway - and 'mostly' have the value printed on the outside.

We ran out of 100 ohms the other day (and we needed just a few more to complete an order, and get it sent off), and 'digging deep' we managed to find some old 5% ones, it was lovely seeing resistors that you can easily read again.

My webpage is only 3k, but it draws 2 nice looking charts that are full screen, and the second page is about the same, only it has 4 guages.

Sounds good - how about posing some screen shots?

 

[dr pepper]

Surething:
Its still in development so the edges need tidying, but its nearly there.
Some of the code is borrowed & adapted & some is mine.
I've done other stuff than weather however that is for 'work' so I dont want to show.

 

[Nigel Goodwin]

Nice - although I do think you could have nicely 'cut out' the addresses, rather than scribbling over them!

 

[dr pepper]

Couldnt be bothered, and works computers are 'locked' up so tightly all I can use is mspaint.

I'm having all kinds of issues with Ntptime, incorrect time, no time reply, function crashes when called, so I'm now working on geting the browser to send the time as an argument when it calls for json data, the time will only update when the site is accessed, but hopefully will be more relaible than Udp.

 

[Nigel Goodwin]

Couldnt be bothered, and works computers are 'locked' up so tightly all I can use is mspaint.

All I use is mspaint as well, it's dead easy and dead quick - to be fair I don't 'cut out' the addresses, I simply crop the part of the screen that includes the part I want, it only take a couple of button presses - like this:

 

[jclaudii]

i like the windows 10 utility called snippit for copy what is on a screen and pasting it almost anywhere.

Dr. Pepper, We both may and try to implement a time library like https://www.pjrc.com/teensy/td_libs_Time.html . We can set the time once every 24 hours or some such logic to compare the esp time to the ntp server to see if it needs updating. This should save time by not having the time function having to grab the time from the internet whenever we need to display our output in our program. Lets let the micro-controller keep up with it after we set it instead of getting it every time

That webpage is what I am looking for, basically a simple gage/number/icon to show current status of pump (on/off), and elapsed pump run time if pump is "on". Some trend charts like you have as well for the pump cycle count and other metrics I mentioned above. Is it possible to adjust the time scale of the trends? LIke have a link above the chart for (hour - 24 - 48 - 72 - week - month) to I'm guessing each link would just go to a separate page for each scale or adjust the scale values that are stored in the file. If you can send me some links or info on how you implemented this I will start playing with it as well.

I'll take a look at my resistor values, could have easily misread the numbers or even been packaged wrong

 

[Nigel Goodwin]

i like the windows 10 utility called snippit for copy what is on a screen and pasting it almost anywhere.

How is that any different to the PrintScrn button on your keyboard?

I simply press PrintScrn, run Mspaint, press ctrl-V, click on 'rectangular select', draw the area I want and click on crop - then save what I've selected.

 

[dr pepper]

I used print screen but come to think of it I have snippet too.

I'm a bit of a time nut, on this site you'll see frequency standards, clocks and the like that use various radio time signals that I've built.
I've seen many times on this site where someone has an issue, totally overcomplicates it then realises its somthing stupid.
tI'm a bonehead too, the sync function I wrote initialises the Ntp system, so I've been repeatedly initialising it, no wonder it messed up.
Correcting that didnt make it work perfect, but its a lot better.
The Ds32321 module is cheap off ebay and it has a temp compensated osc so its accrate to a second a year.

My pages use google developer tools, you can use them free, and download them and install them on the device if you want, I didnt, the browser gets them itself on load.
The gauge is google gauge and the chart google chart, yes you can fix the scales, I have done that on mine today.
The hardest part I found with charts was getting ajax to work, I used jquery.
Of course now it works it all seems easy.
If you dont want your chart to update while the page is open its a lot easier to do, but the chart doesnt change till the page is refreshed.

Google charts:
**broken link removed**
If you copy the examples that start <html> and end </html> into wordpad, then save them with a .html extension, you can then open them in google or whatever browser.
Then you can copy it into spiffs & get the esp to serve them to a browser over the net, if you then manipulate the values you have your own chart.
Sorry about the scribbles, and yes Im listening to the cure.

 

[Nigel Goodwin]

I'm a bit of a time nut, on this site you'll see frequency standards, clocks and the like that use various radio time signals that I've built.
I've seen many times on this site where someone has an issue, totally overcomplicates it then realises its somthing stupid.
tI'm a bonehead too, the sync function I wrote initialises the Ntp system, so I've been repeatedly initialising it, no wonder it messed up.
Correcting that didnt make it work perfect, but its a lot better.
The Ds32321 module is cheap off ebay and it has a temp compensated osc so its accrate to a second a year.

Assuming you have a website?, that provides PHP?, you can write a very simple script that your device calls that simply reads the time and sends it back to you as simple text.

A number of my devices run an RTC using a 32KHz xtal using TMR1 on a PIC, with the PIC in sleep mode - and everything possible shut down.

The 32KHz clock generates an interrupt every second, and the PIC wakes up, does the clock timekeeping, and checks that it's to time to do something yet, and goes back to sleep.

Once it decides it is time, it powers up a SIM800L modem, powers up an I2C memory (power to that is turned OFF during sleep as well), and connects to my time server script, this downloads the time and is used to reset the clock registers in the RTC, and also a flag to tell it if any settings have been altered. If the flag is set, it then downloads all the settings via another PHP/MYQL script and stores them in the non-volatile I2C memory, plus a copy in the normal RAM variables - and also resets the update flag.

It then continues on to send it's readings.

Here is an example of what my time script returns, this is simply cut and pasted from the browser I just used to run it:

"2019","8","24","16","9","47"," 6"

It's just a simple text string, there's no HTML whatsoever, so it's dead easy to process - and you can alter the separators and order to be anything you like in the PHP file. Here's the simple PHP file that generated it:

<?php
// Print the array from getdate()
// Set timezone
date_default_timezone_set("Europe/London");

// Return date/time info of a timestamp; then format the output
$mydate=getdate(date("U"));
//echo "$mydate[weekday], $mydate[month] $mydate[mday], $mydate[year]";
//echo "Year, Month, Day, Hour, Minute, Second, Day Of Week";
//echo "<br>";
echo "\"";
echo "$mydate[year]";
echo "\",\"";
echo "$mydate[mon]";
echo "\",\"";
echo "$mydate[mday]";
echo "\",\"";
echo "$mydate[hours]";
echo "\",\"";
echo "$mydate[minutes]";
echo "\",\"";
echo "$mydate[seconds]";
echo "\",\"";
echo "$mydate[wday]";
echo "\"";
?>

 

[dr pepper]

No I dont have a site.
I'm more interested in frequency than time.
Yes I have a few projects that use a pic with a 32kc xtal, microchip even list a routine to keep time in the datasheet, I've used timer1 and the watchdog timer to wake the ic from sleep.
One of mine uses a roman black 'crystal oven', its ghetto to say the least but works.
Php is a useful thing, I havent used it much.
Getting javascript to calc dew point & heat index was hard enough.

 

[jclaudii]

How is that any different to the PrintScrn button on your keyboard?

I simply press PrintScrn, run Mspaint, press ctrl-V, click on 'rectangular select', draw the area I want and click on crop - then save what I've selected.

You can draw/select the exact "thing" you want from the screen. Similar to the crop bars that pop up when you are editing an image. It saves it to the clipboard just like PrintScr. Below is a cut and past from Dr Pepper's screenshot he took above. I was able to quickly drag my selection which is automatically copied to the clipboard and then just CTRL-f here and it shows up. No need to blur out url's or other tabs you don't want folks to see.

 

[Nigel Goodwin]

You can draw/select the exact "thing" you want from the screen. Similar to the crop bars that pop up when you are editing an image. It saves it to the clipboard just like PrintScr. Below is a cut and past from Dr Pepper's screenshot he took above. I was able to quickly drag my selection which is automatically copied to the clipboard and then just CTRL-f here and it shows up. No need to blur out url's or other tabs you don't want folks to see.
View attachment 120121

OK, see what you mean - I just do the exaxct same thing in MSPaint

 

[jclaudii]

I am looking around trying to find the best tutorial for SPIFF's , please comment if you have one you followed or would like to share. I have found https://diyprojects.io/esp8266-web-...terface-spiffs-area-html-css-js/#.XWQ393t7lhE and will try and follow it to see if I can create what you have above. Hopefully I can get it to render something