AM2315 Temp/Humidity Highly Reliable ESP8266 Arduino Driver

During the development of OurWeather (A complete Weather Kit) and the WeatherPlus weather station controller, we had to work through a number of issues with the I2C AM2315 Temperature / Humidity Sensor in terms of long term reliability.   We would read the AM2315 every 5 seconds and it would fail about every (at best) 8000 reads.   The failure was that it would report bad data (NaN – Not  a Number) for temperature and humidity.   A bad read once in while can be programmed around (detecting the bad data and re-reading the values, for example).   However, in this case the AM2315 would completely lock up (reading bad data continuously) and no amount of fussing with reseting the I2C bus would fix it.    You had to power cycle the device.

We went back and forth with the manufacturer for the AM2315.   They had some good suggestions and asked us lots of questions, but none of the suggestions would work.   We finally solved the issue (OurWeather has been running for 23 days now and has read the AM2315 over 425,000 times with no bad reads.   Our guess on the statistics of when we would say, with a ~99% certainty, that it was fixed was about 100,000 reads.     We are now shipping OurWeather units with the driver installed in the code.

The driver is a substantial modification of the original Adafruit driver with other modifications by EasternStarGeek.

Where are the AM2315 ESP8266 Software Drivers?

You can find these drivers on github here:  https://github.com/switchdoclabs/SDL_ESP8266_HR_AM2315

What Was the Problem?

After a great deal of debugging, we came to the conclusion that some interrupt or activity on the ESP8266 WiFi system was hosing the I2C reads for the AM2315 and the resulting issue, whatever it was, would lock up the AM2315 to the point where we had to power cycle the AM2315.   Not a good issue for a product that is supposed to run for months at a time, if not years.

What Did We Change?

It is the combination of all five of these changes that dramatically improved the reliability of the AM2315 in an ESP8266 system.

Change 1:   Reading the Data Twice to Start

After talking with the manufacturer, we understood the issue raised by a number of different people.   Why do you have to read the data twice?   Well, it turns out you don’t.   However, you need to wake up the AM2315 processor, delay a bit, then you can read the data.   They sleep the internal processor to avoid self-heating the temperature sensor by running the internal processor only when needed and then going back into a sleep.   The disadvantage?   You can only read the sensor every 3 seconds.   Don’t try to read it more often than every 3 seconds.  Strictly speaking, this is not an ESP8266 only change.  This same method will work on a Raspberry Pi or Arduino.

 // Wake up the AM2315
 Wire.beginTransmission(AM2315_I2CADDR);
 Wire.write(AM2315_READREG);
 Wire.endTransmission();

 delayByCPU(50);

 Wire.beginTransmission(AM2315_I2CADDR);
 Wire.write(AM2315_READREG);
 Wire.write(0x00); // start at address 0x0
 Wire.write(4); // request 4 bytes data
 Wire.endTransmission();

Change 2:   Disable as many ESP8266 Interrupts as Possible during the I2C AM2315 Read

When we looked at the distribution of the failures over the course of 20+ runs, we noticed that you could change the average by adding delays and things, but there was still a random component to the failures, well beyond the 1 or 2% we would expect with measurement error.    This told us that there were two asynchronous processes running (WiFi and our main loop we expect) that would occasionally interact causing the AM2315 failure.   While this might seem like a stretch to conclude, our CTO, Dr. John Shovic, has debugged complex asynchronous systems many times, and he recognized the symptoms.   Since the ESP8266 is an multitasking interrupt driven system under the user code, we disabled as many interrupts as possible.

At the beginning of the I2C Read sequence:

 noInterrupts();
 ETS_INTR_LOCK();

and then at the conclusion of the I2C read:

 interrupts();
 ETS_UART_INTR_ENABLE();

This one time improved the reliability of the reads from about 400 reads on the average to about 2000 reads on the average.

Change 3: Delay by CPU usage, not by delay()

When you use the delay() or the yield() function, you are giving the processor back to the ESP8266 operating system to use for the WiFi and other housekeeping tasks.   Your process is stopped, but the underlying processes continue.   You HAVE to delay() or yield() in your code or the ESP8266 will throw a WDT (watchdog timer) reset and crash.  You should really not exceed about 250ms without calling delay() or yield().   The 250ms number is a conservative estimate with ranges from 1sec to 3.2sec as values for the WDT on the net.

Without a doubt, this dealyByCPU function below is an odd looking piece of code.   Why not use use for loops force the CPU to waste time between the I2C reads?   It is because the compiler will optimize empty or non-reference for loops right out of the code, causing no delay at all. We ended up doing some math and then passing the result out of the routine so the compiler wouldn’t just remove the whole bit of the code. Then we could consume the CPU for these short (50msec) bursts allowing no ESP8266 underlying code to get in and hose things. This improved our reliability up to about the 8000 read level. Better but not good enough.

 

int delayByCPU(long delaycount)
{
  unsigned long startMillis;
  unsigned long endMillis;

#define COUNT 100000
  startMillis = millis();

  long index;
  float test;
  long i;
  long j;

  for (i = 0; i < delaycount * 166; i++)
  {
    //Serial.println("outside loop");
    for (j = 0; j < 1000; j++)
    {
      for (index = 0; index < COUNT; index++)
      {

        test = 30.4 + i;
        test = test / 50.0;


      }
      test = test + j;
    }
  }

  endMillis = millis();



  return int(test);
}

By the way, you don’t want to use this type of code to delay() in general. You will trigger then ESP8266 watchdog timer if you delay too long. Keep delayByCPU() under 100ms per call.

Change 4: Increase the speed of the I2C Bus to 400KHz for the AM2315

This is by far the most controversial change we made to the code. But it was also the final element that made our code highly reliable. This call increases the speed of the I2C Bus to 400KHz from he default (set by Wire.begin()) of 100KHz. Virtually all I2C devices can run at 400KHz, as long as the I2C cables are not too long and have too much capacitance on them (it is an interaction between the pulldowns of the device, the capacitance on the lines, the resistance of the wires and the I2C pull-ups on the line, as well as what voltage the I2C bus is running at).

We talked to the manufacturer and they hadn’t tested it and their spec implied that 100KHz was the max. However, this worked like a champ. We did not verify what the clock speed was on the bus (it is a software I2C “bit banging” on the ESP8266) but this worked like a champ. After this change, we have not had a failure.  Not a single one.

  Wire.begin(5, 4);   // some ESP8266 devices require 5, 4 instead of 4,5
  Wire.setClock(400000L);

The HotBox

Yes, we were nervous about implementing this fast I2C bus in a production product.   What were the margins for use in -40C (-40 degrees F) to  +54C Remembering that circuits slow down when heated up, by heating the unit, if it is close to failing, we should be able to make it fall.   To determine if we were on the edge of the device we placed the AM2315 in a hot box and set the temperature to +78C (+172 degrees F).   We did this by taking a plastic outdoor box and placing a 100W bulb inside and then regulating the temperature by opening some vents in the tape around the box.  It sat very close to 176 for hours.

Running the hotbox for days, we had absolutely no failures.    We were not close to failure at our normal operating temperatures.

Conclusion

We now have a reliable set of drivers for the AM2315 Temperature and Humidity sensor.   This is the production driver that has been released in the OurWeather product and in our WeatherPlus controller.