Preventing torn reads with an HCS12 microcontroller

Question

Summary

I'm trying to write an embedded application for an MC9S12VR microcontroller. This is a 16-bit microcontroller but some of the values I deal with are 32 bits wide and while debugging I've captured some anomalous values that seem to be due to torn reads.

I'm writing the firmware for this micro in C89 and running it through the Freescale HC12 compiler, and I'm wondering if anyone has any suggestions on how to prevent them on this particular microcontroller assuming that this is the case.

Details

Part of my application involves driving a motor and estimating its position and speed based on pulses generated by an encoder (a pulse is generated on every full rotation of the motor).

For this to work, I need to configure one of the MCU timers so that I can track the time elapsed between pulses. However, the timer has a clock rate of 3 MHz (after prescaling) and the timer counter register is only 16-bit, so the counter overflows every ~22ms. To compensate, I set up an interrupt handler that fires on a timer counter overflow, and this increments an "overflow" variable by 1:

// TEMP
static volatile unsigned long _timerOverflowsNoReset;

// ...

#ifndef __INTELLISENSE__
__interrupt VectorNumber_Vtimovf
#endif
void timovf_isr(void)
{
  // Clear the interrupt.
  TFLG2_TOF = 1;

  // TEMP
  _timerOverflowsNoReset++;

  // ...
}

I can then work out the current time from this:

// TEMP
unsigned long MOTOR_GetCurrentTime(void)
{
  const unsigned long ticksPerCycle = 0xFFFF;
  const unsigned long ticksPerMicrosecond = 3; // 24 MHZ / 8 (prescaler)
  const unsigned long ticks = _timerOverflowsNoReset * ticksPerCycle + TCNT;
  const unsigned long microseconds = ticks / ticksPerMicrosecond;

  return microseconds;
}

In main.c, I've temporarily written some debugging code that drives the motor in one direction and then takes "snapshots" of various data at regular intervals:

// Test
for (iter = 0; iter < 10; iter++)
{
  nextWait += SECONDS(secondsPerIteration);
  while ((_test2Snapshots[iter].elapsed = MOTOR_GetCurrentTime() - startTime) < nextWait);
  _test2Snapshots[iter].position = MOTOR_GetCount();
  _test2Snapshots[iter].phase = MOTOR_GetPhase();
  _test2Snapshots[iter].time = MOTOR_GetCurrentTime() - startTime;
  // ...

In this test I'm reading MOTOR_GetCurrentTime() in two places very close together in code and assign them to properties of a globally available struct.

In almost every case, I find that the first value read is a few microseconds beyond the point the while loop should terminate, and the second read is a few microseconds after that - this is expected. However, occasionally I find the first read is significantly higher than the point the while loop should terminate at, and then the second read is less than the first value (as well as the termination value).

The screenshot below gives an example of this. It took about 20 repeats of the test before I was able to reproduce it. In the code, <snapshot>.elapsed is written to before <snapshot>.time so I expect it to have a slightly smaller value:

For snapshot[8], my application first reads 20010014 (over 10ms beyond where it should have terminated the busy-loop) and then reads 19988209. As I mentioned above, an overflow occurs every 22ms - specifically, a difference in _timerOverflowsNoReset of one unit will produce a difference of 65535 / 3 in the calculated microsecond value. If we account for this:

A difference of 40 isn't that far off the discrepancy I see between my other pairs of reads (~23/24), so my guess is that there's some kind of tear going on involving an off-by-one read of _timerOverflowsNoReset. As in while busy-looping, it will perform one call to MOTOR_GetCurrentTime() that erroneously sees _timerOverflowsNoReset as one greater than it actually is, causing the loop to end early, and then on the next read after that it sees the correct value again.

I have other problems with my application that I'm having trouble pinning down, and I'm hoping that if I resolve this, it might resolve these other problems as well if they share a similar cause.

Edit: Among other changes, I've changed _timerOverflowsNoReset and some other globals from 32-bit unsigned to 16-bit unsigned in the implementation I now have.

Answer 1

Calculate the tick count, then check if while doing that the overflow changed, and if so repeat;

#define TCNT_BITS 16 ; // TCNT register width

uint32_t MOTOR_GetCurrentTicks(void)
{
   uint32_t ticks = 0 ;
   uint32_t overflow_count = 0;

   do
   {
       overflow_count = _timerOverflowsNoReset ;
       ticks = (overflow_count << TCNT_BITS) | TCNT;
   }
   while( overflow_count != _timerOverflowsNoReset ) ;

   return ticks ;
}

the while loop will iterate either once or twice no more.

Answer 2

You can read this value TWICE:

unsigned long GetTmrOverflowNo()
{
    unsigned long ovfl1, ovfl2;
    do {
        ovfl1 = _timerOverflowsNoReset;
        ovfl2 = _timerOverflowsNoReset;
    } while (ovfl1 != ovfl2);
    return ovfl1;
}

unsigned long MOTOR_GetCurrentTime(void)
{
  const unsigned long ticksPerCycle = 0xFFFF;
  const unsigned long ticksPerMicrosecond = 3; // 24 MHZ / 8 (prescaler)
  const unsigned long ticks = GetTmrOverflowNo() * ticksPerCycle + TCNT;
  const unsigned long microseconds = ticks / ticksPerMicrosecond;

  return microseconds;
}

If _timerOverflowsNoReset increments much slower then execution of GetTmrOverflowNo(), in worst case inner loop runs only two times. In most cases ovfl1 and ovfl2 will be equal after first run of while() loop.

Answer 3

The problem is that the writes to _timerOverflowsNoReset isn't atomic and you don't protect them. This is a bug. Writing atomic from the ISR isn't very important, as the HCS12 blocks the background program during interrupt. But reading atomic in the background program is absolutely necessary.

Also, have in mind that Codewarrior/HCS12 generates somewhat ineffective code for 32 bit arithmetic.

Here is how you can fix it:

• Drop unsigned long for the shared variable. In fact you don't need a counter at all, given that your background program can service the variable within 22ms real-time - should be very easy requirement. Keep your 32 bit counter local and away from the ISR.
• Ensure that reads of the shared variable are atomic. Disassemble! It must be a single MOV instruction or similar; otherwise you must implement semaphores.
• Don't read any volatile variable inside complex expressions. Not only the shared variable but also the TCNT. Your program as it stands has a tight coupling between the slow 32 bit arithmetic algorithm's speed and the timer, which is very bad. You won't be able to reliably read TCNT with any accuracy, and to make things worse you call this function from other complex code.

---

Your code should be changed to something like this:

static volatile bool overflow;


void timovf_isr(void)
{
  // Clear the interrupt.
  TFLG2_TOF = 1;

  // TEMP
  overflow = true;

  // ...
}

unsigned long MOTOR_GetCurrentTime(void)
{
  bool of = overflow;   // read this on a line of its own, ensure this is atomic!
  uint16_t tcnt = TCNT; // read this on a line of its own

  overflow = false; // ensure this is atomic too

  if(of)
  {
    _timerOverflowsNoReset++;
  }

  /* calculations here */

  return microseconds;
}

If you don't end up with atomic reads, you will have to implement semaphores, block the timer interrupt or write the reading code in inline assembler (my recommendation).

Overall I would say that your design relying on TOF is somewhat questionable. I think it would be better to set up a dedicated timer channel and let it count up a known time unit (10ms?). Any reason why you can't use one of the 8 timer channels for this?

Answer 4

The atomic reads are not the main problem here.
It's the problem that the overflow-ISR and TCNT are highly related.
And you get problems when you read first TCNT and then the overflow counter.

Three sample situations:

TCNT=0x0000, Overflow=0   --- okay
TCNT=0xFFFF, Overflow=1   --- fails
TCNT=0x0001, Overflow=1   --- okay again

You got the same problems, when you change the order to: First read overflow, then TCNT.

You could solve it with reading twice the totalOverflow counter.

disable_ints();
uint16_t overflowsA=totalOverflows;
uint16_t cnt = TCNT;
uint16_t overflowsB=totalOverflows;
enable_ints();

uint32_t totalCnt = cnt;

if ( overflowsA != overflowsB )
{
   if (cnt < 0x4000)
      totalCnt += 0x10000;
}

totalCnt += (uint32_t)overflowsA << 16;

If the totalOverflowCounter changed while reading the TCNT, then it's necessary to check if the value in tcnt is already greater 0 (but below ex. 0x4000) or if tcnt is just before the overflow.

Answer 5

It all boils down to the question of how often you do read the timer and how long the maximum interrupt sequence will be in your system (i.e. the maximum time the timer code can be stopped without making "substantial" progress).

Iff you test for time stamps more often than the cycle time of your hardware timer AND those tests have the guarantee that the end of one test is no further apart from the start of its predecessor than one interval (in your case 22ms), all is well. In the case your code is held up for so long that these preconditions don't hold, the following solution will not work - the question then however is whether the time information coming from such a system has any value at all.

The good thing is that you don't need an interrupt at all - any try to compensate for the inability of the system to satisfy two equally hard RT problems - updating your overflow timer and delivering the hardware time is either futile or ugly plus not meeting the basic system properties.

unsigned long MOTOR_GetCurrentTime(void)
{
  static uint16_t last;
  static uint16_t hi;
  volatile uint16_t now = TCNT;

  if (now < last)
  {
     hi++;
  }
  last = now;
  return now + (hi * 65536UL);
}

BTW: I return ticks, not microseconds. Don't mix concerns.

PS: the caveat is that such a function is not reentrant and in a sense a true singleton.

Answer 6

Based on the answers @AlexeyEsaulenko and @jeb provided, I gained understanding into the cause of this problem and how I could tackle it. As both their answers were helpful and the solution I currently have is sort of a mixture of the two, I can't decide which of the two answers to accept, so instead I'll upvote both answers and keep this question open.

This is how I now implement MOTOR_GetCurrentTime:

unsigned long MOTOR_GetCurrentTime(void)
{
  const unsigned long ticksPerMicrosecond = 3; // 24 MHZ / 8 (prescaler)
  unsigned int countA;
  unsigned int countB;
  unsigned int timerOverflowsA;
  unsigned int timerOverflowsB;
  unsigned long ticks;
  unsigned long microseconds;

  // Loops until TCNT and the timer overflow count can be reliably determined.
  do
  {
    timerOverflowsA = _timerOverflowsNoReset;
    countA = TCNT;
    timerOverflowsB = _timerOverflowsNoReset;
    countB = TCNT;
  } while (timerOverflowsA != timerOverflowsB || countA >= countB);

  ticks = ((unsigned long)timerOverflowsA << 16) + countA;
  microseconds = ticks / ticksPerMicrosecond;
  return microseconds;
}

This function might not be as efficient as other proposed answers, but it gives me confidence that it will avoid some of the pitfalls that have been brought to light. It works by repeatedly reading both the timer overflow count and TCNT register twice, and only exiting the loop when the following two conditions are satisfied:

1. the timer overflow count hasn't changed while reading TCNT for the first time in the loop
2. the second count is greater than the first count

This basically means that if MOTOR_GetCurrentTime is called around the time that a timer overflow occurs, we wait until we've safely moved on to the next cycle, indicated by the second TCNT read being greater than the first (e.g. 0x0001 > 0x0000).

This does mean that the function blocks until TCNT increments at least once, but since that occurs every 333 nanoseconds I don't see it being problematic.

I've tried running my test 20 times in a row and haven't noticed any tearing, so I believe this works. I'll continue to test and update this answer if I'm wrong and the issue persists.

Edit: As Vroomfondel points out in the comments below, the check I do involving countA and countB also incidentally works for me and can potentially cause the loop to repeat indefinitely if _timerOverflowsNoReset is read fast enough. I'll update this answer when I've come up with something to address this.

Answer 7

One technique that can be helpful is to maintain two or three values that, collectively, hold overlapping portions of a larger value.

If one knows that a value will be monotonically increasing, and one will never go more than 65,280 counts between calls to "update timer" function, one could use something like:

// Note: Assuming a platform where 16-bit loads and stores are atomic
uint16_t volatile timerHi, timerMed, timerLow;

void updateTimer(void)  // Must be only thing that writes timers!
{
  timerLow = HARDWARE_TIMER;
  timerMed += (uint8_t)((timerLow >> 8) - timerMed);
  timerHi  += (uint8_t)((timerMed >> 8) - timerHi);
}

uint32_t readTimer(void)
{
  uint16_t tempTimerHi  = timerHi;
  uint16_t tempTimerMed = timerMed;
  uint16_t tempTimerLow = timerLow;
  tempTimerMed += (uint8_t)((tempTimerLow >> 8) - tempTimerMed);
  tempTimerHi  += (uint8_t)((tempTimerMed >> 8) - tempTimerHi);
  return ((uint32_t)tempTimerHi) << 16) | tempTimerLow;
}

Note that readTimer reads timerHi before it reads timerLow. It's possible that updateTimer might update timerLow or timerMed between the time readTimer reads timerHi and the time it reads those other values, but if that occurs, it will notice that the lower part of timerHi needs to be incremented to match the upper part of the value that got updated later.

This approach can be cascaded to arbitrary length, and need not use a full 8 bits of overlap. Using 8 bits of overlap, however, makes it possible to form a 32-bit value by using the upper and lower values while simply ignoring the middle one. If less overlap were used, all three values would need to take part in the final computation.