text:
Can I record us (microseconds) with AnalogBinLogger example in SdFat's library?
I am using the AnalogBinLogger from the SdFat example library. I  am using it to record 8 analog channels at 5kHz! which is amazing (thank you SdFat).
Is it possible to include a separate column to provide a time-stamp in microseconds for when the samples were recorded? Or will this impair the speed at which I can log data?
Or; how reliable is the sample rate? To what degree of confidence can I assume that the sketch executes (in my case) every 200 microseconds?
I'm performing frequency analysis on the data so knowing when the data is captured or if there is any drift or skew is important to me.
Thank you in advance Arduino Crew!
...appended...
To answer @jsotola:
The sketch I am using (see below) comes with the SdFat library, which is beyond my basic understanding of code. I can write a for loop and an if statement but the stuff that goes on in this sketch is levels above where I am at right now. Trust me, if I could've I would've.
To answer @Gerben:
The 5kHz sampling rate is the default sampling rate that comes with the example sketch. As long as I can be confident that the samples are taken every 200us then I wont need the timestamp but I don't know how confident I can be or how to write a timestamp into the code.
/**
 * This program logs data from the Arduino ADC to a binary file.
 *
 * Samples are logged at regular intervals. Each Sample consists of the ADC
 * values for the analog pins defined in the PIN_LIST array.  The pins numbers
 * may be in any order.
 *
 * Edit the configuration constants below to set the sample pins, sample rate,
 * and other configuration values.
 *
 * If your SD card has a long write latency, it may be necessary to use
 * slower sample rates.  Using a Mega Arduino helps overcome latency
 * problems since 13 512 byte buffers will be used.
 *
 * Each 512 byte data block in the file has a four byte header followed by up
 * to 508 bytes of data. (508 values in 8-bit mode or 254 values in 10-bit mode)
 * Each block contains an integral number of samples with unused space at the
 * end of the block.
 *
 * Data is written to the file using a SD multiple block write command.
 */
#ifdef __AVR__
#include <SPI.h>
#include "SdFat.h"
#include "FreeStack.h"
#include "AnalogBinLogger.h"
//------------------------------------------------------------------------------
// Analog pin number list for a sample.  Pins may be in any order and pin
// numbers may be repeated.
const uint8_t PIN_LIST[] = {0, 1, 2, 3, 4, 5, 6, 7};
//------------------------------------------------------------------------------
// Sample rate in samples per second.
const float SAMPLE_RATE = 5000;  // Must be 0.25 or greater.

// The interval between samples in seconds, SAMPLE_INTERVAL, may be set to a
// constant instead of being calculated from SAMPLE_RATE.  SAMPLE_RATE is not
// used in the code below.  For example, setting SAMPLE_INTERVAL = 2.0e-4
// will result in a 200 microsecond sample interval.
const float SAMPLE_INTERVAL = 1.0/SAMPLE_RATE;

// Setting ROUND_SAMPLE_INTERVAL non-zero will cause the sample interval to
// be rounded to a a multiple of the ADC clock period and will reduce sample
// time jitter.
#define ROUND_SAMPLE_INTERVAL 1
//------------------------------------------------------------------------------
// ADC clock rate.
// The ADC clock rate is normally calculated from the pin count and sample
// interval.  The calculation attempts to use the lowest possible ADC clock
// rate.
//
// You can select an ADC clock rate by defining the symbol ADC_PRESCALER to
// one of these values.  You must choose an appropriate ADC clock rate for
// your sample interval.
// #define ADC_PRESCALER 7 // F_CPU/128 125 kHz on an Uno
// #define ADC_PRESCALER 6 // F_CPU/64  250 kHz on an Uno
// #define ADC_PRESCALER 5 // F_CPU/32  500 kHz on an Uno
// #define ADC_PRESCALER 4 // F_CPU/16 1000 kHz on an Uno
// #define ADC_PRESCALER 3 // F_CPU/8  2000 kHz on an Uno (8-bit mode only)
//------------------------------------------------------------------------------
// Reference voltage.  See the processor data-sheet for reference details.
// uint8_t const ADC_REF = 0; // External Reference AREF pin.
uint8_t const ADC_REF = (1 << REFS0);  // Vcc Reference.
// uint8_t const ADC_REF = (1 << REFS1);  // Internal 1.1 (only 644 1284P Mega)
// uint8_t const ADC_REF = (1 << REFS1) | (1 << REFS0);  // Internal 1.1 or 2.56
//------------------------------------------------------------------------------
// File definitions.
//
// Maximum file size in blocks.
// The program creates a contiguous file with FILE_BLOCK_COUNT 512 byte blocks.
// This file is flash erased using special SD commands.  The file will be
// truncated if logging is stopped early.
const uint32_t FILE_BLOCK_COUNT = 256000;

// log file base name.  Must be six characters or less.
#define FILE_BASE_NAME "analog"

// Set RECORD_EIGHT_BITS non-zero to record only the high 8-bits of the ADC.
#define RECORD_EIGHT_BITS 0
//------------------------------------------------------------------------------
// Pin definitions.
//
// Digital pin to indicate an error, set to -1 if not used.
// The led blinks for fatal errors. The led goes on solid for SD write
// overrun errors and logging continues.
const int8_t ERROR_LED_PIN = 3;

// SD chip select pin.
const uint8_t SD_CS_PIN = SS;
//------------------------------------------------------------------------------
// Buffer definitions.
//
// The logger will use SdFat's buffer plus BUFFER_BLOCK_COUNT additional
// buffers.  QUEUE_DIM must be a power of two larger than
//(BUFFER_BLOCK_COUNT + 1).
//
#if RAMEND < 0X8FF
#error Too little SRAM
//
#elif RAMEND < 0X10FF
// Use total of two 512 byte buffers.
const uint8_t BUFFER_BLOCK_COUNT = 1;
// Dimension for queues of 512 byte SD blocks.
const uint8_t QUEUE_DIM = 4;  // Must be a power of two!
//
#elif RAMEND < 0X20FF
// Use total of five 512 byte buffers.
const uint8_t BUFFER_BLOCK_COUNT = 4;
// Dimension for queues of 512 byte SD blocks.
const uint8_t QUEUE_DIM = 8;  // Must be a power of two!
//
#elif RAMEND < 0X40FF
// Use total of 13 512 byte buffers.
const uint8_t BUFFER_BLOCK_COUNT = 12;
// Dimension for queues of 512 byte SD blocks.
const uint8_t QUEUE_DIM = 16;  // Must be a power of two!
//
#else  // RAMEND
// Use total of 29 512 byte buffers.
const uint8_t BUFFER_BLOCK_COUNT = 28;
// Dimension for queues of 512 byte SD blocks.
const uint8_t QUEUE_DIM = 32;  // Must be a power of two!
#endif  // RAMEND
//==============================================================================
// End of configuration constants.
//==============================================================================
// Temporary log file.  Will be deleted if a reset or power failure occurs.
#define TMP_FILE_NAME "tmp_log.bin"

// Size of file base name.  Must not be larger than six.
const uint8_t BASE_NAME_SIZE = sizeof(FILE_BASE_NAME) - 1;

// Number of analog pins to log.
const uint8_t PIN_COUNT = sizeof(PIN_LIST)/sizeof(PIN_LIST[0]);

// Minimum ADC clock cycles per sample interval
const uint16_t MIN_ADC_CYCLES = 15;

// Extra cpu cycles to setup ADC with more than one pin per sample.
const uint16_t ISR_SETUP_ADC = PIN_COUNT > 1 ? 100 : 0;

// Maximum cycles for timer0 system interrupt, millis, micros.
const uint16_t ISR_TIMER0 = 160;
//==============================================================================
SdFat sd;

SdBaseFile binFile;

char binName[13] = FILE_BASE_NAME "00.bin";

#if RECORD_EIGHT_BITS
const size_t SAMPLES_PER_BLOCK = DATA_DIM8/PIN_COUNT;
typedef block8_t block_t;
#else  // RECORD_EIGHT_BITS
const size_t SAMPLES_PER_BLOCK = DATA_DIM16/PIN_COUNT;
typedef block16_t block_t;
#endif // RECORD_EIGHT_BITS

block_t* emptyQueue[QUEUE_DIM];
uint8_t emptyHead;
uint8_t emptyTail;

block_t* fullQueue[QUEUE_DIM];
volatile uint8_t fullHead;  // volatile insures non-interrupt code sees changes.
uint8_t fullTail;

// queueNext assumes QUEUE_DIM is a power of two
inline uint8_t queueNext(uint8_t ht) {
  return (ht + 1) & (QUEUE_DIM -1);
}
//==============================================================================
// Interrupt Service Routines

// Pointer to current buffer.
block_t* isrBuf;

// Need new buffer if true.
bool isrBufNeeded = true;

// overrun count
uint16_t isrOver = 0;

// ADC configuration for each pin.
uint8_t adcmux[PIN_COUNT];
uint8_t adcsra[PIN_COUNT];
uint8_t adcsrb[PIN_COUNT];
uint8_t adcindex = 1;

// Insure no timer events are missed.
volatile bool timerError = false;
volatile bool timerFlag = false;
//------------------------------------------------------------------------------
// ADC done interrupt.
ISR(ADC_vect) {
  // Read ADC data.
#if RECORD_EIGHT_BITS
  uint8_t d = ADCH;
#else  // RECORD_EIGHT_BITS
  // This will access ADCL first.
  uint16_t d = ADC;
#endif  // RECORD_EIGHT_BITS

  if (isrBufNeeded && emptyHead == emptyTail) {
    // no buffers - count overrun
    if (isrOver < 0XFFFF) {
      isrOver++;
    }

    // Avoid missed timer error.
    timerFlag = false;
    return;
  }
  // Start ADC
  if (PIN_COUNT > 1) {
    ADMUX = adcmux[adcindex];
    ADCSRB = adcsrb[adcindex];
    ADCSRA = adcsra[adcindex];
    if (adcindex == 0) {
      timerFlag = false;
    }
    adcindex =  adcindex < (PIN_COUNT - 1) ? adcindex + 1 : 0;
  } else {
    timerFlag = false;
  }
  // Check for buffer needed.
  if (isrBufNeeded) {
    // Remove buffer from empty queue.
    isrBuf = emptyQueue[emptyTail];
    emptyTail = queueNext(emptyTail);
    isrBuf->count = 0;
    isrBuf->overrun = isrOver;
    isrBufNeeded = false;
  }
  // Store ADC data.
  isrBuf->data[isrBuf->count++] = d;

  // Check for buffer full.
  if (isrBuf->count >= PIN_COUNT*SAMPLES_PER_BLOCK) {
    // Put buffer isrIn full queue.
    uint8_t tmp = fullHead;  // Avoid extra fetch of volatile fullHead.
    fullQueue[tmp] = (block_t*)isrBuf;
    fullHead = queueNext(tmp);

    // Set buffer needed and clear overruns.
    isrBufNeeded = true;
    isrOver = 0;
  }
}
//------------------------------------------------------------------------------
// timer1 interrupt to clear OCF1B
ISR(TIMER1_COMPB_vect) {
  // Make sure ADC ISR responded to timer event.
  if (timerFlag) {
    timerError = true;
  }
  timerFlag = true;
}
//==============================================================================
// Error messages stored in flash.
#define error(msg) {sd.errorPrint(F(msg));fatalBlink();}
//------------------------------------------------------------------------------
//
void fatalBlink() {
  while (true) {
    if (ERROR_LED_PIN >= 0) {
      digitalWrite(ERROR_LED_PIN, HIGH);
      delay(200);
      digitalWrite(ERROR_LED_PIN, LOW);
      delay(200);
    }
  }
}
//==============================================================================
#if ADPS0 != 0 || ADPS1 != 1 || ADPS2 != 2
#error unexpected ADC prescaler bits
#endif
//------------------------------------------------------------------------------
// initialize ADC and timer1
void adcInit(metadata_t* meta) {
  uint8_t adps;  // prescaler bits for ADCSRA
  uint32_t ticks = F_CPU*SAMPLE_INTERVAL + 0.5;  // Sample interval cpu cycles.

  if (ADC_REF & ~((1 << REFS0) | (1 << REFS1))) {
    error("Invalid ADC reference");
  }
#ifdef ADC_PRESCALER
  if (ADC_PRESCALER > 7 || ADC_PRESCALER < 2) {
    error("Invalid ADC prescaler");
  }
  adps = ADC_PRESCALER;
#else  // ADC_PRESCALER
  // Allow extra cpu cycles to change ADC settings if more than one pin.
  int32_t adcCycles = (ticks - ISR_TIMER0)/PIN_COUNT - ISR_SETUP_ADC;

  for (adps = 7; adps > 0; adps--) {
    if (adcCycles >= (MIN_ADC_CYCLES << adps)) {
      break;
    }
  }
#endif  // ADC_PRESCALER
  meta->adcFrequency = F_CPU >> adps;
  if (meta->adcFrequency > (RECORD_EIGHT_BITS ? 2000000 : 1000000)) {
    error("Sample Rate Too High");
  }
#if ROUND_SAMPLE_INTERVAL
  // Round so interval is multiple of ADC clock.
  ticks += 1 << (adps - 1);
  ticks >>= adps;
  ticks <<= adps;
#endif  // ROUND_SAMPLE_INTERVAL

  if (PIN_COUNT > sizeof(meta->pinNumber)/sizeof(meta->pinNumber[0])) {
    error("Too many pins");
  }
  meta->pinCount = PIN_COUNT;
  meta->recordEightBits = RECORD_EIGHT_BITS;

  for (int i = 0; i < PIN_COUNT; i++) {
    uint8_t pin = PIN_LIST[i];
    if (pin >= NUM_ANALOG_INPUTS) {
      error("Invalid Analog pin number");
    }
    meta->pinNumber[i] = pin;

    // Set ADC reference and low three bits of analog pin number.
    adcmux[i] = (pin & 7) | ADC_REF;
    if (RECORD_EIGHT_BITS) {
      adcmux[i] |= 1 << ADLAR;
    }

    // If this is the first pin, trigger on timer/counter 1 compare match B.
    adcsrb[i] = i == 0 ? (1 << ADTS2) | (1 << ADTS0) : 0;
#ifdef MUX5
    if (pin > 7) {
      adcsrb[i] |= (1 << MUX5);
    }
#endif  // MUX5
    adcsra[i] = (1 << ADEN) | (1 << ADIE) | adps;
    adcsra[i] |= i == 0 ? 1 << ADATE : 1 << ADSC;
  }

  // Setup timer1
  TCCR1A = 0;
  uint8_t tshift;
  if (ticks < 0X10000) {
    // no prescale, CTC mode
    TCCR1B = (1 << WGM13) | (1 << WGM12) | (1 << CS10);
    tshift = 0;
  } else if (ticks < 0X10000*8) {
    // prescale 8, CTC mode
    TCCR1B = (1 << WGM13) | (1 << WGM12) | (1 << CS11);
    tshift = 3;
  } else if (ticks < 0X10000*64) {
    // prescale 64, CTC mode
    TCCR1B = (1 << WGM13) | (1 << WGM12) | (1 << CS11) | (1 << CS10);
    tshift = 6;
  } else if (ticks < 0X10000*256) {
    // prescale 256, CTC mode
    TCCR1B = (1 << WGM13) | (1 << WGM12) | (1 << CS12);
    tshift = 8;
  } else if (ticks < 0X10000*1024) {
    // prescale 1024, CTC mode
    TCCR1B = (1 << WGM13) | (1 << WGM12) | (1 << CS12) | (1 << CS10);
    tshift = 10;
  } else {
    error("Sample Rate Too Slow");
  }
  // divide by prescaler
  ticks >>= tshift;
  // set TOP for timer reset
  ICR1 = ticks - 1;
  // compare for ADC start
  OCR1B = 0;

  // multiply by prescaler
  ticks <<= tshift;

  // Sample interval in CPU clock ticks.
  meta->sampleInterval = ticks;
  meta->cpuFrequency = F_CPU;
  float sampleRate = (float)meta->cpuFrequency/meta->sampleInterval;
  Serial.print(F("Sample pins:"));
  for (uint8_t i = 0; i < meta->pinCount; i++) {
    Serial.print(' ');
    Serial.print(meta->pinNumber[i], DEC);
  }
  Serial.println();
  Serial.print(F("ADC bits: "));
  Serial.println(meta->recordEightBits ? 8 : 10);
  Serial.print(F("ADC clock kHz: "));
  Serial.println(meta->adcFrequency/1000);
  Serial.print(F("Sample Rate: "));
  Serial.println(sampleRate);
  Serial.print(F("Sample interval usec: "));
  Serial.println(1000000.0/sampleRate, 4);
}
//------------------------------------------------------------------------------
// enable ADC and timer1 interrupts
void adcStart() {
  // initialize ISR
  isrBufNeeded = true;
  isrOver = 0;
  adcindex = 1;

  // Clear any pending interrupt.
  ADCSRA |= 1 << ADIF;

  // Setup for first pin.
  ADMUX = adcmux[0];
  ADCSRB = adcsrb[0];
  ADCSRA = adcsra[0];

  // Enable timer1 interrupts.
  timerError = false;
  timerFlag = false;
  TCNT1 = 0;
  TIFR1 = 1 << OCF1B;
  TIMSK1 = 1 << OCIE1B;
}
//------------------------------------------------------------------------------
void adcStop() {
  TIMSK1 = 0;
  ADCSRA = 0;
}
//------------------------------------------------------------------------------
// Convert binary file to csv file.
void binaryToCsv() {
  uint8_t lastPct = 0;
  block_t buf;
  metadata_t* pm;
  uint32_t t0 = millis();
  char csvName[13];
  StdioStream csvStream;

  if (!binFile.isOpen()) {
    Serial.println(F("No current binary file"));
    return;
  }
  binFile.rewind();
  if (binFile.read(&buf , 512) != 512) {
    error("Read metadata failed");
  }
  // Create a new csv file.
  strcpy(csvName, binName);
  strcpy(&csvName[BASE_NAME_SIZE + 3], "csv");

  if (!csvStream.fopen(csvName, "w")) {
    error("open csvStream failed");
  }
  Serial.println();
  Serial.print(F("Writing: "));
  Serial.print(csvName);
  Serial.println(F(" - type any character to stop"));
  pm = (metadata_t*)&buf;
  csvStream.print(F("Interval,"));
  float intervalMicros = 1.0e6*pm->sampleInterval/(float)pm->cpuFrequency;
  csvStream.print(intervalMicros, 4);
  csvStream.println(F(",usec"));
  for (uint8_t i = 0; i < pm->pinCount; i++) {
    if (i) {
      csvStream.putc(',');
    }
    csvStream.print(F("pin"));
    csvStream.print(pm->pinNumber[i]);
  }
  csvStream.println();
  uint32_t tPct = millis();
  while (!Serial.available() && binFile.read(&buf, 512) == 512) {
    if (buf.count == 0) {
      break;
    }
    if (buf.overrun) {
      csvStream.print(F("OVERRUN,"));
      csvStream.println(buf.overrun);
    }
    for (uint16_t j = 0; j < buf.count; j += PIN_COUNT) {
      for (uint16_t i = 0; i < PIN_COUNT; i++) {
        if (i) {
          csvStream.putc(',');
        }
        csvStream.print(buf.data[i + j]);
      }
      csvStream.println();
    }
    if ((millis() - tPct) > 1000) {
      uint8_t pct = binFile.curPosition()/(binFile.fileSize()/100);
      if (pct != lastPct) {
        tPct = millis();
        lastPct = pct;
        Serial.print(pct, DEC);
        Serial.println('%');
      }
    }
    if (Serial.available()) {
      break;
    }
  }
  csvStream.fclose();
  Serial.print(F("Done: "));
  Serial.print(0.001*(millis() - t0));
  Serial.println(F(" Seconds"));
}
//------------------------------------------------------------------------------
// read data file and check for overruns
void checkOverrun() {
  bool headerPrinted = false;
  block_t buf;
  uint32_t bgnBlock, endBlock;
  uint32_t bn = 0;

  if (!binFile.isOpen()) {
    Serial.println(F("No current binary file"));
    return;
  }
  if (!binFile.contiguousRange(&bgnBlock, &endBlock)) {
    error("contiguousRange failed");
  }
  binFile.rewind();
  Serial.println();
  Serial.println(F("Checking overrun errors - type any character to stop"));
  if (binFile.read(&buf , 512) != 512) {
    error("Read metadata failed");
  }
  bn++;
  while (binFile.read(&buf, 512) == 512) {
    if (buf.count == 0) {
      break;
    }
    if (buf.overrun) {
      if (!headerPrinted) {
        Serial.println();
        Serial.println(F("Overruns:"));
        Serial.println(F("fileBlockNumber,sdBlockNumber,overrunCount"));
        headerPrinted = true;
      }
      Serial.print(bn);
      Serial.print(',');
      Serial.print(bgnBlock + bn);
      Serial.print(',');
      Serial.println(buf.overrun);
    }
    bn++;
  }
  if (!headerPrinted) {
    Serial.println(F("No errors found"));
  } else {
    Serial.println(F("Done"));
  }
}
//------------------------------------------------------------------------------
// dump data file to Serial
void dumpData() {
  block_t buf;
  if (!binFile.isOpen()) {
    Serial.println(F("No current binary file"));
    return;
  }
  binFile.rewind();
  if (binFile.read(&buf , 512) != 512) {
    error("Read metadata failed");
  }
  Serial.println();
  Serial.println(F("Type any character to stop"));
  delay(1000);
  while (!Serial.available() && binFile.read(&buf , 512) == 512) {
    if (buf.count == 0) {
      break;
    }
    if (buf.overrun) {
      Serial.print(F("OVERRUN,"));
      Serial.println(buf.overrun);
    }
    for (uint16_t i = 0; i < buf.count; i++) {
      Serial.print(buf.data[i], DEC);
      if ((i+1)%PIN_COUNT) {
        Serial.print(',');
      } else {
        Serial.println();
      }
    }
  }
  Serial.println(F("Done"));
}
//------------------------------------------------------------------------------
// log data
// max number of blocks to erase per erase call
uint32_t const ERASE_SIZE = 262144L;
void logData() {
  uint32_t bgnBlock, endBlock;

  // Allocate extra buffer space.
  block_t block[BUFFER_BLOCK_COUNT];

  Serial.println();

  // Initialize ADC and timer1.
  adcInit((metadata_t*) &block[0]);

  // Find unused file name.
  if (BASE_NAME_SIZE > 6) {
    error("FILE_BASE_NAME too long");
  }
  while (sd.exists(binName)) {
    if (binName[BASE_NAME_SIZE + 1] != '9') {
      binName[BASE_NAME_SIZE + 1]++;
    } else {
      binName[BASE_NAME_SIZE + 1] = '0';
      if (binName[BASE_NAME_SIZE] == '9') {
        error("Can't create file name");
      }
      binName[BASE_NAME_SIZE]++;
    }
  }
  // Delete old tmp file.
  if (sd.exists(TMP_FILE_NAME)) {
    Serial.println(F("Deleting tmp file"));
    if (!sd.remove(TMP_FILE_NAME)) {
      error("Can't remove tmp file");
    }
  }
  // Create new file.
  Serial.println(F("Creating new file"));
  binFile.close();
  if (!binFile.createContiguous(TMP_FILE_NAME, 512 * FILE_BLOCK_COUNT)) {
    error("createContiguous failed");
  }
  // Get the address of the file on the SD.
  if (!binFile.contiguousRange(&bgnBlock, &endBlock)) {
    error("contiguousRange failed");
  }
  // Use SdFat's internal buffer.
  uint8_t* cache = (uint8_t*)sd.vol()->cacheClear();
  if (cache == 0) {
    error("cacheClear failed");
  }

  // Flash erase all data in the file.
  Serial.println(F("Erasing all data"));
  uint32_t bgnErase = bgnBlock;
  uint32_t endErase;
  while (bgnErase < endBlock) {
    endErase = bgnErase + ERASE_SIZE;
    if (endErase > endBlock) {
      endErase = endBlock;
    }
    if (!sd.card()->erase(bgnErase, endErase)) {
      error("erase failed");
    }
    bgnErase = endErase + 1;
  }
  // Start a multiple block write.
  if (!sd.card()->writeStart(bgnBlock, FILE_BLOCK_COUNT)) {
    error("writeBegin failed");
  }
  // Write metadata.
  if (!sd.card()->writeData((uint8_t*)&block[0])) {
    error("Write metadata failed");
  }
  // Initialize queues.
  emptyHead = emptyTail = 0;
  fullHead = fullTail = 0;

  // Use SdFat buffer for one block.
  emptyQueue[emptyHead] = (block_t*)cache;
  emptyHead = queueNext(emptyHead);

  // Put rest of buffers in the empty queue.
  for (uint8_t i = 0; i < BUFFER_BLOCK_COUNT; i++) {
    emptyQueue[emptyHead] = &block[i];
    emptyHead = queueNext(emptyHead);
  }
  // Give SD time to prepare for big write.
  delay(1000);
  Serial.println(F("Logging - type any character to stop"));
  // Wait for Serial Idle.
  Serial.flush();
  delay(10);
  uint32_t bn = 1;
  uint32_t t0 = millis();
  uint32_t t1 = t0;
  uint32_t overruns = 0;
  uint32_t count = 0;
  uint32_t maxLatency = 0;

  // Start logging interrupts.
  adcStart();
  while (1) {
    if (fullHead != fullTail) {
      // Get address of block to write.
      block_t* pBlock = fullQueue[fullTail];

      // Write block to SD.
      uint32_t usec = micros();
      if (!sd.card()->writeData((uint8_t*)pBlock)) {
        error("write data failed");
      }
      usec = micros() - usec;
      t1 = millis();
      if (usec > maxLatency) {
        maxLatency = usec;
      }
      count += pBlock->count;

      // Add overruns and possibly light LED.
      if (pBlock->overrun) {
        overruns += pBlock->overrun;
        if (ERROR_LED_PIN >= 0) {
          digitalWrite(ERROR_LED_PIN, HIGH);
        }
      }
      // Move block to empty queue.
      emptyQueue[emptyHead] = pBlock;
      emptyHead = queueNext(emptyHead);
      fullTail = queueNext(fullTail);
      bn++;
      if (bn == FILE_BLOCK_COUNT) {
        // File full so stop ISR calls.
        adcStop();
        break;
      }
    }
    if (timerError) {
      error("Missed timer event - rate too high");
    }
    if (Serial.available()) {
      // Stop ISR calls.
      adcStop();
      if (isrBuf != 0 && isrBuf->count >= PIN_COUNT) {
        // Truncate to last complete sample.
        isrBuf->count = PIN_COUNT*(isrBuf->count/PIN_COUNT);
        // Put buffer in full queue.
        fullQueue[fullHead] = isrBuf;
        fullHead = queueNext(fullHead);
        isrBuf = 0;
      }
      if (fullHead == fullTail) {
        break;
      }
    }
  }
  if (!sd.card()->writeStop()) {
    error("writeStop failed");
  }
  // Truncate file if recording stopped early.
  if (bn != FILE_BLOCK_COUNT) {
    Serial.println(F("Truncating file"));
    if (!binFile.truncate(512L * bn)) {
      error("Can't truncate file");
    }
  }
  if (!binFile.rename(binName)) {
    error("Can't rename file");
  }
  Serial.print(F("File renamed: "));
  Serial.println(binName);
  Serial.print(F("Max block write usec: "));
  Serial.println(maxLatency);
  Serial.print(F("Record time sec: "));
  Serial.println(0.001*(t1 - t0), 3);
  Serial.print(F("Sample count: "));
  Serial.println(count/PIN_COUNT);
  Serial.print(F("Samples/sec: "));
  Serial.println((1000.0/PIN_COUNT)*count/(t1-t0));
  Serial.print(F("Overruns: "));
  Serial.println(overruns);
  Serial.println(F("Done"));
}
//------------------------------------------------------------------------------
void setup(void) {
  if (ERROR_LED_PIN >= 0) {
    pinMode(ERROR_LED_PIN, OUTPUT);
  }
  Serial.begin(115200);

  // Read the first sample pin to init the ADC.
  analogRead(PIN_LIST[0]);

  Serial.print(F("FreeStack: "));
  Serial.println(FreeStack());

  // Initialize at the highest speed supported by the board that is
  // not over 50 MHz. Try a lower speed if SPI errors occur.
  if (!sd.begin(SD_CS_PIN, SD_SCK_MHZ(50))) {
    sd.initErrorPrint();
    fatalBlink();
  }
}
//------------------------------------------------------------------------------
void loop(void) {
  // Read any Serial data.
  do {
    delay(10);
  } while (Serial.available() && Serial.read() >= 0);
  Serial.println();
  Serial.println(F("type:"));
  Serial.println(F("c - convert file to csv"));
  Serial.println(F("d - dump data to Serial"));
  Serial.println(F("e - overrun error details"));
  Serial.println(F("r - record ADC data"));

  while(!Serial.available()) {
    SysCall::yield();
  }
  char c = tolower(Serial.read());
  if (ERROR_LED_PIN >= 0) {
    digitalWrite(ERROR_LED_PIN, LOW);
  }
  // Read any Serial data.
  do {
    delay(10);
  } while (Serial.available() && Serial.read() >= 0);

  if (c == 'c') {
    binaryToCsv();
  } else if (c == 'd') {
    dumpData();
  } else if (c == 'e') {
    checkOverrun();
  } else if (c == 'r') {
    logData();
  } else {
    Serial.println(F("Invalid entry"));
  }
}
#else  // __AVR__
#error This program is only for AVR.
#endif  // __AVR__

why don't you try it and find out?

How are you achieving the 5kHz? If you are using something like a timer, you should get a pretty solid 5kHz (the only drift would be the drift of the crystal). In that case not useful to add a timestamp to the data, as it would be a constant 200µs after the previous one. Though you'd have to look out for other interrupts, which can cause jitter. You also could run into problems when a SD cards write take a bit longer than usual. In that case you could lose some of your measurements. Adding a timestamp would at least enable you to detect that.


text:
Interfacing AD7606 with Arduino Due. How to data-log?
I have an 16 bit ADC7606. It is in parallel mode. I have a working sketch that allows me to read  8 channels, convert the 16 bit integer to decimal and then serial prints to monitor. I would like to increase the frequency of the sampling rate. The serial monitor can not keep up with a sampling rate faster than 40millis. The Due is connected via USB on COM3.
How do I log the data to a file on my laptop/computer?
All and any help greatly appreciated.
Code below,
#include <EnableInterrupt.h>

#define BUSY 3
#define RD 4  //RD+CS tied together
#define RESET 5
#define CONVST 6 //CONVSTA+CONVSTB soldered together on the board
#define RANGE 7

#define DB0 38
#define DB1 39
#define DB2 40
#define DB3 41
#define DB4 42
#define DB5 43
#define DB6 44
#define DB7 45
#define DB8 46
#define DB9 47
#define DB10 48
#define DB11 49
#define DB12 50
#define DB13 51
#define DB14 52
#define DB15 53

// declare rawData as an array of 16 integers
int rawData[16];

void setup() {

  enableInterrupt(BUSY, ISR_, FALLING);

  pinMode(RESET, OUTPUT);
  pinMode(CONVST, OUTPUT);
  pinMode(RD, OUTPUT);
  pinMode(RANGE, OUTPUT);
  pinMode(BUSY, INPUT);

  Serial.begin(9600);

  //reset ADC to begin conversion
  digitalWrite(RESET, HIGH);
  delayMicroseconds(10);
  digitalWrite(RESET, LOW);

  digitalWrite(CONVST, LOW);
  digitalWrite(RD, HIGH);
  digitalWrite(RANGE, HIGH);
  digitalWrite(BUSY, LOW);

  delayMicroseconds(1000);

}

void loop() {

  disableInterrupt(BUSY);

  delayMicroseconds(40000);
  digitalWrite(CONVST, LOW);
  delayMicroseconds(10);
  digitalWrite(CONVST, HIGH);
  enableInterrupt(BUSY, ISR_, FALLING);

}

void ISR_ ()  {

  //Serial.print(millis());
  //Serial.print(",");

  delayMicroseconds(1);
  digitalWrite(RD, LOW);
  delayMicroseconds(1);
  readDBpins(); // do read
  digitalWrite(RD, HIGH);
  Serial.print(",");

  delayMicroseconds(1);
  digitalWrite(RD, LOW);
  delayMicroseconds(1);
  readDBpins(); // do read
  digitalWrite(RD, HIGH);
  Serial.print(",");

  delayMicroseconds(1);
  digitalWrite(RD, LOW);
  delayMicroseconds(1);
  readDBpins(); // do read
  digitalWrite(RD, HIGH);
  Serial.print(",");

  delayMicroseconds(1);
  digitalWrite(RD, LOW);
  delayMicroseconds(1);
  readDBpins(); // do read
  digitalWrite(RD, HIGH);
  Serial.print(",");

  delayMicroseconds(1);
  digitalWrite(RD, LOW);
  delayMicroseconds(1);
  readDBpins(); // do read
  digitalWrite(RD, HIGH);
  Serial.print(",");

  delayMicroseconds(1);
  digitalWrite(RD, LOW);
  delayMicroseconds(1);
  readDBpins(); // do read
  digitalWrite(RD, HIGH);
  Serial.print(",");

  delayMicroseconds(1);
  digitalWrite(RD, LOW);
  delayMicroseconds(1);
  readDBpins(); // do read
  digitalWrite(RD, HIGH);
  Serial.print(",");

  delayMicroseconds(1);
  digitalWrite(RD, LOW);
  delayMicroseconds(1);
  readDBpins(); // do read
  digitalWrite(RD, HIGH);

  Serial.println(" ");

}

void readDBpins()
{
  rawData[0] = digitalRead(DB15);
  //Serial.print (rawData[0]);
  if (rawData[0] > 0) {rawData[0] = 32768;}
  rawData[1] = digitalRead(DB14);
  //Serial.print (rawData[1]);
  if (rawData[1] > 0) {rawData[1] = 16384;}
  rawData[2] = digitalRead(DB13);
  //Serial.print (rawData[2]);
  if (rawData[2] > 0) {rawData[2] = 8192;}
  rawData[3] = digitalRead(DB12);
  //Serial.print (rawData[3]);
  if (rawData[3] > 0) {rawData[3] = 4096;}
  rawData[4] = digitalRead(DB11);
  //Serial.print (rawData[4]);
  if (rawData[4] > 0) {rawData[4] = 2048;}
  rawData[5] = digitalRead(DB10);
  //Serial.print (rawData[5]);
  if (rawData[5] > 0) {rawData[5] = 1024;}
  rawData[6] = digitalRead(DB9);
  //Serial.print (rawData[6]);
  if (rawData[6] > 0) {rawData[6] = 512;}
  rawData[7] = digitalRead(DB8);
  //Serial.print (rawData[7]);
  if (rawData[7] > 0) {rawData[7] = 256;}
  rawData[8] = digitalRead(DB7);
  //Serial.print (rawData[8]);
  if (rawData[8] > 0) {rawData[8] = 128;}
  rawData[9] = digitalRead(DB6);
  //Serial.print (rawData[9]);
  if (rawData[9] > 0) {rawData[9] = 64;}
  rawData[10] = digitalRead(DB5);
  //Serial.print (rawData[10]);
  if (rawData[10] > 0) {rawData[10] = 32;}
  rawData[11] = digitalRead(DB4);
  //Serial.print (rawData[11]);
  if (rawData[11] > 0) {rawData[11] = 16;}
  rawData[12] = digitalRead(DB3);
  //Serial.print (rawData[12]);
  if (rawData[12] > 0) {rawData[12] = 8;}
  rawData[13] = digitalRead(DB2);
  //Serial.print (rawData[13]);
  if (rawData[13] > 0) {rawData[13] = 4;}
  rawData[14] = digitalRead(DB1);
  //Serial.print (rawData[14]);
  if (rawData[14] > 0) {rawData[14] = 2;}
  rawData[15] = digitalRead(DB0);
  // Serial.print (rawData[15]);
  if (rawData[15] > 0) {rawData[15] = 1;}

  Serial.print (rawData[0]+rawData[1]+rawData[2]+rawData[3]+rawData[4]+rawData[5]+rawData[6]+rawData[7]+rawData[8]+rawData[9]+rawData[10]+rawData[11]+rawData[12]+rawData[13]+rawData[14]+rawData[15]);
}

Its name is a AD7606 (without the C)? http://www.analog.com/en/products/ad7606.html

why 9600 baud?  try 921600 baud

Other Serial terminal programs may have a function to save the received data to a file. Try some of them. And - since the Serial interface doesn't keep up - as Juraj wrote increase the baudrate, so that the data transfer is done with higher speed.

Thanks peeps. I read that anything over 115200 baud is not effective. 115200 is the usual rate I use. Serial Monitor cannot display the readings with anything faster than a 40millis sampling rate so I cannot do what I usually do and copy and paste to file as there is nothing there to copy.

@Juraj and chrisl 921600 baud and anything above 250000 prints gibberish. i now have a pyserial log script but it still bottlenecks, although now at 1.5millis, which is good but not good enough. I now know I messed up the 2's complement conversion in my arduino sketch but even with that commented out it still bottlenecks.

You can use python on the command line to log your serial monitor output. This is described here. The script below has a baud rate of 250000 but can be set according to how you program your arduino.
from time import sleep
import serial

# Change port and baud rate to match your setup
ser = serial.Serial('/dev/ttyACM0', 250000) # Establish the connection on a specific port

counter = 32 # Below 32 everything in ASCII is gibberish

while True:
     counter +=1
     ser.write(str(chr(counter))) # Convert the decimal number to ASCII then send it to the Arduino
     print(ser.readline()) # Read the newest output from the Arduino
     sleep(.1) # Delay for one tenth of a second
     if counter == 255:
         counter = 32

Script from here. You must put in your appropriate port and baud rate. This Mathworks link describes how to find your port number.

In windows look in Device Manager under USB Serial Port.
In linux or OSX open a terminal and search with ls /dev/* or ls /dev/tty*. You can | grep <search> on the same line to search usb or ACM

This is a discussion about maximum baud rate. I have gone up to 250000 on the Due's programming port but you can go faster using the native port. You may also be limited by your computer usb though unlikely. 480 Mbit/s max for USB 2.0.

The Arduino Due has two USB ports available. The Native USB port (which supports CDC serial communication using the SerialUSB object)
  is connected directly to the SAM3X MCU. The other USB port is the
  Programming port. It is connected to an ATMEL 16U2 which acts as a
  USB-to-Serial converter. This Programming port is the default for
  uploading sketches and communicating with the Arduino.
The USB-to-serial converter of the Programming port is connected to
  the first UART of the SAM3X. It's possible to communicate over this
  port using the "Serial" object in the Arduino programming language.
The USB connector of the Native port is directly connected to the USB
  host pins of the SAM3X. Using the Native port enables you to use the
  Due as a client USB peripheral (acting as a mouse or a keyboard
  connected to the computer) or as a USB host device so that devices can
  be connected to the Due (like a mouse, keyboard, or an Android phone).
  This port can also be used as a virtual serial port using the
  "SerialUSB" object in the Arduino programming language.

From the Arduino Due guide you can switch Serial.print to SerialUSB.print, same with other serial functions like println. Then if you plug your usb into the Native port you can read off that port at a higher baud rate. You must also initialize the SerialUSB,
SerialUSB.begin(2000000);    // Initialize Native USB port
while(!SerialUSB);           // Wait until connection is established

It shouldn't matter what you set the baud rate at on the native port, it will run at the maximum bus speed but from what I know it has to be set regardless.
When running on the command line you can use > to send the output to a log file.
For example,
python serial_log.py > out.txt

Thanks for the detailed answer. I've now got a basic sketch output being read by python and displayed in a command prompt python window. I can data log now thanks to the path you set me on, using a different serial logger script I found here https://github.com/diegoherranz/serial-logger

I could not get the serial_log.py that you suggested to work.  I still have the same problem though, logging 8 columns of 16bit integers at a faster rate than 40millis. Do you think your suggestion will be able to log any faster?

I believe the code provided only works with python2. If you got something else similar to work thats just as well. Raising your baud rate will increase your transmission speed, notes on baud rate. The python script may enable baud rates that the arduino ide does not support. If your problem is in fact with the speed of an arduino function you may look into an alternative, this is an example for the analogRead function.

Update: with 250000 baud I can increase the sampling rate to 2 millis, which is good but I need a minimum of 1 millis and ideally I'd hit 0.5 millis. I know there is 500k and 1M baud rates available but anything above 250k baud returns gibberish. There's got to be a way of reading 16 digital pins 8 times in a row and logging the result every 0.5ms.....

Use the native port and increase the baud rate further. Replace Serial.print with SerialUSB.print, see the updated response.

Implemented! The Due can't make up its mind whether it is connected to port 5 or port 6. I can upload the sketch but pyserial logger can't access the port and neither can serial monitor. It's nearing midnight for me so I think I'll have to leave this for tomorrow, if I can sleep that is, I feel like I'm on the edge of a breakthrough! Cheers

Make sure you are only accessing the serial from one location. If you have the Arduino ide accessing the monitor then you may not be able to use a script to access it, or if you have two terminal windows open. Also I would still probably program via the programming port, then switch over to the native port for reading. Hope it helps!

You nailed it! program in the programming port run the pyserial logger in the native USB port and I'm golden! After playing about with my sketch I am now reading at 1millis average. I believe I will be able to claw back some processing time by fine tuning my sketch and finding a more streamlined way of reading 16dig pins, converting to dec, then printing. BUT the problem of fast data logging to file on my computer/laptop has been cracked, the question thoroughly answered. I am very grateful of your time and your help. Thank you!

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
Greyed out port for Raspberry pico in Arduino IDE - Ubuntu 20.04
I'm trying to figure out how to program my new Raspberry Pico with Arduino, but don't see a port to select so I can't upload the blink sketch.
Here's what I've done:

The Pico is working, I can program it with micro Python.
Just in case, I brought the Pico back to factory settings with the nuke file.
Installed support for the Pico in Arduino IDE. Version 2.1.0 of Arduino MBed RP2040 is installed.
While keeping BOOTSEL pressed, I plug in the Pico to my laptop (Ubuntu 20.04)
The board selected is "Raspberry Pi Pico"
But the option to select a port is greyed out :-(

It seems that I am omitting some step during the installation, but can't figure out what it is.
Any ideas?
Thanks!

I've come across this and this is how I worked around it (on Windows, but I suspect the problem and solution is the same).
A brand new Pico or one not programmed by the Arduino IDE does not have a serial port, which is why you can't see one. Arduino sketches are compiled with a serial port automatically, so once a serial port exists, it can be used for future uploads.
The Arduino solution is to use the 'Export compiled binary' option (for any sketch) and upload the UF2 file (which should be generated into the sketch folder) using virtual USB drive interface. When the sketch starts, the serial port will be available and can be used for future sketch uploads.
Another solution is to use an alternative Pico core. This one does not need the port selected, but will find and upload to a Pico in bootloader mode via the virtual USB drive.

Had this issue and tried everything that was described on forums.
The thing that worked for me was: I held the bootsel button until it opened pico  folder, then port button activated in the IDE and I chose UF2 port and then after upload, pico showed at COM7

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
How to upload an arduino sketch from java / processing?
I'm trying to upload a hex file from Java/Processing but running into some issues.
Here is my code so far, based on the Arduino IDE's upload verbose output:
void setup(){
    //C:\Program Files (x86)\Arduino/hardware/tools/avr/bin/avrdude -CC:\Program Files (x86)\Arduino/hardware/tools/avr/etc/avrdude.conf -v -v -v -v -patmega328p -carduino -PCOM96 -b115200 -D -Uflash:w:C:\Users\HM\AppData\Local\Temp\build5909267154049588263.tmp/sketch_aug13a.cpp.hex:i
    String ArduinoPath = "C:\\Program Files (x86)\\Arduino";
    String hexPath = "C:\\Users\\HM\\AppData\\Local\\Temp\\build1023547366107161384.tmp/sketch_aug13a.cpp.hex";
    String port = "COM96";
    runCommand(new String[]{ArduinoPath+"/hardware/tools/avr/bin/avrdude", "-C"+ArduinoPath+"/hardware/tools/avr/etc/avrdude.conf","-v","-v","-v","-v","-patmega328p","-carduino","-P"+port,"-b115200","-D","-Uflash:w:"+hexPath+":i"});
  }
  void runCommand(String[] cmd){

    String s = null;

    try {

      Process p = Runtime.getRuntime().exec(cmd);
      BufferedReader stdInput = new BufferedReader(new InputStreamReader(p.getInputStream()));
      BufferedReader stdError = new BufferedReader(new InputStreamReader(p.getErrorStream()));

      // read the output from the command
      System.out.println("command out:\n");
      while ( (s = stdInput.readLine ()) != null) System.out.println(s);
      System.out.println("errors (if any):\n");
      while ( (s = stdError.readLine ()) != null) System.out.println(s);

    }catch (IOException e) {
      System.out.println("something went wrong: \n");
      e.printStackTrace();
    }

  }

If I run this from command line it uploads without any issues, but from the code above I always get this output:
avrdude: Version 5.11, compiled on Sep  2 2011 at 19:38:36
         Copyright (c) 2000-2005 Brian Dean, http://www.bdmicro.com/
         Copyright (c) 2007-2009 Joerg Wunsch

         System wide configuration file is "C:\Program Files (x86)\Arduino\hardw
are\tools\avr\etc\avrdude.conf"

         Using Port                    : COM96
         Using Programmer              : arduino
         Overriding Baud Rate          : 115200
avrdude: ser_open(): can't open device "\\.\COM96": Access is denied.

avrdude done.  Thank you.

What output do you get? You forgot to post it in your question!

Is "COM96" really the correct device?  If it is, is this sketch being run in the same way as your other test, or it is perhaps running a less privileged mode?  Do you still have the Arduino IDE (perhaps its serial monitor) open and claiming the port?

I've disconnected the port to see if I get a different message from avrdude and I do. COM96 is correct, otherwise I get The system can't find the file specified. If I use the Arduino IDE or command prompt to do the upload, the upload seems to work, it's when trying to replicate it from Java this problem arises. Does the Arduino IDE do anything clever after uploading calling avrdude to release the port ?

Have you tried hard coding the value of port in your command string. try "-PCOM96" instead of "-P"+port. Other than that, it says access is denied, try running it as administrator

oooh...haven't tried that as administrator yet. That kind of rings a bell because Arduino and avrdude are in Program Files so would have admin privileges

Running as administrator doesn't solve it. You can see a screen recording here (manually scroll or save the file locally if it doesn't play in the browser).  Tomorrow I'll try with Python and subprocess. I'm suspecting the code I use to run avrdude isn't quite there yet.

Ok, I've just done done a test in Python: subprocess.check_output([ap+"/hardware/tools/avr/bin/avrdude","-C"+ap+"/hardware/tools/avr/etc/avrdude.conf","-v","-v","-v","-v","-patmega328p","-carduino","-P"+port,"-b115200","-D","-Uflash:w:"+hexPath+":i"],stderr=subprocess.STDOUT,shell=True), where ap is arduino path, hexPath is the path to the hex file and port is "COM96" and that worked. I have to look into the java Runtime exec method and options/better ways of doing this

I'd first would try to find out the right command when running on command line (cmd.exe).
When you launch an external command from Java which generates output, you should/must read the output (p.getInputStream(), p.getErrorStream()) using separate threads. Otherwise your application might hang for no obvious reason.

I've made two separate thread as advised and it worked!

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
How to correctly pack an unsigned long from 3 unsigned chars?
I would like to use 652 unsigned long values on an Arduino Uno, but it doesn't look like there's enough memory. I thought about the splitting unsigned long values to 3 unsigned chars to store in PROGMEM which I can letter recall and join as unsigned long as needed.
I'm not super experienced with byte manipulations and I got myself stuck when trying to merge the data. When I try with one variable at a time, I get the expected output. The part I'm confused about is why do I get different values when using a variable as an array index for each value ? How can I fix this ?
Here's my test code so far:
#include <avr/pgmspace.h>
#include "flashData.h"

void setup() {
  Serial.begin(115200);
  unsigned long fromPROGMEM = (onsets1[0] << 16) | (onsets2[0] << 8) | onsets3[0];
  Serial.println(fromPROGMEM);

  unsigned long fromValues = (0x00 << 16) | (0x01 << 8) | 0xF3;
  Serial.println(fromValues);

  byte b1 = 0x00;
  byte b2 = 0x01;
  byte b3 = 0xF3;

  unsigned long fromBytes = (b1 << 16) | (b2 << 8) | b3;

  Serial.println(fromBytes);
  delay(2000);

  unsigned char c1,c2,c3;
  unsigned long value;

  Serial.println("data using same index:");
  for(int i = 0 ; i < 10; i++){
    c1 = onsets1[3];
    c2 = onsets2[3];
    c3 = onsets3[3];
    value = (c1 << 16) | (c2 << 8) | c3;
    Serial.print("index: ");
    Serial.print(i);
    Serial.print("\tvalue:");
    Serial.println (value);
    delay(10);
  }

  Serial.println("data using counter:");
  for(int i = 0 ; i < 10; i++){
    c1 = onsets1[i];
    c2 = onsets2[i];
    c3 = onsets3[i];
    value = (c1 << 16) | (c2 << 8) | c3;
    Serial.print("index: ");
    Serial.print(i);
    Serial.print("\tvalue:");
    Serial.println (value);
    delay(10);
  }
}

void loop() {

}

and flashData.h:
#ifndef _flashData_h_
#define _flashData_h_

static PROGMEM const unsigned char onsets1[652] = {0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x02,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03,0x03};
static PROGMEM const unsigned char onsets2[652] = {0x01,0x04,0x05,0x06,0x07,0x08,0x09,0x0A,0x0D,0x0E,0x0F,0x10,0x11,0x12,0x14,0x15,0x16,0x17,0x18,0x19,0x1A,0x1B,0x1D,0x1E,0x1F,0x20,0x21,0x22,0x23,0x25,0x26,0x28,0x29,0x2A,0x2B,0x2C,0x2E,0x2F,0x30,0x31,0x32,0x33,0x34,0x35,0x38,0x39,0x3A,0x3B,0x3C,0x3F,0x40,0x41,0x42,0x43,0x44,0x45,0x46,0x48,0x49,0x4A,0x4C,0x4D,0x4E,0x51,0x52,0x53,0x54,0x55,0x56,0x57,0x59,0x5A,0x5B,0x5C,0x5E,0x5F,0x61,0x63,0x64,0x65,0x67,0x6A,0x6B,0x6C,0x6D,0x6E,0x70,0x71,0x73,0x75,0x76,0x77,0x79,0x7C,0x7E,0x7F,0x80,0x82,0x84,0x85,0x86,0x87,0x88,0x89,0x8A,0x8C,0x8D,0x8E,0x8F,0x90,0x91,0x92,0x95,0x96,0x97,0x98,0x99,0x9A,0x9B,0x9C,0x9E,0x9F,0xA0,0xA1,0xA2,0xA3,0xA4,0xA7,0xA8,0xA9,0xAA,0xAB,0xAC,0xAD,0xAF,0xB0,0xB1,0xB2,0xB3,0xB4,0xB5,0xB6,0xB9,0xBA,0xBB,0xBC,0xBD,0xBE,0xC0,0xC2,0xC4,0xC6,0xC9,0xCB,0xCD,0xCF,0xD2,0xD4,0xD6,0xD8,0xDA,0xDB,0xDD,0xDF,0xE0,0xE2,0xE4,0xE6,0xE7,0xE8,0xE9,0xEB,0xEC,0xED,0xEE,0xEF,0xF0,0xF1,0xF2,0xF4,0xF6,0xF7,0xF8,0xF9,0xFA,0xFD,0xFE,0xFF,0x00,0x01,0x02,0x03,0x06,0x07,0x08,0x0A,0x0D,0x0F,0x10,0x11,0x12,0x13,0x14,0x16,0x18,0x19,0x1A,0x1C,0x1D,0x1F,0x21,0x22,0x23,0x25,0x28,0x2A,0x2B,0x2C,0x2E,0x30,0x31,0x33,0x34,0x35,0x38,0x3C,0x3D,0x3E,0x41,0x42,0x44,0x45,0x46,0x47,0x48,0x4A,0x4B,0x4C,0x4D,0x4E,0x4F,0x50,0x53,0x54,0x56,0x57,0x58,0x59,0x5B,0x5C,0x5D,0x5E,0x5F,0x60,0x61,0x63,0x65,0x66,0x67,0x69,0x6A,0x6D,0x6E,0x6F,0x70,0x71,0x72,0x73,0x75,0x77,0x78,0x79,0x7A,0x7B,0x7D,0x7E,0x7F,0x80,0x82,0x84,0x87,0x89,0x8B,0x8E,0x90,0x92,0x94,0x97,0x99,0x9B,0x9D,0xA0,0xA2,0xA4,0xA5,0xA6,0xA8,0xA9,0xAA,0xAB,0xAC,0xAD,0xAE,0xAF,0xB1,0xB2,0xB3,0xB4,0xB5,0xB6,0xB9,0xBA,0xBB,0xBD,0xBF,0xC0,0xC2,0xC4,0xC6,0xC7,0xC8,0xCB,0xCC,0xCD,0xCE,0xCF,0xD1,0xD4,0xD5,0xD6,0xD7,0xD8,0xD9,0xDA,0xDC,0xDD,0xDE,0xDF,0xE0,0xE1,0xE2,0xE4,0xE6,0xE8,0xEA,0xEB,0xED,0xEF,0xF1,0xF2,0xF3,0xF5,0xF6,0xF8,0xF9,0xFA,0xFB,0xFC,0xFF,0x01,0x03,0x04,0x05,0x07,0x08,0x09,0x0A,0x0B,0x0C,0x0D,0x0F,0x10,0x11,0x12,0x13,0x15,0x16,0x18,0x19,0x1A,0x1B,0x1C,0x1D,0x1E,0x20,0x21,0x23,0x24,0x25,0x27,0x29,0x2A,0x2B,0x2C,0x2D,0x2E,0x30,0x32,0x33,0x35,0x36,0x37,0x3A,0x3B,0x3C,0x3E,0x40,0x43,0x44,0x45,0x47,0x48,0x49,0x4A,0x4C,0x4D,0x4E,0x50,0x55,0x56,0x57,0x59,0x5A,0x5B,0x5E,0x5F,0x60,0x61,0x62,0x63,0x65,0x66,0x67,0x68,0x69,0x6B,0x6C,0x6F,0x70,0x71,0x72,0x73,0x74,0x76,0x77,0x79,0x7A,0x7B,0x7D,0x7F,0x80,0x81,0x82,0x83,0x84,0x85,0x86,0x88,0x89,0x8A,0x8B,0x8C,0x8D,0x90,0x91,0x92,0x93,0x94,0x95,0x96,0x97,0x99,0x9B,0x9D,0x9F,0xA2,0xA4,0xA6,0xA8,0xAB,0xAD,0xAF,0xB1,0xB4,0xB6,0xB7,0xB8,0xBB,0xBD,0xBE,0xBF,0xC0,0xC1,0xC2,0xC4,0xC5,0xC6,0xC7,0xC8,0xC9,0xCA,0xCC,0xCD,0xCE,0xCF,0xD1,0xD3,0xD6,0xD8,0xDA,0xDC,0xDE,0xDF,0xE0,0xE1,0xE3,0xE4,0xE5,0xE7,0xE8,0xE9,0xEA,0xEC,0xEF,0xF0,0xF1,0xF2,0xF3,0xF4,0xF5,0xF7,0xF8,0xF9,0xFA,0xFB,0xFC,0xFE,0x01,0x03,0x04,0x05,0x07,0x09,0x0A,0x0B,0x0C,0x0E,0x10,0x12,0x13,0x14,0x15,0x17,0x18,0x1A,0x1C,0x1E,0x1F,0x20,0x21,0x23,0x24,0x25,0x26,0x27,0x28,0x29,0x2B,0x2C,0x2D,0x2E,0x2F,0x30,0x31,0x32,0x34,0x35,0x37,0x39,0x3C,0x3E,0x40,0x42,0x43,0x45,0x47,0x49,0x4A,0x4B,0x4C,0x4E,0x50,0x52,0x53,0x54,0x56,0x57,0x58,0x59,0x5B,0x5C,0x5D,0x5F,0x60,0x61,0x62,0x64,0x66,0x68,0x69,0x6B,0x6C,0x6D,0x6E,0x70,0x71,0x72,0x74,0x75,0x76,0x78,0x79,0x7B,0x7C,0x7D,0x7F,0x80,0x81,0x82,0x83,0x84,0x85,0x86,0x87,0x88,0x8A,0x8B,0x8C,0x8D,0x8F,0x90,0x91,0x93,0x94,0x95,0x96,0x97,0x98,0x9B,0x9D,0x9F,0xA1};
static PROGMEM const unsigned char onsets3[652] = {0xF3,0x31,0x59,0x76,0x98,0xBA,0xDD,0xFF,0x43,0x66,0x88,0xAA,0xC7,0xEF,0x0B,0x2D,0x50,0x72,0x8E,0xB1,0xD3,0xF5,0x17,0x3A,0x5C,0x7E,0xA0,0xC8,0xE5,0x0D,0x29,0x6E,0x90,0xAC,0xD5,0xF7,0x13,0x35,0x58,0x7A,0x9C,0xB9,0xE1,0xFD,0x42,0x6A,0x86,0xA8,0xC5,0x0F,0x31,0x54,0x7C,0x92,0xB5,0xCB,0xF9,0x1B,0x43,0x60,0xA4,0xC1,0xE3,0x28,0x4A,0x72,0x8E,0xB1,0xD3,0xF5,0x17,0x34,0x5C,0x78,0xBD,0xDF,0x01,0x46,0x62,0x84,0xCF,0x0D,0x30,0x52,0x7A,0x91,0xD5,0xF7,0x1A,0x5E,0x86,0xA3,0xE7,0x26,0x5F,0x92,0xAF,0xF3,0x1B,0x32,0x5A,0x7C,0x9F,0xBB,0xE3,0x00,0x28,0x4A,0x66,0x89,0xAB,0xC7,0x0C,0x34,0x4A,0x72,0x95,0xB7,0xD9,0xFB,0x1E,0x3A,0x62,0x7F,0x9B,0xC3,0xE0,0x2A,0x52,0x69,0x91,0xB3,0xD5,0xF7,0x1F,0x36,0x58,0x80,0x9D,0xB9,0xE1,0xF8,0x42,0x65,0x81,0xA9,0xCB,0xE8,0x10,0x54,0x93,0xD2,0x16,0x60,0x9F,0xE4,0x28,0x6D,0xAB,0xF0,0x12,0x34,0x79,0xB2,0xDA,0x02,0x46,0x85,0xAD,0xCF,0xF2,0x0E,0x30,0x4D,0x75,0x97,0xB9,0xD0,0xFE,0x1A,0x65,0x87,0xA3,0xC5,0xE2,0x2C,0x4E,0x6B,0x8D,0xAF,0xD7,0xEE,0x38,0x5B,0x77,0xC1,0x00,0x45,0x67,0x89,0xAB,0xCE,0xF0,0x01,0x57,0x7F,0x95,0xDA,0xFC,0x1E,0x5D,0x7F,0x9C,0xE0,0x25,0x6F,0x91,0xAE,0xF2,0x14,0x3C,0x7B,0x98,0xBA,0x04,0x82,0xAF,0xCC,0x16,0x33,0x77,0x94,0xBC,0xD2,0xFA,0x1D,0x3F,0x61,0x7D,0xA6,0xC8,0xE4,0x29,0x51,0x8F,0xAC,0xCE,0xF0,0x13,0x35,0x57,0x79,0x9C,0xBE,0xE0,0x02,0x47,0x69,0x8B,0xCA,0xF2,0x31,0x53,0x75,0x98,0xBA,0xD6,0xF9,0x15,0x5F,0x82,0xA4,0xC6,0xE8,0x05,0x33,0x60,0x7D,0xBC,0xFA,0x3F,0x83,0xC2,0x0C,0x51,0x95,0xD4,0x18,0x5D,0xA1,0xE0,0x25,0x69,0xA8,0xD0,0xE7,0x0F,0x31,0x53,0x75,0x98,0xB4,0xDC,0xF9,0x1B,0x3D,0x5F,0x7C,0xA4,0xC0,0x05,0x21,0x49,0x88,0xD2,0xF4,0x11,0x55,0x94,0xBC,0xDE,0x1D,0x4B,0x62,0x8F,0xAC,0xF0,0x2F,0x57,0x74,0x96,0xB2,0xD5,0xF7,0x1F,0x3B,0x63,0x86,0xAE,0xC4,0xEC,0x09,0x47,0x8C,0xD0,0xF3,0x15,0x54,0x98,0xC0,0xE2,0x05,0x21,0x66,0x88,0xA4,0xD2,0xEF,0x39,0x72,0xBC,0xD9,0xFB,0x23,0x3F,0x62,0x84,0xA0,0xC3,0xF0,0x07,0x2F,0x4C,0x74,0x90,0xCF,0xF7,0x13,0x3B,0x58,0x74,0x9C,0xB9,0xDB,0x03,0x1F,0x6A,0x86,0xA3,0xED,0x0F,0x31,0x4E,0x76,0x92,0xB5,0xF9,0x27,0x38,0x7C,0xA4,0xC1,0x05,0x28,0x4A,0x8E,0xCD,0x11,0x3A,0x50,0x9B,0xBD,0xDF,0x93,0x24,0x40,0x68,0xA7,0x2A,0x58,0x74,0xB9,0xDB,0xF7,0x3C,0x5E,0x80,0xA3,0xC5,0xE7,0x09,0x2C,0x48,0x70,0x8D,0xD1,0xF9,0x38,0x54,0x77,0x9F,0xBB,0xDD,0x00,0x22,0x60,0x83,0x9F,0xE9,0x11,0x2E,0x50,0x72,0x95,0xB7,0xDF,0xFB,0x18,0x40,0x5C,0x7F,0xA1,0xB8,0x08,0x2A,0x46,0x6E,0x8B,0xAD,0xD5,0xFD,0x14,0x58,0x97,0xDC,0x20,0x59,0xA9,0xEE,0x32,0x71,0xB5,0xF4,0x38,0x77,0x99,0xC1,0x06,0x4A,0x6D,0x8F,0xAB,0xCE,0xF0,0x0C,0x34,0x57,0x79,0x95,0xBD,0xDA,0x02,0x1E,0x46,0x5D,0xA7,0xE6,0x25,0x6F,0xAE,0xF2,0x14,0x37,0x5F,0x7B,0xC0,0xE8,0xFE,0x21,0x43,0x6B,0x8D,0xC6,0x10,0x33,0x55,0x77,0x99,0xBC,0xD8,0x00,0x1D,0x45,0x5B,0x83,0xA0,0xE4,0x29,0x67,0x8F,0xAC,0xF0,0x18,0x35,0x63,0x79,0xB8,0xFD,0xC1,0x4D,0x6F,0x86,0xCA,0xF2,0x09,0x4D,0x92,0xBA,0xD6,0xF9,0x15,0x37,0x59,0x82,0xA4,0xBA,0xE8,0x05,0x27,0x49,0x6B,0x8E,0xAA,0xCC,0xEF,0x17,0x33,0x78,0xB6,0x01,0x3F,0x7E,0xC3,0xEB,0x07,0x4C,0x90,0xB2,0xD5,0xFD,0x19,0x5E,0xA2,0xBE,0xE1,0x03,0x25,0x47,0x64,0xAE,0xD6,0xED,0x15,0x31,0x59,0x76,0xB5,0xF9,0x1B,0x3E,0x7C,0xA4,0xC7,0xE9,0x0B,0x33,0x50,0x8E,0xB6,0xD9,0x01,0x11,0x56,0x7E,0x95,0xDF,0x9F,0x13,0x29,0x46,0x68,0x8A,0xAC,0xC9,0xEB,0x0D,0x2A,0x52,0x6E,0xAD,0xCF,0xF7,0x25,0x30,0x64,0x7B,0xA3,0xBF,0x09,0x42,0x8D,0xCB};

#endif

And here's what I see from Serial:
499
499
499
data using same index:
index: 0    value:1654
index: 1    value:1654
index: 2    value:1654
index: 3    value:1654
index: 4    value:1654
index: 5    value:1654
index: 6    value:1654
index: 7    value:1654
index: 8    value:1654
index: 9    value:1654
data using counter:
index: 0    value:4294945024
index: 1    value:4294962688
index: 2    value:12806
index: 3    value:28928
index: 4    value:4294948096
index: 5    value:4294964224
index: 6    value:14337
index: 7    value:30464
index: 8    value:4294940928
index: 9    value:4294951174

I see what I expect for first 3 values and also for data using same index, but no when using a variable as an index for the PROGMEM arrays.
What is going on and how can I fix this ?
Update
Mikael's answer is the solution, but I will post a working snippet here for reference:
#include <avr/pgmspace.h>
#include "flashData.h"

void setup() {
  Serial.begin(115200);
  delay(2000);

  unsigned char c1,c2,c3;
  unsigned long value;

  Serial.println("data using counter:");
  for(int i = 0 ; i < 652; i++){
    c1 = (unsigned char) pgm_read_byte(&onsets1[i]);
    c2 = (unsigned char) pgm_read_byte(&onsets2[i]);
    c3 = (unsigned char) pgm_read_byte(&onsets3[i]);
    value = (((unsigned long) c1) << 16) | ((unsigned long)c2 << 8) | (unsigned long)c3;
    Serial.print("index: ");
    Serial.print(i);
    Serial.print("\tvalue:");
    Serial.println (value);
    delay(10);
  }

}

void loop() {

}

The key is not only explicitly casting the data read, but also correctly reading the the PROGMEM data.

An unsigned long needs 4 unsigned chars.

Can you please expand ? In my case the data won't need a 4th unsigned char. I've tried (0 << 24) | (c1 << 16) | (c2 << 8) | c3 but the result looks the same.

You don't need to store onsets1. The values it holds are simple to generate at runtime. I'd also look at encoding the values in onsets2 differently, probably as a length and an increment. So the first pair would be 1 and 3, because there's one value (1), followed by an increment of 3 to produce 4. The next pair would be 7 and 3, because there are 7 sequential values (4, 5, 6, 7, 8, 9, A) and the next value is 3 more than the end of that sequence. Etc.

You can try (explict unsigned long values):
unsigned long fromValues = (0x00UL << 16) | (0x01 << 8) | 0xF3;
Serial.println(fromValues);

or:
value = (((unsigned long) c1) << 16) | (c2 << 8) | c3;

The issue is that the expression is implicit int and not long (as you are assuming). You need to tell the compiler about this.
An extra bonus comment: There is actually a 24 bit integer definition in AVR GCC according to the wiki, see Extensions, Types.
Cheers!

Thank you(+1) Mikael, explicitly casting to unsigned long makes sense. However I think this might not be the main issue. This part (unsigned long fromValues = (0x00UL << 16) | (0x01 << 8) | 0xF3; Serial.println(fromValues);) works as expected, this part value = (((unsigned long) c1) << 16) | (c2 << 8) | c3;  only works when I access the PROGMEM arrays using a constant value (e.g. c1 = onsets1[3]; works, c1 = onsets1[i]; produces strange results). I haven't used PROGMEM before, so maybe I'm retrieving/addressing data the wrong way ?

You need to add explicit read of PROGMEM: c1 = (unsigned char) pgm_read_byte(&onsets1[i]). Please see avr/pgmspace.h. I have also put together a more advanced example Sketchbook/ProgramMemory.

I got it to work by typecasting firmly and using multiplications:
  Serial.println("data using counter:");
  for (int i = 0 ; i < 10; i++) {
    c1 = onsets1[i];
    c2 = onsets2[i];
    c3 = onsets3[i];
    value = (uint8_t)c3 + (uint16_t)((uint8_t)c2*256)+(uint32_t)(c1*65536);
    Serial.print("index: ");
    Serial.print(i);
    Serial.print("\t");
    Serial.print(c1, HEX);
    Serial.print(", ");
    Serial.print(c2, HEX);
    Serial.print(", ");
    Serial.print(c3, HEX);
    Serial.print(" ");
    Serial.print("\tvalue:");
    Serial.println (value, HEX);
  }

The result:

data using counter:
index: 0    A1, F6, 0   value:A1F600
index: 1    FC, A7, 0   value:FCA700
index: 2    B5, 7A, B8  value:B57AB8
index: 3    EF, DB, 0   value:EFDB00
index: 4    14, F9, 0   value:14F900
index: 5    D7, 7F, 0   value:D77F00
index: 6    F2, 72, 1   value:F27201
index: 7    6F, DB, 0   value:6FDB00
index: 8    5E, 5E, 0   value:5E5E00
index: 9    F0, E7, B8  value:F0E7B8

Thank you (+1). The issue is that these aren't stored values in PROGMEM, so my issue might actually be about retrieving values from PROGMEM. If you look in the flashData.h, you'll see the first values should be 0x00,0x01,0xF3, not 0xA1,0xF6,0x00.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
How to use ROHM LB202VL display (on BP-006LH-GC board) with Arduino?
I got this 8-unit, 7-segment, 2-digit display board with 24 pins and I am wondering how I can interface with this from my Arduino Uno.
The board is labeled ROHM BP-006LH-GC and it has 8 units of the LB202VL LED display. Please see these photos of the board:

I found the datasheet for the LB202VL unit and I have been trying to make an independent unit on the board to work but I haven't been successful in setting its display.
The datasheet for the LB202VL unit can be found here: http://pdf.datasheetarchive.com/datasheetsmain/Datasheets-27/DSA-525461.pdf
Can anybody help me explain how I can work with this and set the display from my Arduino? After having attempted a few times I feel a little lost with this so any help would be greatly appreciated!

https://www.google.com/search?q=arduino+multiplexed+seven+segment+display

Get a power supply of 5v or so, and a 680 ohm resistor, two lengths of wire, and probe pins until you figure it out.  You can later try a slightly smaller resistor close to the data sheet maximum, especially when you start sequencing the LEDs.

8 pins for the 7 segments and the dot (Anodes). 16 pins for the 16 digits (Cathodes). Looking at the traces I can't see any pattern in how they are connected. Just use Chris's suggestion.

A multimeter set to diode test will also work. I use it to test for working 7-segs when I shop.

I have a similar display, and I found the wiring by inspection and trial and error:
The display consist of 8 pairs of digits (LB-202VL, common cathode).
If contact points are numbered from 1 to 24 from left to right then
the common cathodes for the digits D1-D16 and the segments D.P. and 'a' - 'g' are noted below.
If you i.e. connect 1 to GND and 7 to + then segment 'c' of the first digit will be on. You have to multiplex the digits in order to use it.

D1
D2
D3
D4
D.P.
D5
c
D6
e
D7
a
D8
g
D9
d
D10
f
D11
b
D12
D13
D14
D15
D16


text:
ESP8266 module ESP-12F does not connect to esptool.py
I have bought some blank ESP-12F modules like this: https://www.aliexpress.com/item/1005001520382593.html
For these modules I have made my own adapter PCB to connect them, but now I don't get any connection with the esptool.py to these modules.
The connection is:
TX --------------- RX with an PL2303 USB adapter
RX --------------- TX with an PL2303 USB adapter
EN --------------- 3.3V
GPIO-0 ----------- GND over button
GPIO-15 ---------- GND over 4K7 resistor
RESET ------------ GND over button
GND -------------- GND
VCC -------------- 3.3V from AMS1117-3.3 blocked with Elko 220µF

With a serial terminal I can cleary see with 74880 baud the response for standard boot on TXD (GPIO1):
 ets Jan  8 2013,rst cause:2, boot mode:(7,7)

waiting for host

and on TXD1 (GPIO2)
 ets Jan  8 2013,rst cause:2, boot mode:(7,6)

waiting for host

For flash mode with RESET and GPIO-0 on TXD (GPIO1):
 ets Jan  8 2013,rst cause:2, boot mode:(5,7)

waiting for host

and on TXD1 (GPIO2)
 ets Jan  8 2013,rst cause:2, boot mode:(5,6)

waiting for host

I tried to connect with:
python3 esptool.py --port /dev/ttyUSB0 flash_id
or
python3 esptool.py --chip esp8266 --port /dev/ttyUSB0 flash_id
or
python3 esptool.py --no-stub --port /dev/ttyUSB0 flash_id
but no connection can be established.
"A fatal error occurred: Failed to connect to Espressif device: No serial data received."
With the trace option I get:
TRACE +0.000 Write 46 bytes:
    c000082400000000 0007071220555555 | ...$........ UUU
    5555555555555555 5555555555555555 | UUUUUUUUUUUUUUUU
    5555555555555555 5555555555c0     | UUUUUUUUUUUUU.
TRACE +0.082 Read 1 bytes: 72
TRACE +0.000 Read invalid data: 72
TRACE +0.000 Remaining data in serial buffer: 6c00

I tried the connection with this two versions:
esptool.py v3.3-dev
esptool.py v2.0-beta2
I assume this is simply bad hardware.
Is there anybody who has mastered this problem?

Have you pulled up reset pin?
Check this blog https://circuits4you.com/2016/12/14/reset-programming-esp8266/

The reset pin has an pullup of 4K7

I adressed this problem now to esptool.py https://github.com/espressif/esptool/issues/694

You have io 15 HIGH. Check the pull-down.
The first number in () on first line of the boot log is the state of the boot config pins. First bit is the state of io 2, second the state of io 0 and third is the state of io 15.
You have 5 or 7 which is 101 or 111 in binary. So you have io 15 HIGH.
io 15 HIGH is a mode where the esp8266 doesn't have its own flash memory but boots from the host MCU so you get "waiting for host".
io 15 must be LOW for the esp8266 to use the flash memory.
So you should see for normal mode
ets Jan  8 2013,rst cause:2, boot mode:(3,6)

and for flashing mode
ets Jan  8 2013,rst cause:2, boot mode:(1,6)

Note: it can be (3,7) and (1,7) too

We are talking about GPIO15 that is the next pin (10) after GND in the corner?
This pin is definitely on GND potential - i can measure always about 0,005 V.

Your analyze is correct, although i could not believe it first, because i could measure 5mV at the pin outside, that should be correct.
After a little bit thinking i did check the solder joint to the module and it was cold with a bad contact!
So i resoldered it and now it is running - thank you very much!
Where can i find an explanation of the other numbers?

there is no good documentation. so for me it is a mix of experience and information peaces from different sources which have wrong info mixed with right info. like this one: https://github.com/espressif/esptool/blob/master/docs/en/advanced-topics/boot-mode-selection.rst  it mixes info about esp8266 and esp32

Thanks - it seems not so easy to get all the 'secrets' from chinese products. :-)


text:
Which Bluetooth Module can I use with Arduino Micro/Mini?
I need to use a Bluetooth module with an Arduino Micro/Mini.
Please suggest which one would be the best, HC 05/06, or any other?

It depends on what you want the module to do. For just serial communication the HC modules are fine. For anything more complex you may want to look at BLE.

What research have you done so far?

I have had good results using the Bluefruit LE UART Friend module from Adafruit with a 32u4 micro Arduino type module, there are only a few pins to connect up to get things going (do remember to follow the instructions to pull the CTS pin to ground).
As per the previous answer it really does depend on what you want to do with it - but this module does Low Energy, can be configured as a "Beacon" and there is a companion App for Android/iOS.

Do you require slaves or masters?
Both modules are actually the same, it is just the firmware that is different, and in turn that changes how they use the pins. HC05 can be both Master and Slave, and has a full set of AT commands, whereas HC06 can only be a slave, and has a more limited set of AT commands at its disposal.
Otherwise, for low power projects, then BLE modules, such as HC-10 or HC-11 might be worth considering.
Really, it all depends on what you want to do with them, what is the overall project? You could get away with one HC-05 and one HC-06, but it is really task dependant. It is probably worth you having a read up of some articles on the web, and looking at some example projects to see other people's configurations. This will hopefully help you to see what you actually require for your project.
FWIW, I looked into this a few months back and wrote a short blog, BlueMoon, on it, which contains some example projects and links to video tutorials - this may help give you an insight...


text:
How to make precise square wave?
I am playing with stepper motors and I need a precise square wave generator for f=29.710617283951Hz. I've done this on Raspberry Pi without any problems, but I need something smaller and with less power consumption.
Is it even possible to get that frequency?

How precise do you really need the square wave to be? If you actually need to be precise to the pico Hz that I don't think the Arduino can do that. The clock frequency is 16 MHz, ± a-fair-bit, which means that the smallest time interval that you get is 62.5 pico seconds which is way over your requirement. Can you say more about what you are trying to accomplish?

The frequency you want is  time = .033658 seconds.  Or 33.658 ms (milli-seconds).   Accuracy to the 12 significant digits ?  Like @dlu requested, we might be able to help you better with more information on your intent.

@dlu I think you mean 62.5 nano seconds, not "62.5 pico seconds". However, that doesn't reduce the value of your comment. Trying to generate a signal with an error of 1 part in 10^14 to drive a stepper motor is mind-boggling. Even watches work okay at several parts/million error.

@dlu my project is a "startracker" for astrophotography, my mechanical part is not THAT precise, so I think 29.7106 should be ok.

"I've done this on Raspberry Pi whiteout any problems". How did you get a Raspberry-Pi to generate a signal accurate to better than1 part in 297,106? An R-Pi's crystal is likely accurate to 1 part in 50,000. Did you use an external time source to correct it?

What stepper motor drive are you using to drive the stepper? Is 29.7106Hz the frequency of one motor step, or the frequency of a microstep? If it is the one motor step, then do you want something higher frequency, or have you got that part of the problem solved?

Reducing the power requirement, compared to an R-Pi, should be straightforward, achieving 29.7106Hz is doable, reducing the size is harder (an R-Pi is quite compact).
Some of the available options depend on your electronics skills, and budget.
I'll assume you want to use off-the-shelf hardware as much as practical, but are okay to wire 'modules' together (size could be a bit smaller than an R-Pi using off-the-shelf parts, but could be significantly reduced by a custom made part).
I'll also assume you want to keep costs under $100. (I apologise if I am wrong, but this seems like a level that many of this community might find interesting :-)
Preamble:
Even at 16MHz, 29.7106Hz represents a count of approximately 538,528, which requires 19 bits. So I would not use a chip with 8 or 16 bit timers. I would make my life easier by using a chip with 32 bit timers.
Further, the code will be manipulating quantities with at-least 19 bits, so I would not use an 8 or 16bit CPU. I would make my life easier by using a 32bit CPU that operates on 32bit quantities in a single operation. This may become very important if you opt for a software-heavy approach.
Initial Approach:
Lets assume our 32 bit processor has a clock of 64MHz.
To generate a clock frequency close to 29.710617284Hz (as good as my calculator will do), load a timer with 2,154,112 (maybe 2,154,111, depending on how the counter triggers).
The more accurate divisor is 2,154,112.094.
That count is within 0.05ppm (parts/million) of 29.710617284Hz at a 64MHz clock.
Also, a count of +/- 1 is a change of less than 0.5ppm. That is about 30x better than the more relaxed requirement, '29.7106', mentioned in your comments.
Using 64MHz to drive the count will enable 4x better accuracy, 0.5ppm, than using a 16MHz clock, 2ppm, when we examine ways to implement this in the 'Options' section below.
Summary so far: if the microcontroller had a very accurate clock (much better than 0.5ppm error), then the problem is solved.
However, the crystals used to drive the microcontroller typically have a frequency stability of 30-50ppm, and may be even worse.
Options to reach 0.5ppm:
1. Replace the MCUs crystal with a much better part.
This will typically require some soldering skill, and the parts I've seen (which are likely to replace the existing crystal) are stable at about 10ppm. That might be a factor of 3-5 better than the factory fitted crystal, so may be worth the effort, however there are better options.
2. Feed the MCU with a higher-stability clock source than a crystal.
Most MCU's can accept an external clock source in place of the crystal. Temperature Compensated Crystal Oscillators (TCXO) cost about $5, and can achieve 0.5ppm (0 to 40C, 2ppm -30C to +85C). So an improvement over an 'ordinary' crystal of 30-100x.
If this is appealing, I'd actually recommend looking at ST Micro's Nucleo boards. They are low-cost (under $12), so not much wasted if it doesn't work, and many of the boards already take an external oscillator, so may be easier to change. (They are 32bit CPU's with 32bit timers, and can run at 64MHz,) I imagine folks can recommend other boards.
3. 'Discipline' the square-wave counter with an external high-quality clock
The idea is to use a very accurate external clock, and adjust the square-wave count to get high-accuracy than the existing MCU crystal. While counting the MCU clock for the square wave, use a second counter to count the number of MCU clock cycles a known, high-quality, external timing source event takes. The external clock source could be a TCXO, an Oven Controlled Oscillator (OCXO), which can achieve 0.001ppm) or GPS.
Program code could estimate the error of the MCU clock vs higher-quality clock, and adjust the square-wave count within each square-wave cycle. This may keep the square-wave within 0.5ppm using one or more +1 or -1 adjustment of the counter. This approach has the advantage that the only electronics is connecting the high-quality timing source to a pin on the microcontroller. The rest is (carefully written) software.
Edit:
To be clear, this option 3 can use an unmodified, off-the-shelf, MCU development board. The original crystal is unchanged; there is no need to modify it to replace its crystal.
So it could use a small, e.g. Arduino-Nano size, microcontroller running at a reasonable rate (e.g. 64MHz) connected to a TCXO (or even OCXO), and get close to 0.5ppm error.

You can user Timer 1 on the Atmega328P (such as is found on the Uno) to get you into that ball park:
const unsigned long countTo = 33658;  // F_CPU / 8 / 29.71 / 2

void setup ()
  {
  pinMode (10, OUTPUT);

  // PWM, top at OCR1A
  TCCR1A = bit (WGM10) | bit (WGM11);  // PWM
  TCCR1B = bit (WGM13) | bit (CS11);   // PWM, prescaler of 8
  OCR1A = countTo - 1;                 // zero relative
  OCR1B = (countTo / 2) - 1;           // 50% duty cycle
  bitSet (TCCR1A, COM1B1);   // clear OC1B on compare
  }  // end of setup

void loop ()
  {
  }  // end of loop

I measured a frequency of 29.67 Hz on one device and 29.41 Hz on another. The Uno is driven by a resonator, not a crystal, so a couple of percent error is to be expected.

I have a page about the hardware timers and a cheat-sheet for Timer 1:

The bit() notation is clean.    The bit(WGM13) and bit(WGM10)|bit(WGM11)  chooses the "PWM, Phase correct, top=OCR1A Waveform Generation Mode", which counts TNCT1 both up and down, clearing and setting the pin10 output when it matches OCR1B.  The counting both up and down is what requires the /2 in the frequency calculation.   Out of the box, P  (29.41-29.71)/29.71=-1.01% seems pretty good. One could tune with a countTo=33658.

I agree. You could tweak the counter to more closely match your hardware. I'm a little dubious about my measurements, I'll try for a more accurate frequency measurement. I think the theory is correct.

I measured with my logic analyser and got a period of 33.702056 ms, representing a frequency of 29.6717 Hz, which agrees with the first frequency I posted. I suspect therefore that my Arduino is running a bit slowly.

Nick, on your excellent timer page for Timer 0 CTC mode, you mention that the toggle for OCR0B=200 doesn't work.  I think it is because the timer is cleared at 150.  Maybe try with OCR0B=100.  I'm also curious about the SET and CLEAR modes.

Good point. I've amended that post. You may need to refresh the page to see the new graphic. Please ask a new question if you want to concentrate on the SET and CLEAR modes.


text:
Arduino motor control using digital pins
I am new to these Arduino things, and I am currently using an Uno board for my projects.
I don't know how to control the DC motor with the help of digital pins via L293D. But I cannot use the PWM pins of Arduino, since I have already used them for controlling servos and other things, which need higher accuracy.
I tried using the SoftPWM library to control motor, but it is conflicting with other things like ultrasonic sensor. I don't know what the problem is.
Could someone suggest me a solution?

This is not an answer, but looking at softPWM library disables regular pwm? might help.

You can implement PWM using digital pins by turning the pin on and off with respect to your duty cycle, because PWM (Pulse Width Modulation) depends upon the duty cycle the % amount during which the pin is HIGH or LOW
Let say pin 13 of arduino is connected to your l293D enable pin
Sample Code:
void setup()
{
  pinMode(13, OUTPUT);
}
void loop()
{
  digitalWrite(13, HIGH);
  delayMicroseconds(100); // Approximately 10% duty cycle @ 1KHz
  digitalWrite(13, LOW);
  delayMicroseconds(1000 - 100);
}
Details are also on Arduino Help
https://www.arduino.cc/en/Tutorial/SecretsOfArduinoPWM


text:
Arduino UNO hangs without visible reason
I'm having trouble with Arduino UNO. I bought a cheap clone (what obviously may be the reason) along with display+keypad shield (site in Polish). For my project I wrote a menu library, which (along with simple demo project) may be downloaded or cloned from my Bitbucket repository.
My problem is, that when I push "up" or "down" button and hold it, program behaves as it should for couple of seconds and then board freezes. This is definitely not a case of program hang, because when I used serial debugging, and when such freeze happened, message sent from board was cut in half and no new messages were comming. After such hang I can only reset the board.
I investigated my code a lot, but couldn't find a problem. My questions are:

In what circumstances board may freeze in such way?
If someone owns similar hardware (display, 5-button keypad) and original UNO, could you try running my program and check, if the hang happens as well?

Edit: Complete sources of the project, as requested (now fixed), though the newest version is on repository:
Arduinomenu.ino:
#include "Display.h"
#include "Keypad.h"
#include "Menu.h"

LiquidCrystal lcd(8, 9, 4, 5, 6, 7);
Display display(lcd, 16, 2);
Keypad keypad;

BaseMenuItem * subItems[] =
{
  new ActionMenuItem("Subaction 1", 4),
  new ActionMenuItem("Subaction 2", 5),
  new ActionMenuItem("Subaction 3", 6),
  new IntMenuItem("Int item", 0, 10, 5),
  new BoolMenuItem("Bool item", true)
};

BaseMenuItem * mainItems[] =
{
  new ActionMenuItem("Action 1", 1),
  new ActionMenuItem("Action 2", 2),
  new ActionMenuItem("Action 3", 3),
  new SubMenuItem("Submenu", subItems, sizeof(subItems) / sizeof(BaseMenuItem *))
};

MainMenu mainMenu(display, mainItems, sizeof(mainItems) / sizeof(BaseMenuItem *));

void setup()
{
  Serial.begin(9600);
  while (!Serial);

  mainMenu.Show();
}

void loop()
{
  int key = keypad.ReadKey();
  int result = mainMenu.Action(key);
}

Display.h:
#ifndef DISPLAY_H__
#define DISPLAY_H__

#include <LiquidCrystal.h>

struct Display
{
  LiquidCrystal & lcd;
  unsigned char cols, rows;

  Display(LiquidCrystal & lcd, unsigned char cols, unsigned char rows);
  void Print(unsigned char col, unsigned char row, char * text);
};

#endif

Display.cpp:
#include "Display.h"

Display::Display(LiquidCrystal & lcd, unsigned char cols, unsigned char rows)
  : lcd(lcd)
{
  this->cols = cols;
  this->rows = rows;
  lcd.begin(cols, rows);
}

void Display::Print(unsigned char col, unsigned char row, char * text)
{
  this->lcd.setCursor(col, row);
  this->lcd.print(text);
}

Keypad.h:
#ifndef KEYPAD_H__
#define KEYPAD_H__

#define USE_KEYPAD

#define KEY_RIGHT  0
#define KEY_UP     1
#define KEY_DOWN   2
#define KEY_LEFT   3
#define KEY_SELECT 4
#define KEY_NONE   5

#define LAST_READ_DELAY 250

#define keypadPort 0

class Keypad
{
private:
  unsigned long lastRead;
  unsigned char lastKey;
public:
  Keypad();
  unsigned char ReadKey();
};

#endif

Keypad.cpp:
#include "Keypad.h"
#include "Arduino.h"

Keypad::Keypad()
{
  this->lastRead = millis();
  this->lastKey = KEY_NONE;
}

// Button values (approximate):
// Right    0
// Up       144
// Down     329
// Left     504
// Select   741
unsigned char Keypad::ReadKey()
{
  unsigned long now = millis();
  if (this->lastKey != KEY_NONE && (now - this->lastRead < LAST_READ_DELAY) && (now > this->lastRead))
  {
    return KEY_NONE;
  }

  int key = analogRead(keypadPort);

  if (key > 1000)
    this->lastKey = KEY_NONE;
  else if (key < 50)
    this->lastKey = KEY_RIGHT;
  else if (key < 250)
    this->lastKey = KEY_UP;
  else if (key < 450)
    this->lastKey = KEY_DOWN;
  else if (key < 650)
    this->lastKey = KEY_LEFT;
  else if (key < 850)
    this->lastKey = KEY_SELECT;
  else
    this->lastKey = KEY_NONE;

  this->lastRead = now;

  return this->lastKey;
}

Menu.h:
#ifndef MENU_H__
#define MENU_H__

#include <stdio.h>
#include <stdlib.h>
#include "Display.h"
#include "Arduino.h"

#ifdef USE_KEYPAD
  #define MENU_ACTION_RIGHT KEY_RIGHT
  #define MENU_ACTION_UP KEY_UP
  #define MENU_ACTION_DOWN KEY_DOWN
  #define MENU_ACTION_LEFT KEY_LEFT
  #define MENU_ACTION_SELECT KEY_SELECT
#else
  #define MENU_ACTION_RIGHT 0
  #define MENU_ACTION_UP 1
  #define MENU_ACTION_DOWN 2
  #define MENU_ACTION_LEFT 3
  #define MENU_ACTION_SELECT 4
#endif

#define TEXT_BACK "Back"
#define TEXT_BACK_LEN 4

#ifndef SELECTION_CHAR
#define SELECTION_CHAR '>'
#endif

#define MENU_TYPE_INT 0
#define MENU_TYPE_BOOL 1
#define MENU_TYPE_SUB 2
#define MENU_TYPE_ACTION 3

#ifndef MAX_MENU_LEVELS
#define MAX_MENU_LEVELS 5
#endif

// Forward declarations -------------------------------------------------------

class MainMenu;

// BaseMenuItem ---------------------------------------------------------------

class BaseMenuItem
{
protected:
  const char * name;
  unsigned char len;

public:
  BaseMenuItem(const char * name);
  void ShowListItem(Display & display, int row, bool selected);
  const char * GetName();
  virtual int GetType() = 0;
};

// MenuItem -------------------------------------------------------------------

class MenuItem : public BaseMenuItem
{
friend class MainMenu;

private:
  void ShowEditMode(Display & display);

protected:
  void ShowEditHeader(Display & display);
  virtual void ShowEditValue(Display & display) = 0;
  virtual void HandleAction(int key) = 0;

public:
  MenuItem(const char * name);
};

// IntMenuItem ----------------------------------------------------------------

class IntMenuItem : public MenuItem
{
friend class MainMenu;

private:
  int value;
  int min;
  int max;

protected:
  void ShowEditValue(Display & display);
  void HandleAction(int key);

public:
  IntMenuItem(const char * name, int min, int max, int defaultValue);
  int GetType();
};

// BoolMenuItem ---------------------------------------------------------------

class BoolMenuItem : public MenuItem
{
friend class MainMenu;

private:
  bool value;

protected:
  void ShowEditValue(Display & display);
  void HandleAction(int key);

public:
  BoolMenuItem(const char * name, bool defaultValue);
  int GetType();
};

// SubMenuItem ----------------------------------------------------------------

class SubMenuItem : public BaseMenuItem
{
friend class MainMenu;

private:
  BaseMenuItem ** items;
  int count;

public:
  SubMenuItem(const char * name, BaseMenuItem ** items, int count);
  int GetCount();
  BaseMenuItem * GetItem(int index);
  int GetType();
};

// ActionMenuItem -------------------------------------------------------------

class ActionMenuItem : public BaseMenuItem
{
private:
  int actionId;

public:
  ActionMenuItem(const char * name, int actionId);
  int GetActionId();
  int GetType();
};

// MainMenu -------------------------------------------------------------------

class MainMenu : public SubMenuItem
{
private:
  SubMenuItem ** itemStack;
  short * positionStack;
  unsigned char stackTop;
  short selectedItem;
  short firstItem;
  Display & display;
  MenuItem * editedItem;

  void ShowBack(int row, bool isSelected);

public:
  MainMenu(Display & display, BaseMenuItem ** items, int count);
  ~MainMenu();
  int Action(int keyAction);
  void Show();
};

#endif

Menu.cpp:
#include "Menu.h"

char * NewEmptyLine(int width)
{
  char * line = new char[width + 1];

  memset(line, 32, width);
  line[width] = 0;
  return line;
}

void DeleteLine(char * line)
{
  delete line;
}

// BaseMenuItem ---------------------------------------------------------------

void BaseMenuItem::ShowListItem(Display & display, int row, bool selected)
{
  char * line = NewEmptyLine(display.cols);

  if (selected)
    line[0] = SELECTION_CHAR;

  strncpy(line + 2, this->name, min(display.cols - 2, this->len));
  display.Print(0, row, line);

  DeleteLine(line);
}

BaseMenuItem::BaseMenuItem(const char * name)
{
  this->name = name;
  this->len = strlen(this->name);
}

const char * BaseMenuItem::GetName()
{
  return this->name;
}

// MenuItem -------------------------------------------------------------------

void MenuItem::ShowEditHeader(Display & display)
{
  char * line = NewEmptyLine(display.cols);

  strncpy(line, this->name, min(display.cols, this->len));
  display.Print(0, 0, line);

  DeleteLine(line);
}

void MenuItem::ShowEditMode(Display & display)
{
  this->ShowEditHeader(display);
  this->ShowEditValue(display);
}

MenuItem::MenuItem(const char * name) : BaseMenuItem(name)
{

}

// IntMenuItem ----------------------------------------------------------------

IntMenuItem::IntMenuItem(const char * name, int min, int max, int defaultValue)
  : MenuItem(name)
{
  this->min = min;
  this->max = max;
  this->value = defaultValue < this->min ? this->min : (defaultValue > this->max ? this->max : defaultValue);
}

void IntMenuItem::ShowEditValue(Display & display)
{
  char * line = NewEmptyLine(display.cols);

  line[0] = 127;
  line[display.cols - 1] = 126;

  int i = display.cols - 3;
  int temp = abs(this->value);
  do
  {
    line[i] = temp % 10 + 48;
    temp /= 10;
    i--;
  }
  while (temp > 0);

  if (this->value < 0)
    line[i] = '-';

  display.Print(0, 1, line);

  DeleteLine(line);
}

int IntMenuItem::GetType()
{
  return MENU_TYPE_INT;
}

void IntMenuItem::HandleAction(int key)
{
  if (key == MENU_ACTION_LEFT)
  {
    if (this->value - 1 >= this->min)
      this->value--;
  }
  else if (key == MENU_ACTION_RIGHT)
  {
    if (this->value + 1 <= this->max)
      this->value++;
  }
}

// BoolMenuItem ---------------------------------------------------------------

BoolMenuItem::BoolMenuItem(const char * name, bool defaultValue)
  : MenuItem(name)
{
  value = defaultValue;
}

void BoolMenuItem::ShowEditValue(Display & display)
{
  char * line = NewEmptyLine(display.cols);

  line[0] = 127;
  line[display.cols - 1] = 126;

  if (this->value)
  {
    strncpy(line + display.cols - 4, "On", 2);
  }
  else
  {
    strncpy(line + display.cols - 5, "Off", 3);
  }

  display.Print(0, 1, line);

  DeleteLine(line);
}

int BoolMenuItem::GetType()
{
  return MENU_TYPE_BOOL;
}

void BoolMenuItem::HandleAction(int key)
{
  if (key == MENU_ACTION_LEFT || key == MENU_ACTION_RIGHT)
  {
    value = !value;
  }
}

// SubMenuItem ----------------------------------------------------------------

SubMenuItem::SubMenuItem(const char * name, BaseMenuItem ** items, int count) : BaseMenuItem(name)
{
  this->items = items;
  this->count = count;
}

int SubMenuItem::GetCount()
{
  return this->count;
}

BaseMenuItem * SubMenuItem::GetItem(int index)
{
  return this->items[index];
}

int SubMenuItem::GetType()
{
  return MENU_TYPE_SUB;
}

// ActionMenuItem -------------------------------------------------------------

ActionMenuItem::ActionMenuItem(const char * name, int actionId) : BaseMenuItem(name)
{
  this->actionId = actionId;
}

int ActionMenuItem::GetActionId()
{
  return this->actionId;
}

int ActionMenuItem::GetType()
{
  return MENU_TYPE_ACTION;
}

// MainMenu -------------------------------------------------------------------

void MainMenu::ShowBack(int row, bool isSelected)
{
  char * line = NewEmptyLine(display.cols);
  if (isSelected)
  {
    line[0] = SELECTION_CHAR;
  }
  strncpy(line + 2, TEXT_BACK, TEXT_BACK_LEN);
  display.Print(0, row, line);

  DeleteLine(line);
}

void MainMenu::Show()
{
  if (editedItem == NULL)
  {
    bool isSubMenu = stackTop > 0;

    for (int i = 0; i < display.rows; i++)
    {
      if (isSubMenu && i + firstItem == itemStack[stackTop]->GetCount())
      {
        ShowBack(i, i + firstItem == selectedItem);
      }
      else
      {
        itemStack[stackTop]->GetItem(i + firstItem)->ShowListItem(display, i, i + firstItem == selectedItem);
      }
    }
  }
  else
  {
    editedItem->ShowEditMode(display);
  }
}

MainMenu::MainMenu(Display & display, BaseMenuItem ** items, int count) :
  SubMenuItem(NULL, items, count),
  display(display)
{
  // Assuming initially 5 levels of menu max
  itemStack = new SubMenuItem * [MAX_MENU_LEVELS];
  positionStack = new short[MAX_MENU_LEVELS * 2];
  stackTop = 0;
  itemStack[stackTop] = this;
}

MainMenu::~MainMenu()
{
  delete itemStack;
  delete positionStack;
}

int MainMenu::Action(int keyAction)
{
  int lastItemId = stackTop > 0 ? itemStack[stackTop]->GetCount() : itemStack[stackTop]->GetCount() - 1;

  if (editedItem == NULL)
  {
    if (keyAction == MENU_ACTION_UP || keyAction == MENU_ACTION_DOWN)
    {
      if (keyAction == MENU_ACTION_UP)
        this->selectedItem--;
      else
        this->selectedItem++;

      if (selectedItem < 0)
        selectedItem = lastItemId;
      if (selectedItem > lastItemId)
        selectedItem = 0;

      // Adjust items in view

      while (firstItem > selectedItem)
        firstItem--;
      while (firstItem + display.rows - 1 < selectedItem)
        firstItem++;

      Show();
      return -1;
    }
    else if (keyAction == MENU_ACTION_SELECT)
    {
      if (selectedItem == lastItemId && stackTop > 0)
      {
        // Chosen Back

        stackTop--;
        selectedItem = positionStack[stackTop * 2];
        firstItem = positionStack[stackTop * 2 + 1];

        Serial.print("Restored sel/first: ");
        Serial.print(selectedItem);
        Serial.print("/");
        Serial.print(firstItem);
        Serial.print("\n");

        Show();
      }
      else
      {
        int type = itemStack[stackTop]->GetItem(selectedItem)->GetType();
        switch (type)
        {
          case MENU_TYPE_INT:
          {
            editedItem = (MenuItem *)itemStack[stackTop]->GetItem(selectedItem);
            Show();
            break;
          }
          case MENU_TYPE_BOOL:
          {
            editedItem = (MenuItem *)itemStack[stackTop]->GetItem(selectedItem);
            Show();
            break;
          }
          case MENU_TYPE_SUB:
          {
            Serial.print("Entering ");
            Serial.print(((SubMenuItem *)(itemStack[stackTop]->GetItem(selectedItem)))->GetName());
            Serial.print("\n");

            if (stackTop >= MAX_MENU_LEVELS - 1)
              return -1;

            Serial.print("Storing sel/first: ");
            Serial.print(selectedItem);
            Serial.print("/");
            Serial.print(firstItem);
            Serial.print("\n");

            positionStack[stackTop * 2] = selectedItem;
            positionStack[stackTop * 2 + 1] = firstItem;

            itemStack[stackTop + 1] = ((SubMenuItem *)(itemStack[stackTop]->GetItem(selectedItem)));
            stackTop++;
            selectedItem = 0;
            firstItem = 0;

            Show();

            break;
          }
          case MENU_TYPE_ACTION:
          {
            int actionId = ((ActionMenuItem *)(itemStack[stackTop]->GetItem(selectedItem)))->GetActionId();
            return actionId;
          }
        }
      }
    }
  }
  else
  {
    if (keyAction == MENU_ACTION_LEFT || keyAction == MENU_ACTION_RIGHT || keyAction == MENU_ACTION_UP || keyAction == MENU_ACTION_DOWN)
    {
      Serial.print("Item action\n");
      editedItem->HandleAction(keyAction);
      Show();
    }
    else if (keyAction == MENU_ACTION_SELECT)
    {
      editedItem = NULL;
      Show();
    }
  }
  return -1;
}

Let me guess... You are making heavy use of the String class...?

@Majenko Did you look in the sources? I don't use String at all.

No, I was using my phone at the time. So that rules that common problem out then. Kudos to you for shunning String BTW ;)

@Majenko I've got a project that uses String pretty heavily. Although I haven't experienced any issues, looks like I have some reading to do to get an understanding why all of a sudden I'm seeing that the specific class is frowned upon :)

Stack exchange sites really require that the information needed to understand the problem be included in the post itself.

Yeah, I don't see any code here.

@Michel Keijzers provided overall answer.
And now probable cause:
char * GetEmptyLine(int width)
{
  char * line = new char[width + 1];

In the Menu.cpp. There is no single delete in the code at all. But so many calls of GetEmptyLine.
Using the Arduino String might have done less evil than this kind of memory leak.

Oh my. This is how programmer ends when dealing with automatic memory-managed languages for too long :( But knowing that this is my fault in the end is a relief anyway :)

I wouldn't assume by default that it's a hardware problem.
It seems like a software problem, and there can be many reasons that the Arduino hangs, e.g.:

Too little stack space
Too little heap space
Pointer problems

The Arduino does not have safety against too little memory, that means it will overwrite memory -- causing lots of problems -- and hanging is a normal symptom.
What I suggest is:

Check how many global / local variables you have (and their size), especially arrays and instances of objects; strings can be memory consuming
Check that you don't have an overly deep stack trace
To check, you can print out the available memory in several places

For debugging statements, keep in mind that the debug strings can be cut in half just when the microprocessor hangs; maybe the print statements are still in the buffer but will not be sent, so minimize the (length of the) debug statements in such cases.

There could be a hardware problem as well. Cheap clone board work in most cases, but cheap USB cables are bad. Since the display has a backlight, perhaps there is too much voltage drop in the USB cable.

Good comment (I didn't say it IS a software problem, just not to rule it out).

Using string literals wastes precious RAM. To avoid this you can use the F macro. That way the string is stored in flash memory only. If by freeing up that extra RAM your program doesn't crash or works better you most likely have the answer.
for example:
Serial.print(F("RAM "));          // F() macro frees up 4 bytes of RAM
Serial.println(2048 - freeRam());

a freeRam function:
int freeRam()
{
 extern int __heap_start, *__brkval;
 int v;
 return (int)&v - (__brkval == 0 ? (int)&__heap_start : (int)__brkval);
}

On a project I'm working on I noticed that my Arduino Nano with 2K RAM starts crashing when my free RAM goes below 150 bytes. That free RAM is where the stack is and needs to be accounted for.