text:
Efficient algorithm/data structure to calculate moving averages
Currently I am developing a graphic LCD system to display temperatures, flows, voltages, power and energy in a heat pump system. The use of a graphic LCD means that half of my SRAM and ~75% of my flash have been used up by a screen buffer and strings.
I am currently displaying min/max/average figures for energy At midnight when the daily figure resets, the system checks if the consumption for the day is above or below the previous minimum or maximum, and stores the value. The average is calculated by dividing the cumulative energy consumption by the number of days.
I would like to display the daily average over the last week and month (4 weeks for simplicity) i.e. a rolling average. Currently this involves maintaining an array of values for the last 28 days and calculating an average over the whole array for monthly and last 7 days for weekly.
Initially I was doing this using an array of floats (as the energy is in the form "12.12kWh"), but this was using 28 * 4 bytes = 112 bytes (5.4% of SRAM). I don't mind having only a single decimal point of resolution, so I changed to using uint16_t and multiplying the figure by 100. This means that 12.12 is represented as 1212, and I divide by 100 for display purposes.
The size of the array is now down to 56 bytes (much better!).
There is no trivial way to reduce the figure down to a uint8_t that I can see. I could tolerate the loss of a decimal place ("12.1kWh" instead of "12.12kWh"), but consumption is frequently higher than 25.5kWh (255 being the highest value represented by a 8-bit unsigned integer). Consumption has never been below 10.0kWh or above 35.0kWh, so conceivably I could subtract 10 from the stored figures, but I know that one day we will exceed these limits.
I then tested code to pack 9-bit values into an array. This gives a range of 0-51.2kWh and uses 32 bytes in total. However, accessing an array like this is pretty slow, especially when you have to iterate over all values to calculate an average.
So my question is - is there a more efficient way of calculating a moving average with three windows - lifetime, 28 days and 7 days? Efficiency means smaller in terms of SRAM usage, but without the penalty of huge code. Can I avoid storing all values?

Do you want to calculate a moving average over specific windows or an estimate/approximation of the average would do?

I want a moving average over a 7 day and 28 day window.

you could use a resolution of 0.2kWh (divide and multiply with factor 5) and you still get 0-51.2kWh range in 8 bits

You might end up putting strings and other constants in external RAM or external Flash -- see "What can I do if I run out of Flash memory or SRAM?".

If your data has low standard deviation, then one method would be to sum values over the window, and then keep subtracting the mean from the sum, while adding the new value.
This would work well if there are no outliers, thereby leading to the aggregate error tending to zero over time.
//Pseudocode

count=0
while new_reading and count<7:
    sum += new_reading        //Calculate the sum of first 7 values
    count++

while new_reading:            //Loop till new readings available
    avg = sum / 7             //Calculate average
    sum -= avg                //Subtract average from sum
    sum += new_reading        //Add next reading to sum
    print avg

you can use a different method, you keep the current average and then do
average = (weight1*average+weight2*new_value)/(weight1+weight2);

it's not a true rolling average and has different semantics, but it may fit your needs nonetheless
for a more efficient calculation method for your 9 bits per value solution you could keep the 8 highest bits of the values in an array and separate out the least significant bits:
uint8_t[28] highbits;
uint32_t lowbits;

to set a value you need to split it out
void getvalue(uint8_t index, uint16_t value){
    highbits[index] = value>>1;
    uint32_t flag = (value & 1)<<index;
    highbits|=flag;
    highbits&=~flag;
}

resulting in 2 shifts an AND and an OR and a not
to calculate the average you can use various bit tricks to speed it up:
uint16_t getAverage(){
    uint16_t sum=0;
    for(uint8_t i=0;i<28;i++){
        sum+=highbits[i];
    }
    sum<<=1;//multiply by 2 after the loop
    sum+=bitcount(lowbits);
    return sum/28;
}

you can use an efficient parallel bitcount for the bitcount()

Can you explain more how this would allow me to calculate the 7 and 28 day average?

I've used this approach for smoothing noisy analog values before, and it was certainly fairly effective. I didn't need a lot of precision though, as the resulting values were being put through a very coarse quantiser. I also didn't need historical averages.

This doesnt allow calculating the average for a specific window.

@Cybergibbons you can use different weights to approximate the window so old values become insignificant earlier or later, or keep 7 days for the 7 day window and this moving average for the 28 day average

How about only storing the difference from the previous value? In electronics there is a similar concept called Delta Sigma converter, which is used for DA/AD converters. It relies on the fact that the previous measurement is reasonably near the current one.

Another interesting idea. Unfortunately I am not sure energy consumption will always be like this, as it is a heat pump system and one day could take 30kWh, the next 10kWh. I really need to gather data and see.

Why couldn't you just add the values together as soon as you obtain them. So what I mean is you get the value for day 1, you divide it by 1 and store it and the 1 somewhere. Then you multiply the 1 by the value and add it to the next value and divide them both by 2.
Doing this method would create a rolling average with two or three variable as I can think of. I would write some code but I am new to stackexchange so please bear with me.

I don't understand how this deals with the 7 day and 28 day window?

Keep track of the previous and next values and keep adding and subtracting them from your running average...

So then I am back in the state of needing to remember 27 days of history, surely?

I've been thinking and you are right. So that technically makes my answer incorrect. I'm investing some more time and patience into it. Maybe something out of the box. I'll let you know if I come up with something. We do something like this a lot at my workplace. Let me ask around. Sorry about the confusion.

is there a more efficient way of calculating a moving average with
  ... 28 days and 7 days?
  ...
  needing to remember 27 days of history ... ?

You might get close enough storing 11 values rather than 28 values,
perhaps something like:
// untested code
// static variables
uint16_t daily_energy[7]; // perhaps in units of 0.01 kWh ?
uint16_t weekly_energy[4]; // perhaps in units of 0.1 kWh ?

void print_week_status(){
    Serial.print( F("last week's total energy :") );
    Serial.println( weekly_energy[0] );
    int sum = 0;
    for( int i=0; i<4; i++ ){
        sum += weekly_energy[i];
    };
    Serial.print( F("Total energy over last 4 complete weeks :") );
    Serial.println( sum );
    int average_weekly_energy = sum/4;
    int average_daily_energy = average_weekly_energy/7;
    Serial.print( F("Average daily energy over last 4 weeks :") );
    Serial.println( average_daily_energy );
}
void print_day_status(){
    Serial.print( F("Yesterday's energy :") );
    Serial.println( daily_energy[0] );
    Serial.print( F("average daily energy over the last 7 complete days: ") );
    int sum = 0;
    for( int i=0; i<7; i++ ){
        sum += daily_energy[i];
    };
    int average = sum/7;
    Serial.println( average );
}

In other words, rather than storing every detail of every day for the last 27 days,
(a) store 7 or so values of detailed daily information for the past 7 or so days, and also
(b) store 4 or so "summarized" values of total or average information for each of the past 4 or so weeks.


text:
What overheads and other considerations are there when using a struct vs a class?
C on embedded systems has traditionally use structs to hold structured data.
Arduino brings C++ to the table, so we can use classes instead.
Lets say we have two different data structures which could be considered very similar:
typedef struct
{
    int valueOne;
    int valueTwo;
    int valueThree;
} value_t;

and:
class Value
{
    public:
        int valueOne;
        int valueTwo;
        int valueThree;
}

In terms of memory, what difference will this make?
I still see the struct technique being used heavily - why is this?

if I recall my C++ correctly there is no difference between struct and class besides the default visibility

I would assume structs are used primarily because the great majority of embedded hardware is coded in C, so anyone who spends a lot of time writing code for small MCUs probably knows C much better then C++, and automatically reaches for C idioms before C++ idioms. It's a programmer performance issue.

As taken from the accepted answer from When should you use a class vs a struct in C++?

The only difference between a class and a struct in C++ is that
  structs have default public members and bases and classes have default
  private members and bases. Both classes and structs can have a mixture
  of public and private members, can use inheritance, and can have
  member functions.
I would recommend using structs as plain-old-data structures without
  any class-like features, and using classes as aggregate data
  structures with private data and member functions.

Memory wise the access modifier makes no difference and given the memory constraints of the Arduino, people are less likely to use classes with complex hierarchies, but prefer the POD structs anyway.

Unlike C, an instance of a struct in C++ is an object in exactly the same way as an instance of a class. From the point-of-view of the compiled code, they are identical. Memory usage, alignment, access times etc. are exactly the same (i.e. there are no overheads).
From the programmer's point-of-view, there is a very minor difference. Members of a struct have public visibility by default, whereas members of a class have private visibility by default. Otherwise, all language features work the same on both, such as constructors/destructors, inheritance, polymorphism, templates, and operator overloading. You can even derive a struct from a class, and vice versa.
Despite the similarity, it's quite common to see people deliberately using a struct in C++ for very simple structures, e.g. where it only consists of a few data members, but no functions. A class would be used for anything more complex. This is purely a matter of convention or personal preference though, and can be used as a subtle indication of the structure's intended complexity.

As other answers have pointed out, your particular struct and class are indistinguishable performance wise (There are slight differences in the scopes of the type names, due to the way you defined your struct). The delineation in C++ is not between struct and class, but between types that are POD (plain old data) and types that are not, as explained in this discussion.

In terms of memory, what difference will this make?

None. Structs and classes are the same thing, differing only in protection levels, and instantiating either creates an 'object'.

still see the struct technique being used heavily - why is this?

Less typing if you're not trying to hide data members.

Note that
typedef struct { ... } Foo;

is obsolete in C++ and you should just use
struct Foo { ... };

It's not very clear in the answers but the other effect of structs is public inheritance by default vs private inheritance by default with classes.
As mentioned by others, using struct is generally a convention for POD types.
Also technically 'struct vs object' should be 'struct vs class' (as instantiating a struct or a class still gives an object).

Edited question object->class.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/

sort: votes


text:
Powering Arduino with Solar Panels
Clearly, an Arduino can be powered with solar panels.

What is the most optimal configuration?
Can you recharge and discharge at the same time?
Is LiPO the best battery cache technology, or is there something more durable out there, for fielded deployment?
What are the most economic, and reliable ways to do this?

Helpful pages: http://playground.arduino.cc/Main/IntWithHW-PwrSup, http://www.voltaicsystems.com/solar-arduino-guide.shtml

First of all you want to buy some solar panels or cells. Although they can be found on eBay for very cheap, but they not be as efficient as they say. I would say try to buy from a reputable dealer to see reviews on the product or trust that they work efficiently. Sparkfun datasheets are generally accurate and can be trusted.

Most optimal configuration: Do not use a regulator and some sort of step-up booster. Try the TPS61200 or TPS61202 chips. They will not regulate the voltage, they boost the voltage dynamically. This is much more efficient and therefore generates less heat. They work like a reverse voltage regulator, taking .3V to 5.5V and turning it into anywhere from 1.8V to 5V (xxx0) or 5V exactly (xxx2). Hack a USB B end and just connect the power lines to the Arduino.
Discharge/charge simultaneously: Yes, however I'm not sure how to. I believe that if you hook up a charger IC and a load, the voltage won't go through the battery, it will go through the path of least resistance. However, your battery wouldn't get charged then. This is outside the scope of the question, so I would recommend EE on how exactly to do this.
Is LiPO the best: Try a Lithium-Ion Battery. (A.K.A. Li-ion) According to this site:

Although the lithium-polymer battery is sleeker and thinner, lithium-ion batteries have a higher energy density and cost less to manufacture.

Therefore, it seems like Li-ion is the best choice for you. Note: Since you are dealing with very dense batteries that if overvolted can go up into flames [Note: please mute your speakers!], make sure to go with Sparkfun/Adafruit/etc...
Economic way: If you don't mind a big "homemade" solar panel, go this way. Buy through eBay or a similar site solar cells. They are small plates that are connected to create a solar panel. Try looking for "grade B" or similar solar cells that work fine, they just may have minor imperfections. It won't be as pretty, but you'll save some cash.

See this EE.SE post for more information. One cool thing that I saw was:

This is really cool. You can connect the Li-Ion voltage through an op-amp to generate a pulse wave, which can be picked up by your MPU to tell your bot to go charge –  Raaj Oct 18 '13 at 18:44

I'm not exactly sure how to do this, but it might be useful. I couldn't find anything online about it.

"Is LiPO the best: Try a Lithium-Ion Battery. (A.K.A. Li-ion) According to this site:" well my english not perfect But if I don't get it wrong, this site says the opposite. "Overall, lithium-polymer is slowly replacing lithium-ion in the smartphone industry due to its superior safety, form factor versatility, and weight attributes in high-end and mid-tier devices."

I would actually suggest LiFePo4 for anything DIY. Not only is it safer so one false move won't cause an inferno, but it will last two to four times as long at the CONST of a little extra size and weight.
Since you are just running an arduino, I would use only 1 single cell, and avoid cell balance woes.
I would use a boost regulator to step up the voltage from the cell to the required 5v.
To actually charge the cell, I would probably use a cheap ebay CC-CV charger board, and set the voltage for about 0.15V below the max for your lifepo(you only lose 10% of the energy but You make the battery last much longer), and set the current as appropriate for your panel and battery.
If your circuit is drawing power while the charger is charging, your circuit will take what it needs and any extra the charger puts out will go into the battery.
You might need a diode, to make the battery not discharge back through the charger.
Use a protection pcb like you can get from batteryspace, or better yet a protected lifepo with the pcb built it.
Remember to size your solar panel so that there is enough energy to keep the circuit running. Even with the protected cells overdischarge kills.
What I'd really like to do is design a lifepo4 charge control board that had all this in one unit since everyone seems to want it...

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
Circuit diagram interpretation and electrical flow

I'm new to Arduino and consequently, to circuits. I've been reading Arduino Robotics by Warren and this was a circuit present in the book.
From what Warren has said, electricity in the circuit travels from the negative pole to the positive, BUT to imagine it as going the other way for simplicity. So, looking at this circuit... it seems useless to have the resistor placed after the LED, based on the + --> - model. Wouldn't the LED turn on before going through the resistor, rendering the resistor useless?
Sorry if this seems like a basic concept, I'm entirely new to this.

So, this seems only very tangentially related to Arduino, being a beginner level electronics question. Should it be here?

The OP got this question after reading Arduino-related stuff, so Arduino.SE is a natural target, although I agree this question would probably better in EE.SE

@jfpoilpret Indeed it belongs in EE.SE. Actually, it's already there, so there's no point in migrating, in case anyone is thinking about that.

Should the Resistor be Placed Before or After the LED?
The important thing to remember is that the direct current electricity must flow through the circuit to be useful. Since the resistor is part of the circuit (meaning that there isn't another path for the current to take) it resists the overall flow, thereby helping to protect the LED. Whether before or after the LED, it would still be in the circuit, and would therefore still reduce current flow.
Visualizing the Flow of Water
Think of it like a water hose, where the LED is a flow meter and the resistor is your foot. If you step on the hose before the flow meter it will cause the overall flow to slow down. Stepping on the hose after the flow meter but before the nozzle will have the same result, as the water must pass through the entire length of the hose before being released.


text:
Arduino LED matrix and 2D array, data seems to be corrupting
Duplicate of this question
I'm trying to drive an 8x8 LED matrix with an arduino nano, but I can't get the LEDs to display what I tell it to.
I'm using this LED Matrix
Here's my code:
int row1 = 2;
int row2 = 3;
int row3 = 4;
int row4 = 5;
int row5 = 6;
int row6 = 7;
int row7 = 8;
int row8 = 9;

int col1 = 10;
int col2 = 11;
int col3 = 12;
int col4 = 18;
int col5 = 14;
int col6 = 15;
int col7 = 16;
int col8 = 17;

void setup() {
  int i;
  for(i = 2; i <= 18; i++) {
    pinMode(i,OUTPUT);
  }
  for(i = 2; i <= 9; i++) {
    digitalWrite(i, HIGH);
  }
}

void loop() {

  int display [8][8] = {
  {1,0,0,0,0,0,0,0},
  {0,1,0,0,0,0,0,0},
  {0,0,1,0,0,0,0,0},
  {0,0,0,1,0,0,0,0},
  {0,0,0,0,1,0,0,0},
  {0,0,0,0,0,1,0,0},
  {0,0,0,0,0,0,1,0},
  {0,0,0,0,0,0,0,1}
  };

  updateScreen(display);
}

void updateScreen(int screen[8][8]) {

  Serial.begin(9600);
  int i;
  for (i = row1; i <= row8; i++) {
    Serial.println(i-1);
    digitalWrite(i, LOW);
      digitalWrite(col8, screen[i-2][0]);
      Serial.print(screen[i-2][0]); Serial.print("|");
      digitalWrite(col7, screen[i-2][1]);
      Serial.print(screen[i-2][1]); Serial.print("|");
      digitalWrite(col6, screen[i-2][2]);
      Serial.print(screen[i-2][2]); Serial.print("|");
      digitalWrite(col5, screen[i-2][3]);
      Serial.print(screen[i-2][3]); Serial.print("|");
      digitalWrite(col4, screen[i-2][4]);
      Serial.print(screen[i-2][4]); Serial.print("|");
      digitalWrite(col3, screen[i-2][5]);
      Serial.print(screen[i-2][5]); Serial.print("|");
      digitalWrite(col2, screen[i-2][6]);
      Serial.print(screen[i-2][6]); Serial.print("|");
      digitalWrite(col1, screen[i-2][7]);
      Serial.println(screen[i-2][7]);
      delay(1);
      blank();
    digitalWrite(i, HIGH);
  }

}

void blank() {
  digitalWrite(col1, LOW);
  digitalWrite(col2, LOW);
  digitalWrite(col3, LOW);
  digitalWrite(col4, LOW);
  digitalWrite(col5, LOW);
  digitalWrite(col6, LOW);
  digitalWrite(col7, LOW);
  digitalWrite(col8, LOW);
}

And here is the output:
1
1|0|0|0|0|0|0|0
2
0|1|0|0|0|0|0|0
3
0|0|1|0|0|0|0|0
4
0|0|0|1|0|0|0|0
5
0|0|0|0|1|0|0¾j
6
0|0|0|0|0|1|0|0
7
0|0|0|0|0|0|1|0
8
0|0|0|0|0|0|0|1

So I can tell that the error (probably) isn't my wiring, because those array values are clearly wrong, I have no idea where the ¾j is coming from.

What do they display/what do you expect them to display?

Did you try a longer delay?

With that code as it is works fine, but if I try to display an alternating pattern of LEDs (i.e. like a chess board) like this:
{1,0.1,0,1,0,1,0},
{0.1,0,1,0,1,0,1},
{1,0.1,0,1,0,1,0},
{0.1,0,1,0,1,0,1},
{1,0.1,0,1,0,1,0},
{0.1,0,1,0,1,0,1},
{1,0.1,0,1,0,1,0},
{0.1,0,1,0,1,0,1}

@user168057 If you mean an alternating on-off pattern, then its displaying fine for me. Do you mean on the actual LEDs or only serial monitor?

Nevermind found the issue, I used a few . instead of , in my array declaration. Thanks anyways :)

Because of my psychic abilities*, I am able to tell that you actually used . in a few places you should have used , in your array initialization. Replacing the ,s with .s should fix the problem.
*Okay, okay, not really.

Your initial problem can be sorted out by using a longer delay.
void updateScreen(int screen[8][8]) {

  Serial.begin(9600);
  int i;
  ........
      delay(100); //Longer time duration
      blank();
    digitalWrite(i, HIGH);
  }

}

As far as the chess pattern is concerned, it displays correctly for me.
void loop() {

  int display [8][8] = {
  {1,0,1,0,1,0,1,0},
  {0,1,0,1,0,1,0,1},
  {1,0,1,0,1,0,1,0},
  {0,1,0,1,0,1,0,1},
  {1,0,1,0,1,0,1,0},
  {0,1,0,1,0,1,0,1},
  {1,0,1,0,1,0,1,0},
  {0,1,0,1,0,1,0,1}
  };

  updateScreen(display);
}

Check your array declarations for errors.


text:
How does the Space Replay record audio?

The Space Replay
A hovering object that explores and manipulates transitional public
spaces with particular acoustic properties. By constantly recording
and replaying these ambient sounds, the levitating sphere produces a
delayed echo of human activity.

In this project, the Arduino board and an Adafruit Wave Shield is used. The Wave shield is not designed to record audio however, they say that they use the board for both recording and playback.

The sphere responds sonically to people and its surroundings by means of a battery-powered Arduino, an Adafruit Wave Shield hacked to record and playback audio on-the-fly and a small speaker.

This generates a few questions.

How would this be done?
Where would the microphone be placed inside the balloon for decent audio capture?
Would there be a better way to design this?

No you can't use the board for recording. You can however use the Arduino's own ADC to record sound.

It seems to be possible to record with the shield using a library called waverp

WaveRP is an Arduino library for recording and playing Wave files with
  the Adafruit Wave Shield. It records 8-bit mono files at 4,000 to
  44,100 samples per second.

Some details adafruit blog:

It records 8-bit mono files at 4,000 to 44,100 samples per second.

As jippie said, you can use the Arduinos ADC to record audio, and presumably save that to the Arduino's RAM (the SD card is probably too slow for directly saving it). Then it writes the buffer to the SD card.

But an Arduino has 2K of RAM, so even at low quality sampling rates like 8kHz 8 bit samples, that would cover only about 250ms.

Maybe is kind of late but I found the code for the Space Replay
is in here at the bottom of the last image, specifically in here
/*------------------------------------------------------------------------------

        __-__         Code for Space Replay, a floating black orb that records
      .X+..   Â·       and plays back ambient sounds. This sketch was adapted by
    .Xx+-Â·..    Â·     Francesco Tacchini, with the help of Oliver Smith & Brock
    XXx++-...         Craft, from the original WaveRecordPlay example sketch
    XXxx++--.....Â·    written by Bill Greiman.
    `XXXxxx++---'     This project was created as part of a university project at
      `XXXxxxx'       the Royal College of Art. It's a collaboration with Julinka
         """          Ebhardt and Will Yates-Johnson. See more: vimeo.com/88992597

------------------------------------------------------------------------------*/

#include <SdFat.h>
#include <WaveRP.h>
#include <SdFatUtil.h>
#include <ctype.h>

#define MIC_ANALOG_PIN 0             // Define the analog pin
#define RECORD_RATE 44100            // Define the record rate
#define ADC_REFERENCE ADC_REF_AVCC
#define PRINT_FILE_INFO 0            // Useful for debug: print file info and
#define PRINT_DEBUG_INFO 1           // bad wave file size or SD busy errors
#define MAX_FILE_SIZE 104857600UL    // 100MB or 150 min at 11025 samples/second
#define DISPLAY_RECORD_LEVEL 1

// debugging led
int playLED = 7;
int recLED = 8;
int delStopLED = 9;

// global variables
Sd2Card card;           // SD/SDHC card with support for version 2.00 features
SdVolume vol;           // FAT16 or FAT32 volume
SdFile root;            // volume's root directory
SdFile file;            // current file
WaveRP wave;            // wave file recorder/player
int16_t lastTrack = -1; // Highest track number
uint8_t trackList[32];  // bit list of used tracks

//------------------------------------------------------------------------------
// print error message and halt
void error(char* str) {
  PgmPrint("error: ");
  Serial.println(str);
  if (card.errorCode()) {
    PgmPrint("sdError: ");
    Serial.println(card.errorCode(), HEX);
    PgmPrint("sdData: ");
    Serial.println(card.errorData(), HEX);
  }
  while(1);
}
//------------------------------------------------------------------------------
uint8_t trackName(int16_t number, char* name) {
  if (0 <= number && number <= 255) {
    strcpy_P(name, PSTR("TRACK000.WAV"));
    name[5] = '0' + number/100;
    name[6] = '0' + (number/10)%10;
    name[7] = '0' + number%10;
    return true;
  }
  PgmPrint("Invalid track number: ");
  Serial.println(number);
  return false;
}
//------------------------------------------------------------------------------
// start file playing
uint8_t playBegin(char* name) {
  if (!file.open(&root, name, O_READ)) {
    PgmPrint("Can't open: ");
    Serial.println(name);
    return false;
  }
  if (!wave.play(&file)) {
    PgmPrint("Can't play: ");
    Serial.println(name);
    file.close();
    return false;
  }
#if PRINT_FILE_INFO
  Serial.print(wave.bitsPerSample, DEC);
  PgmPrint("-bit, ");
  Serial.print(wave.sampleRate/1000);
  PgmPrintln(" kps");
#endif // PRINT_FILE_INFO
#if PRINT_DEBUG_INFO
  if (wave.sdEndPosition > file.fileSize()) {
    PgmPrint("play Size mismatch,");
    Serial.print(file.fileSize());
    Serial.print(',');
    Serial.println(wave.sdEndPosition);
  }
#endif // PRINT_DEBUG_INFO
  PgmPrint("Playing: ");
  Serial.print(name);
  return true;
}
//------------------------------------------------------------------------------
// play all files in the root dir
void playTracks(void) {
  dir_t dir;
  char name[13];
  uint8_t np = 0;
  root.rewind();
  while (root.readDir(&dir) == sizeof(dir)) {
    //only play wave files
    if (strncmp_P((char *)&dir.name[8], PSTR("WAV"), 3)) continue;
    // remember current dir position
    uint32_t pos = root.curPosition();
    // format file name
    SdFile::dirName(dir, name);
    if (!playBegin(name)) continue;
    PgmPrintln("");
    while (wave.isPlaying()) {
      uint8_t c = Serial.read();
    }
  }
  file.close();
}
//------------------------------------------------------------------------------
// delete a track
void trackDelete(int16_t track) {
  char name[13];
  if (!trackName(track, name)) return;

  if (SdFile::remove(&root, name)) {
    PgmPrintln("Deleted!");
  }
  else {
    PgmPrintln("Delete failed!");
  }
}
//------------------------------------------------------------------------------
// Finally we have the normal SETUP and LOOP functions -------------------------
//------------------------------------------------------------------------------
void setup(void) {
  Serial.begin(9600);
  if (!card.init()) error("card.init");
  if (!vol.init(&card)) error("vol.init");
  if (!root.openRoot(&vol)) error("openRoot");

  pinMode(playLED, OUTPUT);
  pinMode(recLED, OUTPUT);
  pinMode(delStopLED, OUTPUT);

}
//------------------------------------------------------------------------------
// loop to play and record files.
void loop() {
  digitalWrite(delStopLED, HIGH);
  if (file.isOpen()){
    file.close();      // Make sure file is closed
  }

  int16_t track = -1;
  uint8_t c;
  if (track < 0 ) {
    track = lastTrack;
  }

  char name[13];
  if (track < 0) track = lastTrack + 1;
  if (!trackName(track , name)) return;
  if (file.open(&root, name, O_READ)) {
    file.close();
    trackDelete(track);
    return;
  }
  digitalWrite(delStopLED, LOW);
  digitalWrite(recLED, HIGH);
  Serial.println( "Recording" );    // Record new file...

  if (!file.createContiguous(&root, name, MAX_FILE_SIZE)) {
      PgmPrintln("Create failed");
      return; }
  if(!wave.record(&file, RECORD_RATE, MIC_ANALOG_PIN, ADC_REFERENCE)) {
      PgmPrintln("Record failed");
      file.remove();
      return; }

  delay(5000);     // ...for x milliseconds
  digitalWrite(recLED, LOW);
  wave.stop();
  wave.trim(&file);
  file.close();     // Stop the recording, trim and close the file

  digitalWrite(playLED, HIGH);
  playTracks();     // Play the file
  digitalWrite(playLED, LOW);
}

About your questions,

In the latest WaveRP library there's a folder where it explains how to attach a microphone to the waveshield. And I'm guessing I don't know for sure, but I'd say they're using a contact mic, the contact mics are both, mics and speakers, some cheap walkies also use them. So I'm guessing that they're using only one?
Contact mics pick up vibration so, I think the plastic cone that they're using is for de contact mic to pick up the vibrations of the surroundings sounds.
A better way to design it? I think that it depends. In a way is an art piece. So, a better way to design it... is like to ask VanGogh a better way to paint sunflowers.
But I think always the sound can be better. A way to improve it could be  making a better amplifier circuit for reduce the noise or separating the mic and the speakers... But I don't know, I think it depends what you want to make and what you want to achieve. Because if you want it to float is better to have the leastest components. But if the sound is the priority I think it can be much more better using an amplifier, a better mic, etc.

Also, now in the Adafruit Wave Shield's page there is a project of realtime voice changer, that also record and playbacks. More information here


text:
Connecting Something Big to an Arduino - Sending more than 5V
So I am turning a relay on and off with my Arduino. I am using python to control it but wrote the simple basic code in the Arduino IDE. I am able to send 5 or 0 volts to the pin. My question is, how can I turn something on and off that requires more than 5 volts to turn on?
Thank you.

you need a relay shield

Many relays are controlled w a 5v signal, yet switch 120-240 volts on the second set of contacts.

Use a transistor, amplifier, opto-isolator or if you can drive a 5V relay use that relay to drive a higher voltage/current item item.


text:
Is using malloc() and free() a really bad idea on Arduino?
The use of malloc() and free() seems pretty rare in the Arduino world. It is used in pure AVR C much more often, but still with caution.
Is it a really bad idea to use malloc() and free() with Arduino?

you'll run out of memory really fast otherwise, and if you know how much memory you'll use you might as well statically allocate it anyway

As usual, the right answer is "it depends." You haven't provided enough information to know for sure whether dynamic allocation is right for you.

My general rule for embedded systems is to only malloc() large buffers and only once, at the start of the program, e.g., in setup(). The trouble comes when you allocate and de-allocate memory. Over a long run session, memory becomes fragmented and eventually an allocation fails due to lack of a sufficiently large free area, even though the total free memory is more than adequate for the request.
(Historical perspective, skip if not interested): Depending on the loader implementation, the only advantage of run-time allocation vs. compile-time allocation (intialized globals) is the size of the hex file. When embedded systems were built with off the shelf computers having all volatile memory, the program was often uploaded to the embedded system from a network or an instrumentation computer and the upload time was sometimes an issue. Leaving out buffers full of zeros from the image could shorten the time considerably.)
If I need dynamic memory allocation in an embedded system, I generally malloc(), or preferably, statically allocate, a large pool and divide it into fixed-size buffers (or one pool each of small and large buffers, respectively) and do my own allocation/de-allocation from that pool. Then every request for any amount of memory up to the fixed buffer size is honored with one of those buffers. The calling function doesn't need to know whether it's larger than requested, and by avoiding splitting and re-combining blocks we solve fragmentation. Of course memory leaks can still occur if the program has allocate/de-allocate bugs.
Update 02/16/23:

I'm curious why the Arduino library's malloc implementation doesn't
implement some coalescing of the free blocks like a full OS would.

That's interesting to think about, but let's first be clear that in C/C++, malloc() and free() are implemented as library functions at the application-level not the OS level, even in major OSes. And they are based on fragmenting the heap, just as the Arduinos' malloc() functions do.
Embedded systems, are relatively new, at least as an influential force in the design of OSes, languages, and libraries' design. Which is to say, programs that had to run "forever" without failing (other than, perhaps the OS itself) were outliers. Most programs we run on our desktops or (once upon a time) ran on mainframes, were started, processed a batch of data, and exited.
Secondly, the malloc()/free() method of memory allocation/deallocation was simple, where anticipating the sizes of memory requests was not; and some allocations are used briefly and returned, where some are kept for the duration of the run. How is a library designer to provide for coalescing currently free and newly deallocated memory with no knowledge of the allocate/deallocate usage and sizes of requests? The fixed-size buffer scheme I described above doesn't suffer the fragmentation problem that "malloc() an arbitrary number of bytes" scheme does. It is still limited to available memory but within that limitation, could meet the "run forever" requirement. There may be algorithms that split and coalesce memory, but probably only for a few - statistically predictable - patterns of allocation and deallocation.
Update 02/18/23:
@edgarbonet points out that Arduino's free() does try to coalesce free blocks. This is a good example of an algorithm that works for certain cases: Properly done, it would work for "last-out / first-in" cases, and to some extent for some other cases.
The trouble is, most of our use patterns are not very regular, if they're regular at all. An alogrithm that does no splitting, but merely allocates & deallocates fixed-size pieces (rather like loaning library books) can still run out of memory but won't end up with fragmented resources. In fact, because it doesn't split, it will need more memory to meet the same random-sized requests than even a coalescing malloc() and free().

Another historical note, this quickly led to the BSS segment, which allowed a program to zero its own memory for initialization, without slowly copying the zeros during program load.

I'm curious why the Arduino library's malloc implementation doesn't implement some coalescing of the free blocks like a full OS would.

@vasilescur: The AVR implementation of malloc()/free() does coalesce adjacent free blocks.

I have taken a look at the algorithm used by malloc(), from avr-libc,
and there seems
to be a few usage patterns that are safe from the point of view of heap
fragmentation:
1. Allocate only long-lived buffers
By this I mean: allocate all you need at the beginning of the program,
and never free it. Of course, in this case, you could as well use static
buffers...
2. Allocate only short-lived buffers
Meaning: you free the buffer before allocating anything else. A
reasonable example might look like this:
void foo()
{
    size_t size = figure_out_needs();
    char * buffer = malloc(size);
    if (!buffer) fail();
    do_whatever_with(buffer);
    free(buffer);
}

If there is no malloc inside do_whatever_with(), or if that function
frees whatever it allocates, then you are safe from fragmentation.
3. Always free the last allocated buffer
This is a generalization of the two previous cases. If you use the heap
like a stack (last in is first out), then it will behave like a stack
and not fragment. It should be noted that in this case it is safe to
resize the last allocated buffer with realloc().
4. Always allocate the same size
This will not prevent fragmentation, but it is safe in the sense that
the heap will not grow larger than the maximum used size. If all your
buffers have the same size, you can be sure that, whenever you free one
of them, the slot will be available for subsequent allocations.

Pattern 2 should be avoided as it adds cycles for malloc() and free() when this can be done with "char buffer[size];" (in C++). I would also like to add the anti-pattern "Never from an ISR".

Typically, when writing Arduino sketches, you will avoid dynamic allocation (be it with malloc or new for C++ instances), people rather use global -or static- variables, or local (stack) variables.
Using dynamic allocation can lead to several problems:

memory leaks (if you lose a pointer to a memory you previously allocated, or more likely if you forget to free the allocated memory when you don't need it anymore)
heap fragmentation (after several malloc/free calls) where the heap grows bigger thant the actual amount of memory allocated currently

In most situations I have faced, dynamic allocation was either not necessary, or could be avoided with macros as in the following code sample:
MySketch.ino
#define BUFFER_SIZE 32
#include "Dummy.h"

Dummy.h
class Dummy
{
    byte buffer[BUFFER_SIZE];
    ...
};

Without #define BUFFER_SIZE, if we wanted Dummy class to have a non-fixed buffer size, we would have to use dynamic allocation as follows:
class Dummy
{
    const byte* buffer;

    public:
    Dummy(int size):buffer(new byte[size])
    {
    }

    ~Dummy()
    {
        delete [] bufer;
    }
};

In this case, we have more options than in the first sample (e.g. use different Dummy objects with different buffer size for each), but we may have heap fragmentation issues.
Note the use of a destructor to ensure dynamically allocated memory for buffer will be freed when a Dummy instance is deleted.

I disagree with people who think you shouldn't use it or it is generally unnecessary. I believe it can be dangerous if you don't know the ins and outs of it, but it is useful. I do have cases where I don't know (and shouldn't care to know) the size of a structure or a buffer (at compile time or run time), especially when it comes to libraries I send out into the world. I agree that if you're application is only dealing with a single, known structure, you should just bake in that size at compile time.
Example: I have a serial packet class (a library) that can take arbitrary length data payloads (can be struct, array of uint16_t, etc.). On the sending end of that class you simply tell the Packet.send() method the address of the thing you wish to send and the HardwareSerial port through which you wish to send it. However, on the receiving end I need a dynamically allocated receive buffer to hold that incoming payload, as that payload could be a different structure at any given moment, depending on the application's state, for instance. IF I'm only ever sending a single structure back and forth, I'd just make the buffer the size it needs to be at compile time. But, in the case where packets may be different lengths over time, malloc() and free() are not so bad.
I've run tests with the following code for days, letting it loop continuously, and I've found no evidence of memory fragmentation. After freeing the dynamically allocated memory, the free amount returns to its previous value.
// found at learn.adafruit.com/memories-of-an-arduino/measuring-free-memory
int freeRam () {
    extern int __heap_start, *__brkval;
    int v;
    return (int) &v - (__brkval == 0 ? (int) &__heap_start : (int) __brkval);
}

uint8_t *_tester;

while(1) {
    uint8_t len = random(1, 1000);
    Serial.println("-------------------------------------");
    Serial.println("len is " + String(len, DEC));
    Serial.println("RAM: " + String(freeRam(), DEC));
    Serial.println("_tester = " + String((uint16_t)_tester, DEC));
    Serial.println("alloating _tester memory");
    _tester = (uint8_t *)malloc(len);
    Serial.println("RAM: " + String(freeRam(), DEC));
    Serial.println("_tester = " + String((uint16_t)_tester, DEC));
    Serial.println("Filling _tester");
    for (uint8_t i = 0; i < len; i++) {
        _tester[i] = 255;
    }
    Serial.println("RAM: " + String(freeRam(), DEC));
    Serial.println("freeing _tester memory");
    free(_tester); _tester = NULL;
    Serial.println("RAM: " + String(freeRam(), DEC));
    Serial.println("_tester = " + String((uint16_t)_tester, DEC));
    delay(1000); // quick look
}

I haven't seen any sort of degradation in RAM or in my ability to allocate it dynamically using this method, so I'd say it's a viable tool. FWIW.

Your test code conforms to the usage pattern 2. Allocate only short-lived buffers I described in my previous answer. This is one of those few usage patterns known to be safe.

In other words, the problems will come up when you start sharing the processor with other unknown code - which is precisely the problem you think you are avoiding.  Generally, if you want something that will always work or else fail during linking, you make a fixed allocation of the maximum size and use it over and over again, for example by having your user pass it in to you in initialization.  Remember you are typically running on a chip where everything has to fit in 2048 bytes - maybe more on some boards but also maybe a lot less on others.

@EdgarBonet Yes, exactly. Just wanted to share.

@ChrisStratton It's because of the memory constraints that I believe dynamically allocating the buffer of only the size required in the moment is ideal. If the buffer isn't need during regular operation, the memory is available for something else. But, I see what you're saying: Allocate a reserved space so that something else doesn't take away the space you may need for the buffer. All great information and thoughts.

Dynamically allocating a buffer of only the size needed is risky, as if anything else allocates before you free it you can be left with fragmentation - memory that you can't re-use.  Also, dynamic allocation has tracking overhead.  Fixed allocation doesn't mean you can't multiply use the memory, it just means that you have to work the sharing into the design of your program.  For a buffer with purely local scope, you might also weigh use of the stack.  You haven't checked for the possibility of malloc() failing either.

@ChrisStratton Right, that chunk of code doesn't check if malloc() works or not. Then again, that chunk of code is using notoriously fat Arduino libraries and routines. :) Thanks for your insight. Very reasonable and valuable.

"it can be dangerous if you don't know the ins and outs of it, but it is useful." pretty much sums up all development in C/C++. :-)

Using dynamic allocation (via malloc/free or new/delete) isn't inherently bad as such. In fact, for something like string processing (e.g. via the String object), it's often quite helpful. That's because many sketches use several small fragments of strings, which eventually get combined into a larger one. Using dynamic allocation lets you use only as much memory as you need for each one. In contrast, using a fixed-size static buffer for each one could end up wasting a lot of space (causing it to run out of memory much faster), although it depends entirely on the context.
With all of that being said, it's very important to make sure memory usage is predictable. Allowing the sketch to use arbitrary amounts of memory depending on run-time circumstances (e.g. input) can easily cause a problem sooner or later. In some cases, it might be perfectly safe, e.g. if you know the usage will never add up to much. Sketches can change during the programming process though. An assumption made early-on could be forgotten when something is changed later, resulting in an unforeseen problem.
For robustness, it's usually better to work with fixed-size buffers where possible, and design the sketch to work explicitly with those limits from the outset. That means any future changes to the sketch, or any unexpected run-time circumstances, should hopefully not cause any memory problems.

Is it a really bad idea to use malloc() and free() with Arduino?

The short answer is yes. Below are the reasons why:
It is all about understanding what an MPU is and how to program within the constraints of the available resources. The Arduino Uno uses an ATmega328p MPU with 32KB ISP flash memory, 1024B EEPROM, and 2KB SRAM. That is not a lot of memory resources.
Remember that the 2KB SRAM is used for all global variables, string literals, stack and possible usage of the heap. The stack also needs to have head room for an ISR.
The memory layout is:

Todays PC/laptops have more than 1.000.000 times the amount of memory. A 1 Mbyte default stack space per thread is not uncommon but totally unrealistic on an MPU.
An embedded software project has to do a resource budget. This is estimating  ISR latency, necessary memory space, compute power, instruction cycles, etc. There are unfortunately no free-lunches and hard real-time embedded programming is the most difficult of programming skills to master.

Amen to that: "[H]ard real-time embedded programming is the most difficult of programming skills to master."

Is the execution time of malloc always the same? I can imagine malloc taking more time as it searches further in the available ram for a slot that fits? This would be yet another argument (aside running out of ram) to not allocate memory on the go?

@Paul The heap algorithms (malloc and free) are typically not constant execution time, and not reentrant. The algorithm contains search and data structures that require locks when using threads (concurrency).

In light of the resource limitations of such small devices some would argue that the use of malloc/free as required for the scenarios is a much more efficient use of resources than pre allocating a bunch of fixed size buffers whose size must be large if being conservative (i.e. able to handle ALL scenarios). Lots of wasted allocation most of the time in that usage pattern methinks.

Ok, I know this is an old question but the more I read through the answers the more I keep coming back to an observation that seems salient.
The Halting Problem Is Real
There seems to be a link with Turing's Halting Problem here.  Allowing dynamic allocation increases the odds of said 'halting' so the question becomes one of risk-tolerance. While it is convenient to wave off the possibility of malloc() failing and so forth, it is still a valid outcome.  The question the OP asks only appears to be about technique, and yes the details of the libraries used or the specific MPU do matter; the conversation turns towards reducing the risk of the program halting or any other abnormal end. We need to recognize the existence of environments that tolerate risk vastly differently.  My hobby project to display pretty colors on a LED-strip won't kill someone if something unusual happens but the MCU inside a heart-lung machine likely will.
Hello Mr. Turing My Name is Hubris
For my LED-strip, I don't care if it locks up, I'll just reset it.  If I were on a heart-lung machine controlled by an MCU the consequences of it locking up or failing to operate are literally life and death, so the question about malloc() and free() should be split between how the intended program deals with the possibility of demonstrating Mr. Turing's famous problem. It can be easy to forget that it is a mathematical proof and to convince ourselves that if only we are clever enough we can avoid being a casualty of the limits of computation.
This question should have two accepted answers, one for those who are forced to blink when staring The Halting Problem in the face, and one for all others. While most uses of the arduino are likely not mission critical or life-and-death applications, the distinction is still there regardless of which MPU you may be coding.

I don't think the Halting problem applies in this specific situation considering the fact that heap usage isn't necessarily arbitrary. If used in a well-defined manner then heap usage becomes predictably "safe". The point of the Halting problem was figure out if it can be determined what happens to a necessarily arbitrary and not-so-well-defined algorithm. It really applies much more to programming in a broader sense and as-such I find it to be specifically not very relevant here. I don't even think it's relevant at all to be entirely honest.

I'll admit to some rhetorical exaggeration but the point is really if you want to guarantee behavior, using the heap implys a level of risk that is much higher than sticking to only using the stack.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Is it possible to build a 1080p OSD webcam shield?
What do I need to build a shield capable of receiving 1080p video from USB camera, timestamp each frame and send the frame to memory card?

It would require so much hardware in terms of memory, speed and specialized peripherals, that it renders adding an Arduino to the mix pointless.

You would be better off with a Raspberry Pi connected to the USB port via serial and to send when to start/stop. The YUN might be able to handle it, but you'll have to write your own device drivers. Also, the TRE (not available yet) might work.

None of the ATmega based boards have the hardware (in terms of acting as a USB host) or power (in terms of handling probably compressed 1080p video).
Potentially one of the ARM boards could do this. The Arduino Due can act as a host, but it sounds fairly experimental, and none of them will have drivers to connect to the camera.
If a shield was developed to do this, it would have so much more power than the Arduino that the Arduino would probably better be viewed as a slave to the other device.

The Arduino Due has a port that can act as a USB host. You'd still have driver issues, though.

Raspberry Pi...


text:
attiny85 and arduino - analogRead + pwmOut
I am using the Arduino IDE to program the attiny85. I want to take an incoming analog reading, then based on that reading, output a specific PWM value. Here's my circuit:

and here's my code:
// to run on attiny85

const byte pwmPin = 0;
const byte analogInPin = A2;

void setup() {
  pinMode(pwmPin, OUTPUT);
}

void loop() {
  int analogIn = analogRead(analogInPin);
  analogWrite(pwmPin, analogIn);
}

should be very simple- I have no problems uploading code to the attiny85, and no problems with simple tests like outputting a specific PWM value (not based on the analog read). but when I try to combine the two- read, then write that value, I can't seem to get things to work. In this circuit for example- I get a reading of 1023 (5v) on the arduino micro- instead of a reading of ~ 790 (3.85v) which is what I should expect. I've used a multimeter to verify the voltages in this circuit- so I think I must either be doing something wrong with my expectations of how to wire up or program the attiny85.

A schematic of your circuit would probably be more helpful than an implementation schema with a breadboard. Do you have such a schematic? If yes, add it to your question.

I don't see any clock (quartz or ceramic resonator) for your ATtiny, is this normal?

@jfpoilpret yeah you have a few choices for using the internal clock.

Did you try to add a short delay in your loop? What may happen is that your loop is running too fast, and each new analogWrite() call would "reset" the PWM of the pin, making it always up (no time to use PWM). I would try delay(1000) for a start.

Did your multimeter read 3.85v after the low pass filter? (so right on the Micro's input pin)

i needed to re-range the input to 8 bit. analogIn = map(analogIn, 0, 1023, 0, 255);

it works fine. btw i am using the internal 8mhz - attiny85. also using arduino-tiny core. thanks for all your suggestions.

just to clarify - the adc input on the attiny85 is 10bit. the pwm output is 8bit. thus the map().

@GradeSchool It would be helpful if you could write your solution as an actual answer, so that future visitors can find it more easily. Thanks!

i tried to do that, but this site won't let me for like 8 more hours. i will do so as soon as i can.

Could you throw in some capitals in your text to improve readability? Also a proper circuit diagram, rather than a wiring diagram would help a lot.

Analog read is 10-bits (2^10 = 0-1023 range), analog write is 8-bits (2^8 = 0-255 range). Ditch the lower two bits of the result either by doing:
analogIn = analogIn >> 2;

which can be shortened to:
analogIn >>= 2;

Or you can use the rather complex map function:
analogIn = map( analogIn(0, 1023, 0, 255) )

Sorry, didn't see the answer in the comments.

+1 Not a problem. It's good to have it all spelled out in a proper answer anyway.

i needed to re-range the 10bit input to 8 bit for the pwm out.
// to run on attiny85

const byte pwmPin = 0;
const byte analogInPin = A2;

void setup() {
}

void loop() {
  pinMode(pwmPin, OUTPUT);
  int analogIn = analogRead(analogInPin);
 analogIn = map(analogIn, 0, 1023, 0, 255);
 analogWrite(pwmPin, analogIn);
}

the map function takes a lot of clock cycles compared to the bitshift describes in Cybergibbons' answer.


text:
Arduino time keeping using millis() is not accurate or correct?
I have been using the Arduino to record some data. In my Arduino sketch I also used the millis() function so I can keep track of the time at which each value I am measuring is taken. However, I noticed that the timing isn't correct. For example 30 seconds in real life only comes out as 10 seconds (made up example).
Am I correct in saying that the Arduino delay function affects the time keeping using millis()? In other words suppose I have a delay of 50ms, does that mean the millis() function stops for that duration as well and then continues and so on for the duration of the connection? I noticed this when I tried plotting some data and finding that the frequency of the peaks in my data was too frequent given the time which had passed by. So I want to know if that is the reasoning for this mismatch of timing and if so, how do I fix this so that I can keep the time each sample occurs?
To give some context here is my sketch:

#include <eHealth.h>

unsigned long time;
// The setup routine runs once when you press reset:
void setup() {
  Serial.begin(9600);
}

// The loop routine runs over and over again forever:
void loop() {

  float ECG = eHealth.getECG();
  time = millis();
  Serial.print(time);
  Serial.print(" ");
  Serial.print(ECG, 5);
  Serial.println("");

  delay(50);
}

Are you using one of the official Uno boards?

Actual timing instead of made-up values (a serial monitor that timestamps the lines is ideal) would probably help figure out what's going on.

The computation of millis() is interrupt driven, so delay() should not affect it.

I have the same problem but only when i integrate it (millis()) into complex code. I guess complexity of the code affects its accuracy in way it goes more and more into delay with the complexity of the code. Is there any way to avoid this? maybe using separated RTC module?

millis() is interrupt driven so delay() won't impact it, at least not on an ATmega based board.
That isn't to say that millis() is totally accurate either. Each tick of the timer is not exactly 1ms, but is 1.024ms. This error gradually accumulates until a correction is made. This can be seen in the implementation of the TIMER0_OVF (timer 0 overflow) interrupt handler.
Another source of inaccuracy is the oscillator/crystal itself, which is not exactly 16MHz. It is pretty close though, and as long as temperature doesn't change too much, is relatively stable.
The above means that you might be about 1ms out when using millis(). This doesn't sound like your problem.
Another potential issue would be what getECG() is doing - it might be very slow.
float eHealthClass::getECG(void)
    {
        float analog0;
        // Read from analogic in.
        analog0=analogRead(0);
        // binary to voltage conversion
        return analog0 = (float)analog0 * 5 / 1023.0;
    }

analogRead() is slow, but not so slow as to impact a loop like this.
Another problem I have seen people have is when they change the clock speed but don't correctly change boards.txt. This means that the constants used in the millis() implementation are wrong and the times are wrong.
If you actually want to read values every 50ms, a much better way of implementing this is to do the following
static long lastUpdate;

if (millis() - lastUpdate > 50)
{
    lastUpdate = millis();
    //Do stuff
}

We'd really need to see the timestamps you are getting. If you actually seeing 30s showing as 10s, then there is something else at work.

Please note that, for the Uno, the clock is not crystal-driven, but just uses a ceramic resonator which is less accurate than a crystal.

@jfpoilpret Ah good to know.  Looking at the schematic, this would be the CSTCE16M0V53-R0 Murata CERALOCK device.

Resonators have a poor initial tolerance (often 0.5-2%) and poor temperature stability, but if you calibrate timing loops when using them, they can be fine as long as temperature doesn't move.

Millis() still works on a timer that ticks every 1.024ms, but they added error compensation, in the form of incrementing whenever an error meter variable gets too high. I think it's Roman Black's algorithm actually.  So timing should be much closer to 1ms exactly.https://github.com/arduino/Arduino/blob/master/hardware/arduino/cores/arduino/wiring.c

For those still interested, see the comment I have posted on JRobert's answer, I didn't want to reply with my own answer since I don't have one, I have just rephrased the problem.

A more optimized getECG: float eHealthClass::getECG(void) { return analogRead(0) * 5 / 1023.0; }

If interrupts are turned off for any significant fraction eHealth.getECG() call duration, millis()'s count could fall behind. Otherwise, millis() should return much more accurate time than the 3x errors you described.
You said your sampled signal appears higher in frequency than what you expected, which could happen if your sample rate is lower than you intended. Are you a assuming a 20Hz sample rate? Your loop could be taking a fair bit longer than 50ms, which you would see in the printed times, but those should still track the clock time. If you didn't account for that but assumed 50ms/sample, you'd see an apparent speed up of of the data.
If this is not the issue, then the next step would be to toggle an output while you're in loop(), and measure the frequency of the resulting square wave with a frequency meter (some inexpensive DVMs can do this) or a 'scope. Do the same thing with an empty loop(). The first experiment will your real sampling rate or interval; the second will tell you whether millis() (ie., the timer0 frequency) is what you expected.

I have been playing around with it further and have come to realise that the issue is not with the Arduino, the millis() function for the most part works very well, some of the values are not exactly 8ms (delay) apart but from what you have said that is to be expected. The 3x error I described is true of the Python side of things which I using to receive the data. Any idea what that could be a result of, I am using Python's pyserial and it is slow as hell.

I don't know enough about your implementation to give you more than a 1/2@'d guess, but is the Python side slow enough to be dropping samples?

Same here. I can add that if interruptions are turned off, the time measured is "real time". Anyway, I dont understand why this delay, because if the loop take too long, anyway the millis() should return real time values (only with more distance between each value)

What does "same here" refer to? Answers should stand on their own, as StackExchange can re-order things (unlike a forum). So "same here" could mean anything depending on what reply/question your answer appears underneath.

This post would be more appropriate as a comment, although (admittedly) you lack sufficient reputation .

Sorry, I though when you answer something, its obvious it's refering to the main post, otherwise it would be a comment


text:
How do I know the sampling frequency?
I am beginning to get a bit confused about sampling rates and baudrates etc. I have this Arduino code:
#include <eHealth.h>

extern volatile unsigned long timer0_overflow_count;
float fanalog0;
int analog0;
unsigned long time;

byte serialByte;
void setup() {
  Serial.begin(9600);
}

void loop() {
  while (Serial.available()>0){
    serialByte=Serial.read();
    if (serialByte=='S'){
      while(1){
        fanalog0=eHealth.getECG();
        // Use the timer0 => 1 tick every 4 us
        time=(timer0_overflow_count << 8) + TCNT0;
        // Microseconds conversion.
        time=(time*4);
        //Print in a file for simulation
        //Serial.print(time);
        //Serial.print(" ");
        Serial.print(fanalog0,5);
        Serial.print("\n");

        if (Serial.available()>0){
          serialByte=Serial.read();
          if (serialByte=='F')  break;
        }
      }
    }
  }
}

Since there is no delay interrupt, what is the sampling rate/frequency? Is it based on the Arduino ADC speed? When I increase the baudrate am I increasing the sampling frequency or just the rate at which I send data over the serial port?

The Arduino ADC clock speed is set in ..arduino-1.5.5\hardware\arduino\avr\cores\arduino\wiring.c
Here is the relevant part
#if defined(ADCSRA)
    // Set A/D prescale factor to 128
    // 16 MHz / 128 = 125 KHz, inside the desired 50-200 KHz range.
    // XXX: this will not work properly for other clock speeds, and
    // this code should use F_CPU to determine the prescale factor.
    sbi(ADCSRA, ADPS2);
    sbi(ADCSRA, ADPS1);
    sbi(ADCSRA, ADPS0);

    // Enable A/D conversions
    sbi(ADCSRA, ADEN);
#endif

For a 16 MHz Arduino the ADC clock is set to 16 MHz/128 = 125 KHz. Each conversion in AVR takes 13 ADC clocks so 125 KHz /13 = 9615 Hz.
That is the maximum possible sampling rate, but the actual sampling rate in your application depends on the interval between successive conversions calls.
Since you read the result and send it through the serial port, you are getting a delay that increases as the baud rate decreases. The lower the baud rate the longer it will take to send the same length of data and the longer it will take to call the next ADC conversion.
The actual sampling rate in your application can be determined with the use of a debugger or a simulator, but an easier solution is to toggle a digital pin every time you execute a conversion and measure the frequency that the digital pin toggles at.

Also, the time between my time stamps increases from ~1300 all the way up to ~16400, surely they should stay the same? That is at 9600, at 115200, they increase only to about 1500 after a lot of time.

@user3284376 regarding your time stamp code, I think it cannot work at all times (can be biased by some interrupts at the wrong time). I'd suggest you post a specific question about how to get high precision timing on Arduino and put the relevant part of your code there.

I also wanted to get a high sampling rate, for a project. turns out that the ADPS2,ADPS1,ADPS0 bits of the ADCSRA register can be configured to get a sampling rate of 76923 s/s or 76.8 ks/s. But, beware that I am running my arduino's ADC in free running mode, the following lines worked for me.
#ifndef cbi
#define cbi(sfr, bit) (_SFR_BYTE(sfr) &= ~_BV(bit))
#endif
#ifndef sbi
#define sbi(sfr, bit) (_SFR_BYTE(sfr) |= _BV(bit))
#endif

void setup() {
// Set the Prescaler to 16 (16000KHz/16 = 1MHz)
// WARNING: Above 200KHz 10-bit results are not reliable.
//ADCSRA |= B00000100;
sbi(ADCSRA, ADPS2);
cbi(ADCSRA, ADPS1);
cbi(ADCSRA, ADPS0);

// Set ADIE in ADCSRA (0x7A) to enable the ADC interrupt.
// Without this, the internal interrupt will not trigger.
//ADCSRA |= B00001000;
sbi(ADCSRA,ADIE)
}

At this frequency the usual 10-bit results are not reliable. It means that increasing the sample rate will decrease the precision of the results. So I only use the upper 8 bits because at this prescalar the upper 8-bits are reliable. You can go into more detail on this page, this dude rocks! he made a high sample rate oscilloscpe using UNO http://www.instructables.com/id/Girino-Fast-Arduino-Oscilloscope/

I'm on an ATMEGA 2560, but I figured I'd copypaste in case this worked.  Once I replaced the calls to sbi and cbi with their definitions (my compiler didn't like them), AND commented out the last line, the quality of my stupid audio circuit improved massively.  Thanks!

Sending 11 bits over serial at a baud of 9600, but for the sampling, I store it in an array with as little delay as possible, then once it's done, I send it through the serial port to be read by a python script. I'm doing this for an FFT using matplotlib. I listen to a 0-5V signal, then without using the delay() function, I store the analogRead() values into that array. In a split second the reading is done, then the serial data dump starts. When I calibrated the input frequency using tone() from another connected Arduino, I realized that I had to divide the index by 8915 in order to get accuracy to within .1 Hz. Because one would have to divide by the frequency of the sampling to get the proper index intervals, my guess is the Arduino sampling frequency (at least mine with my code) is 8915Hz.

Each loop you are printing 8 characters over a 9600bps serial link. Each character takes 10 bits (1 start, 8 bits for the character, 1 stop). That means you can only go through this loop ~120 times/sec.
The analogRead() function can sample at about 9600 times/sec in theory, realistically it is about 8600 times/sec.
You are being bounded by the serial communication.

So increasing to 115200, gives 1440 times/sec, is that the sample rate?

Give or take, yes. You need to bear in mind that Serial requires that the other end responds, so you are dependent on the PC responding. This is not deterministic so you will get jitter.

You are right on the Arduino end of things, it all appears fine, but on Python things are much slower, what kind of things would I need to do to increase the performance on the computer end of things?

You need to not look at this as a problem with the serial performance on the PC, but how do you get the sampling to be decoupled from the sending of data.

@Cybergibbons - no, as this is running on an Uno where the USB and serial are decoupled, there is no dependence on the PC beyond issuing the 'S' character and not issuing the 'F' one.  The sketch posted here and platform on which it runs will happily throw serial data at the USB-serial companion micro, blindly oblivious if that or anything on the other end of the USB is keeping up.

Referring to the part about the difference between sample rate and baud rate, they are different measurements.
Sample Rate is the frequency at which the device (arduino) can recreate a digital representation of incoming analog values.
Baud Rate is the rate at which information is transferred in a communication channel. It describes the rate of communication between the microcontroller and the outside world (the computer).
I would recommend this electronics_stack_exchange link.
https://electronics.stackexchange.com/questions/135056/sampling-rate-data-rate-and-bandwidth

8915Hz - it is very close to 125000/14 ~= 8928.6
My initial guess that exactly one gap is required between adjacent conversions
One ADC clock for sampling and 13 ADC clocks for conversion itself.
Small error could be effect of not perfect clock source of Arduino.
I am not sure yet.
This topic is actual for me now as sampled data must feed digital filter.

I'm not sure what you mean when you say "This topic is actual for me now as sampled data must feed digital filter.". Are you having a similar problem?

Each conversion starts on a rising edge of the ADC clock, and at least one ADC clock cycle is lost executing code. So yes, 8928.6 Hz is the fastest you can get by calling analogRead() in a tight loop, v.s. a very consistent 9615.4 Hz in free-running mode.

Have you considered getting A Saleae or similar logic analyzer. The simple inexpensive ones work great and allow you to make these measurements quite easily. This works better then a scope in most of my applications. If there is signal overlap it will show. I simply devote one pin for testing and call it testpin. Just set it high when starting the routine and then low when finished.  It has 8 inputs so you can see the relationship between signals. You could monitor the A/D, starting the trigger and place the serial on another pin. You still have 6 left for other things. Spend a few hours with it and it will become one of your most important tools. I use Linux Mint so I just keep it on another screen so a simple click brings it in view and it will run whether the window is in view or not. Be careful if you have unused inputs they will act as antenna and pick up stray signals especially from switching power supplies. Just select the channels you want or ignore the ones that are not relevant. The built in analyzers make short work of serial signals.


text:
How to use a Pololu 5v AVR ISP Programmer to program an AVR at 3.3v
I have a Pololu 5v AVR ISP which I have successfully used to program several Arduinos (a Mega, an Uno and a Pro Mini 5v). I now have a new Arduino compatible clone which is basically a clone of the pro mini 3.3V.
I am wondering if anyone else has used this programmer to program a 3.3V board without issues. I've seen a forum post on pololu's forum which discusses modifications required - but I don't really understand what is necessary.
I've also found the software tool mentioned in the above forum post here and I have downloaded it and built it.
When I run the aforementioned tool I get these options:
PgmCmd: Configuration and status utility for the Pololu USB AVR Programmer.
Version: 1.0.1.0
Options:
 -l, --list             list available devices
 -d, --device SERIALNUM (optional) select device with given serial number
 -s, --status           display complete device status
     --freq NUM         sets the ISP frequency (in units of kHz)
     --linea ID
       or --lineb ID    set serial control signal associated with line A or B.
                        Valid IDs are: none, cd, dsr, ri, dtr, rts.
                        Warning: dtr and rts are outputs: -f option is required
     --swminor HEXNUM   AVR ISP software version minor (in hex, e.g. A)
     --swmajor HEXNUM   AVR ISP software version major (in hex)
     --hw HEXNUM        AVR ISP software hardware version (in hex)
     --vddmin NUM       set minimum allowed target vdd (units of mV)
     --vddmaxrange NUM  set maximum allowed target vdd range (units of mV)
     --restoredefaults  restore factory settings
     --bootloader       put device in to bootloader (firmware upgrade) mode

I then ran the --status option with the serial number of my programmer and got this:
Serial number:                  00048632
Firmware version:               1.07
Settings:
  ISP Frequency:                200 kHz
  Line A Identity:              None
  Line B Identity:              None
  AVR ISP hardware version:     F
  AVR ISP software version:     2.A
  Target VDD allowed minimum:   4384 mV
  Target VDD allowed max range: 512 mV
Last programming:
  Error: None
  Measured Target VDD Minimum:  N/A
  Measured Target VDD Range:    N/A
SLO-scope:
  State:                        Off
  Line A output:                Off
  Line B output:                Off

From this I deduced that my programmer is currently set at between about 4.38V and 4.89V which I presume corresponds to the 5V approximate voltage of the normal Arduino chips.
Given that the utility program allows me to set both the target minimum Vdd and the target Vdd max range should I therefore set it to something like a minimum of 2704 with a max range of 512 which should put the signal voltages at up to ~3.2V.
Is that all I have to do to get this to work?

Instead of modifying your programmer to wok with 3v you can also consider buying a USBasp programmer that has a jumper to select between 5v/3v. The cost is less than $3 from ebay.

I tried the usbasp method. I bought a cheap one off ebay. When it arrived it didn't work until I gave it a firmware upgrade - fortunately I had the other programmer to do that with. Then having had it's firmware upgrade I tested it out on my arduino with mega2560 and my arduino bt with a 328p. I simply tried the blink sketch - on the bt it worked but not on the 2560. Turns out cheap chinese programmers don't always set the fuses correctly and I had to reload the arduino bootloader using my other programmer. I decided not to risk using the usbasp on my 3.3v things.

That is actually unfortunate to hear. I have a USBasp clone (Baite EvUSBasp specifically) and have had success with it for the most part, but I've not tried it at 3.3V.

fair enough. i guess it's just a pot luck thing on which manufacturers clone you get.

Since AVR ISP is simply a modified form of SPI, the most universal way to handle this is to use a 3/1 level translator such as the MAX3392E. Once you provide it with 5V on VCC and 3V3 on VL it will translate the MOSI, SCK, and nRESET signals to 3V3 and the MISO signal to 5V.

Thx. I now have 2 more questions - why does MISO stay at 5V? and I'm not sure I can easily get hold of the part you mention but I do have several of these: https://www.sparkfun.com/products/retired/8745
Could I use the tx (bi directional despite name) lines of those instead?

I should probably add I come from a Software background rather than hardware and have only really used the serial and I2C interfaces of the Arduinos for communication so haven't come across SPI before.

"MISO" stands for "Master In, Slave Out". It's the signal going from the slave (the device) to the master (the programmer). That translator should work provided that 1) you have 4 bits total, and 2) you fix the 3V3 side as described in the comments.

Thanks for this - eventually though I went with @alexan_e's solution of purchasing a cheap programmer from ebay. I didn't realize they were quite so cheap.

I've come back to this solution because cheap ebay programmers are cheap rubbish.


text:
High precision timing on Arduino for serial communication
I am using an Arduino Uno to send time and voltage information over the serial port to Python to plot. However the interval timings between successive time stamps appears to be increasing over time, affecting my plotting. This is especially true when the baud rate is set to 9600, where my initial time differences maybe 1320 and increases to 16400 after a relatively short period of time. When this rate is put to maximum of 115200 bps the change is slower and less noticeable, from around 1340 to 1500 even after a relatively long run of sending. All times are given in microseconds.
I would like to know if I can reduce or eliminate this effect, and if not understand why it exists. I have read things about interrupts and delays causing this, but I do not fully appreciate the complexity of the electronics at hand and would like to know:

Can I get greater precision in the timing?
What causes this change in timing?

Here is what I currently have:
#include <eHealth.h>

extern volatile unsigned long timer0_overflow_count;
float fanalog0;
int analog0;
unsigned long time;

byte serialByte;
void setup() {
  Serial.begin(9600);
}

void loop() {
  while (Serial.available()>0){
    serialByte=Serial.read();
    if (serialByte=='S'){
      while(1){
        fanalog0=eHealth.getECG();
        // Use the timer0 => 1 tick every 4 us
        time=(timer0_overflow_count << 8) + TCNT0;
        // Microseconds conversion.
        time=(time*4);
        //Print in a file for simulation
        //Serial.print(time);
        //Serial.print(" ");
        Serial.print(fanalog0,5);
        Serial.print("\n");

        if (Serial.available()>0){
          serialByte=Serial.read();
          if (serialByte=='F')  break;
        }
      }
    }
  }
}

What do you mean by "precise"? The times given by the counter are going to be fairly precise, accurate and with a good resolution. Do you want the times to be deterministic (i.e. always the same)?

Apologies, yes I guess that is what I meant, for the difference between them to be consistent and if not, the reason why they are not

Add the timestamp at PC end rather than Arduino end, or use a RTC (real time clock) module. RTC modules are pretty cheap to find on various webstores, just ensure that the store links to the datasheet. Another method is to program a timer and use an interrupt service routine to get reasonably accurate timing.

What does eHealth.getECG() do? Does that call always last the same amount of time?

Can you specify how long a "relatively short period of time" lasts? Is it always the same after restarting the Arduino?

getECG() is calling a function which takes the ECG value from electrodes connected to a subject. It is a voltage value and a short period of time usually encompasses 10 - 15 seconds

Posting the getECG() method could help explain the behavior you observe. Is this a method you wrote yourself or you got it from a library?

Although I don't have the ECG stuff, I have tried to reproduce the behavior you describe (and replaced getECG() with a simple delay(10)). At 9600 bps, I could not observe a shift in the timing as you describe.

Timestamping at the PC end would be good to avoid accumulating errors, but will add some jitter due to the asynchronous packetization over the USB-serial.  So for precise intervals in the short term, it would be better to fix the timestamping at the source.

getECG() just converts the analog value from analog0 to a voltage value from 0 to 5, its a simple calculation.

Use a timer and ISR (interrupt service routine) to make timing more accurate.
Take a look at my 1ms timed interrupt Proof of Concept. The idea is to have a reasonably accurate 1ms 'heartbeat' in the system that can be used to trigger other events. In the PoC it is used to blink an LED at ½Hz, but having access to the new variables millisecondCounter and secondCounter enables you to trigger events in the main loop at arbitrary (but accurately timed) moments.

Your PoC is very interesting but it has a flaw (easy to fix) in the fact that it reads a 2-byte value while interrupts are enabled (in loop()), this value being modified by an ISR. It can happen that loop() reads a bad value (in the middle of a modification by the ISR). I've posted a comment on your blog about it.

@jfpoilpret interesting point you make there, never thought of an interrupt occuring half way retrieving the value from RAM. I'm going to check the disassembly this evening and update the article. Maybe a good reason to write another article too :o)

I created a sample from your PoC and could see the problem occur at least once every 10 seconds on my UNO. But of course in reality it highly depends on what you do in your loop(): my sample just got the milliseconds value, ompare it to the previous read value and if difference > 0 (other than reset counter to 0) , display a message.

@jfpoilpret never really noticed it. I just use it as a heartbeat to monitor the food buckets for my cats and make an LED flash when my cats will be potentially disappointed ... ;o) It definitely will change the way how I use ISR's in future.

@jfpoilpret code is updated. Need to sit down to write a nice blog about this. Thnx

Although an Uno easily achieves a resolution of 1ms its timebase is usually worse than 1000 ppm. With other words it typically drifts more than 1ms per second. It is not uncommon for UNOs to drift more than 1s each 10 minutes.

@UdoKlein For accurate time keeping you should use an RTC anyway, but 1000ppm is a lot for a crystal oscillator. I'd expect order of magnitude 100ppm

The UNO does not have a crystal oscillator for the controller. It only has a crystal oscillator for the serial connection. The main controller is run from a resonator. Thus the typical deviation of a UNO is >> 1000 ppm. Older Arduinos (with crystal) used to be ~30 ppm or better.

@UdoKlein http://arduino.cc/en/uploads/Main/Arduino_Uno_Rev3-schematic.pdf still shows a 16MHz crystal.

It shows a crystal connected to a block ATMEGA16U2 and a resonator connected to ATMEGA328P-PU. The 16U2 is for the serial interface the 328P is "The Arduino". Interestingly enough the 16U2 would be able to push its clock to another chip, e.g. the 328P.

@UdoKlein I searched for the part number and I stand corrected. Awkward design decision.

I can think of a few things that can impact the "consistency" of the serial write timings:

size of the data to be printout

this may be the most obvious thing to think of, but indeed the more you print, the more it'll take to handle it.
Solution: print format the string into a string of known length.

using buffered serial

on unix you can access the serial port using a buffered or an unbuffered way. Using the buffered way for a long time may make it a bit slower as the buffer fills, usually it happens when data is incoming faster than you're reading it…
Solution: use the unbuffered serial line (e.g.: on Darwin/OSX it's /dev/cu.usbmodemXXX instead of /dev/tty.usbmodemXXX)

priority of the timers

it looks like your using a TC interrupt, and AVRs have priorities in the way interrupts are handled, I don't know the order of priority for the Atmega328, and it's not one of the most documented feature around, so I don't know how safe is TC0 versus the UART interrupt.
Solution: look up further in the documentation/datasheet about interrupt priorities and change the timer if needed ; and/or do a test without having the other timer running.

the data you're reading from takes more time to read from over time

some drivers need to average or do some operations over the previous values, so the more values you measure, the longer the buffer is, and the longer it takes to calculate the value, until you've reached the maximum size of the buffer.
Solution: look at the source code of the library you're using, and either optimize it, remove the calcul if there's one or take that increasing processing time into account.

avoiding the arduino framework overhead

but if you really want to optimize serial output from the arduino, you should avoid using the arduino overhead… But it's way less elegant and comfortable to use.
I'm pretty sure there are other points I'm missing, but that's the first things that I'd check before digging further.
HTH

Your code includes the duration of the output in subsequent measurements. Thus depending on the length of the output you will measure different times. This can be fixed by formating to fixed length output.
The next issue is that the UNO has a very poor timebase. Have a look here for a comparison of different Arduino types vs. the DCF77 time reference.
Conclusion: if you need precise timing either get an Arduino with crystal or go for an RTC. I can highly recommend DS3231 / DS3232 RTCs since these usually achieve 2 ppm accuracy out of the box.


text:
XBee shield communication
I need to communicate between two Arduinos using XBee. I am new to XBee and hardware. The two Arduino Unos need to send a message (number) to each other and light an LED or play a sound, when they come close within a 10 meter distance.
So my doubt is which XBee shield and XBee module I need to purchase. I read about XBee series 2, XB24 etc.

Welcome to [arduino.se]! Could you add more details like what is your application? What do you intend to do? What are your requirements? What have you found? Whats confusing you? We can help only if you give us information which can enable us to do so.

Check out the XBee Buying Guide: https://www.sparkfun.com/pages/xbee_guide

Please add more details to your question, there are a lot of XBee modules and each is good a different purpose. Take a look at the link in my previous comment and learn about the different kinds of modules available and then update your question with specific details.

You lay want to give the following details to your question: the environment in which your 2 Arduinos will need to operate, the maximum distance between them, the quantity of information you need to exchange, the exchange frequency, the exchange direction (in case it might be unilateral)...

thanks guys for your reply. My question has been updated with my requirement. Please check

Your edit made your intention with the XBees a bit clearer, yes. But can you easily detect proximity using only the XBees? I don't know, but that's probably the most important part of your question.

the rssi isn't really a great test on the xbees for the purpose you're after. I hve a few around my place, the furthest is 40m away in a metal shed at the back of my yard. Because xbees are mesh, the rssi for that unit is still pretty good. You might want to look more into BLE

The only way I can think of accomplishing this without some sort of proximity sensor is to measure the signal strength of the XBees.
From page 41 of the datasheet:

DB (Received Signal Strength) Command
AT Command: ATDB
Parameter Range [read-only]:  0x17-0x5C (XBee), 0x24-0x64 (XBee-PRO)
 DB parameter is used to read the received signal
  strength (in dBm) of the last RF packet received. Reported values are
  accurate between -40 dBm and the RF module's receiver sensitivity.
  Absolute values are reported. For example: 0x58 = -88 dBm (decimal).
  If no packets have been received (since last reset, power cycle or
  sleep event), “0” will be reported.

You can use this as a rough estimate of the distance between two sensors. You will have to correlate signal strength to actual distance and obstructions and other factors will cause the reported value to be off.
How to measure XBee signal strength?
This page has an example using Processing (on the computer): XBee Signal Strength

Using a single RF signal that may be on a moving object (which has its own RF absorbing and/or amplifying capability) as a proximity measurement sounds dodgy to me. Ten meters with clear line of sight, ten meters with a steel framed wall between the stations, and ten meters with a wood frame wall with electrical wires running through it will all cause unwanted variance in the measurement.

hi thanks for your reply. I am trying to communicate between these. Can you suggest a tutorial or something for this ? Its ruining my time.

The "XBee" brand has two major types. Series 1 are the original ones, and are often NOT referred to by their series number. Series 2 are the newer mesh network brethren.
Most XBee's and their clones are pin compatible (there is even a Bluetooth replacement 'Bee), thus any XBee shield or adapter can be used for either series.
The difference is largely in their communication protocols. The Series 2 allow more robust mesh networking options. But for a beginner - the original (S1) provide simpler out-of-the-box networking.
There are retail packages available that include a pair of S1 XBees, an Arduino compatible breakout board, and a USB adapter to program them. This allows you to get "on the air" and setup the XBees quickly. For communication between two arduinos, a pair of shields will make physical connectivity much easier.

Adding to Ron's answer, the Series 1 is a lot easier to use, because you have to flash firmware onto Series 2s and do all kinds of setup. The S1s work out of the box.
Another thing to keep in mind is that Xbees use 3.3v I/O pins, while the Arduino uses 5v. Most Xbee shields have a Logic Level Converter to handle this, but there are some that don't, and those could damage your Xbee.


text:
Using more than one Ping Ultrasonic Range Finder
I have done various applications with the Ultrasonic example project but it is always using one pinger. I'd like to make a scanner resting on a servo sweep but to reduce sweep time use more than one censor.
the documentation is rather basic ping sonic that states the signal 40khz is used.
Down to the question now. Is there a means to tell one pinger to use 40Khz and another say  45Khz.
This would allow me to scan two areas (different directions) per cycle, reducing the time frame to complete area scan by half.

@NickAlexeev, there actually may not be an answer, the pinger hardware is the key not the Arduino itself - the 40Khz cycle may indeed be fixed by it's hardware.  I will leave it sit a day and then take your suggestion, thank you.

The question might also work on EE.SE, but consider leaving out the Arduino tag if it's not essential.

By the way.  With a single sensor, how long does the scan take?  How many increments?  How much time does the servo adjustment take?  How much time does the actual ping take?

I am sweeping 180 degrees, its about 30 seconds all told, I am guessing from memory I did not time it. But I'd like to reduce the time.

You may want to clarify the question a bit. These ping sensors work on a "pulse driven" method. The Arduino sends the activation pulse, and detects the difference in time from the return (echo). Using two (or more) sensors on separate I/O pins means each sensor is activated (pulsed) separately, their ultrasonic signals do NOT interfere with each other as they will not fire off at the same time.

This question was migrated from Stack Overflow, but it doesn't seem to be about Arduino at all. As per one of the original comments, I'm inclined to migrate it to EE.SE.

There is no way to change the frequency, however there are other ways of doing what you want to accomplish. I think the best thing to do is stagger the timing of the pings so they don't happen at the same time.
Example code, modified by me:
#include <Servo.h>
Servo servo; // create servo object to control a servo

int pingPins[] = {7,8}; //The pins for the two sensors
char *pingString[] = {"Ping1", "Ping2"};

void setup()
{
  servo.attach(10); // attaches a servo to pin 10
  Serial.begin(9600);
}

void loop()
{
  unsigned long ultrasoundValue;
  for(int i = 0; i < ultraSoundSignalPins.length; i++)
  {
    ultrasoundValue = ping(i);
    Serial.print(pingString[i]);
    Serial.print(": ");
    Serial.print(ultrasoundValue);
    Serial.print("in, ");
    delay(50);
  }
  Serial.println();
  delay(50);
 }

//Ping function
unsigned long ping(int i)
{
  unsigned long echo;

  pinMode(pingPins[i], OUTPUT);         // Switch signalpin to output
  digitalWrite(pingPins[i], LOW);       // Send low pulse
  delayMicroseconds(2);                 // Wait for 2 microseconds
  digitalWrite(pingPins[i], HIGH);      // Send high pulse
  delayMicroseconds(5);                 // Wait for 5 microseconds
  digitalWrite(pingPins[i], LOW);       // Holdoff
  pinMode(pingPins[i], INPUT);          // Switch signalpin to input
  digitalWrite(pingPins[i], HIGH);      // Turn on pullup resistor
  echo = pulseIn(pingPins[i], HIGH);    //Listen for echo
  return (echo / 58.138) * .39;         //convert to CM then to inches
}

Sources:

https://stackoverflow.com/questions/20273971/arduino-multiple-ultrasonic-sensors-interference
http://forum.arduino.cc/index.php?topic=17898.0

The Ping))) sensor uses a crystal for timing.
Some of the cheap knock-offs actually use a MAX3232 chip (which is an RS232 serial port/usart) to detect the return pulse, just for the carrier detection. If the parallax is similar, these have a fairly wide range of frequencies they work at, since you never know the baud rate to expect, from 9600 up to 1mhz, although they are deliberately not very fussy about accidentally picking up nearby frequencies.
Since there's probably some other logic on the other chips, and everything is probably run off the one crystal, I'd recommend going to lower frequencies, otherwise you're overclocking.
Also, make sure to send a longer trigger pulse because all the logic will be slower.
From the picture, it looks like a 16MHz crystal, which must be divided down to the 40KHz.
So solder in a 14MHz, and an 11MHz, lengthen the 5 uS trigger pulse accordingly, and you should be good to go.
As an aside, you might also consider scanning further apart. If it's taking you 30 seconds, you're probably checking every degree; maybe try every 15 or 30 degrees (just make sure to give the servo time to move)

I don't think it would be very easy to change the frequency unless Parallax decides to release the firmware for the Ping. It is possible to use two, as long as you use a different data pin for each, but the ultrasonic signals may interfere.


text:
How do I manipulate a large number of moving platforms?
I am trying to build a device that will allow me to control at least a couple hundred raising platforms. The goal of this is to create a real time editor for board games (Dungeons and Dragons).
Ideally:

All the platforms would move simultaneously
The platforms would be transparent to allow us to place LEDs inside them to color the platform so we can make colors like blue for water and brown for dirt.

In our current setup, we use some acrylic prisms that are 1 inch x 1 inch x 6 inch (width, length, height). We wanted to use a CNC motor like system to move something around and move a platform. Obviously, this can only raise/lower one item at a time. We calculated (on paper) quite a long update time (calculated approximately 1 sec per platform) to raise/lower a platform. A CNC linear drive type system is the only solution we found that had a reasonable cost/quality (a CNC motor for every platform also raised the power consumption very high). After a few weeks of looking around, planning and throwing stupid ideas away, we found this.
Clearly, it's way better then our idea but I have no idea how they did this. They have 900 platforms and they all work simultaneously. We also don't need real time interaction. Loading from computer and staying static until we change it is good (though water movement would be really nice).
What would you guys recommend to raise simultaneously a thousand small platforms?

After looking at this document that the devs made, and doing a bit more research I found that the device they used to move the platforms cost approxomently $30; I don't think I need to say this, but I don't have $27,000.
Key difference between their implementation and our goal:

No real-time interaction with the table. Controlled by an Arduino is fine.
Size of the platforms will be closer to 1 inch^2
All sides of the platform should be light up, but I believe this is mostly a question of choosing the right material

What do you plan to use to control the setup?

If possible an arduino as maybe we would want water to be consistently moving. If not a desktop is also a possibility (thx for the edit :)

It would be great if you could find a proper image (not the gif that you have linked) to allow others to visualize what it is you are talking about. It took me a long time to even understand what you were describing :)

I found the video MIT made

Are you sure the pricing is right?

@TheDoctor, more or less, they have 900 platforms and the device they use (revealed in the paper) seems to cost around $30 on various site I found. That's excluding the material, table, lights etc etc
@ annon, thx, i did :)

How many defined states (positions) is each platform supposed to have?

@KlausWarzecha, anythign between max and min

Do you really need 1,000 platforms? That seems like a pretty big game board, but I've never played that game.

@AnnonomusPerson, do you really need 500gb of extra space in your hard drive? :P Not reaaallly but its there in case you do have a situation where the map you wish to describe is actually really big. But I could say that most maps WE play on would fit in 20x20 (400). But the map size depends on a few factor. Maybe my group can play on a 20x20 but another group needs a 50x50 because there are 4 time as many players

In terms of cost, perhaps it would be best to build a 4x4 prototype.
My first thought was to use inexpensive servos with a flexible rod inside a sheath, perhaps a wire, like a bicycle brake/gear.  The amount of movement could then be governed by the size of the armature on an inexpensive servo.
Servos will position themselves after being given the command, so there's no need to wait and do them one-by-one.  Fire off the command, then use some sort of multiplexed output like multiple 74HC595 shift registers to connect to the next one.
If the tiles were spring loaded, such that they returned to the topmost position by themselves, only a simple pull-string is needed to adjust the tile height.  Even a very light filament (like fishing line) would be strong enough for the pull.  This would allow the servos to be positioned even some metres away from the table.