text:
How precise is the timing of pulseIn()?
I've been using the pulseIn() function for processing PWM-based binary data encoding. It works well for distinguishing pulses which are significantly different lengths, e.g. 500us vs. 1500us. That makes it more than sufficient for handling typical IR remotes.
However, I want to make my own IR system which can use more than 2 pulse lengths, so that data transfer can occur faster. Ideally, I'd like to use 8 different pulse lengths for octal encoding (e.g. 200us, 400us, 600us, etc.).
I've noticed quite significant variations in the values returned by pulseIn() though (+/- 10%). I expect at least some of it is introduced by the IR transmitter and receiver modules, but I don't have good enough equipment to verify that.
Assuming I can mitigate that external error, is pulseIn() likely to be precise enough to distinguish such similar pulses?

The pulsein() function is very lossy, in that it is a hard loop and returns a number * the assumed clock cycles it takes for per loop
...
// wait for the pulse to stop
while ((*portInputRegister(port) & bit) == stateMask) {
  if (numloops++ == maxloops)
    return 0;
  width++;
}

// convert the reading to microseconds. The loop has been determined
// to be 20 clock cycles long and have about 16 clocks between the edge
// and the start of the loop. There will be some error introduced by
// the interrupt handlers.
return clockCyclesToMicroseconds(width * 21 + 16);

The most accurate method for capturing the timing of a PIN is use the INPUT CAPTURE FEATURE. Look at the this example . It enables the input capture at 1x of CPU for max resolution and each edge of the input pin captures the timer's clock value for reading from the generated Interrupt service. It also enables the timer overflow interrupt to as to maintain large absolute time, to be capture. Since the 1x will roll rather quickly. The captures store the time into an array for reading by the main loop.

Where for signals over IR the typical library to use is shirriff/Arduino-IRremote library. Where it has several demo's that will read and send the IR from a demodulated signal. To allow one to build a sketch of their own design. This code originally creates a timer interrupt that polls the input pin at a rate determine by
#define USECPERTICK 50  // microseconds per clock interrupt tick

in the IRremote.h file. For my purposes I have change it to 25 us. Where I find this still can be intermittently missing pulse streams.
Note the demodulation is best accomplished within the IR receiver, which in turn outputs this signal of interest. Where to add some background. Using the typical 38KHz modulation, equates to a minimum resolution of 26.3uS per pulse's cycle. sherriff's library shows typically most bauds or bits are in the order of 10+ pulses. Which appear to meet your desired timings.
microtherion/Arduino-IRremote fork of shirriff's work improves the reception by replacing the timer interrupt polling of the pin with the use of PinChangeInterrupts. Which I have merged into my own mpflaga/Arduino-IRremote fork, that adds several other features.
So you could use any of the above libraries. Or create your own app that uses either of the below to capture edges.

polls on a Timer event (e.g. 50uS)
captures the micros() on a PinChangeInterrupt
uses Input Capture Interrupts to grab the exact time

Brilliant answer. The Arduino-IRremote is a very high quality library. Easily readable, and has a lot of well thought out design that means it is very reliable.

Here is some test data of a pulseIn test. One Arduino sent what were supposed to be 14us pulses, and the other spat out this data:

18,18,18,12,18,18,18,18,18,18,18,18,18,18,18,18,18,24,19,18,18,18,18,18,24,18,18,18,19,18,18,12,18,18,19,18,18,18,18,18,18,18,18,18,18,18,19,18,19,24,18,18,18,18,18,18,18,24,18,18,18,18,18,18,18,18,18,18,18,18,18,19,18,18,18,18,18,18,18,18,18,18,19,18,18,18,11,18

As you can see, the pulses are by no means accurate. The time would be more accurate if the sending and recieving ends were written in assembly, or even offloaded to their own processors.


text:
Why is int only 2 bytes?
When using C/C++ on other platforms, the int type is typically 4 bytes (or potentially more). However, on Arduino, it's only 2 bytes.
Why is it different? Does it affect performance if I always use the 4 byte long instead?

note that int is 4 bytes on the Arduino Due. A short will be 2 byte on all existing Ardunios, but I emphasize the others' advice to use int16_t or uint16_t.

The ATmega328 used in a lot of Arduinos is a 8-bit microcontroller. This means that registers are 8-bits, the data bus is 8-bits, the ports are 8-bits. There are some minimal 16-bit aspects to the system (e.g. one of the timers), but nearly everything is 8-bits.
Therefore, most operations handle 8-bits at a time. Working on anything except 8-bits (i.e. 16-bit or 32-bit integers and floating point numbers) requires what could essentially be described as software emulation, where the compiler uses multiple instructions to work on these larger variables.
8-bits is obviously adequate to address a 8-bit port. It's also enough to deal with many loop counters, return values, and ASCII characters. It isn't really enough though when dealing with numbers. A signed 8-bit int (int8_t) can only represent -128 -> +127. Unsigned (uint8_t) can only represent 0 -> 255.
8-bit integers are quite limiting. C/C++ int must represent at least -32,678 -> +32,767 so maps to int16_t - the smallest size that will do so. This gives a good balance of range and efficiency. This is especially important when beginners are learning - overflow is not really something that non-programmers understand.
There is a performance impact of doing this however, because most 16-bit operations take at least twice as long as an 8-bit operation, and use twice as many registers. This may or may not make a difference to you.
Many of us switch to the native types such as int8_t and uint8_t as it gives you far more control.

Just a note: it's not the Arduino Team that mapped int to int16_t, "int" is a C/C++ reserved keyword, and the type mapping is part of the ABI (http://gcc.gnu.org/wiki/avr-gcc) that the avr-gcc compiler's developers decided to follow. Another notably difference is in the "double" type that usually is 64bit wide, while in the avr-gcc is 32bit like "float"

Thanks. Not sure why I wrote that. I know int must represent 32,678 -> +32,767 (though, actually, I think there was a proprietary compiler for one of the NEC processors that didn't follow this). I think it is because I don't like hiding widths on embedded systems - using int16_t is much clearer.

+1 for the use of clear native types! On the Arduino Due, an int is 32-bit! http://arduino.cc/en/Reference/int

One important fact about C and C++ languages is that their respective standards do not define the size (in bytes) of integral and floating point number types.
They just define minimal ranges and relation between these ranges, e.g.
range(short) <= range(int) < range(long)

So the size of e.g. an int will typically depend on:

the target platform (processor)
the compiler itself

are you saying sizeof(short) == sizeof(int) == sizeof(long) is possible?

@ron-e Theoretically, yes, that would be possible. In practice, however, I've never seen that. In most compiler/platforms, one could expect (although it is not imposed) that sizeof(short) < sizeof(long).

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/

sort: votes


text:
Why do I need to include libraries used in other libraries?
I have a simple library which uses another library.
Here is the header:
#ifndef __DERIVEDCLASS_H__
#define __DERIVEDCLASS_H__

#include <HardwareSerial.h>

class DerivedClass
{
private:
    HardwareSerial* serial;

public:
    DerivedClass();
};
#endif

And the CPP:
#include "DerivedClass.h"

DerivedClass::DerivedClass()
{

}

And the sketch:
#include <DerivedClass.h>

void setup()
{
}

void loop()
{
}

This works fine.
However, if I change the library header to this:
#ifndef __DERIVEDCLASS_H__
#define __DERIVEDCLASS_H__

#include <SoftwareSerial.h>

class DerivedClass
{
private:
    SoftwareSerial* serial;

public:
    DerivedClass();
};
#endif

The sketch fails to compile:

/Users/andrew/Documents/Arduino/libraries/DerivedClass/DerivedClass.h:9:
  error: ISO C++ forbids declaration of 'SoftwareSerial' with no type
  /Users/andrew/Documents/Arduino/libraries/DerivedClass/DerivedClass.h:9:
  error: expected ';' before '*' token

If I change the sketch to:
#include <DerivedClass.h>
#include <SoftwareSerial.h>

void setup()
{
}

void loop()
{
}

i.e. including the SoftwareSerial library in the top level sketch, it compiles fine.
If I expand the library out so it actually performs actions, then if it compiles, it works.
Why is this? HardwareSerial is a built-in part of the Arduino, located in hardware/arduino/cores/arduino and SoftwareSerial is in libraries, but why would the build process differentiate between the two.

If I remember correctly this is in fact a limitation of the Arduino IDE.
If you use a better IDE (personally I use Eclipse with the Arduino plugin) it works as it should, i.e. if you include a library A that includes another library B, then in the end you won't get compile-time errors and both B and A will be included in the final binary.

Absolutely right. Just tried Stino and it works fine. I might retreat back to using command line. I can't understand why they would do this.

Indeed, they are trying to fix this in 1.5 https://github.com/arduino/Arduino/pull/1726

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
How can I adapt the DallasTemperature library to accept another 1-Wire class?
A commonly used library is the DallasTemperature library, used to access DS18B20 sensors on a 1-Wire bus. When creating a DallasTemperature object you pass a reference to a OneWire object. OneWire can create a 1-Wire bus on pretty much any port of an Arduino.
An example of this is like so:
// Setup a oneWire instance to communicate with any OneWire devices (not just Maxim/Dallas temperature ICs)
OneWire oneWire(ONE_WIRE_BUS);

// Pass our oneWire reference to Dallas Temperature.
DallasTemperature sensors(&oneWire);

However, there are some issues with using OneWire to drive a 1-Wire bus, mainly around driving longer bus lengths, dealing with shorts, and other errors.
There is a commonly available chip called the DS2482-100 that can drive longer busses (hundreds of feet), deals with shorts and other errors. It is also less CPU heavy and it deals with a number of the slower aspects of 1-Wire.
For this reason, I would like to be able to pass a reference to a library that I have created instead. Currently, I am doing this by:

Calling the library OneWire
Replicating all the methods called by the DallasTemperature library
Swapping out the OneWire library for mine

This is relatively messy. I would ideally like to be able to pass either a reference to my library or a 1-Wire library, so that people can chose which interface to use.
How would I go about doing this?

This might be quite C++ and more appropriate for StackOverflow. But, in defence of this question, this is a very specific Arduino/hardware library, and in trying to solve it, I have come up against some other issues (http://arduino.stackexchange.com/questions/322/why-do-i-need-to-include-libraries-used-in-other-libraries) that seem specific to the Arduino environment.

What do you mean by "there are some issues with using OneWire like this"? How should it then be used actually? I find it hard to understand how the way to instantiate an object as in your code cn have a bad impact as you describe.

Clarified in original question.

Can't you just use the DallasTemperature library exclusively?

The typical way to achieve what you want would normally be to create your own subclass of OneWire class and override the methods you need to change, then pass a pointer to an instance of your class to the DallasTemperature constructor.
However, for this to work, the OneWire class must have been developed with extensibility (subclassing) in mind, i.e. use virtual methods that you can override.
Unfortunately, a quick look at OneWire.h shows that no method is virtual in there.
So that means in the end you will have to perform some hacks, just as you do now.
What you could potentially do (that's still a hack, but maybe a bit less dirty) is:

replace OneWire.h to make all methods virtual (at least those you need to override)
keep the original OneWire.cpp that should still compile with the newly adapted header file
create a new MyOneWire.h / MyOneWire.cpp to define your own OneWire subclass that overrides all necessary methods as you want

Then you can pass either a OneWire or a MyOneWire instance to DallasTemperature constructor.
NB: when I see my answer, this makes me think your question is more a general C++ question than an Arduino-related one.

Thanks. I think your hack is cleaner - and doesn't involve moving files in and out of the library folder. I'm coming to the realisation that the C++ aspects of Arduino aren't really used to great effect. Thankyou for a great answer - I will leave open for a while anyway.

I should add one point to the defense of OneWire library authors: there can be a reason to not use virtual methods in a class: performance of method calls and code size. I haven't tried yet to measure such impact on Arduino (on a PC this is of course negligible and unnoticeable).

The DS2482-100 is an I2C-to-1-Wire®bridge device that interfaces directly to standard (100kHz max) or fast (400kHz max) I2C masters to perform bidirectional protocol conversion between the I2C master and any
  downstream 1-Wire slave devices.

It uses a higher level protocol over I2C which does not directly match the bit-banged OneWire interface above.
A possible solution is to "only" reuse the OneWire interface and reimplement it. Sub-classing will not work as the member functions are not virtual. Unfortunately this will not work directly due to error handing, timing, etc. The DallasTemperature class will also need modifying.
Please see the DS2482-100 product description and protocol.
To achieve your goal of allowing either bit-banged or DS2482-100 based 1-Wire master would require 1) an abstract interface (class) used by 1-Wire master device drivers, 2) sub-classes for each implementation master variant.
As the 1-Wire protocol has a number of common commands a delegation pattern is more useful. Please see the Cosa OWI::Driver and DS18B20 device driver for an example of the delegation pattern.


text:
Why can't I use Jeelib Sleepy on the Teensy 2 along with Serial
I have been using the Jeelib Sleepy functionality for several years now to achieve low power states on the Arduino ATmega328/1280/2560 boards, specifically the LoseSomeTime functionality.
I want to get my Teensy 2 to go into a similar low power state, so I tried a very simple sketch that simply put the device to sleep, flashed an LED, and posted some Serial messages. I did need to comment out a few lines in Jeelib:
// adjust the milli ticks, since we will have missed several
#if defined(__AVR_ATtiny84__) || defined(__AVR_ATtiny85__) || defined (__AVR_ATtiny44__) || defined (__AVR_ATtiny45__)
    extern volatile unsigned long millis_timer_millis;
    millis_timer_millis += msecs - msleft;
#else
    extern volatile unsigned long timer0_millis;
    timer0_millis += msecs - msleft;
#endif

This had to be done as the delay() and millis() functions on the Teensy use different variable names for the adjustment, and it was the quickest way of solving the problem. LoseSomeTime works on the ATmega328 hardware with this change.
I found that although the LoseSomeTime function did sleep for a period, the Serial functionality stopped working entirely - no messages were being printed at all. I have encountered this before on ATmega328 hardware because the microcontroller has gone to sleep before the serial buffer was flushed. I added a number of delay()s after the Serial.writes and also tried using Serial.flush() to make sure the buffer was empty.
LoseSomeTime requires setting an interrupt on the watchdog timer, but I can't see how this would impact Serial operation.

You explicitly mention Teensy 2 for this issue, does it mean you have tried on other boards and did not encounter that problem? It looks so in your question, but it's not 100% clear on that point. It could be useful to mention a list of boards on which the same sketch works or doesn't work, in case you could test on different plqtforms.

Not a problem on any ATmega328, ATmega1280, ATmega2560 boards. I haven't tried on a Teensy 2++. I know it won't work with any of the ARM boards.

Does it have anything to do that serial is based on interrupts now? Could that affect it? I've never used the library though or the Teensy 2, so I don't know much about this situation, but I might look into it if I have time.

I think both an Uno and Teensy use interrupts for serial, don't they? At least since v1 of the IDE.

This is a problem with the Teensy boards - or more correctly that the Jeelib Sleepy library does not support the Teensy boards.
You will need to modify the library to make it work for Teensy boards. Or you could use another low power/sleep library.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Can I link another device to the Uno's clock?
I'm trying to use an unusual 16x2 LCD display with my Uno. As far as I know, it's an EA-X16027AR. It doesn't seem to be compatible with the HD44780 driver, which is typically used for Arduino LCD's.
According to the datasheet, the display's ENB (Enable) pin requires a clock input, described as follows:

Basical Clock to perform Instructions in Controller LSI. The Clock at the same timing as MPU are usually input.

The timing graphs elsewhere in the datasheet show it wants a speed of around 2 MHz.
Is it possible to get the internal clock signal from the Uno, scale it down to 2 MHz, and output it to a GPIO pin or similar? Alternatively, is it possible to emulate a clock signal at that speed on-board, without adding additional timing components?

I don't have time to write an answer, but you can use one of the timers and the output compare functionality to do anything from either full clock speed down or half clock speed down (can't recall which one). This requires bare metal usage of the timers though.

Rather than using microprocessor-specific low-level programming, I was able to get a suitable signal using the core tone() function:
tone(11, 2000000);

It can be used to generate a 50% duty cycle square wave of a custom frequency on any individual GPIO pin (I'm using 11 in this case). Given the name, it's obviously designed for producing simple audio output, and 2 MHz is way above the range of human hearing.
The documentation doesn't list any upper limit on the frequency though, and the code it's based on doesn't appear to introduce any limitations. With that said, it clearly can't run faster than the internal timer it's based on (which I think will typically be 8 or 16 MHz).
It seems reasonable to assume that it will be in-sync with the microprocessor's internal clock, although I haven't been able to confirm that empirically as I don't have suitable equipment at home. Similarly, I haven't been able to confirm that it's actually producing a 2 MHz signal, but the LCD display is working properly so it's likely to be in the right vicinity. I had previously tried running it from regular PWM, but that didn't work, so frequency is obviously important here.
The signal can be stopped by calling:
noTone(11);

There are a couple of caveats to be aware of when using tone(). First of all, it can only work on a single pin at a time, so you can't generate multiple signals with it. And secondly, it will prevent some PWM output from working properly (on pins 3 and 11, according to the documentation). This is because it has to reconfigure one of the internal timers which is normally used for PWM.

The tone() command is using " microprocessor-specific low-level programming". It's just a wrapper that prevents you from learning how the MCU actually works.

@FakeName I'd argue that the core Arduino functions (like tone()) are an abstraction layer. In theory, they can be re-implemented under the surface to behave the same way on any microprocessor which supports the relevant features. By "low-level programming", I was referring more to things like directly manipulating special registers. That's definitely not portable, and is probably too advanced for a lot of the visitors we're likely to get here at Arduino.SE.

A wrapper is an abstraction layer. However, the Arduino abstraction layer tends to be dramatically slower, and is not much easier then dealing with the hardware directly. Personally (and this is IMHO), I think that while it's fine to use the arduino functions if you just want to throw something together, you should still read tha datasheet anyways. The hardware can do much more then is exposed by the abstraction layer, and you cannot really fully utilize it without dropping the intermediary entirely.

Basically, I look at the arduino wrappers like using GOTO, or any of a bunch of other programming structures that are easy to use, but lead to complex, hard-to-troubleshoot interactions as soon as you do anything complex. They're nice for hacking together small stuff, but poison for large or complex tasks, unless you understand in full what they're actually doing.


text:
Can a stepper motor be controlled directly from an Arduino?
I'm planning to build a simple XY plotter, and have been looking at the possibility of using a pair of small 5V 4-phase steppers to provide the movement. In my research, I've often seen stepper driver boards advertised, ranging from very big and expensive, through to quite small and cheap. Some of them actually look like little more than a darlington array on a PCB.
Do I really need one of those external driver boards, or can I control the steppers directly from my Arduino?

Do you have the datasheet for the steppers? What would be the DC resistance of the coils? You probably need a driver anyway, but sizing has to do with the power the stepper draws. I estimate you can only drive max 125mW from an Arduino directly, probably even less.

No part number, unfortunately, and minimal info from the supplier. I figured the current draw was likely to be too much, but thought it was worth asking anyway.

Generally, stepper motors cannot be controlled by an Arduino pin. They have a current draw of over the 20mA any single pin can supply.
Kickback is also a problem. Because the motors have moving magnetic parts inside, they will continue to generate electricity after power has been cut. this will almost certainly cause enough negative voltage that is enough to fry your Arduino.
A solution is to use a motor driver chip or shield. This one is a good example. These have transistor based chips that can drive enough current to power a motor, and diodes to mitigate the kickback from the motors.

You do not need moving components to produce inductive kickback. All you need is inductance, which requires no moving parts at all.


text:
Controlling water temperature accurately
I have following setup:

Arduino Duemilanove
Small water heater connected with optical relay (PWM is ok)
Waterproof DS18B20 temperature sensor
Unknown amount of water (but temperature sensor and heater are always underwater).

How can I accurately control water temperature, based on temperature sensor readings? By accurately, I mean up to accuracy of temperature sensor (in this case, ±0.5°C). Everything should run on Arduino, so no additional data storage or processing power is available. For debugging, I can obviously use computer.
I have tried running PID, but it seems really hard to

Avoid overshooting - going over target temperature either initially, or when (cold) water is added
Stabilize properly after adding cold water.

Did you tune your PID, or were you just using standard values?

I tried to tune PID parameters with various 'how to tune PID' guides, with no luck. I also tried PIDAutotune, but couldn't get anything better out from it. I'm not saying PID could not fit to this.

With water heating, you are going to have a lot of inconsistencies. Where is the heater in your tank relative to the temperature probe?

You'd probably have to add something that mixes the water to homogenize the temperature throughout.  Then you'd have to take into account the thermal properties of your heating element, how long it takes to cool off after disabled, etc... It is dependent on how much water is in the system as well.  If you can control the temperature of the heating element, that would go a long way.

Not part of your issue ATM, but worth noting: Don't confuse accuracy, precision, and repeatability. Accuracy means how close the reported temperature is to the actual temperature. Precision is the smallest difference the device can distinguish (.0625 degC for DS18b20). Repeatability is how much reported measurements of the same temperature vary from one another. The last two are of more concern than the first. Even with +/-0.5degC accurate sensor, you could still hold temperature to within (or at least distinguish differences, which is a necessary condition) of a few 1/16ths degC....

... as long as you control or filter electrical noise, the sensor is repeatable to a 16th, and your system otherwise has the ability to maintain the temperature. Good averaging may well bring the repeatability well below 0.5 degC.

You need to know the response time of your system and the repeatability of your measurements to characterize your data (establish confidence). I suggest taking samples and average them before turning the heater back on.
Imho, water will not happily stay within 0.5°C. Can you allow for coarse and fine modes? Use a higher PWM for great differences and then lower PWM for when close to target.
Once homed in on your target, there are a few means you can try to improve temperature stability. While pre-built things like PID work for many, I normally implement functionality like this manually.


text:
Would an infinite loop inside loop() perform faster?
When you're writing a typical sketch, you usually rely on loop() being called repeatedly for as long as the Arduino is running. Moving in and out of the loop() function must introduce a small overhead though.
To avoid that, you could presumably create your own infinite loop, like this:
void loop()
{
    while (true)
    {
        // do stuff...
    }
}

Is that a viable way to improve performance? Will it cause other problems if loop() never returns?

The part of the code on an ATmega core that does setup() and loop() is at follows:
#include <Arduino.h>

int main(void)
{
        init();

#if defined(USBCON)
        USBDevice.attach();
#endif

        setup();

        for (;;) {
                loop();
                if (serialEventRun) serialEventRun();
        }

        return 0;
}

Pretty simple, but there is the overhead of the serialEventRun(); in there.
Let's compare two simple sketches:
void setup()
{

}

volatile uint8_t x;

void loop()
{

    x = 1;

}

and
void setup()
{

}

volatile uint8_t x;

void loop()
{
    while(true)
    {
        x = 1;
    }
}

The x and volatile is just to ensure it isn't optimised out.
In the ASM produced, you get different results:

You can see the while(true) just performs a rjmp (relative jump) back a few instructions, whereas loop() performs a subtraction, comparison and call. This is 4 instructions vs 1 instruction.
To generate ASM as above, you need to use a tool called avr-objdump. This is included with avr-gcc. Location varies depending on OS so it is easiest to search for it by name.
avr-objdump can operate on .hex files, but these are missing the original source and comments. If you have just built code, you will have a .elf file that does contain this data. Again, the location of these files varies by OS - the easiest way to locate them is to turn on verbose compilation in preferences and see where the output files are being stored.
Run the command as follows:

avr-objdump -S output.elf > asm.txt

And examine the output in a text editor.

OK, but isn't there a reason for calling serialEventRun() function? What is it for?

It's part of the functionality used by HardwareSerial, not sure why it isn't taken out when Serial isn't needed.

Would be helpful to briefly explain how you generated the ASM output so people can check themselves.

@Cybergibbons it is never taken out because it is part of standard main.c used by Arduino IDE. However it does not mean that HardwareSerial library is included to your sketch; actually it is not included if you don't use it (that's why there is if (serialEventRun) in main() function. If you don't use HardwareSerial library then serialEventRun will be null, hence no call.

However HardwareSerial library gets probably included (can someone confirm that?) if you use some arduino functions like Serial.print(). In this situation, what can happen if your loop() loops forever and thus never calls serialEventRun()? I'm afraid Serial may not work in this case.

Yeah, it is part of the main.c as quoted, but I would expect it to be optimised out if not required hence I think aspects of Serial are always included. I frequently write code that will never return from loop() and don't notice issues with Serial.

@Cybergibbons, serialEventRun() calls serialEvent() for the various serial ports. By default, all of these are empty procedures, but they can be overridden by the user to do something useful if desired. If you're not overriding it, there's no need to call it.

Cybergibbons's answer describes quite nicely the assembly code generation and the differences amongst the two techniques. This is intended to be a complementary answer looking at the issue in terms of practical differences, i.e. how much of a difference either approach will make in terms of execution time.

Code Variations
I did an analysis involving the following variations:

Basic void loop() (which gets inlined on compilation)
Un-inlined void loop() (using __attribute__ ((noinline)))
Loop with while(1) (which gets optimized)
Loop with un-optimized while(1) (by adding __asm__ __volatile__("");. This is a nop instruction that prevents optimization of the loop without resulting in additional overheads of a volatile variable)
An un-inlined void loop() with optimized while(1)
An un-inlined void loop() with un-optimized while(1)

The sketches can be found here.
Experiment
I ran each of these sketches for 30 seconds, thereby accumulating 300 data points each. There was a 100 millisecond delay call in each loop (without which bad things happen).
Results
I then calculated the mean execution times of each loop, subtracted 100 milliseconds from each and then plotted the results.
http://raw2.github.com/AsheeshR/Arduino-Loop-Analysis/master/Figures/timeplot.png
Conclusion

An un-optimised while(1) loop within void loop is faster than a compiler optimised  void loop.
The time difference between the un-optimized code and default Arduino  optimized code is insignificant practically. You will be better off compiling manually using avr-gcc and using your own optimisation flags rather than depending on the Arduino IDE to help you with it (if you need microsecond optimisations).

NOTE: The actual time values are not of significance here, the difference between them is. The ~90 microseconds of execution time includes a call to Serial.println, micros and delay.

NOTE2: This was done using the Arduino IDE and the default compiler flags that it supplies.

NOTE3: Analysis (plot and calculations) was done using R.

Good work. Graph has milliseconds not microseconds but not a huge issue.

@Cybergibbons Thats quite unlikely since all measurements are in microseconds and I havent changed scales' anywhere :)


text:
SMPTE Timecode Reader with LCD display coding question
I've managed to adapt some code from the Arduino forums to display timecode on an LCD display.
What I would like some extra help with is first is there a way of defining the one_time_max etc differently depending on the high low state of a pin? This way I could use a switch to change the times so it could switch between NTSC and PAL.
Secondly would anyone be kind enough to explain what is happening in the middle part of this code? I've made some notes but any help would be really appreciated.
Here is a picture of how it's going so far. I'll keep you guys up to date as the project continues :)
[![enter image description here][1]][1]
    // Code from forum post Dec 12, 2007

    // include the library code:
    #include <LiquidCrystal.h>

    // initialize the library with the numbers of the interface pins
    LiquidCrystal lcd(12, 11, 5, 4, 3, 2);
    #define one_time_max 600 // these values are setup for PA video
    #define one_time_min 400
    // It's the durstion of a one and zero with a little bit of room for error.
    #define zero_time_max 1050 //
    #define zero_time_min 950 //

    #define icpPin 8 // ICP input pin on arduino
    //#define one_time_max 475 // these values are setup for NTSC video
    //#define one_time_min 300 // PAL would be around 1000 for 0 and 500 for 1
    //#define zero_time_max 875 // 80bits times 29.97 frames per sec
    //#define zero_time_min 700 // equals 833 (divide by 8 clock pulses)

    #define end_data_position 63
    #define end_sync_position 77
    #define end_smpte_position 80

    volatile unsigned int pin = 13;
    volatile unsigned int bit_time;
    // volatile instructs the variable to be stored in RAM
    volatile boolean valid_tc_word;
    // boolean can be either of two values true or false but default to false
    volatile boolean ones_bit_count;
    // boolean can be either of two values true or false but default to false
    volatile boolean tc_sync;
    // boolean can be either of two values true or false but default to false
    volatile boolean write_tc_out;
    // boolean can be either of two values true or false but default to false
    volatile boolean drop_frame_flag;
    // boolean can be either of two values true or false but default to false
    volatile byte total_bits; //this stores a an 8-bit unsigned number
    volatile byte current_bit; //this stores a an 8-bit unsigned number
    volatile byte sync_count; //this stores a an 8-bit unsigned number

    volatile byte tc[8]; //this stores a an 8-bit unsigned number
    volatile char timeCode[11]; //this stores a an 8-bit unsigned number

    /* ICR interrupt vector */
    ISR(TIMER1_CAPT_vect) {
      //ISR=Interrupt Service Routine, and timer1 capture event
      //toggleCaptureEdge
      TCCR1B ^= _BV(ICES1);
      //toggles the edge that triggers the handler so that the duration of both high and low pulses is measured.
      bit_time = ICR1; //this is the value the timer generates
      //resetTimer1
      TCNT1 = 0;
      // Ignore phase changes < time for 1 bit or > time for zero bit (zero bit phase change time is double that of 1 bit)
      if ((bit_time < one_time_min) || (bit_time > zero_time_max)) {
        // this gets rid of anything that's not what we're looking for
        total_bits = 0;
      } else {
        // If the bit we are reading is a 1 then ignore the first phase change
        if (ones_bit_count == true)
          // only count the second ones pulse
          ones_bit_count = false;
        else {
          // We have already checked the outer times for 1 and zero bits so no see if the inner time is > zero min
          if (bit_time > zero_time_min) {
            // We have a zero bit
            current_bit = 0;
            sync_count = 0; // Not a 1 bit so cannot be part of the 12 bit sync
          } else {
            // It must be a 1 bit then
            ones_bit_count = true; // Flag so we don't read the next edge of a 1 bit
            current_bit = 1;
            sync_count++; // Increment sync bit count
            if (sync_count == 12) {
              // part of the last two bytes of a timecode word
              // We have 12 1's in a row that can only be part of the sync
              sync_count = 0;
              tc_sync = true;
              total_bits = end_sync_position;
            }
          }

          if (total_bits <= end_data_position) {
            // timecode runs least to most so we need
            // to shift things around
            tc[0] = tc[0] >> 1;
            for(int n=1;n<8;n++) {
              //creates tc[1-8]
              if(tc[n] & 1) tc[n-1] |= 0x80;
              tc[n] = tc[n] >> 1;
            }
            if(current_bit == 1) tc[7] |= 0x80;
          }
          total_bits++;
        }
        if (total_bits == end_smpte_position) {
          // we have the 80th bit
          total_bits = 0;
          if (tc_sync) {
            tc_sync = false;
            valid_tc_word = true;
          }
        }
        if (total_bits <= end_data_position) {
          // timecode runs least to most so we need
          // to shift things around
          tc[0] = tc[0] >> 1;
          for(int n=1;n<8;n++) {
            //creates tc[1-8]
            if(tc[n] & 1) tc[n-1] |= 0x80;
            tc[n] = tc[n] >> 1;
          }
          if(current_bit == 1) tc[7] |= 0x80;
        }
        total_bits++;
      }

      if (total_bits == end_smpte_position) {
        // we have the 80th bit
        total_bits = 0;
        if (tc_sync) {
          tc_sync = false;
          valid_tc_word = true;
        }
      }

      if (valid_tc_word) {
        valid_tc_word = false;
        timeCode[10] = (tc[0]&0x0F)+0x30;
        // frames  this converst from binary to decimal giving us the last digit
        timeCode[9] = (tc[1]&0x03)+0x30;
        // 10's of frames this converst from binary to decimal giving us the first digit
        timeCode[8] =  ':';
        timeCode[7] = (tc[2]&0x0F)+0x30; // seconds
        timeCode[6] = (tc[3]&0x07)+0x30; // 10's of seconds
        timeCode[5] =  ':';
        timeCode[4] = (tc[4]&0x0F)+0x30; // minutes
        timeCode[3] = (tc[5]&0x07)+0x30; // 10's of minutes
        timeCode[2] = ':';
        timeCode[1] = (tc[6]&0x0F)+0x30; // hours
        timeCode[0] = (tc[7]&0x03)+0x30; // 10's of hours
        drop_frame_flag = bit_is_set(tc[1], 2);
        //detects whether theree is the drop frame bit.
        write_tc_out = true;
      }
    }

    void setup() {
      lcd.begin (16, 2);
      pinMode(icpPin, INPUT); // ICP pin (digital pin 8 on arduino) as input
      bit_time = 0;
      valid_tc_word = false;
      ones_bit_count = false;
      tc_sync = false;
      write_tc_out = false;
      drop_frame_flag = false;
      total_bits =  0;
      current_bit =  0;
      sync_count =  0;
      lcd.print("FINISHED SETUP");
      delay (1000);
      TCCR1A = B00000000; // clear all
      TCCR1B = B11000010; // ICNC1 noise reduction + ICES1 start on rising edge + CS11 divide by 8
      TCCR1C = B00000000; // clear all
      TIMSK1 = B00100000; // ICIE1 enable the icp
      TCNT1 = 0; // clear timer1
    }

    void loop() {
      if (write_tc_out) {
        write_tc_out = false;
        if (drop_frame_flag)
          lcd.print("TC-[df] ");
        else
          lcd.print("TC-NO DROP FRAME");
        lcd.setCursor(0, 1);
        lcd.print((char*)timeCode);
        lcd.print("\r");
        lcd.setCursor(11, 1);
        lcd.print("......");
        delay (30);
        lcd.clear();
      }
    }
111

  [1]: https://i.stack.imgur.com/V6CxE.jpg

So if a pin were to be high or low, what variables would you change? And what would you change them to?

Hiya, the values i'd want to change are the one_time_min etc. And I want to change them to the values required to decode NTSC rather than PAL

I wonder how the circuit configuration.
I would appreciate a more informed picture or drawing. {email address removed by moderator}

The one_time_max, one_time_min, zero_time_max and zero_time_min are all defines at the top of the file. This will insert them as constant values at compile time. This makes changing the code to deal with them slightly awkward, but not impossible.
Oddly, the one_time_max value isn't used, but the others are. To allow you to switch between PAL and NTSC, I would do the following (this is marginally lazy, but quick):

change one_time_max to ONE_TIME_MAX_NTSC and ONE_TIME_MAX_PAL so that you have two defined constants for each part. Caps is convention for defines.
create a new global variable, a boolean, that is a flag to decide if PAL or NTSC is to be decoded.
every time that one of the constant is used, use a conditional statement based on the global variable to decided which constants are used.
use the input from a button to decided the value of the global variable.

Thank you! i'll give this a go and see what happens. I don't suppose you could explain how adding the 0x0F+0x30 converts it from presumably a hex value to a decimal?

The &ing seems superfluous in this situation. 0x30 is ASCII for 0, so if you represent 0 as 0x00 and 9 as 0x9, then adding 0x30 converts to ASCII.

I am in the process of implementing exactly the same thing. The simplest way around this problem is simply to do away with the one_time_max and one_time_min variables and simply implement a mid_point value, testing the bit_time against this for one and zero values. By simply testing for a mid_point value, it allows the routine to tune itself for integer/non-integer timecode values from 24fps thorough to 30fps.

I am implementing a similar project, but my timecode reader is built into the clapperboard itself.
https://improduction.ru/tags/imslate-1/
The thing is that the bit length depends on the FPS. I just "listened" to the signal from my Zoom F8 recorder and then started the reference data. While receiving data from the cable, I simply compare them and determine the frequency. That's all.
https://improduction.ru/all/elektronnaya-hlopushka-imslate-1-pro-chtenie-taymkoda/

I see that you have a similar project.  You have nice images.  I have no idea how this attempts to address the question though.

I wrote how I solved this problem in general terms and showed what my display looks like. I determine frequencies and DF / NF in real time. But, I will not show the code

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Logging multiple gigabytes of data with Arduino
I have Arduino Uno with SD card shield. I'm planning to log a huge amount of data from various sensors - including accelerometers and 3D compass - to the SD card.
However, I have understood that Arduino, 64GB SD, FAT and large amount of data are not working too well.
Does Arduino work well with this amount of data on FAT? There's no need to read any data from SD on Arduino. If not, is there a better approach? Writing directly to SD without FAT?
This data logger will be on the environment where no computer or network connection is available. Performance is not an issue - around 8kB/s (over three months) is already enough.

I'm not sure where you have the impression it won't work. I haven't used 64GB cards, but have used 32GB cards and they work fine in terms of FAT32 and writing files.
As you have already worked out, reading or searching the file system can get very slow as size increases.
That said, I have found that larger capacity cards are more likely to have problems with the SPI interface being poor quality. Consumer devices do not use the SPI interface, and it seems that some manufacturers are willing to skimp on this side of it. The SPI interface can be buggy or unstable as a result. Genuine name-brand cards should work fine though.

You should consider using sdfatlib. This is currently supported version. Where an early version of this was the predecessor the IDE's stock SD library. SdFat better supports newer and larger SdCards, along with more features.

Updating with new location of sdfatlib


text:
Why do sketches take up so much space and memory?
When I compile this sketch for the Yún:
int led = 7;

void setup() {
  pinMode(led, OUTPUT);
}

void loop() {
  digitalWrite(led, HIGH);
}

I get:

Sketch uses 5,098 bytes (17%) of program storage space.
Maximum is
  28,672 bytes. Global variables use 153 bytes (5%) of dynamic memory,
  leaving 2,407 bytes for local variables. Maximum is 2,560 bytes.

Even when I compile the BareMinimum sketch:
void setup() {
  // setup
}

void loop() {
  // loop
}

I get:

Sketch uses 4,548 bytes (15%) of program storage space.
Maximum is
  28,672 bytes. Global variables use 151 bytes (5%) of dynamic memory,
  leaving 2,409 bytes for local variables. Maximum is 2,560 bytes.

Why does a bare minimum sketch take up 15% of the program storage space allocated? And why does a very simple sketch take up 17% of the program storage space? According to the Arduino website:

It's easy to use it all up by having lots of strings in your program.
  For example, a declaration like: char message[] = "I support the Cape
  Wind project."; puts 33 bytes into SRAM (each character takes a byte,
  plus the '\0' terminator).

However, there aren't any strings declared in either of these sketches.
It seems as if they might import or use other libraries/classes that I don't specify. Maybe it imports a system default library? Or is it something else?

The YUN is a combo. Part Arduino and Part OpenWRT(Linux). Your question is in reference to the Arduino. Where this is actually a ATmega32u4 similar to a Leonardo and not an UNO(ATmega328p). The 32u4(Leo) communicates via Virtual Serial Ports over the USB (short answer: this needs to be supported) , where the UNO has a real Serial Port (aka UART). Below are builds statistics of the different boards types for the AVR processors.
Note on the UNO there is an external chip that converts USB to the Serial port's DTR pin which toggles the ATmega328's reset pin when connected causing a reboot to the bootloader. In contrast the Leo/Yun's USB to Serial is in implemented in the firmware of the 32u4. Hence in order to remotely reboot the Leo or YUN's 32u4 chip the firmware loaded must always support the USB client side driver. Which consumes approximately 4K.
If the USB was NOT needed and no other library resources were called as in the case of BareMinimum.ino on an UNO, only approximately 466 bytes are needed for the core Arduino Library.
compile stats of BareMinimum.ino on a UNO(ATmega328p)
Sketch uses 466 bytes (1%) of program storage space. Maximum is 32,256 bytes.
Global variables use 9 bytes (0%) of dynamic memory, leaving 2,039 bytes for local variables. Maximum is 2,048 bytes.

compile stats of BareMinimum.ino on a Leonardo(ATmega32u4)
Sketch uses 4,554 bytes (15%) of program storage space. Maximum is 28,672 bytes.
Global variables use 151 bytes (5%) of dynamic memory, leaving 2,409 bytes for local variables. Maximum is 2,560 bytes.

compile stats of BareMinimum.ino on a Yun(ATmega32u4)
Sketch uses 4,548 bytes (15%) of program storage space. Maximum is 28,672 bytes.
Global variables use 151 bytes (5%) of dynamic memory, leaving 2,409 bytes for local variables. Maximum is 2,560 bytes.

Arduino compiles in a lot of standard libraries, interrupts, ... etc. For example the pinMode and digitalWrite functions use a lookup table to figure out at run time which GPIO registers to write data to. Another example is that Arduino keeps track of time, it defines some interrupts by default and all this functionality requires some space. You'll notice that if you extend the program, the foot print will only change slightly.
I personally like to program controllers with bare minimum, without "bloat", but you'll quickly enter the world of EE.SE and SO because several easy to use functions will no longer work out of the box. There are some alternative libraries for pinMode and digitalWrite that compile into a smaller footprint, but come with other disadvantages like for example static compiled pins (where led cannot be a variable, but is a constant).

So basically it compiles in all sorts of standard libraries without you asking? Neat.

Yes, I usually call it "bloat", but it is really a usability thing. Arduino is a low entry level environment that just works without too much thought. If you need more, Arduino allows you to use alternative libraries or you can compile against bare metal. The last is probably out of scope for Arduino.SE

See my @mpflaga answer. There is not as much bloat. Or at least in the core library for bare minimum functionality. There is not really much of any standard libraries included, unless called the sketch. Rather the 15% is due to the USB support of the 32u4.

You already have some perfectly good answers. I am posting this only to
share some stats I did one day I asked myself the same sort of
questions: What is taking so much space on a minimal sketch? What is the
minimum needed to achieve the same functionality?
Below are three versions of a minimal blinky program that toggles the
LED on pin 13 every second. All three versions have been compiled for an
Uno (no USB involved) using avr-gcc 4.8.2, avr-libc 1.8.0 and
arduino-core 1.0.5 (I do not use the Arduino IDE).
First, the standard Arduino way:
const uint8_t ledPin = 13;

void setup() {
    pinMode(ledPin, OUTPUT);
}

void loop() {
    digitalWrite(ledPin, HIGH);
    delay(1000);
    digitalWrite(ledPin, LOW);
    delay(1000);
}

This compiles to 1018 bytes. Using both avr-nm and
disassembly,
I broke down that size into individual functions. From
largest to smallest:
 148 A ISR(TIMER0_OVF_vect)
 118 A init
 114 A pinMode
 108 A digitalWrite
 104 C vector table
  82 A turnOffPWM
  76 A delay
  70 A micros
  40 U loop
  26 A main
  20 A digital_pin_to_timer_PGM
  20 A digital_pin_to_port_PGM
  20 A digital_pin_to_bit_mask_PGM
  16 C __do_clear_bss
  12 C __init
  10 A port_to_output_PGM
  10 A port_to_mode_PGM
   8 U setup
   8 C .init9 (call main, jmp exit)
   4 C __bad_interrupt
   4 C _exit
-----------------------------------
1018   TOTAL

In the list above, the first column is the size in bytes, and the second
column tells whether the code comes from the Arduino core library (A,
822 bytes total), the C runtime (C, 148 bytes) or the user (U, 48
bytes).
As can be seen in this list, the largest function is the routine
servicing the timer 0 overflow interrupt. This routine is responsible of
tracking time, and is needed by millis(), micros() and delay().
The second largest function is init(), which sets the hardware timers
for PWM, enables TIMER0_OVF interrupt and disconnects the USART (that
was used by the bootloader). Both this and the previous function are
defined in
<Arduino directory>/hardware/arduino/cores/arduino/wiring.c.
Next is the C + avr-libc version:
#include <avr/io.h>
#include <util/delay.h>

int main(void)
{
    DDRB |= _BV(PB5);     /* set pin PB5 as output */
    for (;;) {
        PINB = _BV(PB5);  /* toggle PB5 */
        _delay_ms(1000);
    }
}

The break-down of the individual sizes:
104 C vector table
 26 U main
 12 C __init
  8 C .init9 (call main, jmp exit)
  4 C __bad_interrupt
  4 C _exit
----------------------------------
158   TOTAL

This is 132 bytes for the C runtime and 26 bytes of user code, including
the inlined function _delay_ms().
It could be noted that, since this program does not use interrupts,
the interrupt vector table is not needed, and regular user code
could be put in its place. The following assembly version does exactly
that:
#include <avr/io.h>
#define io(reg) _SFR_IO_ADDR(reg)

    sbi io(DDRB), 5  ; set PB5 as output
loop:
    sbi io(PINB), 5  ; toggle PB5
    ldi r26, 49      ; delay for 49 * 2^16 * 5 cycles
delay:
    sbiw r24, 1
    sbci r26, 0
    brne delay
    rjmp loop

This is assembled (with avr-gcc -nostdlib) into only 14 bytes,
most of which are used to delay the toggles so that the blink is
visible. If you remove that delay loop, you end up with a 6-byte
program that blinks too fast to be seen (at 2 MHz):
    sbi io(DDRB), 5  ; set PB5 as output
loop:
    sbi io(PINB), 5  ; toggle PB5
    rjmp loop

I wrote a post about Why does it take 1000 bytes to blink one LED?.
The brief answer is: "It doesn't take 2000 bytes to blink two LEDs!"
The longer answer is that the standard Arduino libraries (which you don't have to use if you don't want to) have some nice functionality to simplify your life. For example, you can address pins by number at runtime, where the library converts (say) pin 8 to the correct port and correct bit number. If you hard-code port access you can save that overhead.
Even if you don't use them, the standard libraries include code to count "ticks" so you can find out the current "time" (by calling millis()). To do this it has to add the overhead of some interrupt service routines.
If you simplify down (on the Arduino Uno) to this sketch you get the program memory usage down to 178 bytes (on IDE 1.0.6):
int main ()
  {
  DDRB = bit (5);
  while (true)
    PINB = bit (5);
  }

OK, 178 bytes isn't that much, and of that the first 104 bytes are the hardware interrupt vectors (4 bytes each, for 26 vectors).
So arguably, there are only 74 bytes required to blink an LED. And of that 74 bytes most are really the code generated by the compiler to initialize global memory. If you add enough code to blink two LEDs:
int main ()
  {
  DDRB = bit (5);  // pin 13
  DDRB |= bit (4);  // pin 12

  while (true)
    {
    PINB = bit (5); // pin 13
    PINB = bit (4); // pin 12
    }
  }

Then the code size increases to 186 bytes. So therefore you could argue that it only takes 186 - 178 = 8 bytes to blink an LED.
So, 8 bytes to blink an LED. Sounds pretty efficient to me.

In case you are tempted to try this at home, I should point out that whilst the posted code above blinks two LEDs, it does so very rapidly indeed. In fact, they blink at 2 MHz - see screenshot. Channel 1 (yellow) is pin 12, channel 2 (cyan) is pin 13.

As you can see, the output pins have a square wave with a frequency of 2 MHz. Pin 13 changes state 62.5 ns (one clock cycle) before pin 12, due to the order of the toggling of the pins in the code.
So unless you have much better eyes than mine, you won't actually see any blinking effect.

As an amusing extra, you can actually toggle two pins in the same amount of program space as toggling one pin.
int main ()
  {
  DDRB = bit (4) | bit (5);  // set pins 12 and 13 to output

  while (true)
    PINB =  bit (4) | bit (5); // toggle pins 12 and 13
  } // end of main

That compiles into 178 bytes.
This gives you a higher frequency:

Now we are up to 2.66 MHz.

This makes a ton of sense. So are the standard libraries just headers included automatically at build time? And how were you able to not include them?

The linker aggressively strips out code that isn't used. By not calling init() (as the normal main() does) then the file wiring.c (which has init in it) was not linked in. As a result, the processing for the interrupt handlers (for millis(), micros() etc.) was omitted. It's probably not particularly practical to omit it, unless you never need to time things, but the fact is that the sketch grows in size depending on what you put in it. For example, if you use Serial, both program memory and RAM take a hit.


text:
How to organize my Arduino projects' structure for easy source control?
It's been a long time I've been looking for a good answer to this question.
Typically, any Arduino project but the simplest one will include:

The main source code file MyProject.ino
Libraries specific to the project (MyProjectLibrary1.h, MyProjectLibrary1.cpp...)
Third-party libraries (generally free open source, added manually to Arduino libraries directory)
Schematics, PCB diagrams
Documentation
...

All this makes it hard to keep the whole code and doc of one project under Source Code Management (e.g. on Subversion, Git or GitHub).
Managing source control of your project means managing the version of all files used by the project including 3rd-party libraries.
Now for a single project, I need to define a directory structure that:

Includes all project files as described above
I can entirely commit to a Source Code Management tool (including 3rd-party dependencies)
I can checkout anywhere on my hard drive and build the project from there (does it have to be a single location as imposed by Arduino IDE)
I can zip into a self-contained archive that I can send to a friend for him to build as easily as possible (no extra manual download)

What I find particularly tricky with Arduino projects is the management of external libraries dependencies. Java projects developers have maven repositories for that and that helps a lot in managing all external deps. But we don't have an equivalent system for Arduino libraries.
I would be interested to know how other Arduino project makers deal with these aspects in their own projects.
Also note that I am open to changing my development process, including my IDE (currently I use Eclipse with the Arduino plugin most of the time, and then I ensure my projects can also work directly with the Arduino IDE).

I've also been struggling with this. I have two projects which need different versions of an external library, and currently they are outside of version control.

Please note for further research that those are Package Managers. JavaScript got Node.js/npm and bower, PHP got PEAR and Composer, etc.

Here is the way I finally decided to follow for my projects.
Arduino-CMake
The first important decision I made was the choice of a build tool that could work for my environment (Windows) but was not limited to it (I want my projects easily reusable by other people).
I have tested various open source Arduino make tools:

Guyzmo Makefile (suggested by @zmo answer): this is just a standard Makefile handcrafted for Arduino builds; this is a Unix Makefile but there  is a good port of Unix make for Windows; still, unfortunately this Makefile works only for Unix, of course it could be adapted for Windows, but I wanted a tool that works "out of the box".
Arduino-Makefile (suggested by @adnues answer): this a more advanced project, based on Unix Makefile, that aims to be easily reusable by all Arduino projects; it is documented as working on Mac, Linux and Windows, but Windows support proved wrong in my first experiments (many dependencies on Unix shell).
Graduino (not suggested by any answer): this build tool is based on well-known gradle build tool from the groovy world; the tool seems quite well done but requires some (little) groovy/gradle knowlegde, and has only little documentation; I decided not to go with it due to the burden of installing groovy and gradle just for it (I'd like to avoid too many pre-requisites to people who want to build my projects on their environments).
Arduino-CMake (not suggested by any answer): this seems the best of all, it has a long history, has many supporters and maintainers, is very well documented, comes with simple examples and also has a few good tutorial blog posts on the Web, e.g. here and there. It is based on CMake, a "Cross-Platform Make".

I have also found ArduinoDevel, another Arduino build tool -which I have not experimented- that can generate Unix Makefiles or ant build.xml files; that one seemed interesting but a bit limited in terms of functionality.
Finally I decided to go with Arduino-CMake:

it was easy to setup: just install CMake on your machine and copy Arduino-CMake in some directory that is easily accessible (through relative paths) from your projects directories.
the examples worked out of the box for me (just followed the comments in CMakeLists.txt configuration file to adapt properties that needed to for my environment, e.g. Arduino type, serial port)
you can organize your projects any way you want
it can generate configuration files for various build tools out there (I have only tested Unix Makefiles though), including Eclipse projects.
in the generated make, several targets get created to support:

libraries build
programs build
program upload to boards
serial monitor launch
and a few others I have not tested yet

Project Structure
Since Arduono-CMake does not impose any directory structure for your project, you can choose the one that fits you best.
Here is what I've done personnaly (that still requires further refinement, but I'm happy with it now):

I have decided to put all my projects under a common arduino-stuff directory (which I commit to github as a whole, I know I could use git submodules for a better organisation on github, but had no time to check that yet).
arduino-stuff has the following content:

build: that's a directory where cmake and make will generate all their stuff (makefiles, cache, object files...); this one does ot get committed to github
cmake: that one is just a copy (unmodified) of Arduino-CMake cmake directory. This one gets on github so that it's easier for someone who wants to build my projects
CMakeLists.txt: that's the "global" CMake configuration that declares all defaults for my environment (board, serial port) and the list of build target subdirectories
TaskManager: this is my first project based on Arduino-CMake, this one is a library with examples; this idrectory also contains a CMakeLists.txt that states the targets for the project

Points to improve
The current solution is not perfect though.
Among improvements I see (that's rather for Arduino-CMake project to include these improvements if they see fit):

Feature to copy a library directory from the current project to the Arduino libraries directory
Feature to upload a library to github
Feature to download a library from github

Have you tried PlatformIO yet? It might not have been around when you were asking this question.. http://platformio.org/

It looks like the project died and resurrected a few times: arduino-cmake → forked arduino-cmake → Arduino-CMake-NG → Arduino-CMake-Toolchain.

My way of organizing an arduino project is quite simple, all my project are git repositories so that there is at least the following:

project/

project.ino
Makefile

I have a preference using my favorite editor and a Makefile which I have crafted to work against most use cases (and I even improved on that one that I'm going to share soon).
For the libraries, I prefer to keep them as their own repositories and use git submodule to include them to the project. As many libraries written by the community are shared as git repositories, that's a good generic solution. Then, within the Makefile, I just have to add the libraries path I want to include in the LOCALLIBS variable.
Though, for some projects, it makes sense to encapsulate the libraries into a hardware abstraction layer library crafted for the project, then I prefer to use a path such as:

project

project.ino
Makefile
project_hal_lib

library1
library2
library3
…

Though, with arduino 1.5.x a new way to specify libraries is offered, that will offer a way to create and build arduino projects the same way we already do with pipy and virtualenv in python, i.e. you define the set of libraries you need and they get downloaded.

I was working on a similar answer. You beat me to it!

+1 Thanks! This way looks quite interesting. I'll have to give it a try this week (I need to check out how to setup the Makefile stuff first, though).

@AsheeshR if your answer was similar, that means it still has some differences, right? I'd be interested to know about these!

actually the main changes to come with next version of my Makefile will be the ability to either flash using a programmer or upload using the bootloader. As well as handling merging of bootloader with firmware. I also wrote an in-makefile fuse setter.

@zmo bounty deserved, though your Makefile solution cannot work in my situation (using Windows but I did not specify that point). But I am convinced that using one of the existing makefile solutions available is the way to go. Once I have found one tha tworks for me I'll post my answer here.

well, actually I have designed my Makefile to by somehow windows-compatible… though at the time I did not have a windows to test it! So definitely, if you want to help improving it, you're welcome :-)

So , what's the "arduino 1.5.x a new way to specify libraries"? That's too many words. Could you show an example?

When I wrote that answer that was pretty new, but today it's widely used. The best way to find an example (beyond reading the spec) is to crawl github: https://github.com/search?q=%22library.properties%22 . An example would be the first match. Sadly, I haven't done much Arduino over the last few years, and I'm not up to date anymore with the best practices, so I don't know how far they went with this, and whether a pip like CLI is possible or not.

The simplest way to do this is to copy the header and code files of the library into your source directory and include them.
myproject/
    myproject.ino
    somelib.h
    somelib.cpp

In your code, you can do include "somelib.h"
The down side to this is that the libraries must be in the same folder, not sub folders, so it makes your directory look messy.

Regarding the directory structure of my entire project, including schematics and documentation, mine usually looks like this:
myproject/
  schematics/ - eagle files or whatever you may have
  docs/       - include relevant datasheets here
  test/       - any test cases or other code to test parts of the system.
  myproject/  - since Arduino code must be in a directory of the same name
    myproject.ino
    ...

Another downside is that I will have to copy the same libraries on many projects. Also, it is not clear to me if you put only YOUR libraries in there or also 3rd-party libraries?

First point: That's not really a down side, that's just a side effect of keeping libraries and project source together as you wanted with version control. What if another project needs an updated version of the library? What if you modified it? Second point: both will work.

Agreed. Now where do you put other files for the project and the libraries (scehmatics, documentation)? In a subdir, in the project dir, outside?

@jfpoilpret I updated my answer with my typical directory structure.

Thanks for the update, +1 for your answer. I am curious to see what others do for their own projects.

One more question, after duplicating your own libraries into the different projects that use them, one of these projects may find some bugs or improvements of a duplicated library; then you have to duplicate changes everywhere again, that is error-prone and time-consuming! Also, if you want to "open" one of these libraries on its own (outside any project) for others to use, you also have to duplicate changes into there. Do you have any trick to work around this problem.

I do not. The Arduino IDE is quite limited in many ways. You may want to look into a better environment to work in that has better support for this. People have made custom make files that let you import libraries from other sources as well.

This is not a good way to organize projects from the point of view of software licenses. If you are including third-party libraries in your project, which may have different licenses, then you may be violating them as soon as you start sharing the project file. Different open source licenses are generally not compatible with each other.

@AsheeshR having all your files in one directory so the arduino IDE doesn't complain is not a good way to organize projects at all. It's just a way. Feel free to propose a better solution. I don't know of one that still allows you to use the Arduino software.

Git submodules are extremely powerful when it comes to organizing multiple nested repositories. Handling multiple libraries from different sources, and even handling parts of your own project which may be stored at different sources becomes easy with git submodules.
Directory Structure
A way to organize your projects would be:

projectA - Parent Directory

projectA - Source code directory containing Arduino code

projectA.ino
header.h
implementation.cpp

docs - Your main documentation directory
schematics - these may be maintained separately on a separate Git repo or part of the same repo
libs - This will contain your third party libraries.

libA - These may be maintained as third party repositories
libC - ...

license
README
Makefile - Necessary to handle dependencies across directories

Workflow
You would follow your normal cycle of make changes, add and commit as far as the main repository is concerned. Things get interesting with the sub-repositories.
You have the option of adding a repository into the parent directory of your main repository. This means that any part of you directory structure, i.e. docs, schematics, etc. can be maintained as a separate repository and continuously updated from.
You can do this using the git submodule add <repo.git> command. To keep it up to date, you can use git submodule update <path>.
When it comes to maintaining multiple third party libraries within your repository such that each can be version controlled in itself or each can be kept up to date if need be, git submodule again saves your day!
To add a third party repo to libs, use the command git submodule add <lib1.git> libs/lib1. Then, to maintain the library at a fixed point in the release cycle, checkout the library and make a commit. To keep the library up to date, use the command git submodule update <path>.
Now, you can maintain multiple repositories within a main repository as well as multiple third party libraries in their independent stages of release.
Versus Single Directory Approach
While the single directory approach is the simplest, it is not possible to version control parts of a directory without a lot of pain. Hence, the simple approach fails to accomodate different repositories with varying states in the project.
This approach allows maintaining multiple repositories but brings in the need for a Makefile to handle the compilation and linking process.
Depending on the complexity of your project, the optimal approach can be selected.

+1, but just as a sidenote: Git Submodules are pretty unstable and likely loose track. It makes no difference if you use a single directory or multiples (like vendor, node_modules, etc.). Git references them and keeps track of it.

"It makes no difference if you use a single directory or multiples (like vendor, node_modules, etc.)." I didn't understand this part. Could you elaborate?

MyProject
|_MyProject
  |_MyProject.ino
  |_data
  |  |_documentation
  |  |_PCB
  |  |_schematics
  |_src
     |_MyProjectLibrary1
     |_ThirdPartyLibrary

MyProject folder (repository root)
The reason I suggest the seemingly redundant MyProject root folder is that you mentioned using GitHub. When you download (rather than clone) the contents of a GitHub repository the branch or tag name is appended to the repository name (e.g. MyProject-master). The Arduino IDE requires that the sketch folder name match the sketch file name. If you open an .ino file that is in a folder that does not match the sketch name the Arduino IDE prompts you create an appropriately named sketch folder and move the sketch to that folder. In addition to this not being a very good initial experience for the user, the greater problem is that the Arduino IDE may not copy all the other associated files to the newly created folder, which could cause the program to no longer compile. By putting the sketch in a subfolder you avoid GitHub altering the name of the sketch folder.
If the GitHub file name thing is not an issue for you then the redundant root folder is not necessary.
data folder
I recommend using the data subfolder for your non-code files because the Arduino IDE has a special treatment of subfolders of that name. They are copied to the new location when you do a File > Save As.... Subfolders of any other name are not.
src folder
The src subfolder has the special property of allowing recursive compilation. This means that you can leave the libraries in that folder and include them from your sketch like this:
#include "src/MyProjectLibrary1/MyProjectLibrary1.h"
#include "src/ThirdPartyLibrary/ThirdPartyLibrary.h"

The Arduino 1.5 Library format folder structure is also supported, you only need to adjust your #include statements accordingly.
Note that only Arduino IDE 1.6.10 (arduino-builder 1.3.19) and newer support recursive sketch compilation.
Unfortunately some libraries use the incorrect #include syntax for local file includes (e.g. #include <ThirdPartyLibrary.h> instead of #include "ThirdPartyLibrary.h"). This still works when the library is installed to one of the Arduino libraries folders but does not work when the library is bundled with the sketch. So some libraries may require minor edits to use this way.
I greatly prefer this to the alternative of dumping all the library files in the root of the sketch folder because that is messy and every library file will be shown in the Arduino IDE as tabs when you open the sketch (Of course any source files you do want to be editable from the Arduino IDE should be placed in the sketch root folder).
Being able to use bundled libraries in place is also in line with another of your goals:

send to a friend for him to build as easily as possible

Removing the requirement to manually install libraries makes the project much easier to use.
This will also avoid any chances of conflict with other versions of library files of the same name that may be previously installed.

You could use the makefile
https://github.com/sudar/Arduino-Makefile
for compiling Arduino codes. You don't necessarily need the IDE.

I have tried but unfortunately it will work only on Unix machines and I need Windows support. Currently I am evaluating another project based on CMake but I'm not done with it yet. I'll post an answer when I have decided on a tool.

Probably really late to the game but it's a popular enough question to answer using slightly different methods than those posted already.
If you need to maintain compatibility with Arduino IDE directly you could use something like the one I outlined here:
https://gitlab.com/mikealger/ExampleArduinoProjectStructure/tree/master/ExampleSketchBook
I based most of this off the notes of Arduino -- Project structure and build process, and some tips I have picked up over the years.
I really don't know why this is so hard to find via Arduino pages directly, it seems silly coming from a semi professional background that the build process is so obtuse.
best of luck out there
----edited link to the project structure notes i used as they have been erased
they are now here

The gitlab link appears to be broken

odd does it work with out the the direct link ? i.e. https://gitlab.com/mikealger/ExampleArduinoProjectStructure

Actually both links work in Firefox, but neither work in my outdated Chrome version 49.0.2623.112 (64-bit). Nothing to worry about, I guess. :-)

ok grahamwideman's page is gone now but the wayback machine has a copy at https://web.archive.org/web/20161106122158/https://grahamwideman.wikispaces.com/Arduino+--+Project+structure+and+build+process


text:
How can I rename how my Arduino shows up on the computer?
I have a clone of a Mega 2560, and the Chinese manufacture apparently had a broken shift key. To my annoyance, whenever I connect it to my computer it reads, arduino compatible mega 2560. With my extreme annoyance with slightly misspelled or incorrectly capitalised English, it has been very distracting to me.
However, the reason that I'm asking this question is for more than how to rename it to be capitalised. Think of how many projects could be simplified by this! Or for many Arduinos, you can label they physically and then rename them Arduino #1, etc. so you can tell which port it is on using device manager. If you designed a product with Arduino, then you can even rename it to your product name.
I would imagine that there would be an easy way to do this by reburning the firmware to the [Insert USB chip model #### here]. I've seen it done for keyboards on the Uno, so I would imagine that you could modify it somehow.

Wait, where does this name come up? If it comes up in the Computer view on Windows (not sure on a Mac), couldn't you just right click, click properties, and then rename the device?

How do you expect this to be answered when you don't state which USB chip is utilized?  With a "compatible" board it could be any of the options, and the programming process (if any) depends entirely on which one.

This string is contained in the FT232 USB to UART interface chip and the manufacturer has released the FT_PROG tool to reprogram it. But do take note of the warning on their website: "PLEASE NOTE - The use of some of these utilities by an end user may result in a device being rendered useless". Another word of warning is to ensure there is only one device connected with an FTDI-chip.
Here is a screenshot of the MProg tool:

In that link you posted they claimed they were the "Future Technology Devices International Ltd", yet their design included a VGA cable. Oops! Where do I edit this string? Is it just not visible because no device is connected?

@AnnonomusPerson For usage you'll have to read the docs for the tool and I am pretty sure the device must be attached to read the current config.


text:
How much can I recurse? How much can I recurse? How much ca!@#QFSD@$RFW
The Arduino Uno board has limited RAM which means it has a limited call stack available. Sometimes, recursion is the only quick option to implement a certain algorithm. So, given that the call stack is severely limited, what would be a way to find out that given a certain program running on the board, exactly how many recursive calls can you afford before there is a stack overflow (and bad things happen)?

Can you give an example of an algorithm where the choice between a recursive and non-recursive solution is so clear cut?

@Cybergibbons Most recently, I had to write and test RDP in a short time span, so I went ahead with the recursive implementation. I am not saying that iterative is not possible, just that it may be easier to go ahead with the recursive version.

What does this mean in the question title: How much ca!@#QFSD@$RFW? I'm curious as to why no-one has edited that out to be something more meaningful (in the last 4 years).

The answer is 211 times (depending on many factors) :). See my answer here: https://arduino.stackexchange.com/a/51098/7727. @NickGammon, he is pretend "cursing" I think. It is a play on word for "recurse". Took me a minute to figure that out too. Was pretty confusing at first.

Oh I see. I thought the SE servers might have had a glitch :)

If you really want to recurse (and as @jippie said it is a bad idea; subliminal message:  don't do it) and want to know how much you can recurse, then you will have to perform some calculation and experiments; also you generally will have only an approximation of it as it depends a lot on the memory state at the time your recursive function will be called.
For this, you should first know how SRAM is organized inside AVR-based Arduino (it won't apply to e.g. the Arduino Galileo by Intel). The following diagram from Adafruit shows it clearly:

Then you need to know the total size of your SRAM (depends on Atmel MCU, hence what kind of Arduino board you have).
On this diagram, it is easy to find out ths size of Static Data block as it is known at compile-time and won't change later on.
The Heap size can be more difficult to know as it can vary at runtime, depending on dynamic memory allocations (malloc or new) performed by your sketch or the libraries it uses. Using dynamic memory is quite rare on Arduino, but some standard functions do it (type String uses it, I think).
For the Stack size, it will also vary during runtime, based on the current depth of function calls (each function call takes 2 bytes on the Stack to store the address of the caller) and the number and size of local variables including passed arguments (that are also stored on the Stack) for all the functions called until now.
So let's suppose your recurse() function uses 12 bytes for its local variables and arguments, then each call to this function (the first one from an external caller and the recursive ones) will use 12+2 bytes.
If we suppose that:

you are on Arduino UNO (SRAM = 2K)
your sketch does not use dynamic memory allocation (no Heap)
you know the size of your Static Data (let's say 132 bytes)
when your recurse() function is called from your sketch, the current Stack is 128 bytes long

Then you are left with 2048 - 132 - 128 = 1788 available bytes on the Stack. The number of recursive calls to your function is thus 1788 / 14 = 127, including the initial call (which is not a recursive one).
As you can see, this is very difficult, but not impossible to find what you want.
A simpler way to get the stack size available before recurse() is called would be to use the following function (found on Adafruit learning center; I have not tested it myself):
int freeRam ()
{
  extern int __heap_start, *__brkval;
  int v;
  return (int) &v - (__brkval == 0 ? (int) &__heap_start : (int) __brkval);
}

I strongly encourage you to read this article at Adafruit learning center.

I see peter-r-bloomfield posted his answer while I was writing mine; his answer looks better as it fully describes the content of the stack after a call (I had forgotten the registers state).

Both very good quality answers.

Static data = .bss + .data, and is what is reported by Arduino as "RAM taken up by global variables" or whatever, correct?

@GabrielStaples yes exactly. In more detail .bss represents the global variables with no initial value in your code, whereas data is for global variables with an initial value. But in the end they use the same space: Static Data in the diagram.

@GabrielStaples forgot one thing, technically these are not only global variables going there, you also have variables declared static within a function.

Follow up question: is there such a thing as allocating some ram for stack and some for heap, or is it all available for either? In other words, someone told me that stack size is set in a linker script or something but I thought the entire ram could be used arbitrarily by stack or heap as they need it. What's the truth? Does Arduino use a linker script? If so, where is it?

No, the whole SRAM is dedicated to both Heap and Stack altogether (one is allocated from the top, the other from the bottom). It is developer's responsibility to ensure heap and stack do not collide in each other.

Note that there has been several questions here about Arduino memory, some with very comprehensive answers. It might be worth taking a look at them if you want to know more about this topic.

Note that there are some threading libraries for Arduino floating around. When using these, you do have to explicitly allocate stack space, usually within the heap, for all but the main thread. This, however, is an uncommon situation. The situation you describe is by far the most common.

Recursion is bad practice on a microcontroller as you already stated yourself and you probably want to avoid it whenever possible. On the Arduino site there are some examples and libraries available for checking free RAM size. You can for example use this to figure out when to break recursion or a bit trickier/riskier to profile your sketch and hard code the limit in it. This profile would be required for every change in your program and for every change in Arduino tool chain.

Some of the more high-end compilers, such as IAR (who do support AVR) and Keil (who do not support AVR) have tools to help you monitor and manage stack space. It's really not advisable on something as small as an ATmega328 though.

It depends on the function.
Every time a function is called, a new frame is pushed onto the stack. It will usually contain various critical items, potentially including:

Return address (the point in the code from which the function was called).
The local instance pointer (this) if calling a member function.
Parameters passed into the function.
Register values which need to be restored when the function ends.
Space for local variables inside the called function.

As you can see, the stack space required for a given call depends on the function. For example, if you write a recursive function which only takes an int parameter and uses no local variables, it won't need much more than a few bytes on the stack. That means you can recursively call it far more than a function which takes several parameters and uses a lot of local variables (which will eat up the stack much quicker).
Obviously the state of the stack depends on what else is going on in the code. If you start a recursion directly within the standard loop() function, then there probably won't be a lot on the stack already. However, if you start it nested several levels deep in other functions, then there won't be as much room. That will affect how many times you can recurse without exhausting the stack.
It's worth noting that tail recursion optimisation exists on some compilers (although I'm not sure if avr-gcc supports it). If the recursive call is the very last thing in a function, it means it is sometimes possible to avoid altering the stack frame at all. The compiler can just re-use the existing frame, since the 'parent' call (so to speak) is finished using it. That will mean you can theoretically keep recursing as much as you like, so long as your function doesn't call anything else.

avr-gcc does not support tail recursion.

@AsheeshR - Good to know. Thanks. I figured it was probably unlikely.

You can do a tail call elimination/optimization by refactoring your code instead of hoping the compiler will do it. As long as the recursive call is at the end of the recursive method you can safely re-write the method to use a while/for loop.

The post by @TheDoctor contradicts "avr-gcc does not support tail recursion", as does my test of his code. The compiler did indeed implement tail recursion, which is how he got up to a million recursions. Peter is correct - it is possible for the compiler to replace call/return (as the last call in a function) with simply jump. It has the same end result and does not consume stack space.

I had this exact same question as I was reading Jumping into C++ by Alex Allain, Ch 16: Recursion, p.230, so I ran some tests.
TLDR;
My Arduino Nano (ATmega328 mcu) can do 211 recursive function calls (for the code given below) before it has a stack overflow and crashes.
First off, let me address this claim:

Sometimes, recursion is the only quick option to implement a certain algorithm.

[Update: ah, I skimmed the word "quick". In that case you have some validity. Nevertheless, I think it's worth saying the following.]
No, I don't think that is a true statement. I'm pretty certain all algorithms have both a recursive and a non-recursive solution, without exception. It's just that sometimes it is significantly easier to use a recursive algorithm. Having said that, recursion is very much frowned upon for use on microcontrollers and would probably never be allowed in safety-critical code. Nevertheless, it is possible of course to do it on microcontrollers. To know how "deep" you can go into any given recursive function, just test it! Run it in your real-life application in a real-life test case, and remove your base condition so that it will infinitely recurse. Print out a counter and see for yourself how "deep" you can go so you know whether or not your recursive algorithm is pushing the limits of your RAM too close to be used practially. Here's an example below to force stack overflow on an Arduino.
Now, a few notes:
How many recursive calls, or "stack frames" you can get is determined by a number of factors, including:

The size of your RAM
How much stuff is already on your stack or taken up in your heap (ie: your free RAM matters; free_RAM = total_RAM - stack_used - heap_used, or you might say free_RAM = stack_size_allocated - stack_size_used)
The size of each new "stack frame" which will be placed onto the stack for every new recursive function call. This will depend upon the function being called and its variables and memory requirements, etc.

My results:

20171106-2054hrs - Toshiba Satellite w/16 GB RAM; quad-core, Windows 8.1: final value printed before crash: 43166

took several seconds to crash--maybe 5~10?

20180306-1913hrs Dell high-end laptop w/64 GB RAM; 8-core, Linux Ubuntu 14.04 LTS: final value printed before crash: 261752

followed by the phrase Segmentation fault (core dumped)
took only ~4~5 sec or so to crash

20180306-1930hrs Arduino Nano: TBD---is at ~250000 and still counting---the Arduino optimization settings must have caused it
to optimize out the recursion...??? Yes, that is the case.

Add #pragma GCC optimize ("-O0") to the top of the file and redo:

20180307-0910hrs Arduino Nano: 32 kB flash, 2 kB SRAM, 16 MHz processsor: final value printed before crash: 211

Here are the final print results:
209
210
211  ⸮ 9⸮   3⸮

took only a fraction of a second once it started printing at 115200 serial baud-rate--maybe 1/10 sec
2 kiB = 2048 bytes / 211 stack frames = 9.7 bytes/frame (assuming ALL of your RAM is being used by the stack--which actually isn't the case) --but this seems very reasonable nonetheless.

The code:
The PC application:
/*
stack_overflow
 - a quick program to force a stack overflow in order to see how many stack frames in a small function can be loaded onto the stack before the overflow occurs

By Gabriel Staples
www.ElectricRCAircraftGuy.com
Written: 6 Nov 2017
Updated: 6 Nov 2017

References:
 - Jumping into C++, by Alex Allain, pg. 230 - sample code here in the chapter on recursion

To compile and run:
Compile: g++ -Wall -std=c++11 stack_overflow_1.cpp -o stack_overflow_1
Run in Linux: ./stack_overflow_1
*/

#include <iostream>

void recurse(int count)
{
  std::cout << count << "\n";
  recurse(count + 1);
}

int main()
{
  recurse(1);
}

The Arduino "Sketch" program:
/*
recursion_until_stack_overflow
- do a quick recursion test to see how many times I can make the call before the stack overflows

Gabriel Staples
Written: 6 Mar. 2018
Updated: 7 Mar. 2018

References:
- Jumping Into C++, by Alex Allain, Ch. 16: Recursion, p.230
*/

// Force the compiler to NOT optimize! Otherwise this recursive function below just gets optimized into a count++ type
// incrementer instead of doing actual recursion with new frames on the stack each time. This is required since we are
// trying to force stack overflow.
// - See here for all optimization levels: https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html
//   - They include: -O1, -O2, -O3, -O0, -Os (Arduino's default I believe), -Ofast, & -Og.

// I mention `#pragma GCC optimize` in my article here: http://www.electricrcaircraftguy.com/2014/01/the-power-of-arduino.html
#pragma GCC optimize ("-O0")

void recurse(unsigned long count) // each call gets its own "count" variable in a new stack frame
{
  // delay(1000);
  Serial.println(count);

  // It is not necessary to increment count since each function's variables are separate (so the count in each stack
  // frame will be initialized one greater than the last count)
  recurse (count + 1);

  // GS: notice that there is no base condition; ie: this recursive function, once called, will never finish and return!
}

void setup()
{
  Serial.begin(115200);
  Serial.println(F("\nbegin"));
  // First function call, so it starts at 1
  recurse (1);
}

void loop()
{
}

References:

Jumping into C++ by Alex Allain, Ch 16: Recursion, p.230
http://www.electricrcaircraftguy.com/2014/01/the-power-of-arduino.html - literally: I referenced my own website during this "project" to remind myself how to change Arduino compiler optimization levels for a given file with the #pragma GCC optimize command since I knew I had it documented there.

Note that, according to the docs of the avr-lib, you should never compile without optimization anything that relies on the avr-libc, as some thing are not guaranteed to even work with optimization turned off. I thus advice you against the #pragma you are using there. Instead, you may add __attribute__((optimize("O0"))) to the single function you want to unoptimize.

Thanks, Edgar. Do you know where AVR libc has this documented?

The documentation on <util/delay.h> states: “In order for these functions to work as intended, compiler optimizations must be enabled [...]” (emphasis in original). I'm not quite sure whether any other avr-libc functions have this requirement.

I wrote this simple test program:
void setup() {
  // put your setup code here, to run once:
  Serial.begin(115200);
  recurse(1);
}

void loop() {
  // put your main code here, to run repeatedly:

}

void recurse(long i) {
  Serial.println(i);
  recurse(i+1);
}

I compiled it for the Uno, and as i write it has recursed over 1 million times! I don't know, but the compiler may have optimized this program

Try to return after a set number of calls ~1000. It should create a problem then.

The compiler has cunningly implemented tail-recursion on your sketch, as you will see if you disassemble it. What this means is that it replaces the sequence call xxx / ret by jmp xxx. This amounts to the same thing, except that the compiler's method does not consume the stack. Thus you could recurse billions of times with your code (other things being equal).

You can force the compiler not to optimize out the recursion. I'll come back and post an example later.

Done! Example here: https://arduino.stackexchange.com/a/51098/7727. The secret is to prevent optimization by adding #pragma GCC optimize ("-O0")  to the top of your Arduino program. I believe you have to do this at the top of each file you want it to apply to--but I haven't looked that up in years so research it for yourself to be sure.


text:
How can I use Python from a sketch?
I'd like to be able to use Python from a sketch. According to the Arduino playground, one can use PySerial from a computer to talk with an Arduino via Python.
It looks like from that page you can also use various API's to use Python with an Arduino from a computer. However, I'd like to know if you can execute Python directly from a sketch. It would be nice because then you could execute Python directly from a sketch, and not all of it would have to be written in Python.
If this is not possible, is there an easy way to execute Python on an Arduino? Looking at the various API's it was hard to determine exactly what they did, and if you could execute a .py script from a computer or from anything on the Arduino.

It's not an Arduino board, but you might be interested in Micro Python. It got kickstarted a few months back.

It looks like from that page you can also use various API's to execute
  Python directly on the Arduino.

I don't see that, at all. All those links show you how to communicate with the Arduino over serial using Python on the host side.
You'd need a python interpreter to be able to run python on the Arduino. There is the Python On A Chip project and it seems like they have support for the Mega.

2010/09/01 (1bdb8d31f27b) New platform: Arduino Mega.

Some more info from the PyMite page:

any device in the AtMega family which meets these requirements: 20KiB
  Flash and 4K RAM, will run today

I doubt you'll find anything that'll let you mix Arduino and Python code in the same sketch.

According to the python-on-a-chip website, you need at least 8 kB of ram to run a normal program

Hmm, so basically your options are to either execute Python from a computer, somehow use the Python On A Chip project with an Arduino, or just execute native C\C++ code? Interesting...