text:
Use of progmem in ESP8266 vs AVR and also how to handle large dynamic strings
This is a somewhat tricky question dealing with implementation and best-practices of the ESP8266. I'd ideally hope that it's answered by people with in-depth knowledge on the problem.
As you are hopefully aware, the Arduino (AVR) platform requires the use of the PROGMEM or F() instructions to avoid copying all strings to RAM. The ESP8266 platform also provides a PROGMEM instruction, but is memory-mapped.
I thus have several questions about what the PROGMEM instruction actually does on the ESP8266.
For the next few questions, until otherwise marked, I am referring to character-array-style C strings, not the Arduino String type.

Storage of constant strings: If I were to declare a string (global, for now) on the ESP8266, would it be read from flash only when it's accessed, or will it always take up RAM (like it would on the AVR. Assume that in this case I do NOT declare it as PROGMEM; I wish to find out what the default behavior is on the ESP8266)?
Local Constant Strings: What happens with strings that are used inside a function? I am fairly sure that they ALSO exist in RAM, at least in AVR. Does the ESP8266 do this too, or does it (potentially) store it in flash until it is read (it is fine if it ends up inside a cache somewhere, as long as it can be evicted if needed)?
Use of String(F("mystring")): If I wrap a PROGMEM string in this manner, does this truly convert the string into a RAM-based string only when this line is reached, or does this for some reason initialize the String() early (and consume the RAM anyway)? Do the systems differ in this behavior?
If I define several instances of the above (as in, the identical wrapped string shown above appears several times, perhaps in different functions), do all of them point to the SAME F() instance (since they are identical) or does it get duplicated since it's part of code rather than a referenced variable? On ESP8266? On AVR?
On the ESP8266, I discovered that the server.on() function for the built-in webserver read F()'d constant strings just fine, but failed to work once the file grew beyond the size of the RAM. It was clearly stored correctly, but it could not be read. I thus conjecture that the method was making a local copy to RAM, even if the file was stored in Flash. There was also a on_P version that worked after that point (I believe I used PSTR rather than F at this point as well, since -- correct me if I am wrong -- the two are not compatible). Is this the real point of PROGMEM on ESP8266? That the data is only stored in Flash, but it also tells the function reading it to make as few copies as possible or chunk it?

I am also curious what happens with large strings that aren't constant. The web server can return data that was passed via querystring, for instance, which in my case can be quite large. If I copy that data to a string (and then into a function), does it pass the heap pointer directly, or does each String make a new copy? I am concerned about fragmentation or other fun ways of running out of RAM even if one copy should in theory fit.
Does doing something like bool myFunction(String &str){...} help? Some sort of way to actually ensure that it's the same string, rather than copying (and can I do String &myStr=server.arg(...) -- is a return value different from an assignment)? It's confusing for me, since strings are clearly passed by value, despite being a complex datatype with dynamic memory somewhere in there. So, did they override the assignment operator (and/or the one that handles reference passing, since I think there's more than one) to duplicate the string as pass-by-value of a basic type would normally do, or do they share pointers (and thus memory) as a class passed-by-value would normally do?

On the ESP8266 flash is a very special case. Because the ESP8266 itself does not have any flash, but instead uses an external SPI-connected flash chip, accessing it is not an easy or technically obvious operation.
Reading from the flash chip is slow. Very very slow (by comparison to reading RAM).  Because of this where possible certain things are copied into RAM. Also, there are different sections of RAM that are dedicated to different purposes.

dram0 - This is your "Data" RAM. It's where your variables and such are stored (80kB total).
iram1 - Instruction RAM. It's here that interrupt routines need to be stored for rapid execution (32kB).
icache - Instruction cache. 16kB of data that is cached when reading from the flash chip. Executed code usually gets copied here.

Normally when you execute code the block of code that the instructions you are executing are copied from flash into the icache memory block and then executed. This is reasonably fast (reasonably in that there may be brief delays while copying the data from flash, but fast when actually executing. I don't know how efficient the cache is, whether it's split into different blocks that are loaded in the background by the memory controller as needed, but it seems to be a pretty efficient system that is used in practice).
Because of this disassociation of the CPU and flash string constants, unless specified otherwise, will be copied into dram0 during startup. This makes it very fast and simple to get at the data. Anything that you don't specify as PROGMEM will get copied into dram0. Note that you don't get all 80kB of the RAM to play with since the SDK uses a portion of it. You only get about 50kB at most (station mode - softap mode will use more).
Now... what does PROGMEM do on the ESP8266 then?  Well, it adds an attribute to the definition that tells the linker to place the data in a specific section in flash (.irom.text.<filename>.<line>.<number>). This prevents it being copied into dram0 at startup.
So how do you get at it? That's where the *_P variants of the string functions (and certain overloaded functions in the Arduino core) come into play.
The whole of the flash chip is mapped (or can be, depending on register settings) into the ESP8266's memory map (address 0x40200000). Which means that it's possible to get directly at the data. However since this access doesn't go through the icache RAM block it's slow. Very very slow. About 12x slower by some accounts. Also because data can be in different locations within the chip the addresses of your string data can't be directly hard coded. There has to be some form of "it's over there" redirection when reading the data (for example if you use OTA updates you get two distinct blocks of flash where your code can be stored - the currently executing code and the previously executing version). The *_P variants of the string functions take that into account and look up where in flash the current code is located before reading from it.
So to summarise: Not using PROGMEM gives you fast access to your string data at the cost of RAM, and limits the size of the data you can use. Using PROGMEM removes the size restriction, doesn't use RAM, but is considerably slower.
To specifically answer your questions:

The string data gets copied to RAM at startup.
The string data gets copied to RAM at startup.
The string data gets loaded from flash (using *_P variants of functions) when the String object is constructed (RAM is allocated on the heap at that moment in time). The RAM is freed when the String object is destroyed (goes out of scope).
The compiler has recently had an update that will deduplicate constant PROGMEM strings.
As mentioned above *_P copies directly from flash and is slow, but has no size limitation.

That was very helpful. Annoying that it's not a Harvard system but has the same end effect... :D The one thing I'm still not sure on was when the web server read the PROGMEM'd web page without need for the _P function. But given that it has that option, I'd suspect it streams the data from flash for the latter, and the former makes a RAM copy first, which of course failed because the page is very large.

@RDragonrydr A function doesn't  have to have _P to read from flash - that's just a naming convention in the C library. The web server has multiple overloaded functions that accept different data types - one of which is a "flash string".

Yeah, that's what I had noticed. I figure it's a fair warning that they seem to have optimized the _P version, though. The other works, right up until you try to serve a really big page and it abruptly fails, probably due to an inability to allocate a buffer somewhere. I had to switch to the dedicated progmem version despite the compatibility because of that.

Does string de-duplication happen at all? I saw this (https://forum.arduino.cc/index.php?topic=194603.0) and wondered if that has been implemented for either system or if the "compiler changes" mentioned would have been added.

@RDragonrydr I have just run some tests and yes, it looks like it does now do string deduplication for const char *foo PROGMEM variables.  I just made a little sketch and with two differing strings ("this is foo" and "this is bar") they end up pointing to different locations. With both strings having "this is foo" in them they both point to the same address.


text:
How to stop loop in the serial monitor?
I want to make a serial program with input from the serial monitor and show each character with its binary and ASCII codes.
At the end of the program, I want to show the whole message sent from the serial monitor but I don't know how to stop the loop. Can someone help me please?
Here's my program:
String dataST;

void setup() {
  Serial.begin(9600);
  Serial.println("Message sent by serial monitor : ");
}

void loop() {
  for (int i = 0; i < 400 ; i++) {
    if (Serial.available()) {
      char dataOK = Serial.read();
      dataST += dataOK;
      Serial.print(dataOK);
      Serial.print(" = ");
      delay(200);
      for (int i = 7; i >= 0; i--) {
        byte bytes = bitRead(dataOK, i);
        Serial.print(bytes, BIN);
      }
      Serial.print(" = ");
      int dataAS = dataOK;
      Serial.print(dataAS);
      Serial.println(" ");
      Serial.println(" ");
    }
  }
  Serial.println("Message : ");
  Serial.print(dataST);
  dataST = "";
}

The result from the program above is:
Message sent by serial monitor :
Message :
Message :
Message :

And the result I want is:
Message sent by serial monitor:
H = 01001000 = 72

e = 01100101 = 101

y = 01111001 = 121

Message :
Hey

Please help me.

Please take a look at the documentation of SerialEvent.
This might be exactly what you are looking for (after several small tweaks)
Code from the example:
/*
  Serial Event example

  When new serial data arrives, this sketch adds it to a String.
  When a newline is received, the loop prints the string and clears it.

  A good test for this is to try it with a GPS receiver that sends out
  NMEA 0183 sentences.

  NOTE: The serialEvent() feature is not available on the Leonardo, Micro, or
  other ATmega32U4 based boards.

  created 9 May 2011
  by Tom Igoe

  This example code is in the public domain.

  http://www.arduino.cc/en/Tutorial/SerialEvent
*/

String inputString = "";         // a String to hold incoming data
bool stringComplete = false;  // whether the string is complete

void setup() {
  // initialize serial:
  Serial.begin(9600);
  // reserve 200 bytes for the inputString:
  inputString.reserve(200);
}

void loop() {
  // print the string when a newline arrives:
  if (stringComplete) {
    Serial.println(inputString);
    // clear the string:
    inputString = "";
    stringComplete = false;
  }
}

/*
  SerialEvent occurs whenever a new data comes in the hardware serial RX. This
  routine is run between each time loop() runs, so using delay inside loop can
  delay response. Multiple bytes of data may be available.
*/
void serialEvent() {
  while (Serial.available()) {
    // get the new byte:
    char inChar = (char)Serial.read();
    // add it to the inputString:
    inputString += inChar;
    // if the incoming character is a newline, set a flag so the main loop can
    // do something about it:
    if (inChar == '\n') {
      stringComplete = true;
    }
  }
}

An embedded system is never meant to stop. If/when your loop() function exits (returns), it will be immediately called again.
If you need a process that runs to some state of completion and than stops, you're better off to leave loop() empty and write your own function that does what you need it to do, and call it at the end of setup().
You can either put an infinite loop at the bottom of your function:
for(;;);

is one way to do it, or just let it return to setup(). Then setup() will exit, your empty loop() function will be called (infinitely), which accomplishes the same thing.

Or sleep (most of the time)?

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
16 bit PWM doesn't go to absolute 0V
void setup() {
  Serial.begin(9600);
  setupPWM16();
}

uint16_t icr = 0xffff;

void loop() {
  analogWrite16(9, 0);
}
void setupPWM16() {
  DDRB  |= _BV(PB1) | _BV(PB2);       /* set pins as outputs */
  TCCR1A = _BV(COM1A1) | _BV(COM1B1)  /* non-inverting PWM */
        | _BV(WGM11);                 /* mode 14: fast PWM, TOP=ICR1 */
  TCCR1B = _BV(WGM13) | _BV(WGM12)
        | _BV(CS10);                  /* prescaler 1 */
  ICR1 = icr;                         /* TOP counter value (freeing OCR1A*/
}

/* 16-bit version of analogWrite(). Works only on pins 9 and 10. */
void analogWrite16(uint8_t pin, uint16_t val)
{
  switch (pin) {
    case  9: OCR1A = val; break;
    case 10: OCR1B = val; break;
  }
}

Using the code above I can achieve 16 bit PWM signal, but when writing 0 in analogWrite16 it output ~33mV. I know that I can set the PWM pin low but yet again it don't output 0V.
I'm guessing beside the ATMEGA offset the Arduino UNO board it self ground is raised either because it's being fed from USB or the board circuitry and PCB traces resistance. or both?
because ~33mV offset is a lot, is it normal?
How can I make Arduino to start at 0V or at least reduce that 33mV offset to 1-2mV?

You've got two possibilities:

You can't have full off in "non-inverting PWM" mode
You can't have full on in "inverting PWM" mode

In the Arduino core it's solved by turning off the PWM output and setting logical 0 or 1 for corner values.
And if you need full off but you don't need full on, you can use inverted PWM mode.

How do I use inverted mode? is there a code that I have to modify?

You just have to change COM1A/COM1B (it's in the datasheet) values and compute correct value for OCR1A/OCR1B (something like 0xFFFFU - val)

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Using serial communication in .c-files
I am currently trying to connect my Arduino Uno to my computer and let them communicate via UART. I started using a project from Github (here), which I'm trying to modify to get the next step via UART. Because of that, I'm not able to switch programming languages and also need to override the bootloader.
However, I can't get the serial connection to work.
#include "Joystick.h"
#include <avr/io.h>
#include <util/delay.h>
#include <Arduino.h>

These are the #include statements at the top, but <Arduino.h> can't be found...
If I remove these statements, Serial can't be found (from the main-Method: Serial.begin(9600);)
It's my first time to program with C, did I overlook a stupid mistake? Thanks for any help...

Are you using the Arduino IDE to compile this?

@chrisl No, I'm using the makefile that is contained in the GitHub project. I modify the $PATH variable every time to contain arduinoInstallDir/hardware/tools/avr/bin, which includes the avr-gcc needed for it to compile.

You can very easily combine C and C++ code. Just compile your parts of the code as C++, and include the existing C functions you need as extern "C". If you need to call your C++ function from C code, declare it extern "C" as well. You can then use all of the C++ features from the Arduino core if you want.

@tttapa I've tried my best to work out how that works, but I can't get it. I found examples for how to include one method, but not for an entire library. Is also thought about converting that whole project to C++, but that also won't work as I don't fully understand it. If you could give me an example on how to include a full library like <Arduino.h> in a C file and make it still usable, I would be really thankful.

I started to look into it, but it seems to me that the code is intended to run on the ATmega16U2 of the UNO, not the ATmeg328P. In that case, I don't think there's an Arduino Core available for that chip, so you would have to program the USB CDC stack yourself (should be possible using LUFA, but I don't have the time to look into it). And IMHO, it's absolutely not worth any more time. I'd use a microcontroller that supports USB natively, and has decent software support, like a Teensy 3.x.

@tttapa The project I linked from discord already does what I want, but now I want to extend that program to read the next button press from a serial connection with a PC, but to read from serial, I have to use the library from Arduino which I can't access. So the USB Connection thing is no problem.

I don't think you understood my comment. The UNO has 2 microcontrollers, one can run Arduino code (ATmega328P), the other can not (ATmega16U2). You cannot use Serial.begin on this second microcontroller. The reason that you cannot write Arduino code for it is because nobody has written an Arduino Core library for it. You can do that yourself, you can probably recycle a lot of code from the ATmega32U4 core, but that would be a waste of time, IMHO, and it will probably conflict with the existing LUFA code you have.

@tttapa Would it maybe be possible to let the first controller (ATmega 328P) receive the data from serial and pass them on to the ATmega16U2?

The ATmega16U2 has a UART you have access to, so you could use that, but I thought you wanted Serial over USB?

@tttapa The code from GitHub already uses the USB-Port, so I thought I could put the other communication onto the UART connection. So I'm currently trying to let the 16U2 read from UART, and pass that data on via USB.

I would consider using HoodLoader2 instead of that LUFA project. If you can't do that, you can compile parts of the Arduino Core together with the LUFA project.
The main steps:

Add the necessary Arduino source files to the SRC variable in the Makefile.
Add the Arduino core folder to the include path by appending to the CC_FLAGS and CXX_FLAGS variables.
Move the main function to a new main.cpp file and add it to the Makefile SRC as well.
Include the Joystick.h file in your main file in an extern "C" block.
Include the HardwareSerial.h file in your main file as well.

My files after these changes:
Makefile
# Set the MCU accordingly to your device (e.g. at90usb1286 for a Teensy 2.0++, or atmega16u2 for an Arduino UNO R3)
MCU          = atmega16u2
ARCH         = AVR8
F_CPU        = 16000000
F_USB        = $(F_CPU)
OPTIMIZATION = s
TARGET       = Joystick
SRC          = $(TARGET).c Descriptors.c main.cpp $(LUFA_SRC_USB)
LUFA_PATH    = ./lufa/LUFA
CC_FLAGS     = -DUSE_LUFA_CONFIG_HEADER -IConfig/
LD_FLAGS     =

ARDUINO_PATH = ${HOME}/.arduino15/packages/arduino/hardware/avr/1.8.1/cores/arduino
SRC         += $(ARDUINO_PATH)/HardwareSerial.cpp
SRC         += $(ARDUINO_PATH)/HardwareSerial1.cpp
SRC         += $(ARDUINO_PATH)/Print.cpp
CXX_FLAGS   += -I$(ARDUINO_PATH) -I./HoodLoader2/avr/variants/HoodLoader2
CC_FLAGS    += -I$(ARDUINO_PATH) -I./HoodLoader2/avr/variants/HoodLoader2

...

main.cpp
extern "C" {
#include "Joystick.h"
}

#include <HardwareSerial.h>

// Main entry point.
int main(void) {
    // We'll start by performing hardware and peripheral setup.
    SetupHardware();
    // We'll then enable global interrupts for our use.
    GlobalInterruptEnable();
    // Initialize the UART
    Serial1.begin(115200);
    Serial1.println(F("Hello, world"));
    // Once that's done, we'll enter an infinite loop.
    for (;;)
    {
        // We need to run our task to process and deliver data for our IN and OUT endpoints.
        HID_Task();
        // We also need to run the main USB management task.
        USB_USBTask();
    }
}

Joystick.c
Deleted the main function.
To get it to compile after these changes:
# (inside of the snowball-thrower folder)
git submodule add https://github.com/NicoHood/HoodLoader2.git HoodLoader2
make

This compiles without problems for me, but I have no idea if it actually works.
Edit:
You'll notice that the program uses more RAM than the ATmeg16U2 has available. You could solve this by moving the step array to PROGMEM, or by shrinking the Serial buffers.
avr-size --mcu=atmega16u2 --format=avr Joystick.elf
AVR Memory Usage
----------------
Device: atmega16u2

Program:    5076 bytes (31.0% Full)
(.text + .data + .bootloader)

Data:        527 bytes (102.9% Full)
(.data + .bss + .noinit)

 [INFO]    : Finished building project "Joystick".

I've tried that, but I can't confirm that the program really started. I've connected an UART-to-USB-Bridge and tested it before, but PuTTY doesn't output the starting test entry. Instead, the last thing that was programmed with Arduino gets executed and outputted to PuTTY... I've got around the RAM-Issue by just deleting the last part of the step-array, because I'm trying to completely avoid that by streaming the entries via UART. However, neither the starting output gets displayed nor the switch notices a new controller connected.

I need to correct myself: It get's noticed from the switch if I disconnect TX&RX while flashing the new .hex file. But still no output from PuTTY (and yes, I reconnect TX & RX after finished flashing the image). The connection/interaction with the switch is the same as before, so that is perfectly fine, but the Serial-Connection still does not work.

@MonsterDruide1 Upload an empty sketch to the ATmega328P first (using the original '16U2 Arduino firmware), or connect its reset pin to ground. And also swap your RX/TX pins. The RX of the UART-to-USB adapter should be connected to the RX pin of the Arduino (this is the TX pin of the '16U2), and TX to TX. Internally, the RX/TX pins are connected like this (ignore the ESP8266, imagine that it's the UART-to-USB adapter).

YES!!!! Thank you VERY much for all of that help. Everything now works as expected!


text:
Issues with connecting to an Adafruit FLORA board
I just bought a new Adafruit Flora board, and I haven't been able to get it working successfully. When I plug it into a USB port, the board seems to be getting power (the green PWR LED is illuminated), but there is no other output from the board. The Arduino IDE also does not recognize the board (the "port" menu under "Tools" is grayed out), and I have tried to connect to it on both Windows and Linux (Ubuntu) systems with the appropriate drivers/packages installed (including the Windows drivers and the board manager add-on in both installations of the Arduino IDE). I've also verified that the USB cable I'm using can handle data.
Previously, I was able to at least get the board to be recognized in the IDE on my Linux machine, and the red LED did light up, but after disconnecting it doesn't light up anymore (unless I try to reset the board, which makes the red LED pulse a few times).
Anything that I can try to get it recognized in the IDE?
Edit: I'm able to see the board pop up under "ports" in the IDE immediately after I hit the reset button on the board. However, as soon as the red LED stops pulsing (I think this means that the board is finished rebooting), it disappears from the list of available ports. It's also mounting under /dev/ttyACM0, not the usual /dev/ttyUSB0.

/dev/ttyACM0 is OK. upload Blink while the port is visible

@Juraj Yep, just tried this and it works. Strange. Thanks though!

it is not strange. the ATmega32u4 on board has a built in USB handled by the Arduino core linked with your sketch. if your sketch doesn't run, the USB is not handled. reset starts the bootloader which handles the USB while running.

Seems like whatever code was running on the Flora was corrupted. As soon as I hit the reset button, the board would momentarily pop up in the Arduino IDE, and after doing this a few times, I managed to upload the "Blink" sketch while the board was visible to my system. After that the board started working normally. It reports its name as "Adafruit FLORA" under "get board info", and the serial monitor/onboard LED work just fine.


text:
Measure current across shunt resistor (Power Analysis)
I am trying to reproduce the side-channel power analysis attack described in this paper: https://pdfs.semanticscholar.org/7946/a8ee82658e2ffb017270331f220a55f5ace6.pdf for educational purposes.
The attack (on the HCS3xx family of chips) requires measuring the power consumption (i.e. current) that the chip is using in order to reveal its cryptographic key.
In the paper, they describe their setup as attaching a shunt resistor in the ground path and using a digital oscilloscope (with a sampling rate greater than 1MHz) to measure the voltage drop across this resistor.
I have reproduced this setup to the best of my abilities (pictured in the schematic below), but the problem is I don't have an oscilloscope.

simulate this circuit – Schematic created using CircuitLab
Therefore, I've decided to use an Arduino Uno I have lying around as a poor man's oscilloscope.
Besides the fact that the Arduino ADC has a much lower sampling rate (~77KHz vs. the ideal 20MHz described in the paper), I have been having an issue getting a reliable voltage reading across the shunt.
While trying to use the full 10-bit ADC, and producing samples as (ADCH << 8) | ADCL in my ADC ISR, I do not get any samples that are greater than 0.
However, when I left-align the ADC to only use 8-bit precision (with ADMUX |= 1 << ADLAR and read with just ADCH, I get non-zero values but none with a value greater than one.
Could it be that the amount of power that the chip consumes is so small that the Arduino does not have enough precision to read it? Or could it be that my shunt has too large of a resistance and is causing a reduced current throughout the circuit?

In the graphs they've got 2.5mA tops, so you've got about 2mV on that 1.2R shunt resistor.
And Uno's default ADC reference is 5V divided into 1024 steps each of them roughly 5mV. That's more than peak value of measured voltage.
And with such small sampling frequency you won't be even able to spot fast changes as you'll be measuring much slower that it needs to be.
ADC step precision can be better by using internal 1V1 reference, but it's still way too big step. And there is no other option on UNO. The Mega on the other side has also possibility to configure diferential ADC inputs and with 1x, 20x or 200x gain. But it's not supported by Arduino core, so you'll have to do it directly. However it won't help much if it's sampling too slow.

Oh I totally neglected to look at the y axis on that graph. That's very helpful. I figured that the sampling rate would be too slow but I thought that I might be able to delay starting the measurements and basically interleave multiple power traces. Probably wouldn't have worked though considering the changing counter.


text:
What is the best way to replace the buttons inside a car remote chip so that it can be controlled via Arduino?
I have a car remote that looks like the following

I would like to replace the buttons with some sort of switch or relay that can be programmatically controlled via an arduino unit. Would a simple relay switch work if I de-solder the buttons? Any help would be appreciated. Thanks!

you do not have to remove the switches ... you can just bridge them with relay contacts ... it may also be possible to do this without relays .... measure the voltage between ground and each side of the switch ... use battery negative terminal as ground ... post the result

@jsotola Thanks, I will post it soon. May I ask what you are thinking with regards to no relays?

I currently have the device on me and am measuring the voltage. Should I be pressing the button when measuring voltage?

Thanks, I will do that, is the point is to find sets are the input/output terminals and how much voltage it is taking in? Should I contain a current or resistance reading as well?

I measured it, and found that the voltage was 2.97 V from a black probe placed on negative battery side and a red probe on one of the 4 contacts (the one across from it read 0). The voltage of the battery itself was found to be 3.01 V.  Thank you

the 2.97 V pin is most likely the data pin ... it probably has a pullup resistor to +V  .... pressing the switch shorts the input to ground .... you can use a transistor in an open-collector configuration to short the input to ground ... the switch does not need to be removed ... you could solder a wire to one of the round test pads

Will I create problems if I directly use a switch to bridge the two pins of the button without a resistor? Thank you for the config, how can i tell which test pad is the right one?

the button directly bridges the two pins when you press it   .......... there is only 10 test pads ... use the voltmeter and press a button

Thank you, I believe I found it. The battery provides 3 V, while my arduino has 3.3 V. Do you think using 3.3V instead of the battery will create issues?

probably nor, but it is not possible to be absolutely certain without seeing the datasheet for the control chip

If you put a resistor (100 to 1000 ohm) between the output of the Arduino and the 2.97 V side of the button, then you can pull the Arduino's output to ground to mimic a button press. In the worst case, the 3.3 V on Arduino's output and the 2.97 V on the switch would be connected through the, say 100 ohm, resistor. The current would be less than 3 mA, which both Arduino and the microcontroller on the fob should be able to handle. This assumes the switch was configured with a pull-up resistor on the fob, in which case its pull-up resistance would most likely be much higher than 100 ohm.


text:
Converting string to byte array and byte array to string?
I am trying to create a string message and then encrypt it via AES but the example only accepts byte array as input. How can I convert fullmessage string to message byte array to encrypt it? Checked similar questions but couldn't resolve the problem.
void loop ()
{
  dt = clock.getDateTime();
  String dash = "-";
  String comma = ",";
  String time = dt.year + dash + dt.month + dash + dt.day + dash + dt.hour + dash + dt.minute + dash + dt.second;
  String status = "OK";
  String fullmessage = time + comma + status;
  byte message[]= "anything";
  plainLength = sizeof(message)-1;
  padedLength = plainLength + N_BLOCK - plainLength % N_BLOCK;
  prekey(256,message);
  delay(2000);
}

do you get an error?

"cannot convert ‘String’ to ‘byte {aka unsigned char}’ in assignment" I get this when trying to put fullmessage directly as message. Now I was trying to use message[i] = fullmessage[i] in a for loop, seems like it worked.

strcpy(message, fullMessage.c_str()); allocate message as char message[MESSAGE_SIZE]; https://www.tutorialspoint.com/c_standard_library/string_h.htm


text:
aRest.io - how does it work?
With aRest.io you are able control your Arduino/Esp from all over the world - but how?
In the Client SourceCode (found here) you only find the PubSubClient, which is an MQTT client.
I'm assuming:
My Arduino can (during the aRest.h) subscribe to topics in my aRest.io dashboard (http://dashboard.arest.io/) and if I add something to the dashboard the aRest's Broker will inform my Arduino. But here is my error. How can the Broker contact my Arduino behind the firewall? Does the Arduino use hole punching? Is my attempt wrong?

The broker doesn't contact the Arduino - the Arduino contacts the broker.

So the Arduino is polling for commands ?

Not as such, no. It connects to the broker and sits there waiting for the broker to say something.

I have to test this tomorrow! Could it be really so easy... Thank you for the hint in the right direction

check this out ... visual representation of MQTT messages ... https://shiftr.io/shiftr-io/demo

The Client contacts the broker and
maintain the connection - waitung for topic to be updated.


text:
Improper functioning of Node mcu when connected to source other than PC
I connected EM18 rfid reader to NodeMCU to read rfid card data. It showed card number very quickly and consistently till it was connected to my PC's USB. But when I connected it to 5v power source (mobile phone charger) then it didnt read cards well i.e. sometimes it read and sometime dosent.
Thants very strange.
Grounds might not be the issue as EM18 is powered from NodeMCU and NodeMCU ultimately connected to source(mobile phone charger)
Code is given below :
int count = 0;
char input[12];
boolean flag = 0;

void setup()
{
   pinMode(LED_BUILTIN, OUTPUT);
   // begin serial port with baud rate 9600bps
   Serial.begin(9600);
}

void loop()
{
   if(Serial.available())
   {
      count = 0;
      // Read 12 characters and store them in input array
      while(Serial.available() && count < 12)
      {
         input[count] = Serial.read();
         count++;
         delay(5);
      }
      // turn the LED on (HIGH is the voltage level)
      digitalWrite(LED_BUILTIN, !digitalRead(LED_BUILTIN));
  }
}

Where did you see the card number.? Why are you sure it reads the card? Or did you mean the LED goes on, when you said it had read the card number? The only thing I can imagine is that if the MCU is connected to the PC, the if(Serial.available())  is true and can enter the block (for what reason ever). If you have the EM18 also connected to the same Hardware serial port (it seems so) then this connection might fail. check the setup or add a picture of you setup to the question.

Thanx for reply <3, and yeah you are right ! To check if card is read... in built LED is turned on or off as per above code. When I connect it to PC I have serial monitor thus i see card number and LED blinks accordingly. But when i connect it to power source then cards sometimes read and sometimes not in a single attempt because the in built LED doesn't  on or off to prove the same. It seems as the Tx pin got slowed down (just to imagine as this is so strange issue). If you need pictures I can post but there se nothing to show as the setup is very simple. EM18 is just connected to Nodemcu.

You see the card number in the serial terminal on the PC if this is connected. But you do not write the card number from the MCU to the PC.(at least if that is the original code) I fear you connected the EM18 to the RX TX pins that are also connected to the USB to Serial converter on the board. So  the PC can spy and read the card number. I never thought this is possible but it seems so.  You should use two serial ports one for the EM18 and the standard port for the PC. Hint: https://community.blynk.cc/t/how-to-use-2-serial-ports-or-more-in-nodemcu/20282

I will give it a try and will inform you shortly.


text:
How to fill a triangle using the LCD5110 Graph library?
Good day! I'm currently trying to create a small 3D engine using C++, an Arduino UNO as well as a Nokia 5110 display with a library. Now I have successfully drawn a cube using triangles, but I want to fill the triangles to add proper lighting. My method for drawing triangles is the following:
LCD5110 lcd(8,9,10,12,11);

void DrawTriangle(int x1, int y1, int x2, int y2, int x3, int y3) {
 lcd.drawLine(x1, y1, x2, y2);
 lcd.drawLine(x2, y2, x3, y3);
 lcd.drawLine(x3, y3, x1, y1);
}

My library hasn't got a triangle draw or fill method unfortunately. I have no idea how I could fill a triangle so I hope for help.
Thanks in advance...
NOTE:
Please keep it simple.
NOTE NR.2: I have already asked this question at StackOverflow but I haven't got a good answer so I will try it again.

It looks like nobody is answering...

That is most likely because there is no good answer. If your library doesn't provide a draw filled polygon method you have to draw it yourself. Ok, this is a standard algorithm in computer graphics but it's not easy to explain. E.g. have not enough time to think about how to explain it in a few words. But I can point you to an easy to understand algorithm that is called the "flood fill" algorithm. If I recall correctly there is a C++ implementation called FloodSpill.

I will take a look at FloodSpill.

As I said in the comments of you answer: If the given solution works for you and performance doesn't matter, your solution is much better than my proposal. But don't forget to add the } to the code below. As it might confuse other that want to use your solution.

After tinkering around for hours I have finally found a great way to fill triangles! Here we go:
for(int x = x1; x<=x2; x++) {
    for(int y = y1; y<=y2; y++) {
     lcd.drawLine(x3, y3, x, y);
    }
}

Great!

If you add the missing } it will work. But it's not very efficient. If performance doesn't matter, you are right: It's simple and it works.


text:
How to make VOLTE/LTE or LTE calls using Arduino?
I want to make VOLTE/LTE calls using Arduino. I searched everywhere in Google. At last I found a website LTE Cellular Shield for Arduino with Voice!.
But I cannot find SIM7500 Shield in online shops. So U want to use SIM7000. But I can't find code for it. Can you share any?

have you seen this? ... https://github.com/botletics/SIM7500-LTE-Shield?files=1

See but I can't understand anything

I want the code what I give

I connect SIM7500 to Arduino Uno. and connect speaker mic etc.Okk. Then I have to give a code to Arduino by arduino ide, right. I want this code which I give

now i understand ... you want the code that you load into the Arduino ... the link in my comment above has example code ... you can find it in the README.md

Code is not available on README.md

yes it is ...click on step-by-step wiki ... your internet search skills need some sharpening ... lol


text:
For what is used an ESC with a PWM?
So I was reading a few questions on drone, and I reach this one.
It said that an ESC is used between the PWM Arduinno's port and the motor. I don't really understand what it means. According to what I understand, a PWM is a varying signal and you just need an electric relay to receive this signal so that the motor receives correct voltage.
However, in other article I read that there are many different ESCs, with firmware and many other parameters to be taken into account. It seems quite overkill for what a relay does, so I am wondering:

What is inside an ESC?
More important, what does it do with PWM signal that could not be achieved with a relay and is essential to command the motor?

An ESC is an Electronic Speed Controller. It's purpose is to control the speed and direction of a Brushless DC Motor.
A Bushless DC Motor ("BLDM") is not like other motors. You can't just provide it power and it will spin. Instead it is made up of three (or multiples of three) electromagnets along with a permanent magnet on the rotor. Energising these electromagnets in the right order at the right time (and the right amount, too) pulls the permanent magnet around in a circle, making the motor spin.
It's the ESC that has the job of controlling these electromagnets.
You provide a PWM signal as a digital speed indication. The ESC then interprets that as an indication of how fast you want the motor to turn, and excites the electromagnets in the right way to make that happen.
Oh, and no, you certainly never want to provide a PWM signal to a relay. A PWM signal switches hundreds, if not thousands, of times per second. A relay is capable of switching, at best, maybe 10 or 20 times per second.  Relays don't drive motors. Motor drivers (or at the bare minimum, MOSFETs) drive motors.

You could get a brushed motor instead of the BLDM, so you don't need an ESC. But you'd still need something like a motor driver or MOSFET, since the Arduinos digital pins can supply enough current for a motor.

Thank you, exactly the answer I am looking for. I just wait for some people to see the question before accepting it. I have a question as a result of your explanation: is an ESC specific to a motor (I suppose the architecture of the electromagnets could influence the timeline of energise)?
And could you explain please the word "MOFSET"?

@totalMongot Metal Oxide Semiconductor Field Effect Transistor. Like an electronic switch. Yes, different motors have different characteristics, but in general they are only minor differences and the only key things you need to consider are the current (select an ESC that is powerful enough) and the maximum acceleration rate (which you control with how fast you change your PWM values).


text:
Esp8266 Arduino WiFi Exception Error 29
Background: I know nothing about hardware.
I am writing a small project with Arduino IDE, ESP8266 library, and my board is ESP8266 V3 ESP-12N F NodeMcu Lua CP2102.
Now I get Exception 29 and I don't know why.
Sketch and server code(written in Python) github link (I also posted the client and server code at the bottom)

Module: [NodeMCU 1.0 (ESP-12E Module)]
Flash Size: [4MB]
lwip Variant: [v2 Lower Memory]
Flash Frequency: [40Mhz]
CPU Frequency: [80Mhz]
Upload Using: [SERIAL]
Upload Speed: [115200]

Stack Dump:
Exception 29: StoreProhibited: A store referenced a page mapped with an attribute that does not permit stores
PC: 0x4000e1c3
EXCVADDR: 0x00000018

Decoding stack results
0x40100908: malloc(size_t) at C:\Users\ptq00\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.6.3\cores\esp8266\umm_malloc\umm_malloc.cpp line 511
0x40100b6e: calloc(size_t, size_t) at C:\Users\ptq00\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.6.3\cores\esp8266\umm_malloc\umm_malloc.cpp line 826
0x40100b60: calloc(size_t, size_t) at C:\Users\ptq00\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.6.3\cores\esp8266\umm_malloc\umm_malloc.cpp line 820
0x4020318b: loop_task(ETSEvent*) at C:\Users\ptq00\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.6.3\cores\esp8266\core_esp8266_main.cpp line 188
0x40100154: ets_post(uint8, ETSSignal, ETSParam) at C:\Users\ptq00\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.6.3\cores\esp8266\core_esp8266_main.cpp line 160
0x40100154: ets_post(uint8, ETSSignal, ETSParam) at C:\Users\ptq00\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.6.3\cores\esp8266\core_esp8266_main.cpp line 160
0x40100154: ets_post(uint8, ETSSignal, ETSParam) at C:\Users\ptq00\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.6.3\cores\esp8266\core_esp8266_main.cpp line 160
0x40100154: ets_post(uint8, ETSSignal, ETSParam) at C:\Users\ptq00\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.6.3\cores\esp8266\core_esp8266_main.cpp line 160
0x40100154: ets_post(uint8, ETSSignal, ETSParam) at C:\Users\ptq00\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.6.3\cores\esp8266\core_esp8266_main.cpp line 160
0x40100154: ets_post(uint8, ETSSignal, ETSParam) at C:\Users\ptq00\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.6.3\cores\esp8266\core_esp8266_main.cpp line 160
0x40100154: ets_post(uint8, ETSSignal, ETSParam) at C:\Users\ptq00\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.6.3\cores\esp8266\core_esp8266_main.cpp line 160
0x40100154: ets_post(uint8, ETSSignal, ETSParam) at C:\Users\ptq00\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.6.3\cores\esp8266\core_esp8266_main.cpp line 160
0x40100154: ets_post(uint8, ETSSignal, ETSParam) at C:\Users\ptq00\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.6.3\cores\esp8266\core_esp8266_main.cpp line 160
0x40100154: ets_post(uint8, ETSSignal, ETSParam) at C:\Users\ptq00\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.6.3\cores\esp8266\core_esp8266_main.cpp line 160
0x402109ff: tcp_output at core/tcp_out.c line 1259
0x4020216a: ClientContext::wait_until_sent(int) at C:\Users\ptq00\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.6.3\libraries\ESP8266WiFi\src/include/ClientContext.h line 331
0x40100908: malloc(size_t) at C:\Users\ptq00\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.6.3\cores\esp8266\umm_malloc\umm_malloc.cpp line 511
0x402151c0: mem_malloc at core/mem.c line 210
0x4020f104: pbuf_alloc_LWIP2 at core/pbuf.c line 284
0x40214732: ip4_output_if at core/ipv4/ip4.c line 1550
0x4020fe90: tcp_write at core/tcp_out.c line 263
0x402031dc: __esp_yield() at C:\Users\ptq00\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.6.3\cores\esp8266\core_esp8266_main.cpp line 100
0x4020378e: __delay(unsigned long) at C:\Users\ptq00\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.6.3\cores\esp8266\core_esp8266_wiring.cpp line 54
0x4020230f: ClientContext::_write_from_source(DataSource*) at C:\Users\ptq00\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.6.3\libraries\ESP8266WiFi\src/include/ClientContext.h line 464
0x4020239d: WiFiClient::write(unsigned char const*, unsigned int) at C:\Users\ptq00\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.6.3\libraries\ESP8266WiFi\src/include/ClientContext.h line 364
0x40204c30: WiFiClient::write(unsigned char) at C:\Users\ptq00\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.6.3\libraries\ESP8266WiFi\src\WiFiClient.cpp line 217
0x40201bc4: DataSource::~DataSource() at C:\Users\ptq00\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.6.3\libraries\ESP8266WiFi\src/include/DataSource.h line 12
0x402026c9: WiFiClient::flush(unsigned int) at C:\Users\ptq00\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.6.3\libraries\ESP8266WiFi\src\WiFiClient.cpp line 318
0x40201113: loop() at C:\Users\ptq00\Documents\Arduino\sketch_jan03b/sketch_jan03b.ino line 44
0x40100154: ets_post(uint8, ETSSignal, ETSParam) at C:\Users\ptq00\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.6.3\cores\esp8266\core_esp8266_main.cpp line 160
0x402032f4: loop_wrapper() at C:\Users\ptq00\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.6.3\cores\esp8266\core_esp8266_main.cpp line 180

Sketch(Client) Code:
#include <ESP8266WiFi.h>

const char ssid[] = "luminous";
const char pass[] = "zxcv78()[]";

//Mac IP 192.168.1.101
//PC IP 192.168.1.103

IPAddress server(192,168,0,103);
int port = 12700;

WiFiClient client;

void setup() {
  // put your setup code here, to run once:

  Serial.begin(115200);
  Serial.println("");

  WiFi.begin(ssid, pass);
  delay(500);
  while(WiFi.status() != WL_CONNECTED){
    delay(500);
    Serial.print(".../");
  }

  Serial.println("");
  //WiFi is connected

  if(client.connect(server, port)){
    Serial.println("connected to server");
  }

}

// 3s client read timeout. I set this value casually. The server has a 5s timeout when reading data from client.
const unsigned long clientTimeout = 3000;

void loop() {
  // put your main code here, to run repeatedly:

  if(client.connected()){
    client.write(WiFi.RSSI());
    delay(100); // Add for nothing. Not sure if it's useful
    unsigned long timeout = millis();
    //Client timeout, if client can't reach the server, it should start to reconnect the server
    while(client.available() == 0){
      if (millis() - timeout >= clientTimeout){
        Serial.println(">>> Client Timetout !");
        client.stop();
        break;
      }
    }
    //Client is connected and has data to read
    //discard all data
    client.flush();
  }else{
    //Try to reconnect to the server
    client.connect(server,port);
    delay(500); // Add for nothing. Not sure if it's useful
    while(!client.connected()){
       //failed to reconnect
       Serial.println("Failed to reconnect...");
       delay(500);
       client.connect(server,port);
    }
    //succeed reconnecting
    Serial.println("Succeeded reconnecting..");
  }
}

Server Code:
import socket

HOST = '0.0.0.0'
PORT = 12700

catSafeZone = False
serverTimeout = 3

with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
    s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1);
    s.bind((HOST, PORT))
    s.listen()
    print("Start listening")
    print("Device is not conncted")

    while True:
        conn, addr = s.accept()
        try:
            with conn:
                print("Connected by", addr)
                print("Cat is online")
                conn.settimeout(serverTimeout)

                # client won't try to disconnect on its own.
                # So we only need to capture connection timeout exception
                while True:
                    data = conn.recv(128)
                    if data:
                        if catSafeZone == False:
                            catSafeZone = True
                            print("Cat is in safe zone.")
                        conn.send(b'1')
                    else:
                        print("Unexpected condition")
                        print("Client disconnected on its own.")
                        raise Exception("server data NULL")

                    # Code that is used for wifi signal strength detection:
                    #
                    # if not data:
                    #     break
                    # wifiStrength = int.from_bytes(data, byteorder='big', signed=True)
                    # print(wifiStrength)

        except socket.timeout:
            print("!!!Cat is out of safe zone")
            # Warn user in some ways
            catSafeArea = False

# never reach here
print("Server shutdown")

First: If you haven't already; please change your AP password, as you have included it in the code. Are your neighbors all friends? ;-) It seems as if the MCU tries to store data in a page where it is not allowed. But I confess, I see nothing in the code that could cause this behavior. Does it occur every time you run the code or is it a race condition? From the stack trace it seems as if the stack is used by multiple threads or interrupt routines. Perhaps two allocation accesses disturb each other. Because LUA is active that might also interfere with your code. Just guesses.

Ah, wait I've seen something: I add an answer because I use code to explain.

I fixed the problem. It turned out to be that the flush function in Arduino WiFi library has toally different meaning from the one that defined in ESP8266WiFi librarry.
From https://arduino-esp8266.readthedocs.io/en/latest/esp8266wifi/client-class.html:

flush() returning true indicates that output data have effectively
  been sent, and false that a timeout has occurred.

This acts more like a flush stdout buffer in a typical C program.
Then I replaced client.flush() with:
 while(client.connected() && client.available() > 0){
    client.read();
  }

And everything works well then.

And I use your code with a little bit modification.
#include <ESP8266WiFi.h>

const char ssid[] = "luminous";
const char pass[] = "xxxxxxxx";

IPAddress server(192,168,0,101);
int port = 12700;

WiFiClient client;

void setup() {
  // put your setup code here, to run once:

  Serial.begin(115200);
  Serial.println("");

  WiFi.begin(ssid, pass);
  delay(500);
  while(WiFi.status() != WL_CONNECTED){
    delay(500);
    Serial.print(".../");
  }

  Serial.println("");
  //WiFi is connected

  if(client.connect(server, port)){
    Serial.println("connected to server");
  }else{
    Serial.println("Failed to connnect to server at setup().");
  }

}

int needToReconnect = 0;

void loop() {
  // put your main code here, to run repeatedly:
  while(!client.connected()){
       //failed to reconnect
       needToReconnect = 1;
       Serial.println("Failed to reconnect...");
       delay(500);
       client.connect(server,port);
  }

  // Use a flag to indicate if client was in reconnect procedure
  if(needToReconnect){
    Serial.println("Succeeded reconnecting...");
    needToReconnect = 0;
  }

  client.write(WiFi.RSSI());
  delay(100); // Add for nothing. Not sure if it's useful

  while(client.connected() && client.available() > 0){
    client.read();
  }

}

In your code there is a possibility that you stop the client and then flush it after. That could lead to the observed error.
// Client timeout, if client can't reach the server,
// it should start to reconnect the server
while(client.available() == 0)
{
  if (millis() - timeout >= clientTimeout)
  {
    Serial.println(">>> Client Timetout !");
    client.stop(); // <----- HERE is the STOP !!!!!!
    break;
  }
}
//Client is connected and has data to read
//discard all data
client.flush(); <-- Then a flush on a stopped client !!!!!

I have no chance to try the following code, so use it as an example of what I would suggest you could try out. Attention: I've grayed out the last IP number and the SSID and the password.
#include <ESP8266WiFi.h>

const char ssid[] = "XXXXXXXXXXXXXXX";
const char pass[] = "XXXXXXXXXXXXXXX";

IPAddress server(192,168,0,XXX);
int port = 12700;

WiFiClient client;

void setup()
{
  Serial.begin(115200);

  WiFi.begin(ssid, pass);
  delay(500);
  while(WiFi.status() != WL_CONNECTED)
  {
    delay(500);
    Serial.print(".../");
  }

  // New line after .../.../...
  Serial.println("");

  if( client.connect(server, port) )
  {
    Serial.println("connected to server");
  }

}

// 3s client read timeout. I set this value casually.
// The server has a 5s timeout when reading data from client.
const unsigned long clientTimeout = 3000;

void loop()
{
  while( ! client.connected() )
  {
     //failed to reconnect
     Serial.println("Failed to reconnect...");
     client.connect(server,port);
     delay(500);
  }

  client.write(WiFi.RSSI());

  // Add for nothing. Not sure if it's useful
  // PPK: Me too ;-)
  delay(100);

  unsigned long timemark = millis();
  // Client timeout, if client can't reach the server,
  // it should start to reconnect the server
  while( client.available() == 0 )
  {
    if (millis() - timemark >= clientTimeout)
    {
      Serial.println(">>> Client timed out !");
      client.stop();
      break;
    }
  }

  if ( client.connected() && client.available() > 0 )
  {
    // remove bytes in the read buffer
    client.flush();
  }
}

I still get Exception 29 error with your code

Thank you @Peter. I fixed the problem by checking the 8266 documentation and the source file shown in stack dump. And I use your code which is better than mine. :p

Oh. I also delete the whole client timeout procedure, which I think that while(client.connected()) has already done the work.

@Rick Your welcome. As I said, I could not try the code, I had just an idea ;-). I appreciate your answer. I confess I did not realize that you have to use read instead of flush. So I learned a bit more, thank you. I hope you check your answer as the accepted one when it is possible.


text:
DS3231 High Power Consumption
I plan to use my Arduino with a small battery all the time so I don't want it to use so much power. I was checking power consumption of DS3231 with a multimeter, at first it was fluttering around 1 mA and 0.4 mA with 3.3 volts. I removed the battery on DS3231 to reduce affect of charging but now it's still high, about 0.4 mA. The official paper says it should use a lot lower than that. Is it an error about my multimeter or is there something I missed like a configuration or something?

I think the data sheet talks about the current consumption from the battery. That mean, how much the module draws from the battery if it not connected to a controller. If it is connected to a controller and communicates with it, it needs more current which it draws from the controller. and not from it's own buffer battery.

Yes I connected its power to a digital pin now.


text:
MCU to MCU communication over UART
What would be the best way to send the following information from one MCU to another?
Now I'm just sending the sentence over as is eg. Serial.print(sentence in here)
Example sentence: 1306,60.123456,50.123456,23.765382,40.897558,245,85,20.3,5,65,1500,0.9
Many thanks.

It depends actually on your requirements what is 'best'.
If (transmission) speed is an issue, there are mainly some things you can do to improve this:

Increase the transmission speed (this depends on the distance of the wires and the noise).
Decrease the start/stop bits (this reduces the certainty the values will arrive ok)
Pack your data (takes more processing time possibly, and more programming work, but can be faster).

About the third option: since you are mostly sending values, you can make some improvements. So you can send them as float (by sending the float data, with the length of sizeof(float)). However, note there could be inaccuracies in the last digit.

Well I'm now at 9600baud and what I want to achive is less air time. How would one pack the values?

For example like I said, instead of serial.print, send it as floats

using Serial.print(floatVariable);

And boost the serial speed to 115200, will be 12x faster.

@CrossRoads if the noise/distance allows it, that's an easy improvement


text:
ESP32 seconds from epoch
When using ESP8266 - getting seconds from epoch is done using
NTP.begin(NTPserver, 2, true);
delay(delay_tries);
time_t t = now();

while t stores amount of seconds from 1-1-1970.
When trying to do the same, using ESP32 - I cant get that numeric representation.
code below is from ESP32 example- updating time using NTP server ( process is done correct ). BUT when using now() yields 1970, as if NTP update has never occured.
How can I get time from Epoch ?
#include <WiFi.h>
#include "time.h"
#include <TimeLib.h>    <------- Added to original Code

const char* ssid       = "YOUR_SSID";
const char* password   = "YOUR_PASS";

const char* ntpServer = "pool.ntp.org";
const long  gmtOffset_sec = 3600;
const int   daylightOffset_sec = 3600;

void printLocalTime()
{
  struct tm timeinfo;
  if(!getLocalTime(&timeinfo)){
    Serial.println("Failed to obtain time");
    return;
  }
  Serial.println(&timeinfo, "%A, %B %d %Y %H:%M:%S");
}

void setup()
{
  Serial.begin(115200);

  //connect to WiFi
  Serial.printf("Connecting to %s ", ssid);
  WiFi.begin(ssid, password);
  while (WiFi.status() != WL_CONNECTED) {
      delay(500);
      Serial.print(".");
  }
  Serial.println(" CONNECTED");

  //init and get the time
  configTime(gmtOffset_sec, daylightOffset_sec, ntpServer);
  printLocalTime();

  //disconnect WiFi as it's no longer needed
  WiFi.disconnect(true);
  WiFi.mode(WIFI_OFF);
}

void loop()
{
  delay(1000);
  printLocalTime();
  Serial.println(now());   <-------- Added to original code
}

where is now() defined or even used in the code you posted?

Does your code output "Failed to obtain time"?

try time(nullptr) instead of now() - since now() is not even a valid function in the code you posted

you can use the NTP library on ESP32 and you can use the SDK time functions on esp8266 too

@JaromandaX - sorry, now with 2 additional line to original code

@Juraj - Also with #include <NTPClient.h> and timeClient.begin();     timeClient.setTimeOffset(3600); yeilds the same result ( time is updated, but cant get epoch )\

@Guy.D - just because you added now() to the code you've shown, doesn't make it valid (if I try to use now() in ESP32 code, it doesn't even compile :p

to return 'epoch' seconds there is a function time in time.h
time_t now;
time(&now);

time_t is defined as long
this time is retrieved by the ESP32 SDK from NTP servers configured with  ESP32 Arduino function configTime
EDIT: it is the standard C time() function

since time also returns a time_t you can use it like now() ... i.e.  time(nullptr);


text:
How much time passes between waking up and sleeping?
I use the watchdog timer to make Arduino sleep for 8 seconds and then wake it right after and make it sleep again if it didn't complete the loop for 450 times. I made it so that it will only do its job every 1 hour with a for loop. How much time does the Arduino keep awake until it sleeps again? I think it wouldn't be worth to put power consumption of this to calculation but still need to verify.

you said 8 seconds every hour ... what are you asking?

I mean right after 8 seconds of sleeping it wakes up and sleeps again if it didn't complete loop for 450 times.

That time totally depends on what exactly you're doing when waking up. If you have a delay(10000); in there, it will be more awake than sleeping. How does it check that the loop didn't complete for 450 times?

I tend use 1ms to wake up and go back to sleep in my calculations.

it keeps awake the length of the timeout every hour .... your question still makes no sense ... you are describing a fixed timeout ... within that timeout you count the number of iterations ... you said nothing about what happens it 450 iterations are reached

@jsotola I am asking the delay of sleeping. It must wake up after 8 seconds because of watchdog's limit. After waking up, it will go right into another sleep for 8 seconds. How many seconds or milliseconds pass between not sleeping state and sleeping state is what I ask about.

why is this description different from your posted question? ... are you certain that this is your real question? ... please edit your question above .... anyway, add code that lights up an LED at wakeup time and turns off at sleep time ... run the code ... record a video of the LED .... measure the time of 100 LED cycles ... that equals to 800 seconds of sleep plus the "on time"


text:
Why do people use 115200 instead of 9600?
So I came across lots of Arduino program setting serial speed to 115200 instead of 9600, while many people just kept using 9600 for the same purpose.
Why did people choose to use 115200 or other available speed? Why not just use 9600?

Because they want to communicate faster, or because the other end point is setup to 115200, and they don't want to change it. Or They use UART for MIDI with 31250 baud.

Because choice. 115200 is faster than 9600.

I sometimes use 460800 because it does slow down the sketch less then 115200 baud. and to communicate with esp8266 AT firmware over hardware serial on Mega I use 500 000 baud, to read the data from WiFi as fast as possible

@Juraj Do you know the reason why 460800 makes it less slowing down? related to multiplication values maybe?)

@MichelKeijzers, the TX buffer is 63 bytes long. if it fills, write/print blocks. so the baud rate is 'reflected' to the sketch. if I want to see the debug messages, but the problem is timing related, then sometimes higher baud rate helps

@Juraj Ah ok, thought it was some 'magic' number.

I didn't make a complete table on highest baud rate over different USB chips, but 921600 baud didn't work when I tried

@MichelKeijzers Exactly, why not using a faster speed like Juraj suggested? You see, you just contradicted your sentence.

@Juraj / ShoutOutAndCalculate ... I understand that higher is better, but just wonder what the value came from.

@MichelKeijzers Commonly you can't choose the serial speed to be any number. There are "standard" speeds like 9600 and 115200 that all derived from the first standard speeds like 75. BTW, the speed has the unit "steps per second", and in case of serial transmission one step is one bit. All these standard speeds are integer multiples of 75, most of them by factors of 2. For example: 9600 = 2^7 * 75, 115200 = 3 * 2^9 * 75.

@busybee thanks for the explanation.

https://www.arduino.cc/en/Reference/softwareSerial Because 115200 band was also the fastest communication enabled by the software serial.


text:
make EEPROM.h use custom eeprom partition
I created the below partition table for an esp32 and replaced the 'default_8MB.csv' with it.
# Name,   Type, SubType, Offset,  Size, Flags
nvs,      data, nvs,     0x9000,  0x5000,
otadata,  data, ota,     0xe000,  0x2000,
app0,     app,  ota_0,   0x10000, 0x330000,
app1,     app,  ota_1,   0x340000,0x330000,
eeprom,   data, 0x99,    0x670000,0x1000,
spiffs,   data, spiffs,  0x671000,0x18f000,

After uploading a sketch, I confirmed that the partition table was correctly uploaded to the flash at address 0x8000, using esptool.py --port /dev/ttyACM1 read_flash 0x8000 0x300 partition_table.bin && gen_esp32part.py partition_table.bin && cat partition_table.bin.
Now my question is: what do I have to do so that the EEPROM.h library maps the eeprom emulation to the address that I have specified in the partition table? I naively assumed that EEPROM.h would automatically use the partition named "eeprom", which doesn't seem to be the case.
I uploaded the following test sketch to analyse the situation:
#include <Arduino.h>
#include <EEPROM.h>

String s="this is a unique string";

void setup(){
  EEPROM.begin(1024);

  for (uint32_t i=0;i<s.length();i++){
    EEPROM.write(i,s[i]);
  }
  EEPROM.commit();
  Serial.println("EEPROM stored");
}

void loop() {
  delay(10000);
}

After the sketch has stored my unique string I read the flash memory using esptool and expected to find the string at an offset of 0x670000, but that wasn't the case. Instead it must be stored somewhere else because I can retrieve the string in a different sketch.
One thing that is confusing me: I'm pretty sure that I once ordered ESP-WROOM32s with 4MB flash, but the esptool's auto-flash-detection states that my board has 8MB. Can this flash-detection be trusted or does it simply fail because I try to flash 8MB to a 4MB ESP32? What is supposed to happen if I created a partition table that exceeds the flash memory size? When I put a partition table at an offset of 10M the build process still doesn't fail (is that what should happen?).
My main question: which additional steps do I have to take to get my ESP running with a custom_partion_table.csv (beside creating the table) and when using "EEPROM.h"?

EEPROM.h looks up the partition location by name. More specifically, it uses the SDK call nvs_open to open the partition named eeprom. So it should be stored at the right location.

EEPROM on the ESP32 doesn't work like you would expect.  It uses the SDK's "Non-Volatile Ram" storage system to store the data within the nvs partition.  This works through a system of "Key / Value" pairs, and the data is stored as a "BLOB" (Binary Large OBject) for the key eeprom in the namespace eeprom.
Quite what the internal layout of the data within the partition itself is I have no clue (the SDK is closed source), but it won't just have your string located at the start of the partition - it will be at the start of the BLOB, wherever that happens to be in the partition.
The eeprom partition is no longer used. It's just wasted space.

Can this flash-detection be trusted or does it simply fail because I try to flash 8MB to a 4MB ESP32?

I can't comment on that I'm afraid. I would have thought it would look at the chip's JEDEC data, but it may just look at the partition table information instead.

What is supposed to happen if I created a partition table that exceeds the flash memory size?

Probably not a lot, since much of the partition table refers to data storage, or a (presently) unused OTA partition.
When you compile your code you're only compiling it for the partition in which it will reside (app0 for example). When you upload the code you are only uploading to that partition.  Anything outside that range is irrelevant to the compilation / uploading.

When I put a partition table at an offset of 10M the build process still doesn't fail (is that what should happen?).

As I mentioned, it won't care. The partition will not be accessible, since it doesn't actually exist, but it's just an entry in a table. The compiler doesn't know or care what it is.

"it will be at the start of the BLOB, wherever that happens to be in the partition" - until now I had this working the way I expected (the EEPROM structure was just like in my sketch), but unfortunately I had to update the arduino IDE and now the partition scheme I used to work with does no longer exist... How do you upload the eeprom content?

@SimSon I guess the core changed from manually working with the flash chip to using the SDK's built in system. Makes it simpler for the core to implement, but harder for the user to do low-level stuff.  Best to leave EEPROM as internal to the sketch only, and if you want to have data that you can upload from outside a sketch then use SPIFFS.

I'm a bit confused, how do I use "SPIFFS from outside"? What does esptool.py [...] write_flash [...] do (does it depend on the partition type I'm uploading to)? As I understand, I can upload any data to arbitrary memory locations, so what does that have to do with SPIFFS, EEPROM or whatever partition I'd like to write to? On the other hand, what implications does it have to use "SPIFFS from outside"? I seem to lack some important understanding, but I don't know where...

SPIFFS is a filesystem stored in a flash partition. There are tools to create and manipulate an SPIFFS filesystem image on the computer, which you can then upload to the ESP32 at the correct memory location for your SPIFFS partition.

Sorry, I still don't understand: why can't I simply upload the persistent data (which I'd like to access via EEPROM.h from within the sketch) using the esptool like I did until now?

and do I actually need a partition named "eeprom" when I use EEPROM.h in my sketch? If no, where would EEPROM.h emulate the eeprom instead? In SPIFFS maybe?

You can't because the data is formatted in a different way with the current version of the core. And yes you absolutely need a partition named eeprom when you use EEPROM.h in your sketch.

So, on my computer I need to create a SPIFFS partition (which is formated like the eeprom partition that EEPROM.h expects) and then I have to upload this SPIFFS partition to the address of my eeprom partition on the esp? Did I get it?

No. Not quite. SPIFFS has nothing to do with EEPROM.h.  It's a filesystem. You use it like an SD card (but without the SD card).  On your computer you will create (in a new folder) a file with data in it (what format that takes is entirely up to you). You can make multiple files if you like. Then you use mkspiffs to turn that folder into a SPIFFS image, which you then upload to the memory address of the spiffs partition. Then you use SPIFFS.h to access it like you would an SD card.

There's loads of ESP32 tutorials here - many are about SPIFFS.

Let us continue this discussion in chat.


text:
Using MouseX in processing to control LEDs. Error in code - LEDs remain off
I am fairly new to processing and Arduino - I have a coursework deadline for both on Monday (6th January).
I am trying to write a script that uses the mouseX values to turn LEDs on and off. I haven't got any error messages on either of the scripts but currently, nothing happens to the LEDs (they remain off) when I move my mouse. I have tested the circuit with a simpler Arduino code and it is correct so there must be an error in my code!
I would be very grateful for some help. Thanks!
Processing code:
import processing.serial.*;
Serial port;
byte lightsOn;

void setup()
{
  size(600,300);
  port=new Serial(this,"/dev/cu.usbmodem1411",9600);
}

void draw()
{
  background(32,78,95);
  rect(300,0,300,300);
  noStroke();
  fill(185,215,216);

  lightsOn=byte(mouseX);
  port.write(lightsOn);

  println(lightsOn);
}

Arduino code:
#include <Servo.h>

Servo myservo;
int ledpinA=12;
int ledpinB=13;

void setup()
{
  Serial.begin(9600);
  myservo.attach(ledpinA&&ledpinB);
  pinMode(ledpinA,OUTPUT);
  pinMode(ledpinB,OUTPUT);
}

void loop()
{
  if(0<Serial.available())
  {
    int data = Serial.read();

  if (data>300)
  {
    digitalWrite(ledpinA, HIGH);
    digitalWrite(ledpinB, LOW);
  }
  else
  {
    digitalWrite(ledpinB, HIGH);
    digitalWrite(ledpinA, LOW);
  }
  }
}

lightsOn=byte(mouseX);

A byte can be any number from -128 to 127 (Java has no concept of "unsigned" values). The Arduino will reinterpret that as a value from 0 to 255.
if (data>300)

255, the maximum value, can never be more than 300 - so this will always fail.
You need to re-think what data you are sending and how you want to send it.


text:
esp32 arduino wifi disconnect
I have this code running on ESP32 connected to SVM30-J
#include <WiFi.h>
#include <PubSubClient.h>
#include <Arduino_JSON.h>
#include <svm30.h>

// Replace the next variables with your SSID/Password combination
const char* ssid = "xxxx";
const char* password = "xxxx";

// Add your MQTT Broker IP address, example:
const char* mqtt_server = "xxxx";
const int mqtt_port = 18712;
const char* mqtt_user = "xxxx";
const char* mqtt_password = "xxxx";

/* define driver debug
 * false : no messages
 * true : display driver debug messages
 */
#define DEBUG true

// define number of seconds between display results
#define DELAY 10

// function prototypes (sometimes the pre-processor does not create prototypes themself on ESPxx)
void read_id();
void read_featureSet();
void Errorloop(char *mess);
void read_values();
void read_baseline();
void KeepTrigger(uint8_t del);

// create instance
SVM30 svm;

WiFiClient espClient;
PubSubClient client(espClient);

void setup() {

  Serial.begin(115200);

  setup_wifi();
  client.setServer(mqtt_server, mqtt_port);
  //client.setCallback(callback);

  Serial.println(F("Reading information from the SVM30"));

  // enable debug messages
  svm.EnableDebugging(DEBUG);

  svm.begin();

  Serial.print(F("Driver version : "));
  Serial.println(svm.GetDriverVersion());

  // try to detect SVM30 sensors
  if (svm.probe() == false) Errorloop("could not detect SVM30 sensors");
  else Serial.println(F("SVM30 detected"));

  // read and display the ID
  read_id();

  // read SGP30 feature set
  read_featureSet();
}

void loop() {
  char chipId[13]; // Don't forget one byte for the terminating NULL...
  uint64_t mac = ESP.getEfuseMac();
  sprintf(chipId, "%012x", mac);
  Serial.println(chipId);

  JSONVar data;

  if (!client.connected()) {
    reconnect();
  }
  client.loop();

  // read measurement values
  read_values();

  // wait x seconds
  KeepTrigger(DELAY);
}

void setup_wifi() {
  delay(10);
  // We start by connecting to a WiFi network
  Serial.println();
  Serial.print("Connecting to ");
  Serial.println(ssid);

  WiFi.begin(ssid, password);
  WiFi.setSleep(false);

  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }

  Serial.println("");
  Serial.println("WiFi connected");
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());
}

void callback(char* topic, byte* message, unsigned int length) {
  Serial.print("Message arrived on topic: ");
  Serial.print(topic);
  Serial.print(". Message: ");
  String messageTemp;

  for (int i = 0; i < length; i++) {
    Serial.print((char)message[i]);
    messageTemp += (char)message[i];
  }
  Serial.println();

  // If a message is received on the topic esp32/output, you check if the message is either "on" or "off".
  // Changes the output state according to the message
  if (String(topic) == "esp32/output") {
    Serial.print("Changing output to ");
    if(messageTemp == "on"){
      Serial.println("on");
      //digitalWrite(ledPin, HIGH);
    }
    else if(messageTemp == "off"){
      Serial.println("off");
      //digitalWrite(ledPin, LOW);
    }
  }
}

void reconnect() {
  // Loop until we're reconnected
  while (!client.connected()) {
    Serial.print("Attempting MQTT connection...");
    // Attempt to connect
    if (client.connect("ESP32Client", mqtt_user, mqtt_password)) {
      Serial.println("connected");
      // Subscribe
      //client.subscribe("esp32/output");
    } else {
      Serial.print("failed, rc=");
      Serial.print(client.state());
      Serial.println(" try again in 10 seconds");
      // Wait 10 seconds before retrying
      delay(10000);
    }
  }
}

/*
 * @brief : keep triggering SGP30 while waiting
 *
 * @param del : number of seconds to wait
 *
 * source datasheet SVM30:
 * The on-chip baseline compensation algorithm has been optimized
 * for 1HZ sampling rate. The sensor shows best performance when
 * used with this sampling rate.
 *
 */
void KeepTrigger(uint8_t del)
{
  uint8_t w_seconds = del;
  unsigned long startMillis;

  if (w_seconds == 0) w_seconds = 1;

  while (w_seconds--)
  {
    startMillis = millis();

    if (! svm.TriggerSGP30())
      Errorloop("Error during trigger waiting");

    // this gives 1Hz /1000ms (aboutisch)
    while(millis() - startMillis < 1000);
  }
}

/*
 * @brief : read and display the values from the SVM30
 *
 */
void read_values() {
  struct svm_values v;
  uint16_t baseline;

  if (! svm.GetValues(&v))
      Errorloop("Error during reading values");

  JSONVar data;

  Serial.print(F("CO2 equivalent : "));
  Serial.print(v.CO2eq);
  data["co2_equivalent"] = v.CO2eq;

  Serial.print(F(", TVOC : "));
  Serial.print(v.TVOC);
  data["tvoc"] = v.TVOC;

  Serial.print(F(", H2_signal : "));
  Serial.print(v.H2_signal);
  data["h2_signal"] = v.H2_signal;

  Serial.print(F(", Ethanol_signal : "));
  Serial.print(v.Ethanol_signal);
  data["ethanol_signal"] = v.Ethanol_signal;

  Serial.print(F(", Humidity : "));
  Serial.print((float) v.humidity/1000);
  data["humidity"] = (float) v.humidity/1000;

  Serial.print(F(", temperature : "));
  Serial.print((float) v.temperature/1000);
  data["temperature"] = (float) v.temperature/1000;

  Serial.print(F(", absolute humidity : "));
  Serial.print(v.absolute_hum);
  data["absolute_hum"] = v.absolute_hum;

  if (! svm.GetBaseLine_CO2(&baseline))
      Errorloop("could not read SGP30 CO2 baseline");

  Serial.print(F(" CO2 equivalent baseline : 0x"));
  Serial.print(baseline, HEX);
  data["co2_equivalent_baseline"] = (baseline, HEX);

  if (! svm.GetBaseLine_TVOC(&baseline))
      Errorloop("could not read SGP30 TVOC baseline");

  Serial.print(F(", TVOC baseline : 0x"));
  Serial.println(baseline, HEX);
  data["tvoc_baseline"] = (baseline, HEX);

  char chipId[13]; // Don't forget one byte for the terminating NULL...
  uint64_t mac = ESP.getEfuseMac();
  sprintf(chipId, "%012x", mac);
  data["chipid"] = chipId;

  Serial.print("Data = ");
  Serial.println(data);

  String data_json = JSON.stringify(data);
  char mqtt_publish_topic[] = "esp32/data";
  client.publish(mqtt_publish_topic, data_json.c_str());

}

/*
 * @brief: read and display the id of the SGP30 and SHTC1 sensors
 */
void read_id() {

  uint16_t buf[3];    // SGP30 needs 3 words, SHTC1 is 1 word
  char  id[15];

  if ( ! svm.GetId(SGP30, buf))
      Errorloop("could not read SGP30 id");

  Serial.print(F("\nSGP30 id : "));
  sprintf(id, "%04x %04x %04x", buf[0], buf[1], buf[2]);
  Serial.println(id);

  if (svm.GetId(SHTC1, buf) == false)
      Errorloop("could not read SHTC1 id");

  Serial.print(F("SHTC1 id : "));
  // only bit 5:0 matter (source: datasheet)
  sprintf(id, "%04x", buf[0] & 0x3f);
  Serial.println(id);
}

/*
 * @brief: read and display the product feature set of the SGP30 sensor
 */
void read_featureSet(){

  char buf[2];

  if ( ! svm.GetFeatureSet(buf))
      Errorloop("could not read SGP30 feature set");

  Serial.print(F("\nSGP30 product type : "));
  Serial.print((buf[0] & 0xf0), HEX);

  Serial.print(F(", Product version : "));
  Serial.println(buf[1], HEX);
  Serial.println();
}

/**
 *  @brief : continued loop after fatal error
 *  @param mess : message to display
 */
void Errorloop(char *mess)
{
  Serial.println(mess);
  Serial.println(F("Program on hold"));
  for(;;) delay(100000);
}

This code run as expected, except get following in serial monitor
000010a4ae30
Attempting MQTT connection...connected
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x90 0x00 0x00 length: 4
Sending to 0x58: 0x20 0x50 , Received: 0x31 0x8D 0x47 0x9D length: 4
Sending to 0x70: 0x78 0x66 , Received: 0x61 0xBD 0x7B 0xBE length: 4
CO2 equivalent : 400, TVOC : 0, H2_signal : 12685, Ethanol_signal : 18333, Humidity : 48.34, temperature : 21.81, absolute humidity : 9.28Sending to 0x58: 0x20 0x15 , Received: 0x00 0x00 0x00 0x00 length: 4
 CO2 equivalent baseline : 0x0Sending to 0x58: 0x20 0x15 , Received: 0x00 0x00 0x00 0x00 length: 4
, TVOC baseline : 0x0
Data = {"co2_equivalent":400,"tvoc":0,"h2_signal":12685,"ethanol_signal":18333,"humidity":48.33599853515625,"temperature":21.812999725341797,"absolute_hum":9.2800455093383789,"co2_equivalent_baseline":16,"tvoc_baseline":16,"chipid":"000010a4ae30"}
[D][WiFiClient.cpp:509] connected(): Disconnected: RES: 0, ERR: 128
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x90 0x00 0x00 length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x90 0x00 0x00 length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x90 0x00 0x00 length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x90 0x00 0x00 length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x90 0x00 0x00 length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x90 0x00 0x00 length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x90 0x00 0x00 length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x90 0x00 0x00 length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x90 0x00 0x00 length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x90 0x00 0x00 length: 4
000010a4ae30
Attempting MQTT connection...connected
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x90 0x00 0x00 length: 4
Sending to 0x58: 0x20 0x50 , Received: 0x31 0x94 0x47 0xAB length: 4
Sending to 0x70: 0x78 0x66 , Received: 0x61 0xC4 0x7A 0xDC length: 4
CO2 equivalent : 400, TVOC : 0, H2_signal : 12692, Ethanol_signal : 18347, Humidity : 47.99, temperature : 21.83, absolute humidity : 9.22Sending to 0x58: 0x20 0x15 , Received: 0x00 0x00 0x00 0x00 length: 4
 CO2 equivalent baseline : 0x0Sending to 0x58: 0x20 0x15 , Received: 0x00 0x00 0x00 0x00 length: 4
, TVOC baseline : 0x0
Data = {"co2_equivalent":400,"tvoc":0,"h2_signal":12692,"ethanol_signal":18347,"humidity":47.991001129150391,"temperature":21.830999374389648,"absolute_hum":9.223388671875,"co2_equivalent_baseline":16,"tvoc_baseline":16,"chipid":"000010a4ae30"}
[D][WiFiClient.cpp:509] connected(): Disconnected: RES: 0, ERR: 128
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x90 0x00 0x00 length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x90 0x00 0x00 length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x90 0x00 0x00 length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x90 0x00 0x00 length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x90 0x00 0x00 length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x98 0x00 0x05 length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x90 0x00 0x03 length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x92 0x00 0x03 length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x96 0x00 0x01 length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x90 0x00 0x00 length: 4
000010a4ae30
Attempting MQTT connection...connected
Sending to 0x58: 0x20 0x08 , Received: 0x01 0xA1 0x00 0x0D length: 4
Sending to 0x58: 0x20 0x50 , Received: 0x31 0x94 0x47 0xA8 length: 4
Sending to 0x70: 0x78 0x66 , Received: 0x61 0xBD 0x7A 0xCC length: 4
CO2 equivalent : 417, TVOC : 13, H2_signal : 12692, Ethanol_signal : 18344, Humidity : 47.97, temperature : 21.81, absolute humidity : 9.21Sending to 0x58: 0x20 0x15 , Received: 0x86 0x6C 0x8C 0x58 length: 4
 CO2 equivalent baseline : 0x8C58Sending to 0x58: 0x20 0x15 , Received: 0x86 0x6C 0x8C 0x58 length: 4
, TVOC baseline : 0x866C
Data = {"co2_equivalent":417,"tvoc":13,"h2_signal":12692,"ethanol_signal":18344,"humidity":47.966999053955078,"temperature":21.812999725341797,"absolute_hum":9.20920181274414,"co2_equivalent_baseline":16,"tvoc_baseline":16,"chipid":"000010a4ae30"}
[D][WiFiClient.cpp:509] connected(): Disconnected: RES: 0, ERR: 128
Sending to 0x58: 0x20 0x08 , Received: 0x01 0xA6 0x00 0x0F length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x98 0x00 0x05 length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x9E 0x00 0x0F length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x98 0x00 0x0C length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x9C 0x00 0x0D length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x9A 0x00 0x11 length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x9A 0x00 0x0F length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x9D 0x00 0x09 length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x96 0x00 0x0A length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x90 0x00 0x06 length: 4

Here ESP try to connect to MQTT because it looses wifi connection
Attempting MQTT connection...connected
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x90 0x00 0x00 length: 4
Sending to 0x58: 0x20 0x50 , Received: 0x31 0x8D 0x47 0x9D length: 4
Sending to 0x70: 0x78 0x66 , Received: 0x61 0xBD 0x7B 0xBE length: 4
CO2 equivalent : 400, TVOC : 0, H2_signal : 12685, Ethanol_signal : 18333, Humidity : 48.34, temperature : 21.81, absolute humidity : 9.28Sending to 0x58: 0x20 0x15 , Received: 0x00 0x00 0x00 0x00 length: 4
 CO2 equivalent baseline : 0x0Sending to 0x58: 0x20 0x15 , Received: 0x00 0x00 0x00 0x00 length: 4
, TVOC baseline : 0x0
Data = {"co2_equivalent":400,"tvoc":0,"h2_signal":12685,"ethanol_signal":18333,"humidity":48.33599853515625,"temperature":21.812999725341797,"absolute_hum":9.2800455093383789,"co2_equivalent_baseline":16,"tvoc_baseline":16,"chipid":"000010a4ae30"}
[D][WiFiClient.cpp:509] connected(): Disconnected: RES: 0, ERR: 128

I am not using deep sleep yet, which would have disconnected from wifi.
Update :
I had ESP32-CAM connected to same AP, when I turn off ESP32-CAM then this code works without any error (ERR: 128)
Dont know how ESP32-CAM WiFi connection affects ESP32 WiFi connectivity ?

what happens if you set DELAY to a lower value

For DELAY 0, DELAY 1, DELAY 5, DELAY 10 getting  same [D][WiFiClient.cpp:509] connected(): Disconnected: RES: 0, ERR: 128

perhaps this post in an issue on github sheds some light - I don't know - it seems your not disconnecting from wifi though - nothing in the log suggests that, does it?

If you are actually disconnectin gfrom Wifi, I know some ESP8266's had trouble keeping a wifi connection with a small set of wifi routers in an older version of the ESP8266 library - not sure if the ESP32 suffered the same issue, but out of interest, what make/model is the access point/wifi router you are connecting to?

I have esp32-cam which connect to same AP with no issue.

OK - was just a thought - so how about that link https://github.com/espressif/arduino-esp32/issues/1921#issuecomment-471346624 - anything there help?

I have added some update to my question.

so a completely different ESP32, with a camera attached, affects the connection of this other ESP32? Is this ESP32CAM using the same MQTT server and perhaps sending huge amounts of data to it?

Yes different ESP32 with CAM sending data to same MQTT. But not sending huge amount of data.

the id to client.connect .... it's not ESP32Client in both cases is it?


text:
Plant irrigation system
I'm very new to the Arduino game I wanted to make a small auto watering system for my plants. I bought these items

5V 1 CHANNEL RELAY MODULE FOR ARDUINO PIC AVR DSP ARMB
9 VOLT BATTERY SNAP CONNECTOR B
ARDUINO UNO R3
CAMELION 9V BATTERY SUPER HEAVY DUTY
PIN TO HOLE DUPONT 40 PIN 30CM ARDUINO JUMPER WIRE 12 INCH
SOIL MOISTURE METER FOR ARDUINO.

I followed a video on youtube I attached the guys circuit diagram as well. After following instructions my pump wont turn on I also programmed it as well. Please help if anyone can show me how to rewire the device and codes or if I'm doing something wrong. Please help.
int Relay = 13;
int sensor = 8;
int val;
void setup() {
  pinMode(13,OUTPUT); //Set pin 13 as OUTPUT pin, to send signal to relay
  pinMode(8,INPUT); //Set pin 8 as input pin, to receive data from Soil moisture sensor.
}

void loop() {
  val = digitalRead(8);
  if(val == LOW)
  {
    digitalWrite(13,LOW); //if soil moisture sensor provides LOW value send LOW value to relay
  }
  else
  {
    digitalWrite(13,HIGH); //if soil moisture sensor provides HIGH value send HIGH value to relay
  }
  delay(400);
}

Those 9V batteries aren't ideal. They are low in capacity (don't last long), and can provide that much current. My guess is that the battery can't supply enough current to turn on the relay. Try removing the relay, and see if the led on pin 13 ('L' on the Uno) turns on/off based on the moisture on the sensor.

Hey thanks for replying. The 9v battery is for powering the uno. The lights turn on when i turn it on. The relay also shows the moisture sensor is working as the lights on it turn on and off when I dip it into a cup of water. I'll try removing the relay and see if that helps thanks alot. If you could share a circuit diagram I will happily follow it :)

It sounds like everything on the Arduino side is working. It might be that you wired to water pump/valve the wrong way.

How did you wire the output side of relay for the pump? You do know you have to supply power to it, right?-) If you have a volt-meter, check that the power to the pump is switching properly. If you don't have a volt-meter, get one.

@haresfur I figured that out this morning thanks tbo :) I had wired the relay with the pump wrong. Everything is working well now.

I've seen so much fake projects from "Viral science". I'm not sure whether he/she simply don't know what he/she does or if he/she simply tries to trap people. This project is a race condition itself ;-).
Besides your battery is empty. I don't think it is a big problem to use a 9V block battery, but Gerben is right, it won't do its job for very long. Have you thought about using a LiPo or LiOn Accu and a Solar loader?
There are worse things I see:
The sensor is powered by 3.3V but you read the ON/OFF signal with a 5-Volt Pin. That could be a problem, if there are other voltage drops that lead to a signal the PIN can not recognize as HIGH. Check the documentation of the moisture sensor and if it allows 5V use 5V to power it. All sensors I have found on Google have at least a range from 3.3 Volt to 5 Volt.
Check the current consumption of the relay. It must not draw more current than the Arduino PIN can provide (40mA for the UNO). And, don't use PIN13 to switch the relay. There are so many others and PIN 13 is connected to the onboard LED, which draws 10 - 20 mA on its own.

Note that it's supposed to be a relay module, which includes the driving transistor.

@EdgarBonet Yes, that's right. That's why I wrote "Check the current consumption of the relay." I don't know whether it is a MOSFET or a NPN, so the impedence might not be enough for a PIN13 that has to drive the on board LED also. And you are complete right: If it is a MOSFET even PIN13 would work.

@Peter Paul Kiefer thanks for replying! well that sucks lol I am thinking of using these batteries but i dont know how to connect them the 9v block battery is only for powering the uno device but I still haven't figured out how to connect the lion or lipo to the motor is there a way I could use rechargeable batteries and plug the uno directly into a dc power source without the use of any batteries. Or if you can help with a new circuit diagram and codes? I would appreciate it. Note all of the products I bought are Chinese apart from the uno.

@Nizammulk Important note: I recommend powering the sensor only during the measurement and then disconnecting the power supply (eg using a transistor). This significantly extends the life of the electrodes.


text:
How to make BLE data accessible to a particular app?
I used the BLE example from here.
But any device can read and write to the advertised characteristic. Is there a way to make the data inaccessible to other devices except for my app somehow. What I would really like to do is something like SDP and then connect and exchange data over RFCOMM but it seems impossible to do so in esp32. I have done it using bluez library on Linux but in case of esp32 I found only one library btstack that is impossible to use with esp-idf because of lack of CMakeLists.txt. Is there some way I can achieve what I want?

In general terms, you could simply require a client to authenticate first. This would require a bit of a deviation from the example you linked to (which actually only handles reading of the characteristic) to handle multiple connections and their state, but it should be pretty straightforward.
The Arduino BLE library seems to lack sending quite a bit of information via the callbacks, so you just know there's a connect or disconnect or a write to a characteristic, and not which devices did. This means you won't be able to handle multiple connections at the same time, but making sure you reset your authentication state whenever you receive a connect or disconnect should be enough to prevent anyone but your own client from "talking" to your device.

thanks, I'll try implementing this flow and update you as soon as I can.


text:
UPDATE: Fault in processing code - doesn't receive correct data from Arduino
I am receiving data from an LDR which I am inputting into processing. The data seems to be received to Arduino correctly and I have looked at the data being sent via the serial monitor and it all seems correct. However, the data that is being received in processing doesn't reflect the data received by the LDR so I think I have done something wrong here but not sure what as I am fairly new to this all. (Please see image for what is printing in the processing serial monitor).
Any help identifying where I have gone wrong would be so so helpful!
ARDUINO CODE
int rval=0;
int lightval;

//Recorded minimum and maximum values
int minval=0;
int maxval=950;

void setup()
{
  Serial.begin(9600);
}

void loop()
{
  rval = analogRead(1);
  //Serial.println(rval);
  rval=constrain(rval,minval,maxval);
  lightval=map(rval,minval,maxval,0,255);
  //Serial.println(lightval);
  Serial.write(lightval);
}

PROCESSING CODE
import processing.serial.*;

Splash paintsplash;
Serial port;

float value=0;

void setup()
{
  size(600,600);
  smooth();
  background(255);
  paintsplash= new Splash();
  port=new Serial(this, Serial.list()[6],9600);
  //printArray(Serial.list());
}

void draw()
{
  paintsplash.display();
  paintsplash.splash();
  delay(200);

  if(0<port.available())
  {
    value=port.read();
  }

  println(value);

  if(value<=75)
  {
    fill(255,137,132);
  }

  else if(value>75 && value<=125)
  {
    fill(32,78,95);
  }

  else
  {
    fill(183,215,216);
  }
}

PROCESSING CLASS - PAINT_SPLASH
class Splash
{
  float x;
  float y;
  float size;

  Splash()
  {
    x=0;
    y=0;
    size=int(random(35));
  }

  void display()
  {
    //fill(255,137,132);
    noStroke();
    ellipse(x,y,size,size);
  }

  void splash()
  {
  float xcord=int(random(600));
  float ycord=int(random(600));
  pushMatrix();
  translate(xcord,ycord);

     for ( float i =3; i < 29; i ++)
     {
       float splashX = x + random(100);
       float splashY = y + random(100);

       ellipse(splashX,splashY,size-i,(size-i+random(3)));
     }

   popMatrix();
  }
}

Make sure the output you show belongs to the code you show. I doubt you have Serial.println(lightval); in the version whose output we see.

Sorry yes I had commented out Serial.println(lightval) when I ran the script and the letters are still there

Then edit the question to comment out that line. At the risk of repeating myself, please make sure the output you show belongs to the code you show.

You're sending your lightval twice - once as a textual representation of the number, and once as a character representing the lower 8 bits:
Serial.println(lightval);
Serial.write(lightval);

For example:
Q308
R304
Q306

If you take the 308 and AND it with 255 you get 52.  52 as an ASCII character is R.
So you print 308, including a newline, then you print "R".  Then you print 304, which would be 48 after ANDing with 255 - which of course is a Q.

Thank you so much for this! You were correct, I am now getting the correct values in the Arduino serial monitor!
I am now having trouble with the processing side of the code. If you can lend any more of your genius I would be very grateful!


text:
How to transmit data from Arduino to Raspberry Pi?
I am planning to build a network with five Arduino Mega devices, each of which will collect data using multiple sensors. Each Arduino will periodically transmit the sensor data to a gateway Raspberry Pi using XBees. As it is the first time I am working with Arduino and Raspberry Pi, I need help with the following:

Which protocol to use to transmit data from Arduinos to the Raspberry Pi? Do I have to write code for both Arduino and Raspberry Pi or are there existing solutions that I can use for this purpose?
Does MQTT, Node-RED or Node.js have any role in the above?

Thank you for your advice!
A.

this looks like a repeat of your previous question ..https://arduino.stackexchange.com/questions/71291/software-for-communication-between-arduino-raspberry-pi-and-computer-server

@jsotola Yes, but I tried to be more focused this time. I hope I have been successful.

I have not bought the equipment yet. Before doing this, I would like to know the technologies involved and level of depth I would need to go into this. I think it should be straight forward for people who have experience with Arduino and Raspberry Pi to come with some good advice. Thank you for the edit.

MQTT messages are usually set over internet connections.  RaspberryPi boards have this type of interface.  Most Arduinos need to have extra hardware.  An MQTT broker can receive and send out messages.  If you want to do something with the messages between the sender and receiver NodeRED is a good choice.  You will likely want to run the MQTT broker (probably mosquitto) and NodeRED on the RaspberryPi.  Let me know if this is what you are looking for and I'll flesh out the particulars and post an answer to your question.

BTW, I don't know different your original post is, but consider making a question better before resorting to making another post.  The point of most if not all of the stackexchange web sites are to have a bunch of good searchable questions.

@st2000 Arduinos will use XBees for communication. So, no other hardware needed to send data to the RaspberryPi.

@AdnanAli, there are so many Arduino platforms that the only assumption that can be made when the OP does not specify a particular platform is to assume it is a Revision 3 Arduino Uno with no XBees or no Ethernet.   But I believe you are correct, that either can be added to the Revision 3 Arduino Uno to support sending and receiving MQTT messages.

Which protocol to use to transmit data from Arduinos to the Raspberry Pi? Do I have to write code for both Arduino and Raspberry Pi or are there existing solutions which I can use for this purpose?
Does MQTT, Node-RED or Node.js have any role in the above?

I have searched, explored, read and found that MQTT can be used to send and receive data from Arduinos to RaspberryPi as in [1] and [2]. So, no need to design and develop a new protocol.
MQTT libraries, such as PubSubClient [3] are available for Arduino. So Arduino code can use these libraries and no need to write much code either.
Simple and straight!
A.
[1] https://funprojects.blog/2018/03/04/arduino-talking-mqtt-to-node-red/
[2] https://iotdesignpro.com/projects/interface-arduino-with-node-red-to-send-sensor-data-on-webpage
[3] https://www.arduinolibraries.info/libraries/pub-sub-client