text:
Need help figuring out how to organise a system to run a stepper motor at low speed without Steps-skipping
TL;DR : How to prevent step-skipping with a NEMA gearbox motor at low speed.
I am in the process of creating a system in which I want an object to turn on a plate and to show a letter depending on the angle of the rotation. The object is quite heavy hence the rotation should be quite slow, this is why I took a Planetary Gearbox NEMA Stepper.
My system consists in:

An Arduino Uno
A NEMA Stepper Motor
A A4988 Driver
Some wires
2 Iron plates
A ball bearing element in between the Iron plates
Screws to fit everything together.

This is the code I ended up with:
// Define stepper motor connections and steps per revolution:
#define dirPin 6
#define stepPin 7
#define stepsPerRevolution 5373
int degree_init = 1;

int mouvement=0;

// create a mapping between letters and degrees
int letterToDegree[33] ;
int degree;
// function to map letters to degrees
int mapLetterToDegree(char letter) {
  if (letter=='.'){
    return letterToDegree[27];
  }
  else if (letter==':'){
    return letterToDegree[28];
  }
  else if (letter=="\'"){
    return letterToDegree[29];
  }
  else if (letter==' '){
    return letterToDegree[30];
  }
  else {
  int letterIndex = letter - 'A'; // get the index of the letter in the alphabet
  return letterToDegree[letterIndex]; // return the degree associated with that letter
  }
}

void setup() {
  // Declare pins as output:
  pinMode(stepPin, OUTPUT);
  pinMode(dirPin, OUTPUT);
    Serial.begin(9600);
    for (int i = 0; i < 33; i++){
    letterToDegree[i]= degree_init;
    degree_init=degree_init+10;
  }
}

void loop() {
  int p= NULL;
  // Set the spinning direction clockwise:
  digitalWrite(dirPin, HIGH);
  String input= "NOW THE NOW THE AAAAA";
  // loop through the letters
  for (int i = 0; i < input.length(); i++) {
    // get the degree for each letter
    degree = mapLetterToDegree(input[i]);
    Serial.print("Letter: ");
    Serial.print(input[i]);
    Serial.print(" Degrees: ");
    Serial.println(degree);
    // wait for the movement to complete
    delay(1000);

  int p_new= map(degree,0,360,0,5373);
  Serial.print(" position: ");
  Serial.println(p_new);
  if (p!=NULL){
    mouvement=p_new-p;
    p=p_new;
  Serial.print(" mouvement: ");
  Serial.println(mouvement);
  }
  else{
    p = p_new;
  }
  if (mouvement<0){
    digitalWrite(dirPin, LOW);
    mouvement=abs(mouvement);
    Serial.println("change de direction");
  }
  Serial.print(" stepper: ");
  Serial.println(mouvement);

  // Spin the stepper motor 1 revolution slowly:
  for (int i = 0; i < mouvement; i++) {
    // These four lines result in 1 step:
    digitalWrite(stepPin, HIGH);
    delay(20);
    digitalWrite(stepPin, LOW);
    delay(20);
  }

  delay(1000);
  digitalWrite(dirPin, HIGH);
}
}

I tried following this tutorial and it works in broad terms but sometimes the motor skips steps, especially when changing direction.
I can't seem to understand how torque works within this system and how to minimise the stress I put on the motor.
My hypothesis is that since the system is at equilibrium and the friction is minimised due to the ball bearing, there should be no need for a lot of power from the motor, and that this motor should suffice.
My hypothesis of why steps are skipped:

Too high a speed for the motor
Not enough torque power supplied
Not good enough electricity supply to the motor.

I am not sure how to go about and test these hypothesis and if those are correct.
I am also having a lot of trouble understanding the relationship between the time delay in the code and the movement of the motor.
Especially this bit:
  for (int i = 0; i < mouvement; i++) {
    // These four lines result in 1 step:
    digitalWrite(stepPin, HIGH);
    delay(20);
    digitalWrite(stepPin, LOW);
    delay(20);
  }

Is it ok to change them as I did? do they need to be symetrical (same delay for high and low)?
I am also looking for an explanation of how to apply acceleration and decceleration algorithms for stepper motor, even though I am not sure it makes sense at this speed.
My questions are:

How to calculate the best speed for such a system and how to implement it?
Is there need for acceleration/ decceleration?
Does microstepping makes sense when using a gearbox?
Are there some obvious flaws in my system/ code that I am missing out?

If the object is heavy instant start/stop/direction change can let the motor skip due to the objects inertia. Do you want to do the acceleration algorithm yourself? If not, try the AccelStepper library, which will do that for you.

you have un-necessary code in the if (p != NULL) { block ... else is not needed ... p = p_new; is always executed ... put p = p_new; after the if block

you have not said anything about the desired rotation rate

A general question: Did you consider using a limit switch (home position) and a rotary encoder (track the steps)?

@jsotola Thanks for noticing it, I changed it. The rotation rate should be really slow.

@chrisl I tried with the Accelstepper but it did not help

@RolandJ. This is the solution I am investigating at the moment, thanks for proposing it.

How to calculate the best speed for such a system and how to implement it?

There is no one best speed. This depends on your requirements, the motor torque with the supplied power and driver and the rotational inertia of the object on the motor axis. You can measure the rotational inertia and set yourself a value for the acceleration (even if you don't write the program for acceleration, it will accelerate on the first steps), that you want to have. Then you can check, if your motor has enough torque for this.
Speaking about the motor: Check the ratings of the motor. A good supplier should also give you information on how much torque your motor has at specific currents. Then check if your driver can handle the current (these small stick drivers often cannot handle much current, like 1 or 2A max). These drivers also have a small potentiometer on board, which controls the motor current. Your supplier should also provide information on how to set it correctly (by checking the voltage on one of the pins). You can even find information about this online, when you search for your type of driver. Setting this wrong can lead to either an underperforming motor or the motor overheating. So when setting the motor current with the potentiometer, check the motor temperature afterwards while it is running. Every motor has its own temperature rating, but I would suggest not letting it get too hot to touch.
Having done all above you can drive a test for checking motor speeds. Forget about your current code for a while. Just drive the motor with specific speeds, starting from only a few steps per second and going up until you notice the motor skipping. Then you know how fast you can drive the motor in your configuration without skipping and acceleration. If that is fast enough for you, you can just stick to that.

Is there need for acceleration/ decceleration?

That depends on the above. If the non-skip speed is not fast enough, you can accelerate by gradually increasing the steps per second (aka decreasing the delay between step pulses). To halt at a specific position you then need to decelerate, so you need to plan the movement and control the steps per second accordingly. If you don't want to do that yourself I would suggest using the AccelStepper library. There you can set the target speed and the maximum acceleration and the library will then handle the motor for you.
You wrote in a comment, that you tried the AccelStepper library and that it didn't help. Though you didn't state how exactly it didn't help and what exactly you tried. Maybe you just didn't use the library correctly.

Does microstepping makes sense when using a gearbox?

With microstepping you can get a smoother movement, though I doubt, that this is the problem here. Activating microstepping will of course mean, that you need to send more pulses to the driver for the same speed. Just keep that in mind.

Is it ok to change them as I did? do they need to be symetrical (same delay for high and low)?

The first delay establishes the pulse for the driver. It needs to be big enough, that the driver recognizes the pulse. 20ms is very long for that. If you look at the datasheet of the your driver chip, there should be some information on that. I used a delay of about 50us in the past (so delayMicroseconds(50)), though you might even get shorter.
The second delay then handles the time between pulses, aka the speed of the motor. Though keep in mind, that the first delay is also still a constant in that equation. The speed in steps per second would then be 1/(delay1+delay2) (ignoring the time for the execution of the rest of the code, because only relevant at high speeds).

Thanks for the detailled response, I eventually chose a bigger, close-loop motor but it helped me understand how to go about it.


text:
Multi dimming led - figuring out which led is available for dimming
TL;DR: I tried to have a random fading pattern on several leds and it leads to flickering.
Edited following @chrisl answer.
I am in the process of creating a simple lamp that show some light patterns using 6 led and an arduino uno. The goal is that depending on some inputs and a bit of randomisation, the patterns will change. For instance, only 2 lights at the same time but fading up and down randomly on all 6 led. Or 4 lights fading up and down randomly on the 6 leds.
I decided to create a struct that would be a light pattern named a particle, and this particle have a variable named led_position that shows which led it is on at a certain moment. The particle can change its position to light up an other led.
I know this is not the clearest of explanation but I hope my code will be more straightforward:
Main code
        #include "MyTypes.h"
int led1 = 9;    // the PWM pin the LED is attached to
int led2 = 10;
int led3 = 11;
int led4 = 6;
int led5 = 3;
int led6 = 5;

//int led;
int brightness;
int fadeAmount;

// Definition of the different particles
particle_type particle[]={
  { 0,led1,0, 2,   false },
  { 1,led2,50, 2,   false },
  { 2,led3,100,  2,   false },
  { 3,led4,80,  2,   false },
  { 4,led5,30,  2,   false},
  { 5, led6,70,  2,   false},
};

led_type led[]={
  { 3,false },
  { 5,false },
  { 6,false },
  { 9,false },
  { 10,false },
  { 11,false },
};

//initiation of the particles

void initiate_particle(particle_type& particle_example,led_type led[]){

 //take a random led from list of led
  int number=random(5);
  led[number];
  if (led[number].busy == false){ //if led non-occupied
    particle_example.led_position=led[number].led_position;
    led[number].busy = true ;//take this led as output
    }

  else{
//    particle_example.led_position=particle_example.led_position; //keep same led
  }

  particle_example.reset = false; //reset turned off
 // delay(30);

}

void update_light(particle_type& particle_example,led_type led[]){ //updating led brightness

  analogWrite(particle_example.led_position, particle_example.brightness);  //updating led brightness
  if (particle_example.brightness >=  160) {
    particle_example.fadeAmount = -particle_example.fadeAmount;

  }

  if (particle_example.brightness <=  160 && particle_example.brightness >=  100 ) {
    particle_example.brightness = particle_example.brightness + 2*particle_example.fadeAmount;

  }

  if (particle_example.brightness >= 50 && particle_example.brightness <=  100){
      particle_example.brightness = particle_example.brightness + particle_example.fadeAmount;

  }

  if (particle_example.brightness >= 0 && particle_example.brightness <=  50){
      particle_example.brightness = particle_example.brightness + 0.5*particle_example.fadeAmount;

  }

  if (particle_example.brightness <= 0){
    particle_example.reset= true;
    int number = particle_example.led_position;
    led[number].busy=false;
    particle_example.fadeAmount = -particle_example.fadeAmount;
  }

}

void activate_particle(particle_type& particle_example,led_type led[]){ //function to keep led activated

  if (particle_example.reset==true){ //if led not initiated or has finished its dimming down
    initiate_particle(particle_example, led); //initiate it again
  ;}

  else {
   update_light(particle_example,led); //update its brightness
  }
}

void setup() { //create led outputs

  pinMode(led1, OUTPUT);
  pinMode(led2, OUTPUT);
  pinMode(led3, OUTPUT);
  pinMode(led4, OUTPUT);
  pinMode(led5, OUTPUT);
  pinMode(led6, OUTPUT);
  Serial.begin(9600); //serial for debug

}

void loop() {

  activate_particle(particle[0],led);
//  activate_particle(particle[1],led);

//  activate_particle(particle[2],led);
//  activate_particle(particle[3],led);
//  activate_particle(particle[4],led);
//  activate_particle(particle[5],led);
//

 delay(5); // 20-17-15-12-10-7-2

}

Mytypes.h
typedef struct {
    int id;
    int led_position;
    int brightness;
    int fadeAmount;
    bool reset;
} particle_type;

typedef struct {
    int led_position;
    bool busy;
} led_type;

This seems to work more or less but I still have the problem of particles "choosing" the same led to light up and it make the led flickers.
I am struggling on this. I have tried to add a data field for busy and create a new struct for led to no avail.
Even when only one particle is on, the different leds seems to deactivate after a few rounds.
Let me know if you figure it out, thanks!
EDIT 2:
    #include "MyTypes.h"

int led0 = 3;    // the PWM pin the LED is attached to
int led1 = 5;
int led2 = 6;
int led3 = 9;
int led4 = 10;
int led5 = 11;

//int led;
int brightness;
int fadeAmount;

// Definition of the different particles
particle_type particle[]={
  { 0,led0,0, 4,   false },
  { 1,led1,50, 4,   false },
  { 2,led2,100,  4,   false },
  { 3,led3,80,  4,   false },
  { 4,led4,30, 4,   false},
  { 5,led5,70,  4,   false},
};

led_type led[]={
  { 3,false },
  { 5,false },
  { 6,false },
  { 9,false },
  { 10,false },
  { 11,false },
};

//initiation of the particles

void initiate_particle(particle_type& particle_example,led_type led[]){

 //take a random led from list of led
  int number=random(6);
  Serial.println(number);
  led[number];
  if (led[number].busy == false){ //if led non-occupied
    for (int i=0;i<6;i++){
      if (led[i].led_position==particle_example.led_position){
      led[i].busy=false;
      Serial.println("success");
      Serial.println(led[i].led_position);
      }
    }
    particle_example.led_position=led[number].led_position;
    Serial.println(particle_example.led_position);
    led[number].busy = true ;//take this led as output
    particle_example.reset = false; //reset turned off
    }

  else{
//  initiate_particle( particle_example,led);
}

  delay(30);

}

void update_light(particle_type& particle_example,led_type led[]){ //updating led brightness
      analogWrite(particle_example.led_position, particle_example.brightness);  //updating led brightness

  if (particle_example.brightness >=  160) {
    particle_example.fadeAmount = -particle_example.fadeAmount;
    particle_example.brightness = particle_example.brightness + 2*particle_example.fadeAmount;
  }

  else if (particle_example.brightness <  160 && particle_example.brightness >=  100 ) {
    particle_example.brightness = particle_example.brightness + 1.5*particle_example.fadeAmount;

  }

  else if (particle_example.brightness > 50 && particle_example.brightness <=  100){
      particle_example.brightness = particle_example.brightness + particle_example.fadeAmount;

  }

  else if (particle_example.brightness > 0 && particle_example.brightness <=  50){
      particle_example.brightness = particle_example.brightness + 0.25*particle_example.fadeAmount;

  }

  else if (particle_example.brightness <= 0){
    particle_example.reset= true;
    particle_example.brightness =0;
    int number = particle_example.led_position;

    if (particle_example.fadeAmount<0){
      particle_example.brightness = 0;
      particle_example.fadeAmount = -particle_example.fadeAmount;
    }
    else{
      particle_example.brightness = particle_example.brightness + 0.5*particle_example.fadeAmount;
    }

  }

}

void activate_particle(particle_type& particle_example,led_type led[]){ //function to keep led activated

  if (particle_example.reset==true){ //if led not initiated or has finished its dimming down
    initiate_particle(particle_example, led); //initiate it again
  ;}

  else {
   update_light(particle_example,led); //update its brightness
  }
}

void setup() { //create led outputs
  pinMode(led0, OUTPUT);
  pinMode(led1, OUTPUT);
  pinMode(led2, OUTPUT);
  pinMode(led3, OUTPUT);
  pinMode(led4, OUTPUT);
  pinMode(led5, OUTPUT);

  Serial.begin(9600); //serial for debug

}

void loop() {

  activate_particle(particle[0],led);
  activate_particle(particle[1],led);
  activate_particle(particle[2],led);
//  activate_particle(particle[3],led);
//  activate_particle(particle[4],led);
//  activate_particle(particle[5],led);
//

 delay(10); // 20-17-15-12-10-7-2

}

void loop() {

  activate_particle(particle[0],led);
  activate_particle(particle[1],led);
  activate_particle(particle[2],led);
  activate_particle(particle[3],led);
//  activate_particle(particle[4],led);
//  activate_particle(particle[5],led);
//

 delay(10); // 20-17-15-12-10-7-2

}

Now the code works well up to 3 particles but messes things up for 4,5,6 particles with flickering and absence of light.

That seems like a logic problem for me. You define a "particle", which is not bound to a special LED (like a particle, that is jumping from one position to another). Then you included a busy flag, that should prevent, that a LED is used by more than one particle at a time. But you included this flag in the particle. Being busy is a state of the LED, not of the particle, so including this in the particle won't do, what you want.
If you want to use this logic, you need to create a struct, which represents the individual LED. The struct's fields would be the pin number and the busy flag. You can create the LED list with instances of this struct. Then to choose a new position for the particle, you need to check if the busy flag of that position is set. If not, choose this position and set the busy flag. Clear the flag, when you remove the particle from the position.

After your edit I see more problems. First a general problem: Currently a new particle, that want's to change it's position, needs a free (not busy) position to move to. You have 6 LED positions, so with 6 particles, there won't be any move. You can change this by freeing a LED position (setting it's busy flag to false), when the fading of a particle is finished. Still there might be timing problems with this way, depending on the overall timing behavior of the sketch (The first particle to free a LED position will not find a different free position and has to wait. This might lead to 2 particles just exchanging there positions).
Then in void update_light() you have multiple if statements, which all test the particles brightness. But inside these statements you change the brightness. So it is possible, that multiple of the if statements will execute. You should use if else if instead, so that only one of the statements can execute.
And in void initiate_particle() you check for the chosen LED position to be busy. But if the position is already busy, you are setting the particles reset flag to false anyway. That means that activate_particles() will update the particles brightness despite the fact, that the particles does not have a valid LED position currently. You should not set reset to false, if the particles hasn't got a valid LED position. And the delay at the end of the initiate function does not serve a function in my eyes.

Hi chrisl, sorry for the flood :). Thanks for your answer, now the system works better with 1 to 3 particles but still has trouble with 4,5,6 at the same time.
Let me know if you have an idea.

@Guillaume_slize I added further explanations to my answer

Hi Chris, thanks a lot for the long explanation. It makes things much clearer, I changed the code according to your advices but it is now experiencing some weird behavior (some led lighting just a little).
Two led exchanging their positions or staying in the same when 6 lights/ particles are up is fine, I was hoping for a system that could sustain all configuration from 1 to 6, but might have to code another one for 6.

Ok, I corrected a few mistakes and now it seems to work fine up until 4. I think 5 and 6 is too unstable for so few leds.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
Menus not changing when button is pressed TFT 128 x 128
I've just followed a tutorial so that when a button is pressed it goes through a menu. Click once, next slide. Only issue is it isn't appearing on my screen at all. It has uploaded but I'm a total novice when it comes to breadboards etc. Here is my code:
    #include "Ucglib.h" // library for this display
#include <TFT.h> // Hardware-specific library
#include <SPI.h>

#define CS   10
#define DC   8
#define RESET  9

#define up 5
#define down 6

TFT TFTscreen = TFT(CS, DC, RESET);
const int menuSize = 4;
String menuItems(menuSize);

bool upLastState = HIGH;
bool downLastState = HIGH;

int currentMenu = 0;
String temp;
char currentPrintOut(10);

void setup()
{
  Serial.begin(9600);

  pinMode(up, INPUT_PULLUP);
  pinMode(down, INPUT_PULLUP);

  menuItems[0] = ":)";
  menuItems[1] = ":(";
  menuItems[2] = "X(";
  menuItems[3] = ":S";
  menuItems[4] = ":T";

  //initialize the library
  TFTscreen.begin();

  MenuChanged();
}

void loop() {
  if (digitalRead(up) != upLastState) {
    upLastState = !upLastState;
  }

  if (!upLastState) {
    if (currentMenu > 0) {
      currentMenu--;
    } else {
      currentMenu = menuSize - 1;
    }
    MenuChanged();
  } else {
  }

 if (digitalRead(down) !=downLastState) {
  downLastState = !downLastState;

  if (!downLastState) {
    if (currentMenu < menuSize - 1) {
      currentMenu++;
    } else {
      currentMenu = 0;
    }
    //Serial.printIn(menuItems[currentMenu]);
    MenuChanged();
  } else {
    //Release
  }
 }

   delay(50);
 }

 void MenuChanged() {
  Serial.println(menuItems[currentMenu]);

// clear the screen with a black background
  TFTscreen.background(0,0,0);
  TFTscreen.setTextSize(9);
  TFTscreen.stroke(255,255,255);

  temp = String(menuItems[currentMenu]);
  temp.toCharArray(currentPrintOut, 10);

  TFTscreen.text(currentPrintOut, 40, 32);
}

Edited back to original as per request

remove #define up 5 and #define down 6, we already defined pin 5 for the button. also remove TFTscreen.text(":)", 40, 32); which is in front of defualt:. the serial.begin() and serial.println() and delay(100) are just for testing the code, you can remove them as well.

Hi! I've done all that now. It's still racing through every single face so I'm going to try the de bounce and see if it's that. I can't even press a button, it just does it itself haha.  Sorry I wasn't sure if I needed to write anything for default! After this I only have one major thing left to do in my code (that you'll probably wind up helping me again im so sorry I'm so new at this). I'll try my best to fix the face speed through

Please revert back your original code so others can see what was the problem, you basically cleared the question...

sure thing, I also had another problem now though haha. I tried to debounce but the faces are even faster now. It's going inbetween faces and won't actually stop on one and still wont let me click the button to activate it :/

updated the answer with de-bounce, if your questions in this site got an answer which solved your problem please pick them as correct answer by clicking on the "tick" next to the answer. is your push-button normally open or is it normally close?

Sorry to bother you so much. I guess it wasn't the debounce (I wasnt wrong for once! I did it right go me), it's still going through all the cases just slower. Is there a way to stop on that face if the button is actually pressed? I'm trying to work it out on my own but it just keeps repeating going through the faces and not actually pausing on the one I want when the button is pressed :/

i suspect it's because you have a normally close push-button, which means push-button is connected until you press it. link. is it a normally close push button?

Ah I see that makes sense. I guess I can try to make the space wider with resistors etc? Even thought that's still the electrical current but can't hurt to try

change the push button with a normally open push button OR change this line: if (buttonStatus == LOW) {} to if (buttonStatus == HIGH) {}

Hey newbie, I put the button in the middle of the board (like originally) and put both the wires on each end. It somewhat works! I need to double tap the button because sometimes it decides to do it again but I'll try your way too. I'll leave you be for now <3. Thank you so much for your help. I appreciate it more than you know. Here is my victory photo.

you're welcome.

Edit: I managed to fix my problem by putting the wires diagonally (ground left pin, 5 right side) if anyone in future has my problem. Back to the victory dance =)

You can make things much simpler:
int count = 0;
int button = 5;
int buttonStatus;
unsigned long timer = 0;

void setup() {
  Serial.begin(9600);
  pinMode(button, INPUT_PULLUP);
}

void loop() {
  buttonStatus = digitalRead(button); // read the button status
  if (buttonStatus == LOW) {
    if (millis() - timer > 200) { //de-bouncing the button
      count++; // increase +1 each time button is pressed
      // clear screen function should be here
      timer = millis();
    }
  }
  if (count > 3){ // if count is bigger than 3 start over.
    count = 0;
    // clear screen function should be here
  }
  switch (count){
    case 1: // if count == 1
    // show charecter :)
    break; // break out of this case!
    case 2:
    // show charecter :(
    break;
    case 3:
    // show charecter :D
    break;
    default: // if none of above cases is valid
    count = 0;
    // clear screen function should be here
  }
  Serial.println(count);
  delay(100);
}

simulate this circuit – Schematic created using CircuitLab
So each time button is pressed count value will increase by 1. you can also add another button to decrease this value...
You might also notice when pushing the button it'll count more than once, you'll also need to De-Bounce the button. tutorial.
Edit: added de-bounce function.

Hi again newbie. So with this, I say like Case 1: TFTscreen.stroke(0,0,0);
TFTscreen.text(":)"); with all my colour settings? Do I also put in void set up that I want the background to be black? I just want to make sure this is what you meant by your thing you sent me, or do you mean to still have the button in the middle but the wires on the other side? link. My TFT is just blank and wont show up anything before so I wan't to clarify before I start. Thanks again

@Ijustneedtopassprobs Only two wire goes to the button (what's the third wire in that picture?), ground and pin 5 in this case. yes define your libraries at top like you did in your original code and the sittings you want in void setup. also read the comments in the code that i posted.

I did remove that one after, sorry I figured that out eventually. It's going through all the faces at the moment. I made up to 5 cases. It's pretty much spamming. I haven't touched the button yet. I'm going to edit my code so you can see my new code to see if I'm extra dumb or not =), once again ty for this

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Powering Peltier using solar panels
I am a beginner to electronics and trying to make a portable air conditioner using a Peltier cooling pad (12V). I am planning on using a solar panel to power the Peltier and looking forward to your expert opinion  for below;
- Is this possible?
-What the specifications I have to consider prior to purchasing the solar panel?
- What is the weight of the solar panel?
- Can I operate the Peltier through an Arduino using Solar panel as the power source( without using batteries to power Arduino and Peltier). My idea is to control the Peltier cooler via Arduino using a phone App.
Really sorry if these questions are basic, yet your guidance will be very much helpful for me as well as for the other beginners like me.

If you don't want to use batteries, but a direct relation between solar radiation and the peltier effect, what's the purpose of an arduino?

Limit current based on temp in cooler, it will be more efficient. You don't need to use 12V, go lower. DS18B20 is a good option if you're digital, else consider a dielectric temp switch.

@user2497, Thank you for the suggestion and I will be able to achieve the required maximum cooling level even with 6V. Controlling the cooling level of the Peltier with the help of a temperature sensor is an interesting idea. I will definitely try this.

@Indee An LDR and a potentiometer, along with a P-FET should make a decent driver circuit.

A microcontroller like an Arduino needs a constant source of regulated 5V (or 3.3V for some models.) The voltage of a solar panel varies depending on the amount of light, and in low light conditions there won't be enough power to run the Arduino.
So the short answer is no, you won't be able to run the Arduino reliably from a solar panel without a battery. You probably want a battery and a buck/boost voltage regulator that can convert a wide range of voltages to the 5V the Arduino needs.
What do you want the Arduino for? Temperature regulation? You'd probably be better off using a bimetallic thermostat that would turn the power to the Peltier pad on and off without needing electronics.
As for running a Peltier junction cooling pad from a solar panel, that should work. It would depend on the specs of the Peltier unit and of the panel. Again, you'll probably get the maximum cooling if you use a buck/boost voltage regulator to convert the variable output of your solar panel to a steady 12V. You'll need a big enough solar panel to provide the current that your Peltier pad needs.
All the talk about Peltier pads and solar panels is not Arduino-specific, however.
Edit:
Research solar chargers for 12V batteries. Buy a solar panel made to charge a 12V battery. Make sure you select a solar panel that puts out enough current to run the Peltier panel at full power when it is in the sun. Ideally, select a solar panel that provides several times more current than needed to run the Peltier pad. That way the solar panel will charge the battery AND run the Peltier pad while the sun is shining, and when the sun is behind the clouds or at night, the Peltier pad will continue to run from the battery.
Start from that. Buy a buck/boost power supply that can convert a wide range of voltages to 5V. Use that to power the Arduino.
Buy a digital thermometer (like a DHT11 for example). Follow the instructions to wire it into your Arduino, and write a sketch that measures the temp in the thing you are trying to cool.
Buy a power MOSFET transistor that is made to work from a 5V logic signal. This one, for example: https://www.sparkfun.com/products/10213
Wire the MOSFET so a logic signal from one of the digital pins on the Arduino is connected to the gate of the MOSFET and it switches the power to the Peltier pad on and off. (You should be able to find a wiring diagram online.)
Write a sketch that measures the temp from the digital thermometer every few seconds, and turns on the power to the MOSFET if the temp rises above a threshold temp, and turns it off if it drops below a colder temp. (Say it turns on the MOSFET when the temp rises above 24°C, and turns the MOSFET off when the temp drops to 20°C. (This is really simple code to write.)
Note that if the battery gets too low, it won't provide enough power to the voltage regulator which supplies the Arduino and it will shut down. That will cause the Peltier pad to turn off. Once the solar panel charges the battery up enough, the voltage regulator will start providing 5V to the Arduino again, it will restart, your sketch will start running, and the cycle will continue. When the battery barely has enough power to drive the Arduino, it may restart the Arduino, try to turn on the Peltier pad, and immediately die again, every few seconds. That would be bad, and would put a lot of stress on all the electronics.
You might want to add a voltage sensor¹  to the battery and have the Arduino measure the battery voltage, and not try to power the Peltier pad unless and until the battery has a reasonable charge. Instead it would monitor the battery charge level and wait until there was enough power stored in the battery to run the Peltier pad. You could also have the Arduino detect that the battery charge level was getting low, and shut off the Peltier pad before it got too low.
A lead/acid battery would be a good choice since it's cheap, holds a lot of power, and it's easy to estimate the remaining power in a healthy lead/acid cell from its output voltage.
If you do use a lead-acid battery, it's important to add a battery charge level sensor and make your sketch avoid draining the battery too low. Lead-acid batteries are damaged when you discharge them too much.
Edit #2:
You said in a comment to another answer that you're looking to provide small-scale cooling while traveling in a hot climate: The fact that you're trying to handle travel poses some interesting challenges, and perhaps some opportunities. If you're traveling in a car or truck you could tap the 12V power that is almost always available in a motor vehicle. If you don't have an external source of power then the shifting and turning you will likely experience while traveling will greatly reduce the efficiency of your solar panel. They only give maximum power when the panel is directly perpendicular to (or "normal to") the sun. As the angle of the panel to the sun shifts from the ideal angle the power output of your panel will drop significantly.

¹ A voltage sensor could be as simple as a voltage divider that reduces the input voltage from a max of a little under 14V down to 5 V, fed into an analog input on the Arudino. You might need to disconnect the solar panel input before measuring the battery charge level however, since input from the solar panel will make the battery voltage seem higher than it actually is. Also note that your battery charge level estimates are going to be crude, since you should really let a lead-acid battery rest with no input and no output for several minutes before taking a measurements in order to get an accurate reading.

Hi Duncan,  Your answer is very helpful and informative. I will work on the solar Peltier circuit. I wanted the Arduino to control the cooling effect of the Peltier using a phone App. Seems like I have to use a battery and voltage regulator to obtain this.It

It would be a great support if you could help me with the Solar, Arduino, Voltage regulator and battery circuit?

Hi DataFiddle, I need the Arduino to control the Peltier using a phone App.

See the edit to my answer. I gave you some guidelines on what to do. I'm not going to design the project for you in detail however.

Hi Duncan, The answer is very helpful and have enough information for me to try it by myself. Thank you very much.

@Indee I added a little more detail about lead-acid batteries.

Hi Duncan, Thank you very much and the guidance you provided is highly appreciated.

Your first decision is to figure out the heat load on the space. How many BTU/hr you need to remove to keep the temperature you want. Then figure out how large / how many Peltier you need to remove that much heat. Do you need some reserve capacity for unusual circumstances or is a higher inside temperature Ok sometimes? Then you need a power budget - how much power will it take to remove that much heat with Peltier panels. How much PV area will you need to provide that power? If you need cooling when the sun isn't out, your power budget has to include the 24-hour demand, and the intermittent supply, perhaps again with some headroom. Power storage - batteries or the power grid - will be a requirement to match the two. The same storage can power your Arduino. The Arduino's power budget will be minuscule compared to the rest of the installation and you can totally ignore it. Do you have to power your internal network that connects the Arduino to the Internet, too? Figure that in. By this point, you'll have the answers to most of your questions. Just a heads-up: A Peltier beer-cooler is quite doable. A whole house A/C might be doable but no practical - you probably won't like the numbers! I'm guessing your project lies somewhere between the two.

Well, yeah. If you need to solve a real-world cooling problem, that is indeed the way to go. (voted.) BUT BTUs? Really?!?

@JRobert: Thank you very much for your answer. My project is closer to 'a Peltier beer-cooler' as I'm planning on to develop a Peltier cooler to provide cooling to food and beverages when travelling under hot climate conditions. Seems  like reserving energy is an important aspect which I need to consider.


text:
Stuck in a loop when reading from EEPROM, works if I clear it first
EDIT: The problem was caused by storing a pointer in the EEPROM. Thanks to @KIIV for helping out
Sorry for the enormous of wall of text, I'm trying to pare it down a bit.
I have two Arduino Megas that are misbehaving. Both boards produce identical output, so I'm certain it's not a hardware issue.
I'm trying to store user configuration data on the EEPROM and load it when the arduino boots. I have an object which contains flags for user settings like baud rates and serial port modes.
During void setup(), it fails to load the config from the EEPROM. It gets stuck looping through a few functions, even though those functions are not loops at all.(more details below)
However if I clear the EEPROM during void setup(), I can then save and load configurations later on, using menus invoked by the user during void main(). This works flawlessly. I'm calling the same load function in both cases, it just causes the arduino to shit itself if I call it during setup().
Here is a condensed version of my code:
struct UserCfg {/* config options */}

void ConfigController::load_eeprom_cfg()
{
    Serial3.println("loading from eeprom");
    EEPROM.get(gp::config_address, UserCfg);
}

void ConfigController::write_user_config()
{
    Serial3.println("writing user config to eeprom");
    EEPROM.put(gp::config_address, UserCfg);
}

void ConfigController::apply_user_config()
{
    /* other serial ports omitted */
    Serial3.begin(UserCfg.serial_cfg_3.ser_speed, UserCfg.serial_cfg_3.ser_mode);
    Serial3.write("serial 3 up\n\r");

    // Allows users to set the admin terminal to any serial port
    Admin_Serial = UserCfg.admin_serial_port;
    Admin_Serial->write("Admin terminal up on this port\n\r");
}

/// checks and loads config at system boot
void ConfigController::startup()
{
    Serial3.begin(9600);
    Serial3.println("loading...");

    load_eeprom_cfg(); //<----- Not working when called here.

    // prevents loading non-existent config. Not robust, but it works for now.
    if (UserCfg.admin_port_str == nullptr)
    {
        Serial3.println("No valid config found. Using defaults.");
        write_default_cfg();    // puts hard coded defaults in the EEPROM.
        load_eeprom_cfg();      // works just fine when called here
    }
    Serial3.println("Applying...");

    apply_user_config(); //<----- Also not working when called here.
}

// Main code:

void setup()
{
    // erase_EEPROM() is is copied from the official arduino tutorial.
    ConfigController.erase_EEPROM();     // If I comment this line out, it breaks.
    ConfigController.startup();
    Serial3.print("startup done");
}

If I remove the call to erase_EEPROM from void setup(), the arduino gets stuck in a loop:
loading...
loading from eeprom
Applying...
serial 3 up

Those lines just repeat over and over.
However, when I call it exactly as you see above in void setup(), it works perfectly. The EEPROM is erased, the IF statement in ConfigController::startup() catches the bogus value, and writes the default config to the EEPROM. Once it's done, it reads it back from the EEPROM into the UserCfg, which is displayed in the menus. I can then save / load values to my hearts content from the menu like this:
write_user_config();
load_eeprom_cfg();
apply_user_config();

or for loading defaults:
write_default_cfg();
load_eeprom_cfg();
apply_user_config();
print_menu_prompt();  // This includes values from UserCfg, so I can confirm that changes are saved and then loaded.

I fail to see a functional difference between calling load_eeprom() during void setup(), vs calling it during void main(). It's the exact same function that put the value in the EEPROM to begin with.
Any help?

My guess it isn't looping at all, it resets whole mcu, or calling that Admin_serial->write with invalid pointer jumps somewhere near to reset vector. Definitely not good idea to call some variable exactly the same as type (=> UserCfg)

Earlier I tried adding in a bunch of null pointer checks to Admin_Serial, like this:
if(Admin_Serial == nullptr) Serial3.print("bad pointer to ser3");
No dice. If the pointer is bad, it is at least not null.

It doesn't have to be nullptr. It can be anything

Of course! I should add a check to see if it's equal to Serial, Serial1, Serial2, or Serial3.

You've gotten me thinking about something. I'm storing a pointer in the EEPROM. That seems like a really bad idea. I cannot imagine there being any guarantee whatsoever that a pointer will be at the same memory address every time.
Even if this is not the cause of the problem, I should probalby switch to storing an enum value, and using that to set the serial pointer when I load the settings.

I'd go for port nuber instead and getting pointer in each run. You can get different pointers if you upload changed code.

Btw, you skipped that UserCfg definition and it might be much faster to figure it out you are storing pointers.


text:
ESP32-CAM not reading BME680
I am trying to read BME680 on ESP32-CAM with following code :
(Before trying BME680, I have suscessfully connected BME280 to ESP32-CAM over S/W SPI)
#include <Wire.h>
#include <SPI.h>
#include <Adafruit_Sensor.h>
#include "Adafruit_BME680.h"

#define SEALEVELPRESSURE_HPA (1013.25)

#define BME_SCK 12 //Connected SCL to IO12
#define BME_MISO 15 //Connected SDO to IO15
#define BME_MOSI 13 //Connected SDA to IO13
#define BME_CS 14 //Connected CS to IO14

//Adafruit_BME280 bme; // I2C
Adafruit_BME680 bme(BME_CS, BME_MOSI, BME_MISO, BME_SCK); // software SPI
//float temperature = 0;
//float humidity = 0;
//float pressure = 0;

void setup() {
  Serial.begin(115200);
  while (!Serial);
  Serial.println(F("BME680 test"));

  if (!bme.begin(0x76))
  {
    Serial.println("Could not find a valid BME680 sensor, check wiring!");
    while (1);
  }

  // Set up oversampling and filter initialization
  bme.setTemperatureOversampling(BME680_OS_8X);
  bme.setHumidityOversampling(BME680_OS_2X);
  bme.setPressureOversampling(BME680_OS_4X);
  bme.setIIRFilterSize(BME680_FILTER_SIZE_3);
  bme.setGasHeater(320, 150); // 320*C for 150 ms
}

void loop()
{
  if (! bme.performReading())
  {
    Serial.println("Failed to perform reading :(");
    return;
  }
  Serial.print("Temperature = ");
  Serial.print(bme.temperature);
  Serial.println(" *C");

  Serial.print("Pressure = ");
  Serial.print(bme.pressure / 100.0);
  Serial.println(" hPa");

  Serial.print("Humidity = ");
  Serial.print(bme.humidity);
  Serial.println(" %");

  Serial.print("Gas = ");
  Serial.print(bme.gas_resistance / 1000.0);
  Serial.println(" KOhms");

  Serial.print("Approx. Altitude = ");
  Serial.print(bme.readAltitude(SEALEVELPRESSURE_HPA));
  Serial.println(" m");

  Serial.println();
  delay(2000);
}

When I try to upload the code to ESP32-CAM while BME680 is connected I get following error
esptool.py v2.6
Serial port /dev/ttyUSB0
Connecting....
Chip is ESP32D0WDQ6 (revision 1)
Features: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None
MAC: a4:cf:12:99:b5:70
Uploading stub...
Running stub...
Stub running...
Configuring flash size...
Warning: Could not auto-detect Flash size (FlashID=0xffffff, SizeID=0xff), defaulting to 4MB
Compressed 8192 bytes to 47...

A fatal error occurred: Timed out waiting for packet content
A fatal error occurred: Timed out waiting for packet content

Then I tried to disconnect BME680 and uploaded the code to ESP32-CAM which worked, then connected BME680 back to ESP32-CAM.
But now I see following error in Serial Monitor :
ets Jun  8 2016 00:22:57

rst:0x1 (POWERON_RESET),boot:0x33 (SPI_FAST_FLASH_BOOT)
flash read err, 1000
ets_main.c 371
ets Jn  8 2016 00:22:57

rst:0x10 (RTCWDT_RTC_RESET),boot:0x33 (SPI_FAST_FLASH_BOOT)
flash read err, 1000
ets_main.c 371
ets Jun  8 2016 00:22:57

rst:0x10 (RTCWDT_RTC_RESET),boot0x33 SPI_FAST_FLASH_BOT)
lash read err, 1000
et_main.c 371
ets Jun  8 2016 00:22:57

rst:0x10 (RTCWDT_RTC_RESET),boot:0x33 (SPI_FAST_FLASH_BOOT)
flash read err, 1000
ets_main.c 371
ets Jun  8 2016 00:22:57

rst:0x10 (RTCWDT_RTC_RESET),boot:0x33 (SPI_FAST_FLASH_BOOT)
flash read err, 1000
ets_main.c 371
ets Jun  8 2016 00:22:57

Any idea what is wrong here ?
Update :
I switched #define BME_SCK 12 to #define BME_SCK 16, now getting following message
ts Jun  8 2016 00:2:57

rst:0x1 (POWEON_REST),boot:0x13 (PI_FAT_FLASH_BOOT)
cnfigsi: 0, SPIWP:0xee
clk_dv:0x00,q_drv:0x0,d_drv0x00,cs0_drv:0x00,d_drv0x00,wp_drv:0x0
mod:DIO, clock div:
loa:0x3fff0018,len:4
load0x3fff001c,len:1216
ho 0 tail 12 room 4
lod:0x40078000,len:9720
h 0 tail 12 room 4
load0x40080400,len:6352
etry 0x400806b8
[D][esp32-hal-psram.c:47] psramInit(): PSRAM enabled

If you read the schematic of ESP32_cam, you will noticed that both GPIO12 and GPIO16 are used by the microSD.

@hcheung, SPI is a bus, but yes if a card would be inserted the SPI wiring must match the wiring of the card

I am not using SD Card. Also BME280 worked perfectly using BME_SCK=14, BME_MISO=12, BME_MOSI=13, BME_CS=15.

GPIO12 must not be pulled high during boot. It's possible there's a pull-up resistor on the BME680 breakout board you're using that's pulling SCK high and interfering with the boot process. I would avoid using GPIO12 here.
If you're out of pins on the ESP32 you can connect the BME680 via I2C rather than SPI. I2C only needs SDA and SCK (and power and ground, of course). The Adafruit library for the BME680 has an example of how to call the constructor for an I2C connection.

I am using ESP32-CAM, not all IO pins are available also I2C not available, so only software SPI is option. I switched BME_SCK fro 12 to 16, getting new error.

You use pins
#define BME_SCK 12 //Connected SCL to IO12
#define BME_MISO 15 //Connected SDO to IO15
#define BME_MOSI 13 //Connected SDA to IO13
#define BME_CS 14 //Connected CS to IO14

for software SPI, but the esp32 can use any set of pins for hardware SPI.
And your set of pins are even the dedicated mux pins for SPI2 (they have the direct connection without mux). But they are in different mapping then your software SPI.
SPI2:
SCLK    14
MISO    12
MOSI    13
SS      15

Source: https://docs.espressif.com/projects/esp-idf/en/latest/api-reference/peripherals/spi_master.html
SPI2 is the default SPI used by the ESP32 Arduino SPI library, so the Adafruit library should have no problem to use hardware SPI with the SPI library.
Adafruit_BME680 bme(BME_CS); // use hardware SPI

and wire the sensor to SPI2 pins according to their function

@roy, I enhanced the answer

With this suggested change I get the last error message mentioned in my question.

try to connect the wire to pin 12 only after the esp32 powerup. it could be the problem in answer by John

Tried this as well, still same error

as next I would try the sensor with some simple board like Uno or Nano


text:
esp32 arduino wifi disconnect
I have this code running on ESP32 connected to SVM30-J
#include <WiFi.h>
#include <PubSubClient.h>
#include <Arduino_JSON.h>
#include <svm30.h>

// Replace the next variables with your SSID/Password combination
const char* ssid = "xxxx";
const char* password = "xxxx";

// Add your MQTT Broker IP address, example:
const char* mqtt_server = "xxxx";
const int mqtt_port = 18712;
const char* mqtt_user = "xxxx";
const char* mqtt_password = "xxxx";

/* define driver debug
 * false : no messages
 * true : display driver debug messages
 */
#define DEBUG true

// define number of seconds between display results
#define DELAY 10

// function prototypes (sometimes the pre-processor does not create prototypes themself on ESPxx)
void read_id();
void read_featureSet();
void Errorloop(char *mess);
void read_values();
void read_baseline();
void KeepTrigger(uint8_t del);

// create instance
SVM30 svm;

WiFiClient espClient;
PubSubClient client(espClient);

void setup() {

  Serial.begin(115200);

  setup_wifi();
  client.setServer(mqtt_server, mqtt_port);
  //client.setCallback(callback);

  Serial.println(F("Reading information from the SVM30"));

  // enable debug messages
  svm.EnableDebugging(DEBUG);

  svm.begin();

  Serial.print(F("Driver version : "));
  Serial.println(svm.GetDriverVersion());

  // try to detect SVM30 sensors
  if (svm.probe() == false) Errorloop("could not detect SVM30 sensors");
  else Serial.println(F("SVM30 detected"));

  // read and display the ID
  read_id();

  // read SGP30 feature set
  read_featureSet();
}

void loop() {
  char chipId[13]; // Don't forget one byte for the terminating NULL...
  uint64_t mac = ESP.getEfuseMac();
  sprintf(chipId, "%012x", mac);
  Serial.println(chipId);

  JSONVar data;

  if (!client.connected()) {
    reconnect();
  }
  client.loop();

  // read measurement values
  read_values();

  // wait x seconds
  KeepTrigger(DELAY);
}

void setup_wifi() {
  delay(10);
  // We start by connecting to a WiFi network
  Serial.println();
  Serial.print("Connecting to ");
  Serial.println(ssid);

  WiFi.begin(ssid, password);
  WiFi.setSleep(false);

  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }

  Serial.println("");
  Serial.println("WiFi connected");
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());
}

void callback(char* topic, byte* message, unsigned int length) {
  Serial.print("Message arrived on topic: ");
  Serial.print(topic);
  Serial.print(". Message: ");
  String messageTemp;

  for (int i = 0; i < length; i++) {
    Serial.print((char)message[i]);
    messageTemp += (char)message[i];
  }
  Serial.println();

  // If a message is received on the topic esp32/output, you check if the message is either "on" or "off".
  // Changes the output state according to the message
  if (String(topic) == "esp32/output") {
    Serial.print("Changing output to ");
    if(messageTemp == "on"){
      Serial.println("on");
      //digitalWrite(ledPin, HIGH);
    }
    else if(messageTemp == "off"){
      Serial.println("off");
      //digitalWrite(ledPin, LOW);
    }
  }
}

void reconnect() {
  // Loop until we're reconnected
  while (!client.connected()) {
    Serial.print("Attempting MQTT connection...");
    // Attempt to connect
    if (client.connect("ESP32Client", mqtt_user, mqtt_password)) {
      Serial.println("connected");
      // Subscribe
      //client.subscribe("esp32/output");
    } else {
      Serial.print("failed, rc=");
      Serial.print(client.state());
      Serial.println(" try again in 10 seconds");
      // Wait 10 seconds before retrying
      delay(10000);
    }
  }
}

/*
 * @brief : keep triggering SGP30 while waiting
 *
 * @param del : number of seconds to wait
 *
 * source datasheet SVM30:
 * The on-chip baseline compensation algorithm has been optimized
 * for 1HZ sampling rate. The sensor shows best performance when
 * used with this sampling rate.
 *
 */
void KeepTrigger(uint8_t del)
{
  uint8_t w_seconds = del;
  unsigned long startMillis;

  if (w_seconds == 0) w_seconds = 1;

  while (w_seconds--)
  {
    startMillis = millis();

    if (! svm.TriggerSGP30())
      Errorloop("Error during trigger waiting");

    // this gives 1Hz /1000ms (aboutisch)
    while(millis() - startMillis < 1000);
  }
}

/*
 * @brief : read and display the values from the SVM30
 *
 */
void read_values() {
  struct svm_values v;
  uint16_t baseline;

  if (! svm.GetValues(&v))
      Errorloop("Error during reading values");

  JSONVar data;

  Serial.print(F("CO2 equivalent : "));
  Serial.print(v.CO2eq);
  data["co2_equivalent"] = v.CO2eq;

  Serial.print(F(", TVOC : "));
  Serial.print(v.TVOC);
  data["tvoc"] = v.TVOC;

  Serial.print(F(", H2_signal : "));
  Serial.print(v.H2_signal);
  data["h2_signal"] = v.H2_signal;

  Serial.print(F(", Ethanol_signal : "));
  Serial.print(v.Ethanol_signal);
  data["ethanol_signal"] = v.Ethanol_signal;

  Serial.print(F(", Humidity : "));
  Serial.print((float) v.humidity/1000);
  data["humidity"] = (float) v.humidity/1000;

  Serial.print(F(", temperature : "));
  Serial.print((float) v.temperature/1000);
  data["temperature"] = (float) v.temperature/1000;

  Serial.print(F(", absolute humidity : "));
  Serial.print(v.absolute_hum);
  data["absolute_hum"] = v.absolute_hum;

  if (! svm.GetBaseLine_CO2(&baseline))
      Errorloop("could not read SGP30 CO2 baseline");

  Serial.print(F(" CO2 equivalent baseline : 0x"));
  Serial.print(baseline, HEX);
  data["co2_equivalent_baseline"] = (baseline, HEX);

  if (! svm.GetBaseLine_TVOC(&baseline))
      Errorloop("could not read SGP30 TVOC baseline");

  Serial.print(F(", TVOC baseline : 0x"));
  Serial.println(baseline, HEX);
  data["tvoc_baseline"] = (baseline, HEX);

  char chipId[13]; // Don't forget one byte for the terminating NULL...
  uint64_t mac = ESP.getEfuseMac();
  sprintf(chipId, "%012x", mac);
  data["chipid"] = chipId;

  Serial.print("Data = ");
  Serial.println(data);

  String data_json = JSON.stringify(data);
  char mqtt_publish_topic[] = "esp32/data";
  client.publish(mqtt_publish_topic, data_json.c_str());

}

/*
 * @brief: read and display the id of the SGP30 and SHTC1 sensors
 */
void read_id() {

  uint16_t buf[3];    // SGP30 needs 3 words, SHTC1 is 1 word
  char  id[15];

  if ( ! svm.GetId(SGP30, buf))
      Errorloop("could not read SGP30 id");

  Serial.print(F("\nSGP30 id : "));
  sprintf(id, "%04x %04x %04x", buf[0], buf[1], buf[2]);
  Serial.println(id);

  if (svm.GetId(SHTC1, buf) == false)
      Errorloop("could not read SHTC1 id");

  Serial.print(F("SHTC1 id : "));
  // only bit 5:0 matter (source: datasheet)
  sprintf(id, "%04x", buf[0] & 0x3f);
  Serial.println(id);
}

/*
 * @brief: read and display the product feature set of the SGP30 sensor
 */
void read_featureSet(){

  char buf[2];

  if ( ! svm.GetFeatureSet(buf))
      Errorloop("could not read SGP30 feature set");

  Serial.print(F("\nSGP30 product type : "));
  Serial.print((buf[0] & 0xf0), HEX);

  Serial.print(F(", Product version : "));
  Serial.println(buf[1], HEX);
  Serial.println();
}

/**
 *  @brief : continued loop after fatal error
 *  @param mess : message to display
 */
void Errorloop(char *mess)
{
  Serial.println(mess);
  Serial.println(F("Program on hold"));
  for(;;) delay(100000);
}

This code run as expected, except get following in serial monitor
000010a4ae30
Attempting MQTT connection...connected
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x90 0x00 0x00 length: 4
Sending to 0x58: 0x20 0x50 , Received: 0x31 0x8D 0x47 0x9D length: 4
Sending to 0x70: 0x78 0x66 , Received: 0x61 0xBD 0x7B 0xBE length: 4
CO2 equivalent : 400, TVOC : 0, H2_signal : 12685, Ethanol_signal : 18333, Humidity : 48.34, temperature : 21.81, absolute humidity : 9.28Sending to 0x58: 0x20 0x15 , Received: 0x00 0x00 0x00 0x00 length: 4
 CO2 equivalent baseline : 0x0Sending to 0x58: 0x20 0x15 , Received: 0x00 0x00 0x00 0x00 length: 4
, TVOC baseline : 0x0
Data = {"co2_equivalent":400,"tvoc":0,"h2_signal":12685,"ethanol_signal":18333,"humidity":48.33599853515625,"temperature":21.812999725341797,"absolute_hum":9.2800455093383789,"co2_equivalent_baseline":16,"tvoc_baseline":16,"chipid":"000010a4ae30"}
[D][WiFiClient.cpp:509] connected(): Disconnected: RES: 0, ERR: 128
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x90 0x00 0x00 length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x90 0x00 0x00 length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x90 0x00 0x00 length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x90 0x00 0x00 length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x90 0x00 0x00 length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x90 0x00 0x00 length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x90 0x00 0x00 length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x90 0x00 0x00 length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x90 0x00 0x00 length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x90 0x00 0x00 length: 4
000010a4ae30
Attempting MQTT connection...connected
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x90 0x00 0x00 length: 4
Sending to 0x58: 0x20 0x50 , Received: 0x31 0x94 0x47 0xAB length: 4
Sending to 0x70: 0x78 0x66 , Received: 0x61 0xC4 0x7A 0xDC length: 4
CO2 equivalent : 400, TVOC : 0, H2_signal : 12692, Ethanol_signal : 18347, Humidity : 47.99, temperature : 21.83, absolute humidity : 9.22Sending to 0x58: 0x20 0x15 , Received: 0x00 0x00 0x00 0x00 length: 4
 CO2 equivalent baseline : 0x0Sending to 0x58: 0x20 0x15 , Received: 0x00 0x00 0x00 0x00 length: 4
, TVOC baseline : 0x0
Data = {"co2_equivalent":400,"tvoc":0,"h2_signal":12692,"ethanol_signal":18347,"humidity":47.991001129150391,"temperature":21.830999374389648,"absolute_hum":9.223388671875,"co2_equivalent_baseline":16,"tvoc_baseline":16,"chipid":"000010a4ae30"}
[D][WiFiClient.cpp:509] connected(): Disconnected: RES: 0, ERR: 128
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x90 0x00 0x00 length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x90 0x00 0x00 length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x90 0x00 0x00 length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x90 0x00 0x00 length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x90 0x00 0x00 length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x98 0x00 0x05 length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x90 0x00 0x03 length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x92 0x00 0x03 length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x96 0x00 0x01 length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x90 0x00 0x00 length: 4
000010a4ae30
Attempting MQTT connection...connected
Sending to 0x58: 0x20 0x08 , Received: 0x01 0xA1 0x00 0x0D length: 4
Sending to 0x58: 0x20 0x50 , Received: 0x31 0x94 0x47 0xA8 length: 4
Sending to 0x70: 0x78 0x66 , Received: 0x61 0xBD 0x7A 0xCC length: 4
CO2 equivalent : 417, TVOC : 13, H2_signal : 12692, Ethanol_signal : 18344, Humidity : 47.97, temperature : 21.81, absolute humidity : 9.21Sending to 0x58: 0x20 0x15 , Received: 0x86 0x6C 0x8C 0x58 length: 4
 CO2 equivalent baseline : 0x8C58Sending to 0x58: 0x20 0x15 , Received: 0x86 0x6C 0x8C 0x58 length: 4
, TVOC baseline : 0x866C
Data = {"co2_equivalent":417,"tvoc":13,"h2_signal":12692,"ethanol_signal":18344,"humidity":47.966999053955078,"temperature":21.812999725341797,"absolute_hum":9.20920181274414,"co2_equivalent_baseline":16,"tvoc_baseline":16,"chipid":"000010a4ae30"}
[D][WiFiClient.cpp:509] connected(): Disconnected: RES: 0, ERR: 128
Sending to 0x58: 0x20 0x08 , Received: 0x01 0xA6 0x00 0x0F length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x98 0x00 0x05 length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x9E 0x00 0x0F length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x98 0x00 0x0C length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x9C 0x00 0x0D length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x9A 0x00 0x11 length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x9A 0x00 0x0F length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x9D 0x00 0x09 length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x96 0x00 0x0A length: 4
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x90 0x00 0x06 length: 4

Here ESP try to connect to MQTT because it looses wifi connection
Attempting MQTT connection...connected
Sending to 0x58: 0x20 0x08 , Received: 0x01 0x90 0x00 0x00 length: 4
Sending to 0x58: 0x20 0x50 , Received: 0x31 0x8D 0x47 0x9D length: 4
Sending to 0x70: 0x78 0x66 , Received: 0x61 0xBD 0x7B 0xBE length: 4
CO2 equivalent : 400, TVOC : 0, H2_signal : 12685, Ethanol_signal : 18333, Humidity : 48.34, temperature : 21.81, absolute humidity : 9.28Sending to 0x58: 0x20 0x15 , Received: 0x00 0x00 0x00 0x00 length: 4
 CO2 equivalent baseline : 0x0Sending to 0x58: 0x20 0x15 , Received: 0x00 0x00 0x00 0x00 length: 4
, TVOC baseline : 0x0
Data = {"co2_equivalent":400,"tvoc":0,"h2_signal":12685,"ethanol_signal":18333,"humidity":48.33599853515625,"temperature":21.812999725341797,"absolute_hum":9.2800455093383789,"co2_equivalent_baseline":16,"tvoc_baseline":16,"chipid":"000010a4ae30"}
[D][WiFiClient.cpp:509] connected(): Disconnected: RES: 0, ERR: 128

I am not using deep sleep yet, which would have disconnected from wifi.
Update :
I had ESP32-CAM connected to same AP, when I turn off ESP32-CAM then this code works without any error (ERR: 128)
Dont know how ESP32-CAM WiFi connection affects ESP32 WiFi connectivity ?

what happens if you set DELAY to a lower value

For DELAY 0, DELAY 1, DELAY 5, DELAY 10 getting  same [D][WiFiClient.cpp:509] connected(): Disconnected: RES: 0, ERR: 128

perhaps this post in an issue on github sheds some light - I don't know - it seems your not disconnecting from wifi though - nothing in the log suggests that, does it?

If you are actually disconnectin gfrom Wifi, I know some ESP8266's had trouble keeping a wifi connection with a small set of wifi routers in an older version of the ESP8266 library - not sure if the ESP32 suffered the same issue, but out of interest, what make/model is the access point/wifi router you are connecting to?

I have esp32-cam which connect to same AP with no issue.

OK - was just a thought - so how about that link https://github.com/espressif/arduino-esp32/issues/1921#issuecomment-471346624 - anything there help?

I have added some update to my question.

so a completely different ESP32, with a camera attached, affects the connection of this other ESP32? Is this ESP32CAM using the same MQTT server and perhaps sending huge amounts of data to it?

Yes different ESP32 with CAM sending data to same MQTT. But not sending huge amount of data.

the id to client.connect .... it's not ESP32Client in both cases is it?


text:
Adding a custom platform to PlatformIO
How can I add the STM32duino core as a custom platform to PlatformIO?

See https://docs.platformio.org/en/latest/platforms/ststm32.html

Creating custom board support for PlatformIO
PlatformIO provides a very comprehensive documentation on how to use PlatformIO for programming STM32 with various platforms. You can find "Custom Platform & Boards" on the side menu, which will lead you to this section of the document, which again is well documented.
It basically involves creating a JSON file that describes your custom platform or custom board. The easier way is to copy one of the family MCUs JSON configuration file that's already supported by the PlatformIO or STM32duino framework, and make modifications from there. For example, if your custom board is STM32G070xx, which is not supported by PlatformIO yet, you can find the board description JSON from platform-stm32 github repository for Nucleo_G070RB board and use it as a template to create your own custom board JSON structure.
Creating custom board support for Arduino IDE
The official STM32duino wiki provides a description on how to create a custom board variant on Arduino IDE for the framework. I also have an example of a custom variant for supporting my two custom boards in my github that you could take a look at in case you are interested to create custom board support under Arduino IDE.

the question is not about custom board

@juraj, One of the links that I provided including the information for creating custom platform manifest file under PlatformIO. OP mentioned STMduino which means a platform and a specific framework (arduino) has been defined, so I assumed what he means for "custom platform" actually means for "custom board". This is also why I at the beginning only provide a broader link instead of a specific link.  I will let the OP to decide what he is looking for.  If this is not what he is looking for, I'm sure OP will speak out.


text:
Using different variables in a function
I have a function SetTFgr(float T, int FGr) to set the time and filtergrade of my LED enlarger.
It works without a problem and changes the values of T or Fgr according to the buttons I press. Both T and FGr are global variables.
Now I want to use the same funtion with different variables.
When I call the function with SetTFgr(TExp[i], FGrExp[i]) I would expect the function to work with the values of TExp[i] or FGrExp[i] and increase/decrease these values. It does not, it appears to use the value of T or Fgr instead.
How can I get what I am after?

Please show a full code example with your problem

So to be clear, you want to read the values of buttons (meaning unspecified) and use those values to set a float and an int variable. Further, you want to write a function that takes a couple of parameters that specify where you want to store those button values?

To be clear, you want to read the values of buttons (meaning unspecified) and use those values to set a float and an int variable. Further, you want to write a function that takes a couple of parameters that specify where you want to store those button values?
That is an unusual way to use function parameters. Usually function parameters specify an input value, and then you use that value to do something. Instead, you want to have the inputs be globally available readings (button values) and have the function parameters specify where to store those values.
C and C++ function parameters are usually passed "by value". That means that what gets sent to the function is a value, not a pointer to the variable that contains that variable. Thus the called function can't change the original variable.
If you want to pass a pointer to the variable, you have to use the & "address of" operator to pass the variable "by reference".
Your function might look like this using C++ pass by reference syntax:
void SetTFgr(float &T, int &FGr) {
    float input_T = //figure out button value for T
    T = input_T;
    int input_FGr = //figure out button value for Far
    FGr = input_FGr;
}

Then you'd invoke it like this:
SeTFgr(T, FGr);

You could also accomplish the same thing using C pointer syntax.

Parameters of functions in C++ are by value. So T in your function is not the variable you use to call the function. It only has the same value. Use float& T to pass the reference to the variable.


text:
How to move a servo quickly and without delay function
Im trying to move a servo from one place to another while using the typical for loop you find in the servo's library example:
int lightON = 180;
int lightOFF = 90;

for (pos1 = lightOFF; pos1 <= lightON; pos1 += 1) {
  servo1.write(pos1);
  delay(15);
}

for (pos1 = lightON; pos1 >= lightOFF; pos1 -= 1) {
  servo1.write(pos1);
  delay(15);
}

The thing is that while this does work, I want to be able to move the servo a little quicker, but I can't use the delay() function because of some connectivity issues with another library (BLYNK).
How could I move the servo from lightOFF to lightON the quickest way possible without using the delay() function?

The quickest way possible from LightON to LightOFF is simply servo1.write(lightOFF); without loops or delays, in one go; no need to do it degree by degree. For example:
int lightON = 180;
int lightOFF = 90;

for (pos1 = lightOFF; pos1 <= lightON; pos1++) {
  servo1.write(pos1);
  delay(15);
}

servo1.write(lightOFF);

If you want it to move a few degrees every loop and do other things in the loop while it moves, you can time it with millis() instead of delay(). The code below should move the servo 5 degrees every 0.5s:
int moveTime = 500; // Move every 0.5s
unsigned long int startMillis = 0;
int position = lightOFF;
int step = 5; // Move 5 degrees

void loop() {
  if (millis() - startMillis >= moveTime) {
    servo1.write(position);
    if (position <= lightON) {
      position = position + step;
    }
    startMillis = millis();
  }
  // Do other things
}

Untested and incomplete, but you get the idea.

The best way is to use map() function.
void loop() {
  unsigned long progress = millis() - startMillis;

  int angle = map(progress, 0, MOVING_PEDIOD, lightOFF , lightON);
  servo1.write(angle);
}

The above code use the millis() function, therefore you do not need to worry about blocking other code.
Full instruction is available on How to control speed of servo motor

Nice idea! It should be combined with constrain() in order to avoid overshooting (map() does not implicitly constrain()).


text:
Arduino native USB port and Simulink
I am trying to communicate through the native USB port of the Arduino Due with a PC.
The native port itself works, I have tested this. The problem is, that every time I upload my Simulink model via the programming port onto the Arduino, Windows loses the connection to the native (Bossa) port.
It does not recocnize the port anymore after programming the Arduino with Simulink (Arduino Support Package). The only way to get Windows to recognise the port again is holding reset+erase on the Arduino, which obviously deletes my application.
Do you know any solution? Is this possible to get the native port working for communication after using Simulink to deploy my application?

ditch simulink for arduino programming and use arduino ide and code?  simulink should be able to produce C code you can then patch into the ide in the form of headers.

the native USB is handled by the program running in the MCU. if you upload a program which doesn't do it or doesn't work at all, then there will be no working USB on native port

Are you initializing a serial connection in the Simulink code? If not there won't be one and you loose connection when the actual code starts. The main code needs to do that.please show us what you have got in Simulink. Though it might be that we cannot help you since we are no experts in Simulink

Got it working myself by simply writing the communication part in C for the Due, not using the Simulink blocks.


text:
Can an MCP2515 CANBUS controller be used for two-way communication?
I tried searching code for an MCP2515 with Arduino on Arduino Forums, Instructables, Electronics Hub, YouTube, etc.
All of them showed code for a transmitter node and a receiver node, sensors on one node, and result on the other end.
Can we connect two Arduinos using an MCP2515 and communicate TWO WAY?

---Yes---
It should work nicely. They operate only in simplex mode (sending or receiving) to the outside world. Internally they also monitor what they are sending for error checking etc. From the user point they would appear to be bidirectional because they can switch between send and receive messages rapidly, message by message. Of course the bus speed helps. You can get upto 1 Meg with these. I use Cory Fowler's "mcp_can.h". You can get it at: "https://github.com/coryjfowler/MCP_CAN_lib". The library has both send and receive .ino files, get them to work first.  Since CAN is priority Based NDA Non Destructive Arbitration the higher priority message will prevail regardless of who else is on the bus or is sending a message.
Do not forget to put jumpers on the module to terminate the bus. Note the bus gets terminated only at the physical ends. The bus design impedance is 60 Ohm, the termination resistors are 120 ohms.
You simply combine the send and receive sections of code in your module, nothing special to do. You initialize both send and receive at the same time with basically the same information.  You will find the information for this in the setup() section of the code. Be sure to check crystal frequencies, I have seen both 8 and 16 Mhz crystals on these boards.
Per the Bosch specification, which is readily available,  The CAN bus does arbitration automatically by a defined priority on a bit by bit basis. To send a message with the MPC2515 you load it and when the bus becomes available it will transmit it, if there is an error it will retry many times (internal counter) and if it fails you will know. When a message is sent it must receive an ACK in a specified part of the can frame. This can be generated by any node. Your internal protocol needs to determine if the right node got the message.
By the time your external logic can determine it wants to send a message and send a signal it will have already been sent in a standard CAN setup. Any number of modules can start transmitting but the arbitration will weed them out and the highest priority message will get through. Every node will receive every message, depending on its filtering etc it may accept it or ignore it. You can pole or take an interrupt any time it receives a message etc.

I never used CAN bus myself, but what you could do is one of the following:

Letting the transmitter send a specific (short) message to the receiver. In case the receiver has something to transmit, it sends a specific reply back, so the transmitter can request the receiver to send the actual message to be sent. The disadvantage is that polling is needed.
If you want the receiver to be able to send its message as fast as possible, add a separate wire next to the CAN bus communication. If that line is LOW or HIGH (decide yourself), then the transmitter can 'immediately' send a message to have the receiver send its actual message.

Thanks! Tx Arduino runs only tx code, and Rx Arduino runs only rx code. So, it's not possible for Rx to transmit anything back. And, using CAN is for reducing no. of wires used, but that's a nice idea.

By your first point, I'll try to merge the tx & rx commands, and temporarily make TX Arduino receive data, and vice versa.

Let us know how you do!

Per the Bosch specification, which is readily available,  The CAN bus does arbitration automatically by a defined priority on a bit by bit basis. To send a message with the MPC2515 you load it and when the bus becomes available it will transmit it, if there is an error it will retry many times (internal counter) and if it fails you will know. When a message is sent it must receive an ACK in a specified part of the can frame. This can be generated by any node. Your internal protocol needs to determine if the right node got the message.


text:
Row-column scanning to control an 8x8 LED matrix
https://www.arduino.cc/en/Tutorial/RowColumnScanning
This code really confused me because I don't know if the pixels[x][y] is HIGH or LOW in void refreshScreen().
As you look at the code, pixels[x][y] is HIGH and then LOW in void setup(), and then pixels[x][y] is HIGH again in void loop(). Could anyone explain this to me, please？

The pixels array in initialized to all HIGH in setup(). Then one element of the pixel array is set to LOW in readSensors()  which turns on one LED.
The tricky bit is you have to remember to turn off the previous LED. That is why in readSensors() there is the line where an element of pixels is set HIGH while the pixels' array pointers are still pointing at the former (already lit) LED.


text:
SIM800L GSM Module, no network signal RSSI
I am using a GSM 800L module to send SMS messages. When I test the module with Arduino, it shows a 0 : -115dbm signal strength.
So basically there is no signal. SIM800L is a quad-band GSM/GPRS module that works on frequencies GSM850MHz, EGSM900MHz, DSC1800Mhz, and PCS1900MHz.
My SIM card carrier uses the frequency bands GSM 900, GSM 1800 for 2G-capablities.
What might be the possible reasons my GSM module doesn't work properly with my SIM card? I have tried different locations, still no signal.
The code I use for testing is provided by Adafruit-Fona.

Where do you live? 2G has been switched off in some countries.

Check antenna cable/connection, maybe shorted?

there is no cable for the antenna. I don't really what is the problem

Please can you accept your own answer (using the green tick below the voting buttons) in order to remove your question from the unanswered list. Thank you.

The Problem was that I used 5 V Dc to power the module , it worked but in order to function I must use lithium battery. ( its mentioned in the manual)

Please can you accept your own answer (using the green tick below the voting buttons) in order to remove your question from the unanswered list. Thank you.

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
Convert JSON into string for MQTT
I am trying to publish json object into string with following snippet
JSONVar data;
data["chipid"] = chipId;
data["co2_equivalent"] = co2_equivalent.c_str();
data["tvoc"] = tvoc.c_str();
data["h2_signal"] = h2_signal;
data["ethanol_signal"] = ethanol_signal;
data["humidity"] = humidity;
data["temperature"] = temperature;
data["absolute_hum"] = v.absolute_hum;
data["co2_equivalent_baseline"] = co2_equivalent_baseline;
data["tvoc_baseline"] = tvoc_baseline;

String data_json = JSON.stringify(data);
Serial.print(data_json);
Serial.println();
client.publish("/devices/data", data_json);

This give me following error during compilation.
  /Users/ubuntu/Desktop/Arduino/data_collection/esp32-svm30-mqtt.ino/esp32-svm30-mqtt.ino.ino: In function 'void loop()':
  esp32-svm30-mqtt.ino:271:46: error: no matching function for call to 'PubSubClient::publish(const char [14], String&)'
       client.publish("/devices/data", data_json);
                                                ^
  In file included from /Users/ubuntu/Desktop/Arduino/data_collection/esp32-svm30-mqtt.ino/esp32-svm30-mqtt.ino.ino:2:0:
  /Users/ubuntu/Documents/Arduino/libraries/PubSubClient/src/PubSubClient.h:141:12: note: candidate: boolean PubSubClient::publish(const char*, const char*)
      boolean publish(const char* topic, const char* payload);
              ^
  /Users/ubuntu/Documents/Arduino/libraries/PubSubClient/src/PubSubClient.h:141:12: note:   no known conversion for argument 2 from 'String' to 'const char*'
  /Users/ubuntu/Documents/Arduino/libraries/PubSubClient/src/PubSubClient.h:142:12: note: candidate: boolean PubSubClient::publish(const char*, const char*, boolean)
      boolean publish(const char* topic, const char* payload, boolean retained);
              ^
  /Users/ubuntu/Documents/Arduino/libraries/PubSubClient/src/PubSubClient.h:142:12: note:   candidate expects 3 arguments, 2 provided
  /Users/ubuntu/Documents/Arduino/libraries/PubSubClient/src/PubSubClient.h:143:12: note: candidate: boolean PubSubClient::publish(const char*, const uint8_t*, unsigned int)
      boolean publish(const char* topic, const uint8_t * payload, unsigned int plength);
              ^
  /Users/ubuntu/Documents/Arduino/libraries/PubSubClient/src/PubSubClient.h:143:12: note:   candidate expects 3 arguments, 2 provided
  /Users/ubuntu/Documents/Arduino/libraries/PubSubClient/src/PubSubClient.h:144:12: note: candidate: boolean PubSubClient::publish(const char*, const uint8_t*, unsigned int, boolean)
      boolean publish(const char* topic, const uint8_t * payload, unsigned int plength, boolean retained);
              ^
  /Users/ubuntu/Documents/Arduino/libraries/PubSubClient/src/PubSubClient.h:144:12: note:   candidate expects 4 arguments, 2 provided
  Multiple libraries were found for "PubSubClient.h"
   Used: /Users/ubuntu/Documents/Arduino/libraries/PubSubClient
  Multiple libraries were found for "Arduino_JSON.h"
   Used: /Users/ubuntu/Documents/Arduino/libraries/Arduino_JSON
  Multiple libraries were found for "svm30.h"
   Used: /Users/ubuntu/Documents/Arduino/libraries/svm30
  Multiple libraries were found for "Wire.h"
   Used: /Users/ubuntu/Library/Arduino15/packages/esp32/hardware/esp32/1.0.4/libraries/Wire
  Multiple libraries were found for "WiFi.h"
   Used: /Users/ubuntu/Library/Arduino15/packages/esp32/hardware/esp32/1.0.4/libraries/WiFi
   Not used: /Applications/Arduino.app/Contents/Java/libraries/WiFi
  exit status 1
  no matching function for call to 'PubSubClient::publish(const char [14], String&)'

Line Serial.print(data_json) does print it in serial monitor.
Update :
If I doclient.publish("/devices/data", data_json.c_str());, it gets compiled, but data doesn't appear in mqtt.
Serial.print(data_json.length()); prints 236
Is my json to string conversion correct ?

Please note: the question originally had no mention of trying to use c_str() - so I'm not a retard

Point 1 JSON actually is a string
Point 2 your data_json varibale is a String
Point 3 client.publish doesn't work take String as the payload argument, it takes either const char * or const uint8_t *
So, what you can simply do is use Strings c_str method to pas a const char * as the payload
i.e.
client.publish("/devices/data", data_json.c_str());

Point 4 - if you read the pubsubclient documentation, you'll see that

The maximum message size, including header, is 128 bytes by default. This is configurable via MQTT_MAX_PACKET_SIZE in PubSubClient.h. Longer messages can also be sent with the publish_P() method.

so what you may be experiencing now is a message size too big for pubsubclient to handle - set a larger packet size before including the PubSubClientHeader.h
#define MQTT_MAX_PACKET_SIZE 1024
#include <pubsubclient.h>

If that also doesn't work, try
client.publish_P("/devices/data", data_json.c_str());

I tried data_json.c_str() but still data doesn't appear in mqtt.

@roy - but it compiles now?

Yes its getting compiled, but data not getting to mqtt.

@roy - updated the answer with another possible problem - which you have confirmed with your latest update that the payload is over 128 length ... meaning the whole message is WELL over 128

I tried adding #define MQTT_MAX_PACKET_SIZE 1024, still no data appearing in mqtt.

@roy - what if you set a fixed payload? I can't recall if you tried that since you keep editing the question - you did add that define BEFORE you included the header file?

If i just publish fixed data it works like this client.publish("/devices/data", "1234567890");

and now that you've increased the packet size, it still doesn't work? it's odd, because if Serial.print(data_json); prints a string, and the string is far less than 1024, this should've worked - can you Serial.print(MQTT_MAX_PACKET_SIZE); just to be sure

try using client.publish_P("/devices/data", data_json.c_str());

You have almost tried everything and @Jaromanda has given an answer that should possibly work. Can you please try the following things const char *buffer = NULL; //declaring const char * buffer = strdup(data_json.c_str()); //duplicating the string client.publish("/devices/data", buffer); //publish it MQTT