Feature suggestion: Micropython binding

Micropython is becoming mainstream on IOT and embedded development in general. It is available for quite a few of embedded architectures (this and this, partial list), and can be ported to new ones.
Today there already exist several architecture-specific micropython display modules (for example this and this), but none of them as feature rich and general as littlevgl.
I think it could be useful to provide micropython API to littlevgl Objects and Styles.
Main benefits are:

Exposure to a larger audience in the embedded development world.
Interactive developement. Use the Micropython interpreter to exercise littlevgl over a live embedded board. This would allow rapid GUI development without the build-flash cycle.
Some would argue Python code is easier to develop and maintain (not sure I agree about this one...)

If it turns out there is an agreement about this feature, I might try sending PR if I find the time for it.

Hi,
I also see a big potential in it. Both reaching a larger audience and the benefits of Micropython sounds very well. Earlier I made some experiments with Lua binding but Micropython is more mainstream right now.
So this feature would be very welcome!
How can this binding look like? One or more c file(s) with the interface functions?

@turoksama There will definitely be overhead from using Python due to the need to convert objects back and forth. I guess the hope is that Python will be used on platforms which can afford the additional overhead.

Here is what I had in mind.
The lvgl micropython module would provide a constructor for each object type, and each object would provide its methods.
Here is the "Hello World" example:
import lvgl
label1 = lvgl.label(lvgl.get_scr_act())
label1.set_text("Hello world!")
label1.align(None, lvgl.ALIGN_CENTER, 0, 0)

Creating micropython extensions requires lots of boilerplate. I don't think we should write them manually. Writing them manually would be not only tedious to start with, but also hard to maintain when objects are changed and added.
Instead I suggest creating these extensions automatically from the source code (the C headers) relying on the naming conventions and maybe some hints.
During the make process a script will run, scan the lvgl object headers (with pycparser for example) and generate the Micropython extensions in a C file that can be later compiled and linked on Micropython-enabled projects.

@amirgon  Sound good to me! I also vote for the automated way.
From your example, we can see how to call LittelvGL from Micropython. But how to call Micropython from LittelvGL? It is required when clicking a  button and a user-defined action should be executed.

@kisvegabor Calling a Micropython function from C code is covered by Micropython.
You can use mp_call_function_* defined here, so we get this one for free.
In case of a button (or any other) set_action micropython extension, we can give it a python function as a parameter since functions are first-class objects in python, can be passed around, used in lists dicts etc. so I think this should be pretty simple.
One thing we'll need to figure out is how to separate display/board specific code from generic lvgl code.
We would need to provide Micropython extensions to register display driver, set IO pins, configure SPI, DMA, etc.

@amirgon The only problem with your suggestion is that the performance overhead of handling the low-level display code through Python might be too high. What do you think?

@embeddedt It's true that Micropython is slower than C, and costs more resources (RAM, Flash).
However, I believe Micropython is still beneficial for platforms that can afford the extra resources (such as ESP32, for example):

Micropython would only be used for the Objects/Styles API layer (for setting object properties), not for the graphics rending itself which remains in C.  Think about a button, for example. How many cycles are spent on creating the button object and setting its properties (action, state, style) vs. cycles spent on actually writing pixels to the display to render it? My guess is that rending would take most of the cycles (and rendering remains in C), while setting object properties would only consume a very small portion of the cycle budget.
Also take into account that in most cases setting object properties is done once per GUI "screen" and rendering happens continuously while the user interacts with that certain "screen".

Micropython is optimized for architectures with limited resources (CPU and memory). More information can be found here. Micropython compiles code into bytecode and has an efficient VM that executes it. Micropython modules may also be implemented as frozen bytecode modules saving the need to compile them by the MCU before running them.

Micropython support for lvgl would be optional, and you would need to explicitly opt in. If you do nothing you don't pay any cycles or memory. If you are developing for a platform where you want to save cycles and memory, this feature will not get in your way, and you can still write all your code in C as today and get the same performance.

@amirgon I agree, calling lv_set_... functions from Micropython would have only a minor overhead if the rendering remains in C. I suppose we can keep the disp_fush function in C as well, right?

@kisvegabor I agree, disp_flush should remain a low level C function.

@amirgon
Did you already make some experiments with LittlevGL in Micropython?

@amirgon
Great! I'm really waiting for that :)

Looks amazing! My old dream to make an interactive online GUI demo tool but didn't find the right way to do this.
I'm looking forward to try Micropython!

@kisvegabor Have you considered Emscripten? It converts C to JavaScript and might be faster than using Unicorn and MicroPython for a demo.

@embeddedt
I tried Emscripten once but it was extremely slow and wasn't stable (sometimes not started properly). Probably it is worth an other try.

@embeddedt if you do have time to experiment with Emscripten, please try to build Micropython as part of lvgl sources.
This has been done before.

Not ready for a PR yet, but I have a working script.
It scans all *.h files in lv_objx dir and generates an Mpy module called lvgl which contains object for each lvgl object, method functions etc.
It relies heavily on naming conventions. (For example, it assumes that an object is constructed by lv_%s_create with two specific arguments.
Luckily, these naming conventions are followed very well on lvgl sources.
Here is a working example I just run on my device:
>>> import lvgl
ILI9341 initialization.
Enable backlight.
>>> label1 = lvgl.label(lvgl.scr_act())
>>> label1.set_text("Hello world!")
>>> label1.align(None, 0, 0, 0)

A nice thing about Mpy REPL is the automatic completion (start typing and press tab), the help function etc.
 In the future it should not be committed, but generated by the Makefile instead. It provides the Mpy module mp_module_lvgl which should then be registered in Mpy mpconfigport.h.
It's already quite large, about 9000 generated source lines.

I'll be happy to receive comments.

@amirgon
It looks amazing!
Do you know how much flash space is required by lv_mpy.c?
 I fond this: https://github.com/micropython/micropython/wiki/Board-STM32F429
Do you think lv_mpy.c can be integrated into that project? Or is it possible for me who is not familiar with Micropython? :)

@amirgon
I'm planning to try Micropytthon on Linux with framebuffer as a display. I started to put the project together. Some parts of the build process are already working I don't know what is best way to link lvgl to micropython.  Now I got a lot of similar errors:
build/../../../lvgl/mpy/lv_mpy.o: In function `mp_lv_txt_ins':
lv_mpy.c:(.text.mp_lv_txt_ins+0x31): undefined reference to `lv_txt_ins'

I suppose I need to add the source files of lvgl to the makefile but don't know what is the best to do it. Can you help with that?

@kisvegabor  Yes you need to add lvgl sources to the Makefile.
First, I configured lvgl as an optional component, the following way:

Edit Kconfig.projbuild and add the following section under menu "Modules"

	    config MICROPY_USE_LVGL
	       bool "Use LVGL module"
	       default n
	       help
	       Include LittlevGL Embedded GUI Library

Run make menuconfig and enable use LVGL module under Micropython --> Modules. Alternatively you could edit sdkconfig directly and add CONFIG_MICROPY_USE_LVGL=y

Edit components/micropython/component.mk and add lvgl sources like this:

ifdef CONFIG_MICROPY_USE_LVGL
SRC_C += <...lvgl C files..>
endif

Alternatively you could add the source files recursively, but make sure you also read this. A better technique is something like this.
On ESP32 all it takes is adding component.mk (even empty one!) on every directory that contains files you want to include in the build (as explained here).
If you build Micropython on Linux that is probably different.

@kisvegabor Please make sure you take the latest version when you try it out.
Apart from bug fixes, I added the following features:

Object Inheritance. For example, when accessing a btn object, you can use btn methods, cont methods and obj methods, which are all the ancestors of btn. (The script deduces the heirarchy from the ..._ext_t structs)
Enums. Each enum is created as its own type, either as a child of an object type or of the lvgl module. For example,

ALIGN is a member of lvgl
>>> help(lvgl.ALIGN)
object <class 'LV_ALIGN'> is of type type
  OUT_LEFT_BOTTOM -- 17
  OUT_TOP_LEFT -- 9
  IN_RIGHT_MID -- 8
  OUT_RIGHT_MID -- 19
  IN_BOTTOM_MID -- 5
  CENTER -- 0
  OUT_RIGHT_TOP -- 18
  OUT_BOTTOM_MID -- 13
  OUT_LEFT_MID -- 16
  IN_TOP_MID -- 2
  IN_BOTTOM_LEFT -- 4
  IN_TOP_RIGHT -- 3
  OUT_TOP_MID -- 10
  OUT_BOTTOM_LEFT -- 12
  OUT_RIGHT_BOTTOM -- 20
  OUT_LEFT_TOP -- 15
  IN_BOTTOM_RIGHT -- 6
  OUT_TOP_RIGHT -- 11
  OUT_BOTTOM_RIGHT -- 14
  IN_TOP_LEFT -- 1
  IN_LEFT_MID -- 7

but STATE is a member of btn:
>>> help(lvgl.btn.STATE)
object <class 'LV_BTN_STATE'> is of type type
  TGL_REL -- 2
  NUM -- 5
  TGL_PR -- 3
  PR -- 1
  INA -- 4
  REL -- 0
>>>

@amirgon Thank you very much! I will try it on Sunday!

@amirgon The issue you describe sounds similar to #316 and the solution would probably apply to both.
I think your idea of adding a context argument is probably a simple and clean solution. @kisvegabor and I recently had a brief discussion about changes that break compatiblity: https://github.com/littlevgl/lvgl/issues/543#issuecomment-443180096

@embeddedt while the problem sounds similar, the suggested solution of #316 is different. Adding something like lv_action_id is not general enough for callbacks.
I would prefer the generic solution of context/user_data parameter on each callback. It's also simpler and clear for anyone who ever worked with callbacks on other libraries.
Actually I noticed this is already used on lvgl on some places! For example, lv_indev_data_t contains user_data which was passed originally when registering the driver by lv_indev_drv_init.
Why not generalize user_data for all callbacks?

@amirgon
Test results
I made it work on PC. Finally, I used SDL like to normal PC simulator. I also tested the new enums.
label1 = lvgl.label(lvgl.scr_act())
label1.set_text("Hello world")
label1.align(lvgl.scr_act(), lvgl.ALIGN.CENTER, 0, 0)

However, I faced an issue. lv_mp_init(); was not called during import lvgl.
To make it work I added lv_mp_init(); to lv_init().
And I added lv_task_handler to the monitor refresh tread.
My Micropython project with SDL
I uploaded my SDL-Micropython project here: https://drive.google.com/file/d/16htsMz3L1knvfHWdPsmwg3YyiOtSbpF0/view?usp=sharing
Instructions:

Go to: micropython/linux/micropython/ports/unix
make (ignore the warnings :) )
./micropython it should start Micropython COmmand Line interface
import lvgl
lvgl.init() it should open a new white window
label1 = lvgl.label(lvgl.scr_act())
label1.set_text("Hello world")
...continue testing... :)

Actions
You can use lv_obj_set_free_ptr(obj, &context_data) and lv_obj_set_free_num(obj, some_id) to add an arbitrary pointer or number to an object which can be get in the action using lv_obj_get_free_ptr() and lv_obj_get_free_num(). It similar to the user_data at input devices.

@kisvegabor I'm glad to hear you had a success with Micropython!
Regarding lv_mp_init(), it's registered as __init__ on mp_module_lvgl globals, so it should be called automatically (and is actually called on my application on ESP32).
I'm not sure what the problem on your side is, but please make sure you are using a relatively recent version of Micropython. I think it was not called on older versions of Micropython.
Another option to work around this it is to call lvgl.__init__() explicitly. Did you try this?
Currently I'm developing and testing Micropython integration with lvgl on ESP32, I think it's good you are trying this integration on another platform.
Regarding Actions -
Using free ptr is very much like option (3) I suggested above (using an existing lv_obj_t data member instead of adding a new one).
The disadvantage of using free ptr instead of context parameter is that once the callback is called, I can know which object it relates to, but not which specific callback was registered.
So if the same object has many callbacks, I have to define a function for each instead of using a single function to handle callbacks.
As I explained above, implementing it this way would require more RAM, require more ROM (many callback functions in the code), and it would allow only one callback per object per callback type (for example you can't register multiple callbacks and call them all when the event happens. I don't know if you do this anywhere on lvgl).

I'm using v1.9.4 which is the latest release. lvgl.__init__() is working. Strange.
In mpconfigport.h I registered lvgl like this
{ MP_OBJ_NEW_QSTR(MP_QSTR_lvgl), (mp_obj_t)&mp_module_lvgl },
Is it correct?
With @embeddedt we are talking about having only one action callback per object but with an enum parameter about the action type, such as LV_ACTION_CLICK, LV_ACTION_PRESS etc.
So the signature would be:

Register a action: void lv_obj_set_action(lv_obj_t * obj, lv_action_t action);
The callback: lv_res_t my_action(lv_obj_t * obj, lv_action_t action)

This way the user has the opportunity to have a free_prt like this:
static void * contexts[LV_ACTION_NUM];
contexts[LV_ACTION_CLICK] = &x;
contexts[LV_ACTION_PRESS] = &y;

lv_obj_set_free_ptr(obj, contexts);

So we can decide now which direction to choose for v6.0 where we can have API breaking changes. Please let us know what do you think about the actions.

@kisvegabor

In mpconfigport.h I registered lvgl like this
{ MP_OBJ_NEW_QSTR(MP_QSTR_lvgl), (mp_obj_t)&mp_module_lvgl },
Is it correct?

Looks ok. I do this in a similar way, see here.

I found it. I needed to add #define MICROPY_MODULE_BUILTIN_INIT 1 to mpconfigport.h. This define was complately missing from the unix port.

The updated SDL project with mouse support is here: https://drive.google.com/open?id=1YjuaEFBrj1E0OVYZ_fX2wPvoaC_4Gw42

@kisvegabor

I found it. I needed to add #define MICROPY_MODULE_BUILTIN_INIT 1 to mpconfigport.h. This define was complately missing from the unix port.

Nice. I wasn't aware of this macro.
What do think would be the minimal set of additional features Micropython-lvgl-binding should have for a PR?
I was thinking of:

Callbacks (lv_action_t)
Styles
Specific structs such as lv_color_t

Also, I think we should add gen_mpy.py script to the Makefile. It would generate different outputs depending on v_conf.h (and also any change in the lvgl obj H files), so I don't think we want lv_mpy.c commited in git, in the long run.

Thank you for the comments. I updated my previous post.
Yes, I agree if actions, styles, and structures would be added we can already add it to lvgl. It will allow getting feedback from people. I'll add it the PC simulator too in a cleaner way than my current SDL project :)
However, I think we can commit lv_mpy.c too. If the parts would be protected with #ifs according to lv_conf.h we can leave it. Or why not leave it? Is there any disadvantage of leaving it in the code? IMO making it visible and instantly accessible is a good point.

@kisvegabor I think we can leave lv_mpy.c only as an example. The problem is that if the user changes lv_conf.h and removes some feature, lv_mpy.c will stop compiling as it assumes this feature is present.
Adding #ifs according to lv_conf.h is hard to do automatically in gen_mpy.py . The reason is that the C parsing can only be done after C-preprocessing, after which these #ifs are no longer present in the code.

@amirgon  I see. In this case, as you suggested, let's leave a full featured lv_mpy.c as an example.

@kisvegabor @amirgon I think leaving an autogenerated file in Git is a bad idea. Personally I prefer to be able to compile LittlevGL+Micropython without having changes in my working tree.
https://stackoverflow.com/a/894042

@embeddedt I only meant leaving it there as an example, think of it as a text file for reference which is not meant to be compiled.
Generated C files are not like obj files. You may want to browse lv_mpy.c to see which functions were generated, which were not and why (it contains many comments, for this purpose) even before you sync from git and build your project.
I still think we should have a makefile and run gen_mpy.py as part of the build. You will not have changes in your working tree because the committed example will not be committed to the same directory where the file is generated.

@amirgon How "easy" should be to also support Circuitpython?

@C47D

@amirgon How "easy" should be also support Circuitpython?

I'm not familiar with Circuitpython, but from what I read it is a derivative of Micropython. The differences from Micropython seem small enough such that you might be able to use lvgl Micropython binding as is without a change! (I didn't see any difference in the way new extensions are added).
Why don't you try and let us know how it goes?

In case of actions LV_RES_INV means that the object is deleted in the action and it's invalid when the action returns. (Use case: a button is pressed to change screen -> the current screen is deleted along with the button, and the new screen is created in the action). However, as you pointed out if the action is called later the object will be still sure valid even if it's deleted later.
The other functions might have a meaningful return value. lv_anim_path_t is a good example for that. However, I think it's not a big issue if custom animation paths or image decoder interfaces can't be declared in Micropython. IMO, only actions will be used in 99.9% of the cases.
The ultimate solution would be to stop C code while the Micropython callback is executed but you said: " I don't think we want to do that". Do you concern because of the delay in execution? (I don't know how big it is)
By the way, how should we call (and/or protect) lv_task_handler() from the scheduled call of actions? I mean if an action LittlevGL related functions are executed while lv_task_handler also works with the same resources things will be messed up. In "normal" case you should use a mutex around lv_task_handler and your lv_... function calls. Or is Micropython thread-safe in this regard?

@amirgon
I'll be off from work until next Friday. See https://github.com/littlevgl/lvgl/issues/543#issuecomment-445067815
Until Friday, maybe it would be more practical to deal with styles if too many questions arise about actions and callback. Styles seems more straightforward to me.

I'll be off from work until next Friday. See #543 (comment)

Congratulations! Good luck to you and your wife with your new baby!

@amirgon Thank you :)

@kisvegabor

The ultimate solution would be to stop C code while the Micropython callback is executed but you said: " I don't think we want to do that". Do you concern because of the delay in execution? (I don't know how big it is)

In the general case, we can't assume the platform is multi-threaded. Micropython can run on a single thread and schedule its action using its internal scheduler. In such case you are not allowed to block ("stop") the C code.
Even if you are in a multi-threaded platform, Micropython usually runs in a single thread. In such case you are not allowed to run a Micropython callback from another thread (AFAIK).

By the way, how should we call (and/or protect) lv_task_handler() from the scheduled call of actions? I mean if an action LittlevGL related functions are executed while lv_task_handler also works with the same resources things will be messed up. In "normal" case you should use a mutex around lv_task_handler and your lv_... function calls. Or is Micropython thread-safe in this regard?

This is a general problem, not related only to Micropython, right?
How do you protect it today, without Micropython?
I'm using FreeRTOS and I call lv_task_handler periodically from another task (thread).  Should I protect every call to lvgl by mutex (as well as  lv_task_handler) to make sure they never preempts each other?

So here is a quick update: I've added limited support for actions (callbacks).
Here is an example I just ran on my board:

>>> import lvgl
ILI9341 initialization.
Enable backlight.
>>> b = lvgl.btn(lvgl.scr_act())
>>> b.align(None, lvgl.ALIGN.CENTER, 0, 0)
>>> def f(arg):
...       print('Button clicked')
...
...
...
>>> b.set_action(b.ACTION.CLICK, f)
>>>
>>>
>>> Button clicked
Button clicked
Button clicked
Button clicked
Button clicked

Limitations:

It uses the internal Micropython scheduler (calls mp_sched_schedule to schedule a callback).
Apparently the Linux port of micropython does not enable the scheduler. I tried (naively) to enable it and the thing crashed. I'm not sure why and what is the effort to support this on the Linux port.

Callbacks return before the action actually executes, so we assume actions never fail. (This is also related to the usage of the Micropython scheduler).

It only supports callbacks with certain prototype. (need to return lv_res_t, first argument of callback function must be lv_obj_t, all arguments are convertible to python). Luckily, action callbacks (lv_action_t, lv_btnm_action_t , lv_tabview_action_t) use the convention and are supported.

It only supports one callback per object instance. So don't try to register more than one action for the same button. This is in line with #316 for v6.0 where you plan single callback for each object instance anyway.

@amirgon

I'm using FreeRTOS and I call lv_task_handler periodically from another task (thread). Should I protect every call to lvgl by mutex (as well as lv_task_handler) to make sure they never preempts each other?

Yes, you should use a mutex. See: https://doc.littlevgl.com/#Porting (Using with an operating system
section)
Actions
As the return value is used only to indicate that the object is deleted in the action, in v6.0 I can add some improvements to automatically detect if the object was deleted. So the library will know if the object was deleted in the action and handle it as invalid. Would it solve the issue?

To avoid a mutex, I would like to use Micropython scheduler same way as callbacks (call mp_sched_schedule to schedule a call to lv_task_handler every X milliseconds).

It sounds good but you said mp_sched_schedule is not always enabled. Like it's not enabled in the unix build either.

Now I'm always returning LV_RES_OK (when mp_sched_schedule succeeds) . How would that affect the user?

If the action callback is executed later i.e. not when you click the button then it still should work. However, you can try it if you call lvgl.obj_del(btn) in the action callback. You should set LV_MEM_ADD_JUNK   1 in lv_mem.c to fill the freed memories with junk to be sure the system fails is it wants. :)

@kisvegabor

It sounds good but you said mp_sched_schedule is not always enabled. Like it's not enabled in the unix build either.

I think we must enable Micropython scheduler in order to use Actions. Since Action is a Micropython callback, it cannot be called from another thread or in the middle of another Micropython command, so it must be scheduled correctly. For that reason I think we should provide Micropython lvgl biding only under the assumption that Micropython scheduler is enabled.
If we have the scheduler enabled for Actions, we might as well use it for lv_task_handler calls instead of using a mutex on every lvgl Micropython extension function.

BTW, in lack of mp_sched_schedule in the unix port I won't be able to test the actions on PC, right?

I'm not sure about this. When I tried enabling it (setting MICROPY_ENABLE_SCHEDULER macro) it crashed, but I didn't try to debug it much.
On the other hand, I noticed that on the unix Micropython port there are has several mpconfigport.h files that can be used, one of them defines MICROPY_ENABLE_SCHEDULER as 1, so it may be possible to enable the scheduler on the unix port after all. I think it's worth another try.

@kisvegabor
Good news!
I gave it another try, and the unix port supports the Micropython scheduler. I was able to run my example of Action above with your unix port!
Here are the main issues I came across:

The crash was not related to Micropython really. This was because in your code you changed lv_mpy.c and called lv_init in the generated file. Once I re-generated lv_mpy.c, lv_init was no longer called and that was the reason for the crash. So please.... don't change generated files....

Needed to add a periodic call to mp_sched_schedule which runs lv_task_handler.

Needed to change the read from stdin to nonblocking and add calls to mp_handle_pending which were missing in the unix port. Otherwise, Micropython is not called upon async events.

I used the mpconfigport_coverage.h file for configuring Micropython with scheduling.

I use a slightly older version of Micropython with my ESP32 board. To make lv_mpy.c compile I needed to change the calls to mp_sched_schedule (remove the last parameter). I'm planning to change gen_mpy.py to support both prototypes of mp_sched_schedule.

I suggest that instead of a zip file on google drive, add a project on github. Now I would like to show you my changes to your source code and I can't really use git diff. I'll try to manually paste the interesting parts, hopefully this would help you run it on your side:
Make command:
make CFLAGS_EXTRA='-DMP_CONFIGFILE="<mpconfigport_coverage.h>" -DMICROPY_USE_READLINE=1' DEBUG=1 -j4

Changes on lvgl/mpy/lv_mpy_hal.c:
#include "py/runtime.h"
#include "py/obj.h"

STATIC mp_obj_t mp_lv_task_handler(mp_obj_t arg)
{
    lv_task_handler();
    return mp_const_none;
}

STATIC MP_DEFINE_CONST_FUN_OBJ_1(mp_lv_task_handler_obj, mp_lv_task_handler);

/**
 * A task to measure the elapsed time for LittlevGL
 * @param data unused
 * @return never return
 */
static int tick_thread(void * data)
{
    (void)data;

    while(1) {
        SDL_Delay(5);   /*Sleep for 5 millisecond*/
        lv_tick_inc(5); /*Tell LittelvGL that 5 milliseconds were elapsed*/
	mp_sched_schedule((mp_obj_t)&mp_lv_task_handler_obj, mp_const_none);
    }

    return 0;
}

...

void lv_mp_init(void)
{
    lv_init();
    hal_init();
}

Changes on unix_mphal.c (inputAvailable taken from here)
static bool inputAvailable()
{
  struct timeval tv;
  fd_set fds;
  tv.tv_sec = 0;
  tv.tv_usec = 0;
  FD_ZERO(&fds);
  FD_SET(STDIN_FILENO, &fds);
  select(STDIN_FILENO+1, &fds, NULL, NULL, &tv);
  return (FD_ISSET(0, &fds));
}

int mp_hal_stdin_rx_chr(void) {
    unsigned char c;
#if MICROPY_PY_OS_DUPTERM
    // TODO only support dupterm one slot at the moment
    if (MP_STATE_VM(dupterm_objs[0]) != MP_OBJ_NULL) {
        int c;
        do {
             c = call_dupterm_read(0);
        } while (c == -2);
        if (c == -1) {
            goto main_term;
        }
        if (c == '\n') {
            c = '\r';
        }
        return c;
    } else {
        main_term:;
#endif
	while (!inputAvailable())
	{
	    mp_handle_pending();
	    usleep(5);
	}
        int ret = read(0, &c, 1);
        if (ret == 0) {
            c = 4; // EOF, ctrl-D
        } else if (c == '\n') {
            c = '\r';
        }
        return c;
#if MICROPY_PY_OS_DUPTERM
    }
#endif
}

@amirgon I'm not familiar with MicroPython's scheduler. Does it require the platform to have thread support?

@amirgon

No. Micropython runs on a single thread no matter if MicroPython's scheduler is enabled or not.

No problem then. :)

@kisvegabor

I've created a repo here: https://github.com/kisvegabor/lv_mpy

Fixed and tested that Actions are working. See PR.

Merged, thank you!
I suggested releasing v5.3 (#543) and start to work on v6.0. So we will have more freedom to customize lvgl if needed.

@kisvegabor

I suggested releasing v5.3 (#543) and start to work on v6.0. So we will have more freedom to customize lvgl if needed.

How about releasing it on v5.3 before I add support for styles and structs? (actually I'm going to add support for generic structs first, as it's the general case of styles, hopefully next week), or do you want to postpone Micropython for v6.0? What is the schedule for v6.0 release?
The advantage of releasing it early is letting the users know about this, and giving them the opportunity to try it out even if some features are still missing.
What do we do about Makefile/build script that would run gen_mpy.py? Or do you suggest the user would run it manually, not as part of an automatic build process?

I vote new branch for python implementation.

How about releasing it on v5.3 before I add support for styles and structs? (actually I'm going to add support for generic structs first, as it's the general case of styles, hopefully next week), or do you want to postpone Micropython for v6.0?

Good point, I agree to add Micropython support in v5.3 to allow people to try it.

What is the schedule for v6.0 release?

There is no strict deadline. But I'm sure it will take at least 3 months to add all the planned features.

What do we do about Makefile/build script that would run gen_mpy.py? Or do you suggest the user would run it manually, not as part of an automatic build process?

The call of gen_mpy.py should be added to the Micropython's Makefile, right? So when you build Micropython for e.g. ESP32 or UNIX always get an up to date binding.
Another thing:
@rreilink added "normal" Python binding to LittlevGL. Now he is creating an example to test the binding.
So if we'd have more bindings I suggest having an lv_bindings folder where we can add the interface files and scripts.

@turoksama

I vote new branch for python implementation.

My view was to release Micropython binding with v5.3 as it is and update it in dev-6.0 after the release of v5.3. What would be the advantage of using a separate branch instead of keeping things in one place? Because a bug in the binding does not break LittelvGL itself.

It sounds good to bind as what you said.
One thing i concerned was integrity of the library, and it seems too much.

@turoksama

One thing i concerned was integrity of the library, and it seems too much.

What do you mean "integrity of the library"? Let's say there is some bug. lvgl is not affected since Micropython binding does not change anything in lvgl sources, it's just a python script that creates a .c file. So the worst thing that can happen is that someone who tries to use Micropython binding would see some problem (specific to Micropython), and that's a good thing as he would report this and we would fix that early.

Another thing:
@rreilink added "normal" Python binding to LittlevGL. Now he is creating an example to test the binding.

I wish we know about this earlier... :unamused:
I don't see a point in making the same work twice. What's the point in two different implementations that achieve the same goal?
@kisvegabor I don't think it's a good thing for lvgl to integrate two different implementations for Micropython. How would the user select one? Why maintain two scripts instead of one? Why provide two different python interfaces for the same library?

@kisvegabor
Python and Micropython, from the user perspective, are not so different. At least not from the core language aspect.
A user that uses Python binding and wants to migrate to Micropython would have to change his code since the API looks different, things are implemented differently etc.
Ideally we could think of one python script that could generate both Python and Micropython binding, and most of its code would be common for both.
Benefits of such script are unified python API for lvgl from the user perspective, single source code to add features / fix bugs related to binding etc.

@amirgon, @kisvegabor

I wish we knew about this earlier...
Yes, I do to 😒

Ideally we could think of one python script that could generate both Python and Micropython binding, and most of its code would be common for both.
Benefits of such script are unified python API for lvgl from the user perspective, single source code to add features / fix bugs related to binding etc.

I totally agree.
I have not looked into the MicroPython code generation, I will do so this week, and propose a way to integrate the two bindings-generators to achieve what you propose: unified API and the most commonality in the source code.

@amirgon @rreilink If you guys believe that the same script can work for Micropython and Python too, of course, it would be the best! :smile:

@amirgon
@rreilink added two examples with Python3. You can check them here: https://github.com/rreilink/pylvgl

@kisvegabor
In some places you have #defines instead of an enum.
For example, colors and opacities:
#define LV_COLOR_WHITE   LV_COLOR_MAKE(0xFF,0xFF,0xFF)
#define LV_COLOR_SILVER  LV_COLOR_MAKE(0xC0,0xC0,0xC0)
#define LV_COLOR_GRAY    LV_COLOR_MAKE(0x80,0x80,0x80)
...
#define LV_OPA_TRANSP    0
#define LV_OPA_0         0
#define LV_OPA_10        25
#define LV_OPA_20        51

What is the reason for that?
Since I preprocess the sources with standard C preprocessor, I can't parse defines. I can only parse  true C constructs (such as enums).
It would help a lot to converting them to enums.
What do you think?

@kisvegabor Not sure that all C compilers will optimize const lv_color_t to a constant value. C++ compilers would but LittlevGL is not a C++ project.
@amirgon I think colors could safely be duplicated between the Python binding and LittlevGL itself.

@kisvegabor @amirgon
I am currently refactoring my code to use pycparser as @amirgon does (I like that approach better than my flaky regex parser). pycparser comes with a C-preprocessor that is implemented in Python using PLY. Using that preprocessor, I am able to also extract all defines from the source code. I'll try to upload some code tonight.

@rreilink

Using that preprocessor, I am able to also extract all defines from the source code

That would definitely help :grinning:
I'll check it out.

@kisvegabor @amirgon
See sourceparser.py for how I managed to use the pycparser C-preprocessor to parse the lvgl source.
I completely refactored my  bindings-generator, using pycparser. I also split out language-specific and the generic code, which would allow me to merge @amirgon 's code
Next thing I'll do, is see if I can generate an lv_mpy.c identical to @amirgon 's from the same bindings generator. That should pave the way towards unification of the two codebases.

@rreilink  Happy to hear this great progress! :+1: I'm curiously waiting for @amirgon 's thoughts :smile:

But first I'll help my girlfriend prepare some Christmas deserts tomorrow

Yeah, not always just working :smile:

See sourceparser.py for how I managed to use the pycparser C-preprocessor to parse the lvgl source

@rreilink Did you consider contributing to pycparser instead of extending pycparser.ply.cpp.Preprocessor with your CPP class?

@kisvegabor
Oh, it's Christmas time!
Merry X'mas.
Have a NICE VACATION.

@rreilink Did you consider contributing to pycparser instead of extending pycparser.ply.cpp.Preprocessor with your CPP class?

@amirgon I had not, but I think it's a great idea so I'll do that (at least for the generic issues that are solved by my extension)

@amirgon I am trying to reproduce the lv_mpy.c file from your repo, but I cannot get the same result. I am using python2.7, and am executing the command as mentioned in lv_mpy.c, within amirgon/lv_mpy, revision e8954c6  . See attached diff: diff.txt
(I am using the lv_mpy repository since the generated file depends on an lv_conf.h, which is not in the amirgon/lvgl repository)
Any ideas?

@amirgon I am trying to reproduce the lv_mpy.c file from your repo, but I cannot get the same result.

@rreilink It seems you are not running the script on the same lvgl version I was running it to generate my output file. For example, on your version there was calendar object, which is missing from my version.
Please try to follow this:

Fork https://github.com/amirgon/lv_mpy
cd lib/lvgl/micropython
python gen_mpy.py -X anim -X group -I pycparser/utils/fake_libc_include ../lv_objx/*.h > lv_mpy.c

You should get the same result: lv_mpy.c  on the same directory would be generated the same.
I'm working now on processing structs generically (which would also take care of styles) so expect lots of more changes :wink:

Hi @Amirgon,
I had already tried using your lv_mpy repo. I have run the command exactly as specified in lv_mpy.c, and I do get the extra calendar object found. (Which seems to make sense as it is indeed present in the lvgl version that is in your repo).
A couple of things come to mind:

What Python version are you using? The generated file depends on the order of items in a dictionary, which is undefined in Python <3.6

Do you have pycparser also installed on your Python installation? Since you append ‘./pycparser’ to sys.path instead of inserting it at the front

Could you try to see if with a clean checkout, you still get the same lv_mpy.c that is in the repo?
gen_mpy.txt

Could you try to use Python 3.6 or 3.7 to generate lv_mpy.c and send me the result? There is only one change required to make your gen_mpy.c work under Python 3 (and still keep it working under Python 2):

-s = subprocess.check_output(pp_cmd.split())
+s = subprocess.check_output(pp_cmd.split()).decode()
(patch is attached)
If you are starting on the structs, I have already done quite some stuff for getting the styles to work under normal Python, so I’d be happy to help you out there; be sure to have a look in my pylvgl repo.
I’d like to get to integrating our efforts. My first step would be to merge the code generating scripts but still generate exactly the same code; so that’s why I am trying to replicate lv_mpy.c .
Regards
Rob

@kisvegabor @amirgon
I have managed to combine the efforts of amirgon and myself into a template-based bindings generator that generates the same lv_mpy code as amirgon's script does. It can be found here.
Within the process, I have come across several things that amirgon and myself have approached differently. I think we should try to agree on one single approach, and I would like to get your opinion on these. I have listed my personal preference at the bottom. Please let me know you thoughts.

How to access enums?
a) lvgl.LV_CURSOR_NONE
b) lvgl.LV_CURSOR.NONE
c) lvgl.CURSOR_NONE
d) lvgl.CURSOR.NONE (current lv_mpy)

Where to store object-specific enums like LV_DDLIST_STYLE_SEL?
a) only store as module global (according to choice of 1) (current pylvgl)
b) lvgl.Ddlist.STYLE_SEL (like 1c)
c) lvgl.Ddlist.STYLE.SEL (like 1d) (current lv_mpy)

in case of 2b/ 2c, also store a module global enum?
a) yes
b) no (current lv_mpy)

what about inconsistently named enums?
lv_mpy uses naming of enums using common prefix, however:

enum {
    LV_PRELOAD_TYPE_SPINNING_ARC,
};
typedef uint8_t lv_preloader_type_t;

--> enum is called LV_PRELOAD_TYPE_SPINNING

a) this is not an issue in case of only storing enum constants as module globals (1a + 2a)
b) lv_mpy method (common prefix) --> in this case LV_PRELOAD_TYPE_SPINNING or PRELOAD_TYPE_SPINNING
c) use typedef name, convert to upper-case style --> in this case LV_PRELOADER_TYPE or PRELOADER_TYPE

Generate bindings for all functions which are not object-related?
Some are completely irrelevant to Python (e.g. lv_mem_xx, lv_ll_xx), others may be useful e.g. lv_color_xxx_to_yyy
a) no, only a few like scr_load etc (hal-related stuff would be done in c anyway) (current pylvgl)
b) yes, but only as far as we can with those argument/return types we already deal with (current lv_mpy)
c) yes, for all

Which source files to use to generate bindings?
lv_mpy uses declarations from H-files; pylvgl uses definitions from C-files. lv_mpy uses definition of lv_xxx_ext_t to determine ancestor; pylvgl uses lv_create_parent calls within the lv_create_xxx method to determine ancestor. Using C files takes a lot more time, since each C-file includes many H files, which need to be re-processed
for every C file.
a) Use H files (current lv_mpy)
b) Use C files (current pylvgl)

What to do for methods for which we cannot (yet) generate bindings?
a) raise RuntimeError at runtime (current pylvgl)
b) Do not generate mapping (method does not exist in Python/MicroPython); add remark in C source-file (current lv_mpy)
c) Do not generate mapping, no remark in C source-file
In any case, the bindings generator script should list which mappings were not generated

How to access style struct members
a) flattened: lvgl.style_plain.body_main_color (pylvgl)
b) structured: lvgl.style.plain.body.main.color

How to get children
a) lv_obj_get_child/lv_obj_get_child (lv_mpy)
b) Implement get_children, which returns a list (pylvgl)

My suggestion:
1d since it is the most Pythonic
2c since grouping the enums with the object keeps the module globals clean
3b since there should be only one obvious way to do things
4c since it is the cleanest; it allows for enums which have names which do not start with the enum name
5a since things like HAL, memory management, linked list (lv_ll) etc are not relevant for Python
6a since it is way faster and provides the same functionality
It would be good to add a test script that checks that C and H function definition and declaration are consistent
7a since it keeps the interface (object methods) the same if we support new argtypes/restypes. It also makes the
bindings generator code simpeler since there is no need to track for which methods code was generated and for which not
8a since it reduces the number of Python objects that need to be defined (only one for the Style struct; not one for every sub stuct)
9b since it is more Pythonic
alternatively: 1c / 2b

@rreilink it shows that you made some effort, I appreciate it.
On my side I'm still working and changing gen_mpy.py script, specifically, adding missing features such as structs and styles.
When writing gen_mpy.py, as a general guideline, I tried to keep my code as generic as possible: I don't care what a function or an object is doing, I just expose it syntactically to the Micropython user.
When running gen_mpy.py the user can configure the result by changing #defines on lv_conf and by providing input to the script (the -X option). But in the script itself, I tried hard to avoid references to lvgl specific types functions and objects.
Even for Style and Font, I'm making an effort to treat them as just another (more complex) struct and not handle them explicitly in the code.
The reason is that API tends to change and evolve, and changing the python script is harder than changing the inputs for it. It would make the script more "adaptable" to changes and additions.
@rreilink, what is your opinion about this? I noticed that your code makes more assumptions. For example, lv_coord_t lv_color_t and lv_opa_t appear explicitly as types, and you have an explicit Style object so you are probably treating it different than a generic struct.

About your questions:
(1), (2), (3), (8) - In general I prefer hierarchy whenever it's possible, that is the reason for the nested access to enums.
From the user perspective, clicking TAB (or help) and getting gazillion functions/globals/enums is less favorable than getting a shorter list of only the "top-level" ones and drilling down as necessary.
I'm also working on nested access to structs, even when the structs are nested in C. For example style.body.border.color. I'm not a big fan of "getters/setters" and prefer the more Pythonic "Attribute" access (so you could refer to a field in a struct more naturally, as you would in C for a struct field).
(4) I think we should consider solving inconsistencies by fixing lvgl itself. Looking forward, lvgl does not promise backward compatibility across major releases (see for example #316 where V6.0 would include a breaking change related to action handlers). Especially when this is a simple naming convention error.
(5), (9) As I mentioned, I tried not to care what is the purpose of the function. As long as it is defined in an H file that is given as an input to the script, it can probably be useful to the user. Even memory handling and file access functions. If the user would like to remove them, this can be done either by configuring lvgl not to preprocess them, or by some runtime flags to the script.
The downside of not knowing the function purpose is that it's harder to handle some cases in a more natural (or "pythonic") way (like your example of get_children, which returns a list). A solution for that could be defining and implementing some naming convention to describe iterators (and changing lvgl sources) such that the script could deduce the fact that this is an iterator automatically without knowing anything about the purpose of the function. The advantage would be that when a new iterator is added to lvgl it would be automatically supported.
(6) I think that H files should be sufficient for creating the binding. We only care about API, not implementation. The user provides the API H file to the script and the script generates the binding for that API.
(7) Ideally I would like to be able to generate "everything", the complete Python API for the given set of H files. Practically I'm working incrementally so some functions won't be generated (usually because of missing conversions, but possibly because of conventions or ambiguity issues). From the user perspective when a function is missing, the (power-) user would browse the C sources and try to understand what went wrong. A comment there ("we wanted to create that function and didn't do that because ____") is very useful.
I don't think the script should stop with an error when not able to generate something. It would make it less resistant to changes in lvgl. A small change in lvgl may cause some function not to be available. But the Python binding can still be useful even if it doesn't include everything.

@amirgon ,
I like the idea of minimizing the amount of 'lvgl specifics' into the script. Also, the idea of generalizing struct access seems a good idea to me. I am going to have a look on how to implement that in Python.

I'm not a big fan of "getters/setters" and prefer the more Pythonic "Attribute" access (so you could refer to a field in a struct more naturally, as you would in C for a struct field).

I agree. Struct access should be similar between C and Python, so attribute access it is.

(4) I think we should consider solving inconsistencies by fixing lvgl itself.

I agree, however if it is practical for the time being, we could implement a fix in the bindings generator for the time being I think (especially if we want this script to work for lvgl 5.x series where changing names is not so straight forward as for a new major release)

(5) ... Another option is some naming convention that would tell us that this is some "internal" function that we should not care about.
(6) I think that H files should be sufficient for creating the binding. We only care about API, not implementation.

I agree. I will change my script to use the H-files only. It will speed up processing, and it will ensure only the API is exposed. At least all internal functions that are not in the H-files will not be converted this way.

(7) ... I don't think the script should stop with an error when not able to generate something.

I agree. Maybe I was not clear, my script doesn't generate an error when it is not able to generate a binding; instead it generates a Python method which in turn raises an error when it is called. So my question here would be whether we generate a method that raises an error, or whether we generate no method at all.
Finally,

The downside of not knowing the function purpose is that it's harder to handle some cases in a more natural (or "pythonic") way (like your example of get_children, which returns a list)

For get_children, we could add it, aside from lv_obj_get_child.
I think we're on the same page for most or all of the issues, so I'll have a look into implementing these as discussed.

@rreilink

I agree, however if it is practical for the time being, we could implement a fix in the bindings generator for the time being I think (especially if we want this script to work for lvgl 5.x series where changing names is not so straight forward as for a new major release)

LittlevGL v5.3 is the last 5.x release and will be released somewhere in January (at most a month from now).  Work will then start on v6.0. If you were willing to wait until February, we could skip Python support for 5.3 and make the necessary changes in 6.0.

@rreilink I fully agree with the approaches in your latest post.

I agree. Maybe I was not clear, my script doesn't generate an error when it is not able to generate a binding; instead it generates a Python method which in turn raises an error when it is called. So my question here would be whether we generate a method that raises an error, or whether we generate no method at all.

I vote for raising an error if an unimplemented lvgl function is referred. This way the user will know what is going on. Else he won't know why the method is not there and he could think that it has an other name, it's not implemented or it's just a configuration issue. So, in my opinion, it's clear to raise an error an tell the function is not implemented.

For get_children, we could add it, aside from lv_obj_get_child.

I think it's a good idea to make possible to add some custom functions too. get_children is a good example for it because it's much easier to use in Python than the original function in C. So I think it's worth to add it and probably others too.

@kisvegabor

I think it's a good idea to make possible to add some custom functions too. get_children is a good example for it because it's much easier to use in Python than the original function in C. So I think it's worth to add it and probably others too.

My opinion is that in many cases there is a better approach than "custom functions". For example, in case of get_children we can come up with a naming convention that will hint the parser that this is an iterator (it will require changing the relevant function name and function parameters on lvgl).
Based on this naming convention, the parser could automatically generate a list or a generator that will represent it in a more "pythonian" way.
The advantage of this over "custom functions" is that if someone ever wants to add a new iterator, we wouldn't need to add another custom function. As long as the iterator-naming-convention is kept, the parser could represent the new iterator correctly without any changes to the parser script.
What do you think?

@amirgon, @kisvegabor , regarding 'custom functions':
In general, I agree that 'generic' is better than 'custom'. However, while implementing pylvgl I have found that there are several places which are 'just a bit different than all the others', i.e. exceptions. Of course, one would want to avoid those if possible, but sometimes the extra effort of writing something generic does not outweigh the advantages.
For example, lv_btnm_set_map is the only function that takes a char ** as argument, which then also needs to be kept available (i.e. it cannot be on the stack). It needs to convert a Python list-of-str to char** and store a copy on the heap, and then of course free that memory when appropriate. See here for how I implemented that.
As a counter-example, I also implemented lv_label_get_letter_pos as a custom implementation, which
Concluding, I think we should strive for generic implementations, but decide on a case-by-case basis whether generic or custom is the best way to go.

@rreilink
Iterators
I agree naming conventions can work for iterators. For example, we can change:

lv_obj_get_child -> lv_obj_iterate_child
lv_obj_get_child_back -> lv_obj_iterate_child_back
lv_list_get_next_btn -> lv_list_iterate_btn
lv_list_get_prev_btn -> lv_list_iterate_btn_back

char * * in button matrix
A year ago tried to add Lua binding to LittelvGL and found that the most C specific thing in the library is the char ** variable of the button matrix. And now you found it also difficult to implement. So I'm open to get rid of it somehow or provide an other set function which allocates the map too.
generic / custom
In general, we can modify the names and even extend LittlevGL to better support generic generation and "compatibility" with high-level languages. I believe it will be sufficient in most of the cases. But as @rreilink  said, "sometimes the extra effort of writing something generic does not outweigh the advantages". I hope it will be the minority of the cases.

@kisvegabor Didn't the API of lv_btnm_set_map also become a problem in #562 when we were discussing multi-language support?

@embeddedt Yes, it caused difficulties there too because it"s the only char ** in the lib and it's difficult the handle. Maybe using separator characters (e.g. \t -> "abc\tABC\t1\t2\t3") would solve both issues. This way we would have a simple string here too.

@kisvegabor What is the current behavior of lv_draw_label with regards to tabs? Are they ignored or printed as spaces?

@embeddedt Ignored. But I thought then lv_btnm_design would have a harder work to extract the button's text from the long string.

@embeddedt Yes, it caused difficulties there too because it"s the only char ** in the lib and it's difficult the handle. Maybe using separator characters (e.g. \t -> "abc\tABC\t1\t2\t3") would solve both issues. This way we would have a simple string here too.

Would it work if we would use \0 as separator? Like: char * map = "A" "\0" "B" "\0" "\n" "\0" "C" "\0" "D" "\0"; This way, every individual button label is usable as a NUL-terminated C-string, and the end-of-map is recognizable as an empty string (i.e. two subsequent NUL chars).
Since it is one contiguous char array, allocation etc is a lot easier.
Note that I am using separate "\0" literals and not like e.g. "1\023\0" which would be interpreted as octal character \023 whereas you would probably be meant as "1" "\0" "23" "\0"

@rreilink It's really a good idea and easier to handle during drawing than my \t separator.
@embeddedt  I think examples in the documentation and in the source code are sufficient. Do you think we need more?

@kisvegabor  @amirgon

Iterators
I agree naming conventions can work for iterators. For example, we can change:

lv_obj_get_child -> lv_obj_iterate_child
lv_obj_get_child_back -> lv_obj_iterate_child_back
lv_list_get_next_btn -> lv_list_iterate_btn
lv_list_get_prev_btn -> lv_list_iterate_btn_back

In case of iterators, the Python method would not need a ..._back iterator. Also, as discussed, we would like to still keep the exact C-method available as well.
So in case of lv_obj_iterate_child, we could have lvgl.Obj.iterate_child and lvgl.Obj.iterate_child_back which take the same arguments like C, and additionally a method like lvgl.Obj.iterate_child_list which returns a Python list (which only exists in Python/MicroPython).
Additional things that I'd like to encode (maybe in the argument names) is:

whether a pointer argument may be NULL or not. Suggestion: use like lv_obj_t *obj_or_null if it may be NULL.

whether a pointed-to argument should stay available after the  function call or not (e.g. the lv_obj_set_style does require this, but lv_label_set_text does not). Suggestion: use char *text_volatile or something the like if the pointed-to argument may disappear after the function call.

whether a char* argument is input (usually) or output (like in lv_ddlist_get_selected_str). Suggestion: use char * buf_output in case it is an output.

Actually the latter method is also a bit evil since it does not allow the required buffer size to be known; the user must know that the output will never be longer than the longest item text. If for dd_list we would use "\0" terminated item names, lv_ddlist_get_selected_str could just return a char*. I'm not sure if there are other cases where something similar exists? At least we could ensure that we don't generate a binding for any method that writes data to char* input arguments.

@kisvegabor I guess we could just point out in the documentation that the "\0" literals have to be placed separately.

@kisvegabor @embeddedt

I think examples in the documentation and in the source code are sufficient. Do you think we need more?

Maybe also some clever macro that helps to construct the string? And indeed, for sure mention this caveat in the documentation.

@rreilink I thought of using a macro but I don't see how it could work for button matrixes of arbritary length.

@rreilink
The lvgl.Obj.iterate_child_list extension sound good to me!

Additional things that I'd like to encode (maybe in the argument names) is:

whether a pointer argument may be NULL or not. Suggestion: use like lv_obj_t *obj_or_null if it may be NULL.

What about processing the description above the function declaration? We can have special "tags" in it like $NULL or $CAN_BE_NULL

whether a pointed-to argument should stay available after the function call or not (e.g. the lv_obj_set_style does require this, but lv_label_set_text does not). Suggestion: use char *text_volatile or something the like if the pointed-to argument may disappear after the function call.

I think it would be better to mark the non-volatile arguments because there are only a few of them. Most of the passed data will be saved by the object.
What about an other tag in the description, like $NOT_SAVED?

whether a char* argument is input (usually) or output (like in lv_ddlist_get_selected_str). Suggestion: use char * buf_output in case it is an output.

I agree.

Actually the latter method is also a bit evil since it does not allow the required buffer size to be known; the user must know that the output will never be longer than the longest item text. If for dd_list we would use "\0" terminated item names, lv_ddlist_get_selected_str could just return a char*. I'm not sure if there are other cases where something similar exists? At least we could ensure that we don't generate a binding for any method that writes data to char* input arguments.

In the case of ddlist the separators are \ns because to save memory all the options are one label in multiple lines. So, unfortunately, the  \0 terminated string idea is not working here.
It's not a solution but, passing the buffer length too can make lv_ddlist_get_selected_str safer.

@kisvegabor @rreilink

What about processing the description above the function declaration? We can have special "tags" in it like $NULL or $CAN_BE_NULL

Adding "tags" in comments requires parsing the source before preprocessing it.
While the C parsing library (pycparser) is very good, it does not have a good support for preprocessing. The preprocessing library is not part of pycparser and looks buggy.
I think it's better to add hints that can be obtained after preprocessing (not on comments or macros), for example as typedefs or by naming conventions.

@amirgon @rreilink
I see, then we have no choice. Let's use the parameter names.
The parameter names can be updated even in v.5.3. What do you think, shall we choose descriptive, readable names as @rreilink suggested (e.g. "obj_or_null" ) or some abbreviations (e.g. "obj_cn" means "obj_can_null")?
I vote for abbreviations because in case of lv_obj_set_style the style parameters can be NULL and only its pointer is saved. So a readable name could look like style_or_null_pointer_saved. An abbreviated is style_cn_ps.

@kisvegabor

I vote for abbreviations because in case of lv_obj_set_style the style parameters can be NULL and only its pointer is saved. So a readable name could look like style_or_null_pointer_saved. An abbreviated is style_cn_ps.

I'm also in favor of the abbreviations.

@amirgon @kisvegabor
I would favor readability (readability counts) especially since if we use these conventions consistently (and we should if we want to generate bindings correctly), C-only users of the library will also benefit, since they too will know what the library expects.
How about:

"_or_null" if it can be null (I don't really like "can_null" since in correct English it would be "can_be_null" or "may_be_null" but that's such a long scentence
"_retain" like the Objective-C keyword if the caller needs to retain the referenced argument (char array or style struct) after the call has been made. (I would vote against "pointer saved" since the fact that the pointer is saved doesn't really matter; what matters is that the referenced argument is not saved. So it would be "only_pointer_saved" or something but that would be really long)

So then it would be style_or_null_retain. Still not very short, but I would say workable, and a lot more readable/explicit than style_cn_ps.

The preprocessing library is not part of pycparser and looks buggy and incomplete.

For what it's worth: the preprocessing library is certainly not complete, but with two minor fixes (which I have discussed with the owner and for which I have pushed fixes) I have found it well suited for the lvgl sources. A benefit over an 'external' preprocessor would be that we get exactly the same result cross-platform (i.e. also on Windows where Python modules generally need to be compiled with MSVC). But that choice is completely separate from all other choices to be made, and can easily be reconsidered at any time.

@kisvegabor @amirgon

I think it's better to add hints that can be obtained after preprocessing (not on comments or macros), for example as typedefs or by naming conventions.

Just another idea: what if we would do:
#ifdef LV_GENERATE_BINDINGS
#define LV_NOT_NULL(x) bindings_not_null(x)
#define LV_RETAIN(x) bindings_retain(x)
#else
#define LV_NOT_NULL(x) assert (x)
#define LV_RETAIN(x)
#endif

and then in lv_obj.c:
void lv_obj_set_style(lv_obj_t * obj, lv_style_t * style) {
    NOT_NULL(obj);
    RETAIN(style);
    ....
}

(this is a bit of a lame example since the first argument lv_obj_t *obj may never be null, but anyway). Now if we define LV_GENERATE_BINDINGS when preprocessing for bindings generation, we will find function calls in the preprocessed output, which we can use to control bindings generation. As a bonus, when using the library (even C-only), we get a run-time check.

@rreilink
This is an interesting idea.
But why do we need LV_GENERATE_BINDINGS at all?
You can define bindings_not_null and bindings_retain as static inline functions that do nothing. We would still identify them when parsing C, but the compiler will optimize them out.
Another problem is that we wanted to parse only the H files. You suggest adding these hints to the function definition, which is usually on a C file.

@rreilink

I have found it well suited for the lvgl sources.

And then someone adds to lvgl some innocent macro/comment and everything breaks. I don't think we should rely on unreliable libraries which are not well maintained.

A benefit over an 'external' preprocessor would be that we get exactly the same result cross-platform

Actually C preprocessor is standard across platforms and OSs, even windows. Even MSVC tries to.
If you don't want to use /experimental:preprocessor, you can always use gcc. It works very well under windows with MinGW/MSYS2.
So I don't expect that cross-platform should be a concern.

@amirgon

And then someone adds to lvgl some innocent macro/comment and everything breaks. I don't think we should rely on unreliable libraries which are not well maintained.

Fair point, I agree.

You can define bindings_not_null and bindings_retain as static inline functions that do nothing. We would still identify them when parsing C, but the compiler will optimize them out.

Nice idea.

Another problem is that we wanted to parse only the H files. You suggest adding these hints to the function definition, which is usually on a C file.

Yes, good point. Two ways I see:

Parse the C-files anyway. We can check that function declarations and definitions match. Maybe first use only the H-files to derive which functions etc exist, then use the C-files to find the additional info

Use macros in the H files:

void lv_obj_set_style(lv_obj_t * obj, lv_style_t * style)
LV_TAG(  bindings_not_null(obj), bindings_retain(style) ) ;

And then for bindings generation:
#define LV_TAG(...) { __VA_ARGS__ }

and during normal use:
#define LV_TAG(...)

This makes the line expand to a function definition when generating bindings, and to a declaration during normal use. By using the , operator we can use the macro with any number of tags.
I like 2) most since it provides the features we need while still allowing us to only process the H-files. As a bonus, it does not alter the generated code during 'normal' compilation, since the empty macro definition makes the tags disappear. We don't even need to define the bindings_not_null etc functions.

@rreilink
Yes (2) is a nice idea. I prefer not to parse the C files.
One problem I see, is that it might not compile (link, actually) correctly with bindings generation, since the same function lv_obj_set_style would have more than one definition when the H file it's declared on is included more than once in the same application.
Another thing I don't like is the two "modes" of compilation (with/without LV_GENERATE_BINDINGS). Every build we would need to build twice to make sure it compiles both ways, to make sure there's no syntax error in one of them.
How about separating tags from the function completely? Something like:
void lv_obj_set_style(lv_obj_t * obj, lv_style_t * style);
LV_TAG(lv_obj_set_style(bindings_not_null(), bindings_retain()));

and we can define LV_TAG, bindings_not_null, bindings_retain like this:
void* bindings_not_null();
void* bindings_retain();

#define LV_TAG(...) static inline void __tag ## __LINE__() { __VA_ARGS__; }

Actually... looking at this again, I tend to prefer a simple naming convention of parameters instead of tags.
I still prefer the abbreviated version @kisvegabor suggested over your more verbose "readability counts" suggestion . It's shorter, less verbose, and you can get used to it. Imagine, for example, writing every time unsigned_integer_8_bit_type instead of u8_t or uint8_t :wink:

@amirgon @rreilink I think @kisvegabor's abbrevations are better as well. style_or_null_retain looks very long and unmanageable.

@rreilink
Yes (2) is a nice idea. I prefer not to parse the C files.
One problem I see, is that it might not compile (link, actually) correctly with bindings generation, since the same function lv_obj_set_style would have more than one definition when the H file it's declared on is included more than once in the same application.

The H files which are preprocessed for bindings generation, would not be linked, so that's no issue.

Another thing I don't like is the two "modes" of compilation (with/without LV_GENERATE_BINDINGS). Every build we would need to build twice to make sure it compiles both ways, to make sure there's no syntax error in one of them.

That's where continuous integration should help... We would need to check correct behaviour for all bindings anyway. As a bonus, the generated bindings can be used to check correct behaviour of the C-library using a high-level language testing framework.

How about separating tags from the function completely? Something like:
void lv_obj_set_style(lv_obj_t * obj, lv_style_t * style);
LV_TAG(lv_obj_set_style(bindings_not_null(), bindings_retain()));

Actually... looking at this again, I tend to prefer a simple naming convention of parameters instead of tags.

I agree separating isn't a nice solution. I still see a few issues with the naming convention. The main one is that, with interpreted languages, the names of the function arguments matter, as in obj.set_style(style=x) Thus, renaming argument names will easily break backwards compatibility. With a naming-convention-based solution, renaming arguments may be required in case:

we made a mistake and need to fix it(i.e. forgot to add the retain marker)
some functionality is added and now NULL is allowed while it was not allowed before (e.g. if the feature that lv_obj_align defaults to parent with base==NULL had not yet been implemented and is added)
we find the need for some other marker than null allowed yes/no and retain yes/no

Furthermore, the naming-convention scheme is limited in case we find the need for a marker that is not just a boolean.

@rreilink

I still see a few issues with the naming convention. The main one is that, with interpreted languages, the names of the function arguments matter, as in obj.set_style(style=x) Thus, renaming argument names will easily break backwards compatibility

In Python (and MicroPython) you can choose to use positional arguments (args) or keyword arguments (kwargs). Let's stick to args and avoid kwargs. It will solve compatibility issues with argument names, and is closer to C. Eventually, we provide an API to a C library, so I think it's advantageous to keep the python prototype and usage as close to C as possible, positional arguments being one aspect of that.

Furthermore, the naming-convention scheme is limited in case we find the need for a marker that is not just a boolean.

Do you have an example? How common would that be?
My opinion is that the simplicity of using naming convention is a big factor vs. other tagging techniques. Tagging would affect lvgl API widely so it's best to keep it as simple, straightforward and minimal (short) as possible.

@rreilink @amirgon @embeddedt
I agree people can get used to it. For example, if a function looks like this:
/**
 * Set a new style for an object
 * @param obj pointer to an object
 * @param style_cn_ps pointer to the new style. (Can Null, only the Pointer Saved)
 */
void lv_obj_set_style(lv_obj_t * obj, lv_style_t * style_cn_ps)

So from the comments, it's easy to learn what the abbreviations mean.

I still see a few issues with the naming convention. The main one is that, with interpreted languages, the names of the function arguments matter, as in obj.set_style(style=x) Thus, renaming argument names will easily break backwards compatibility

In Python (and MicroPython) you can choose to use positional arguments (args) or keyword (named) arguments (kwargs). Let's stick to args and avoid kwargs. It will solve compatibility issues with argument names, and is closer to C. Eventually, we provide an API to a C library, so I think it's advantageous to keep the python prototype and usage as close to C as possible, positional arguments being one aspect of that.

As function usually has one or two parameters and their order is quite logical, So I'm not sure  obj.set_style(style=x) appoach is important in thapractice. What do you think?

@kisvegabor, when an lvgl function receives a pointer to a struct type (lv_style_t* for example), this pointer could represent either a single struct (instead of passing the whole struct by value), or it may represent a list of structs where the pointer is to the first one (in such case another parameter would probably contain the number of elements).
In lvgl, how common is the second case of list of structs? Is there a naming convention for the pointer type and the count parameter? If not, we should probably add such naming convention for parsing it correctly.

@amirgon There are only a few functions like this. For example
void lv_line_set_points(lv_obj_t * line, const lv_point_t * point_a, uint16_t point_num)
I suppose in the Python API we don't need the point_num parameter (or count in general) because we have len(points)

I suppose in the Python API we don't need the point_num parameter (or count in general) because we have len(points)

@kisvegabor that's right, but my point is that we would need some naming convention to identify the fact that this is a vector and not a scalar, and correlate the "pointer" to the "size" argument.
Something like:
void lv_line_set_points(lv_obj_t * line, const lv_point_t * point_a_vec, uint16_t point_num_vecsize)

Assuming there is no more than one vector per function prototype.

@amirgon We can do something like this! Can you show a simple dummy Python code snippet about how to use this function? I mean the equivalent of this:
lv_point_t p[]={{10,15}, {100, 105}};
lv_line_set_points(line, p, 2);

Progress update:
We have a working Micropython on ESP32 + LittlevGL + ILI9341.
See littlevgl/lv_binding_micropython#7.

That's quite exciting, I look forward to trying this out! I have an M5Stack which seems to be an ideal test device (ESP32+ILI9341).
The main concern I have with this approach - bindings to a C library - is that the API is likely to reflect the C API too closely. ie not feel very pythonic. The advantage, of course, is to be able to build on top of the large existing codebase... I guess we'll have to see how it progresses!
In any case thanks again for your efforts and I'll try it out as soon as I have some time.

That's quite exciting, I look forward to trying this out!

@mattytrentini That's great! I'd love to hear some user feedback.

The main concern I have with this approach - bindings to a C library - is that the API is likely to reflect the C API too closely. ie not feel very pythonic

Don't be fooled by the fact that it's written in C.
LittlevGL is an object oriented component-based library. There is a base object lv_obj which all other components inherit from, and a hierarchy between the components.  Objects have their method functions, inherit their parent methods etc.
I made an effort to take advantage of this design and make it feel more natural (pythonic?) to the Micropython user. You can create your own (pure Python) composite components from existing LittlevGL components by inheritance.
Here is a simple example:
class SymbolButton(lv.btn):
    def __init__(self, parent, symbol, text):
        super().__init__(parent)
        self.symbol = lv.label(self)
        self.symbol.set_text(symbol)
        self.symbol.set_style(symbolstyle)
        self.symbol.align(self, lv.ALIGN.CENTER,0,0)

        self.label = lv.label(self)
        self.label.set_text(text)
        self.label.align(self, lv.ALIGN.CENTER,20,0)

In this example (originally taken from here, thanks @rreilink!) we create a reusable composite component that inherits from a native LittlevGL btn component. It consists of a button and two labels, one text and the other a symbol.
For more examples please have a look at the Micropython demo script.
The same script works on both "unix+SDL" and "ESP32+ILI9341" platforms. It detects the platform automatically by trying to import the driver and catch an ImportError exception if the driver is not supported.
So, I would like to hear more about your concern.
Could you provide some examples of things that should be done in a more "Pythonic" way?
I believe there are many advantages for starting from LittlevGL instead of building something totally new in pure MicroPython:

LittlevGL is a popular library and very well maintained. It has many existing components, bugs are fixed very fast, questions are answered, new releases are published in a consistent way. @kisvegabor and @embeddedt are doing a great job! They are also very open to changes and additions to the library, a lot of pull requests are accepted to the library after a short review cycle.
Performance: Eventually C is faster, lower footprint etc. I prefer to have GUI rendering logic in C and not in Python.
Exposure: The more people use a library, the better the library is.
A pure Micropython library will not be used by architectures that are not using (or cannot use) Micropython.
But everything supports C. So the same library is used by more architectures and more users. There is also an effort being done to add LittlevGL bindings to more languages (Python  and Ada) which has the potential to grow the user base even more.

Micropython is now an official LittlevGL feature!
I would like to thank @kisvegabor and @embeddedt for their support in this effort. It is a pleasure to work with you!
I would also like to thank @rreilink for his comments, his work on the Python binding and his advanced_demo.py example script which I use with minor changes as my primary example script.
I really hope you would find time to continue working on the Python binding!
Here is a blog post about Micropython + LittlevGL.
Closing the issue.

Thank you very much for adding this awesome feature! Having Micropython support really makes a difference! I'm sure that there is big potential in it!
:clap: :clap: :clap:

There's a blogpost on Hackaday.

@manison
A few days ago I wrote to them about LittlevGL and it seems they have chosen the Micropython binding to highlight.

@kisvegabor perfect marketing 👍

Actually, my friend suggested it so it's his merit. :)


Reorganize tekton-results

Reorganize tekton-results manifest in order to:

Prefer patchesStrategicMerge with kustomize rather than patchesJson6902, the first is clearer.
Move s3 logging config from developer folder to operator one
reflect s3 secrets layout from staging/prod

/retest

/retest

/retest

/retest pipeline-service-test

/retest

/retest

@Mo3m3n , Sorry, but I provided for you information that you should revert changes regarding to minio ca sertificate https://github.com/openshift-pipelines/pipeline-service/pull/545/files#r1132263560 , because that's a breaking change, and connection beetween tekton-results api server and minio tenant server will be broken. But you skip it. Could you explain, please?

@Mo3m3n I proposed https://github.com/openshift-pipelines/pipeline-service/pull/552


Add xdebug during installation.

Fixed in: https://github.com/wotta/dotfiles/commit/9c380f1962bb714dcb38c42b915c53675e79e900


[Message] Messages of the maximum size are not fully displayed.

Environment: Windows 10, 64-bit, Google Chrome 107.0.5304.107.
Reproducible: always.
Preconditions

Go to https://oos.dmytrominochkin.cloud/#
Log in as Parent.
Select a workshop and on the workshop page click "Надіслати повідомлення".
Write a message of 256 characters and press the "Відправити" button.

Steps to reproduce

Log in as a service provider of the workshop chosen by the parent.
Go to "Повідомлення" on the top right corner drop down list.
Click on the message from the parent and pay attention to the sent message on the chat page that opens.
Write a reply of 256 characters in the message input line and click "Відправити".
Log in as Parent.
Go to "Повідомлення" on the top right corner drop down list.
Pay attention to the received message.

Actual result
Messages of the maximum size on the chat page in the parent's cabinet (sent by the father) and the service provider's cabinet  (sent by the SP) are not fully displayed. On the chat pages, scrolling works to the right and messages sent by the parent or service provider are not fully displayed. On the message page, large messages go beyond the frame and break the structure of the page.

Expected result
On the message page, large messages do not break the structure of the page. On the chat page, large messages are displayed in several lines.
Labels to be added
"Bug", Priority ("pri: high"), Severity ("severity: high"), Type ("UI, "Functional").

Fixed


Remove unnecessary outlives bounds

cc rust-lang/rust#61172

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @alexcrichton (or someone else) soon.
If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.
Please see the contribution instructions for more information.

@bors: r+

:pushpin: Commit 46615d290131f4032c653b251a7adb54c0147723 has been approved by alexcrichton

:hourglass: Testing commit 46615d290131f4032c653b251a7adb54c0147723 with merge f97b11c9a32dab52e69f2b87364acbdcbda331c1...

:sunny: Test successful - checks-travis, status-appveyor
Approved by: alexcrichton
Pushing f97b11c9a32dab52e69f2b87364acbdcbda331c1 to master...


[C++][Parquet] Hide thrift dependency in parquet-cpp

Pulling in thrift compiled headers tend to pull in a lot of things. It would be nice to not expose them in the parquet library (the application should be able to use a different version of thrift, etc).
We can also see if it is practical to not depend on thrift at all and replicate the logic we need. Thrift is fairly stable at this point so this might be feasible. This would allow us to do things like not rely on boost.
Reporter: Nong Li / @nongli
Assignee: Wes McKinney](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=wesm) / @wesm
Note: This issue was originally created as PARQUET-446. Please see the migration documentation for further details.

Julien Le Dem / @julienledem:
Issue resolved by pull request 49
https://github.com/apache/parquet-cpp/pull/49

Wes McKinney / @wesm:
Makes sense; it would be nice to extract the TCompactProtocol bits from Thrift into a less heavyweight form. The only downside with this would be taking on responsible for portability issues (this will all need to compile with MSVC).

Wes McKinney / @wesm:
I'm going to tackle this and PARQUET-277 jointly today. I believe that may also resolve PARQUET-470 but I will verify after.

Wes McKinney / @wesm:
While it doesn't get rid of the Thrift linking dependency, I'm working on limiting the exposure of the compiled Thrift headers as much as possible, so many of the compilation units will not have to include parquet_types.h at all.
This is similar to the approach taken in parquet-mr.

Wes McKinney / @wesm:
see patch: https://github.com/apache/parquet-cpp/pull/49

Wes McKinney / @wesm:
see patch for PARQUET-442 which cuts down the number of compilation units touching Thrift significantly (but still more to do): https://github.com/apache/parquet-cpp/pull/38


Use Date sorting to properly sort dates

Use Date sorting to properly sort dates
No ticket, tests were failing because we were using the wrong service.

Before getting review, please check the following:

[x] Does frontend functionality render and work correctly in IE?
[x] Have we load-tested any new pages or internal API endpoints that will receive significant traffic?
[x] Are interactive elements accessible to screen readers?
[x] Have you checked for tech debt you can address in the area you're working in?
[x] If this change involves routes, does it work correctly with pertinent "unusual" routes such as the combined Green Line, Silver Line, Foxboro commuter rail, and single-direction bus routes like the 170?
[x] Are the changes organized into self-contained commits with descriptive and well-formatted commit messages?

Hold on this, will require a different approach for Elixir 1.8

@thecristen Sorry about that. Converted to Elixir 1.8-friendly code and it's passing now.

If anyone is interested, more info about this can be found in this blog post.


setQueryKey() is not compatible with terminators that execute several queries - such as Paginate()

UPDATE: Previously I thought it was propel's caching system in general that was causing this error, but I've found it's just paginate() (or potentially any terminator that uses more than one Query to run).
When using a caching system (I tested both APC & Memcache) the system does actually store successfully (on both the official APC & memcache methods) but it only stores the first SELECT COUNT(*) query. This behavior has occurred on both a Windows & Linux machine.
In other words, look at this behavior, with Memcache disabled (but setQueryCache still live in the code):
Without Cache
[2014-12-29 01:58:40] defaultLogger.INFO: SELECT COUNT(*) FROM (SELECT article.id, article.title, article.body, article.tags, article.positive_votes, article.negative_votes, article.theme_id, article.created_at, article.updated_at, article.slug FROM article) propelmatch4cnt [] []
[2014-12-29 01:58:40] defaultLogger.INFO: SELECT article.id, article.title, article.body, article.tags, article.positive_votes, article.negative_votes, article.theme_id, article.created_at, article.updated_at, article.slug FROM article ORDER BY article.id DESC [] []
First, Propel counts the number of objects, THEN executes the query to actually receive rows - which paginate() needs to do before receiving these rows so it can get the right number of results.
This chain of events is the same if the cache is enabled, except that the first COUNT SQL statement is stored. When Propel follows through to get a paginated list of rows it suddenly finds that a key has been created already, so it just uses the cache. So, in result, Propel executes a COUNT statement twice.
With Cache
[2014-12-29 01:57:12] defaultLogger.INFO: SELECT COUNT(*) FROM (SELECT article.id, article.title, article.body, article.tags, article.positive_votes, article.negative_votes, article.theme_id, article.created_at, article.updated_at, article.slug FROM article) propelmatch4cnt
[2014-12-29 01:57:12] defaultLogger.INFO: SELECT COUNT(*) FROM (SELECT article.id, article.title, article.body, article.tags, article.positive_votes, article.negative_votes, article.theme_id, article.created_at, article.updated_at, article.slug FROM article) propelmatch4cnt [] []
Maybe I am doing something incredibly wrong, but I don't think so, because I followed the documentation, even using the example on paper, and the same problem occurs.
Here is one example of the code that fails with this:
	$posts = ArticleQuery::create()
		->setQueryKey("all_posts")
		->orderById('DESC')
		->paginate($page, $perPage);

Whilst you might say that caching such a general query is unnecessary, it is a test, and a test that is failing. I have ensured that spaces in memcache key names are converted to underscores, etc, so it is definitely a propel issue.
Tested with both
	$posts = new ArticleQuery();
	$posts = $posts
		->setQueryKey("homepage")
		->paginate($page, $perPage);

And the $posts = ArticleQuery::create() way. Adding filler parameters like orderById() has no affect on the result.
Which should have done the same thing anyway.
My suggestion is to either:

Add an optional parameter (defaulting to true) called "cacheable" that is checked before executing the cache behavior.
When using termination methods that require additional queries, append "_paginate" to the initial count query key so that both are cached separately.

@SacredSkull  Why did you close this ? does the referenced PR fix your issue?


FUNCTIONS_RUNTIME_VERSION is not a valid Application settings

In section changing-version-of-apps, it says version is set through FUNCTIONS_RUNTIME_VERSION instead of FUNCTIONS_EXTENSION_VERSION my app is actually using. Is it a mistake or some naming modification to be made in the future?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 1df5e2bd-489d-19b2-87d3-d993ebb82021
Version Independent ID: 9e9d9ded-8f40-22be-4654-48c87f2c59e8
Content: Azure Functions runtime versions overview
Content Source: articles/azure-functions/functions-versions.md
Service: azure-functions
GitHub Login: @ggailey777
Microsoft Alias: glenga

@JohnReeson Thank you for reporting this issue! We will investigate and make any necessary changes.

@JohnReeson Thanks for reporting this bug (good catch)--it's being fixed as we speak!

Fixed in the private repo (#please-close)


Update npm

Update the dockerfile because of a bug in the versioning in the package.json of the dev branch of genenotebook.

Due to fibers not working for Node >= 16.0, we can at most increase to node 15.14
(https://github.com/laverdet/node-fibers)


Add missing license headers

The addlicense tool was recently updated to include .scss files. This PR adds the missing license headers for the .scss files in this project.

/gcbrun

/gcbrun

/gcbrun


ssh 的链接地址不可以使用

ssh 的地址显示的为：
git@<IP_ADDRESS>:gdop/client_bin.git ，
但实际应该是
git@<IP_ADDRESS>:gogs-repositories/gdop/client_bin.git ，
而且 必须要输入 git 用户的密码，而不是 gogs 的用户名密码，请问这个要怎么修改下呢？
谢谢

通过 ln-s 建立软连接的方法暂时解决这个问题了

I met exactly the same problem. Nothing useful from troubleshooting page :-(

Hi @harry91

I met exactly the same problem. Nothing useful from troubleshooting page :-(

So many people says that and turns out they didn't config SSH server right or manually added public keys to .ssh/authorize_keys file.

I haven't modified .ssh/authorize_keys manually. Maybe you can write a wiki about how to generate public/private keys and add public key instead of a simple link to github wiki. I'm still working on it. Clone through HTTP works fine.

Maybe you can write a wiki about how to generate public/private keys and add public key instead of a simple link to github wiki.

SSH key generate process is all the same, and there is a operation on URL /admin allows you to completely rewrite SSH auth file based on Gogs database.
Also, what is the error message you get when you say cannot push or whatever it is?

Sorry, I didn't say cannot push. I cannot clone.
Key creating process(under cygwin):
$ ssh-keygen -t rsa -b 4096 -C<EMAIL_ADDRESS>Generating public/private rsa key pair.
Enter file in which to save the key (/home/Guangyu/.ssh/id_rsa):
/home/Guangyu/.ssh/id_rsa already exists.
Overwrite (y/n)? y
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/Guangyu/.ssh/id_rsa.
Your public key has been saved in /home/Guangyu/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:uDeN5Evxqz........wpChyjZLLfhJIk<EMAIL_ADDRESS>The key's randomart image is:
+---[RSA 4096]----+
balabala......
|    o.oo+..      |
+----[SHA256]-----+
Clone procedure:
Guangyu@GY91 /cygdrive/c/Users/Guangyu/Desktop/abcdefg
$ git clone guangyu@<IP_ADDRESS>:gogsadmin/gogo_park.git
Cloning into 'gogo_park'...
<EMAIL_ADDRESS>password:
Permission denied, please try again.
<EMAIL_ADDRESS>password:
Permission denied, please try again.
<EMAIL_ADDRESS>password:
Permission denied (publickey,password).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
No log from logs under log/ folder or console.

I'm checking with the auth.log and there's something wrong inside. I'll give you a feedback later.

Well... based on the error, there are two possible known situations:

You doesn't run Gogs by correct user, e.g. you config as git but actually run as root.
Your .ssh/authorize_keys file permission isn't 0600.

Good news~! I've finally solved the problem.
I noticed the error information in /var/log/auth.log. It says " Authentication refused: bad ownership or modes for directory /home/guangyu/.ssh"
So I changed the .ssh/ folder to 0700 and .ssh/authorize_keys to 0600. It works.
Sorry, it was my incorrect configuration. Thank you very much~!

@harry91 no problem, good to hear!


allow setting of some ibrowse options

in particular this allows;
[replicator]
ibrowse_options = [{prefer_ipv6, true}]

which allows the replicator to replicate with an ipv6-only source or target.
Closes: https://github.com/apache/couchdb/issues/4668

@rnewson could you help me undrstand when fix will be backported to 3.3.x branch? Or older branch.

@sergey-safarov sorry, I do not know when there will be a release that includes this fix. I expect it to be backported to 3.3.x but not further.


 Here is the traceback:
'NoneType' object is not callable
Traceback (most recent call last):
  File "/opt/az/lib/python3.6/site-packages/knack/cli.py", line 231, in invoke
    cmd_result = self.invocation.execute(args)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 657, in execute
    raise ex
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 720, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 691, in _run_job
    result = cmd_copy(params)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 328, in __call__
    return self.handler(*args, **kwargs)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/command_operation.py", line 121, in handler
    return op(**command_args)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/command_modules/acr/run.py", line 93, in acr_run
    variant=platform_variant
TypeError: 'NoneType' object is not callable

To Reproduce:
Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.

Put any pre-requisite steps here...
az acr run --cmd {} --registry {} {}

Expected Behavior
Environment Summary
Linux-4.15.0-1106-aws-x86_64-with-debian-9.12
Python 3.6.10
Installer: DEB

azure-cli 2.26.0

Additional Context

route to service team

Same issue as https://github.com/Azure/azure-cli/issues/18857 and https://github.com/Azure/azure-cli/issues/18841...
I too am experiencing this ...

Solution for anyone reading this: just don't use the Azure Portal cloud shell for the time being to perform the ACR builds. Install the Azure CLI into some other Linux system, log in, and issue the ACR commands from there.

@daniel347x really?  I'm not using cloud shell but on a private agent running in docker/aks.

A hotfix for the cli has been released.  Please use Version 2.26.1 for the cli:
https://docs.microsoft.com/en-us/cli/azure/release-notes-azure-cli?tabs=azure-cli#july-14-2021

Instructions to update azure-cli can be found here: https://github.com/actions/virtual-environments/issues/3735#issuecomment-880623386
These instruction got my hosted Ubuntu agent working again.


OD-4337 UnicodeEncodeError: 'ascii' codec can't encode

@mwspoton just needs a version bump in init.py!


Failure to load a model doesn't throw an exception

I have no idea why, in the loader at line 53 when returning a model, the check if (!teapot) stops working.

Is it possible that related path is incorrect to the binary? You can try use absolute path to check it.

That's exactly what happened here, a path wasn't right. However, for me, there's no good reason to explode a program if a resource isn't found. I'd leave explosions to something bad, that shouldn't ever happen. Especially, in the future, when the engine has a notion of a 'project' and stores all the paths to the resources in this project file. An attempt to open a project that doesn't have all resources available on the disk (they might've been removed or moved) shouldn't break anything.


[MacOS] mojave 10.14.4 training tools can't be build.(text2image stopped)

I have ran

brew install automake autoconf autoconf-archive libtool
brew install pkgconfig
brew install icu4c
brew install leptonica
brew install gcc
brew install pango
brew install cairo

PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:/usr/local/opt/libffi/lib/pkgconfig:/usr/local/opt/icu4c/lib/pkgconfig
./autogen.sh
./configure CC=gcc-8 CXX=g++-8 CPPFLAGS=-I/usr/local/opt/icu4c/include LDFLAGS=-L/usr/local/opt/icu4c/lib
make -j 4
make training

Text2Image can't be compile which keep shows libpng16.la not found.

Tesseract version: 4.10-rc1
platform:
Darwin 18.5.0 Darwin Kernel Version 18.5.0: Mon Mar 11 20:40:32 PDT 2019; root:xnu-4903.251.3~3/RELEASE_X86_64 x86_64 i386 MacBookPro14,3 Darwin

CurrentBehavior:

libtool:   error: cannot find the library '/Users/username/.conda/envs/tensorflow-env/lib/libpng16.la' or unhandled argument '/Users/username/.conda/envs/tensorflow-env/lib/libpng16.la'
make[1]: *** [Makefile:923: text2image] Error 1
make[1]: Leaving directory '/Users/username/tesseract/src/training'
make: *** [Makefile:924: training] Error 2

I don't know why it keep searching for my anaconda environment libpng16.la which compile with source.
libpng16.la -> /Users/username/libpng-1.6.36/libpng16.la
it stopped at
how can I fixed this?

This is not tesseract bug, but setting of your (system) environment.  We can do nothing with it.
If you want to compile ANY sw from source you need to be familiar with your OS and YOUR setting.


latest update 3.2.13 breaks the model_function column

Just updated to CRUD 3.2.13 and now it requires to have 'name' for column model_function.
This was working fine previously and doc doesn't mention this new requirement.

I agree. I've spend some time looking for this bug...

Yes, I agree with it too.
model_function columns do not work.

Up, same problem downgraded to 3.2.12 works ok

You don't need to downgrade, just add
'name' => 'whatever_name',
in your CRUD Controller and it will work. But pretty surprised this wasn't backwards compatible.

But if you have many controllers which use  model_function it can be big work :)
That's why this issue has to decide in the near future.

Of course this is why I'vve opened this issue. For me fix is simple, if 'name' doesn't exist then create a fake one.

I have the same issue and as I use a custom CrudController and CrudPanel I was able to fix this in my custom CrudPanel class.
I have commented on the commit (https://github.com/Laravel-Backpack/CRUD/commit/1003c9984b0062f9466a2843c541c17e02017dfd) that caused this issue.

Damn. So sorry guys. This did not come up in my testing at all. This is why the PRs get so long to merge - it's always super-tricky to test everything, every time.
I've just pushed @splodgeco 's fix on dev - thanks a lot. Could you guys please confirm this fixes it for you? Just use dev-dev for Backpack\CRUD in your composer.json file and run composer install. It should pick it up right away.
I guess in the future there should be more of us using the dev branch during development, so we can spot these earlier on? Would be a big help, if any of you can do it.

Thanks for the fix, I looked at the code and this is exactly what we wanted ;)

@tabacitu the fix doesn't cover my case:
I have a custom column type. I used it 3 times in my controller. I had to add name attribute to column even I didn't need it, because it was throwing an exception If I don't add it.
So I was using id as name attribute. It was working okay. But now, only one of them shows in columns (the last one only).

@tabacitu Apparently, the "random column name" fix doesn't work on AjaxTables. We need to exclude it from column names.