Compilation error Cosmic

Basic Project ... I'm seeing these errors...
----------- Project STM8_Template - STM8 Cosmic - Configuration Debug -------------
Compiling ........\library\base\gpio.c...
cxstm8 +warn +modsl +debug -pxp -no -pp -l -dUSE_TIM4_ISR -dDEBUG -i........\library\base -i.. -i....\stm8_lib -i..\stm8_lib -i........\library\muboard -i..\stvd_cosmic -i"C:\Program Files (x86)\COSMIC\FSE_Compilers\CXSTM8\Hstm8"  -clDebug\ -coDebug\ ........\library\base\gpio.c
#error cpstm8 ........\library\base\stm8as.h:195 bad # operand
#error cpstm8 ........\library\base\stm8as.h:196 bad # operand
........\library\base\gpio.c:
Apparently cxstm8 doesn't like #asm, #endasm in the define statment
#error cpstm8 ........\library\base\stm8as.h:195 bad # operand
#error cpstm8 ........\library\base\stm8as.h:196 bad # operand
This is a great project! Thanks for posting.

hi RandoSY,
thanks a lot for pointing this out! However, lately I have been extending the functionality of the STM8_templates quite a bit, including SPL support, more STM8 families, and auto-dependency detection (similar to Arduino). Therefore there haven't been updates for quite a while - sorry for that!
Unfortunately I recently ran into a stupid, non-technical issue. Basically my intention was to replace my custom "stm8as.h" by the device headers from the SPL --> support all STM8 families without having to create my own headers (which is a LOT of effort). However, as  pointed out by spth the SPL licence still is not OS --> shouldn't use it here :-(
I am still struggling about how to proceed from here, ideally without having to scrap the work of the past months...
Ok, but to come back to your question: I will look into that and try to fix it. Thanks again for pointing it out!

hi RandoSY,
I have just updated the repository with my current status. As you can see I have made up my mind to stick to my own headers (i.e. not use SPL), but to provide an example how to mix with SPL.
All examples have been tested with the latest SDCC and Cosmic-Eval compilers (latter with exception of "STM8_StdPeriphLib"). Please let me know what you think of the changes, and if it now works for you. Thanks a lot in advance!
Greetings, Georg
PS: with the new ISR structure, I believe things are now much more robust. At least updating the examples to the new structure, and mixing with SPL, turned out to be a breeze so far... :-)

Hi Georg,
Wow! Thanks so much for the consideration of your prompt reply.
I am testing with a fresh install of ST Visual Develop, and recently licensed COSMIC stm8 C compiler (cxstm8).
Still seeing  L
----------- Project STM8_Template - STM8 Cosmic - Configuration Release -------------
Compiling ........\library\base\src\gpio.c...
cxstm8 +mods0 -pp -i.. -i........\library\base\inc -i........\library\stm8s_discovery\inc -i"C:\Program Files\COSMIC\CXSTM8_EVAL\Hstm8"  -clRelease\ -coRelease\ ........\library\base\src\gpio.c
#error cpstm8 ........\library\base\inc\stm8as.h:195 bad # operand
#error cpstm8 ........\library\base\inc\stm8as.h:196 bad # operand
........\library\base\src\gpio.c:
The command: "cxstm8 +mods0 -pp -i.. -i........\library\base\inc -i........\library\stm8s_discovery\inc -i"C:\Program Files\COSMIC\CXSTM8_EVAL\Hstm8"  -clRelease\ -coRelease\ ........\library\base\src\gpio.c" has failed, the returned value is: 1
exit code=1.
STM8_Template.elf - 4 error(s), 0 warning(s)
I wonder if it’s something  wrong in my setup???
Thanks again, all your STM8 work is GREAT! I really need to get over to Linux and take a look at SDCC and your excellent project…
Best,
Randy
PS. Happy Holidays!!!
From: Georg Icking-Konert<EMAIL_ADDRESS>Sent: Tuesday, December 26, 2017 6:32 PM
To: gicking/STM8_templates
Cc: RandoSY; Author
Subject: Re: [gicking/STM8_templates] Compilation error Cosmic (#1)
hi RandoSY,
I have just updated the repository with my current status. As you can see I have made up my mind to stick to my own headers (i.e. not use SPL), but to provide an example how to mix with SPL.
All examples have been tested with the latest SDCC and Cosmic-Eval compilers (latter with exception of "STM8_StdPeriphLib"). Please let me know what you think of the changes, and if it now works for you. Thanks a lot in advance!
Greetings, Georg
PS: with the new ISR structure, I believe things are now much more robust. At least updating the examples to the new structure, and mixing with SPL, turned out to be a breeze so far... :-)
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/gicking/STM8_templates/issues/1#issuecomment-354024833 , or mute the thread https://github.com/notifications/unsubscribe-auth/AAlOm_J4F_4UlTLXHeufIhEyk1YqBqYFks5tEYH1gaJpZM4RLKoc .  https://github.com/notifications/beacon/AAlOm7l6QpSI7InKMAZ9D0nWHc5NYkB-ks5tEYH1gaJpZM4RLKoc.gif

Georg,
OK, finally have correct compiler in path, simplest template example. Sorry I am SO SLOW today.  Way, way too much party over XMAS.
Removing intermediate files ... Done!
----------- Project STM8_Template - STM8 Cosmic - Configuration Release -------------
Compiling ........\library\base\src\gpio.c...
cxstm8 +mods0 -pp -i.. -i........\library\base\inc -i........\library\stm8s_discovery\inc -i"C:\Program Files (x86)\COSMIC\FSE_Compilers\CXSTM8\Hstm8"  -cl..\stvd_cosmic\ -co..\stvd_cosmic\ ........\library\base\src\gpio.c
#error cpstm8 ........\library\base\inc\stm8as.h:195 bad # operand
#error cpstm8 ........\library\base\inc\stm8as.h:196 bad # operand
........\library\base\src\gpio.c:
The command: "cxstm8 +mods0 -pp -i.. -i........\library\base\inc -i........\library\stm8s_discovery\inc -i"C:\Program Files (x86)\COSMIC\FSE_Compilers\CXSTM8\Hstm8"  -cl..\stvd_cosmic\ -co..\stvd_cosmic\ ........\library\base\src\gpio.c" has failed, the returned value is: 1
exit code=1.
STM8_Template.elf - 3 error(s), 0 warning(s)
From
/**********************
simple C-project without Arduino-like initialization and
setup()/loop(). No assumption is made, no interrupts used.
Can be used as starting point for projects with extreme
memory limitations or real-time requirements.
Functionality:

init FCPU to 16MHz

configure LED pin as output

toggle LED every 500ms w/o timer -> blocking

**********************/
/*----------------------------------------------------------
INCLUDE FILES

----------------------------------------------------------*/
#include "stm8as.h"     // STM8 peripheral registers
#include "gpio.h"       // pin access routines
#include "sw_delay.h"   // dummy pause without timer
// define board LED pin for STM8S Discovery Board
#define LED   pinSet(PORT_D, pin0)
// main routine
void main(void) {
// switch to 16MHz (default is 2MHz)
CLK.CKDIVR.byte = 0x00;
// configure LED pin as output
pinMode(PORT_D, pin0, OUTPUT);
// main loop
while (1) {
// toggle LED

LED ^= 1;

// wait some time

sw_delay(500);

} // main loop
} // main
Best,
Randy
From: Georg Icking-Konert<EMAIL_ADDRESS>Sent: Tuesday, December 26, 2017 6:32 PM
To: gicking/STM8_templates
Cc: RandoSY; Author
Subject: Re: [gicking/STM8_templates] Compilation error Cosmic (#1)
hi RandoSY,
I have just updated the repository with my current status. As you can see I have made up my mind to stick to my own headers (i.e. not use SPL), but to provide an example how to mix with SPL.
All examples have been tested with the latest SDCC and Cosmic-Eval compilers (latter with exception of "STM8_StdPeriphLib"). Please let me know what you think of the changes, and if it now works for you. Thanks a lot in advance!
Greetings, Georg
PS: with the new ISR structure, I believe things are now much more robust. At least updating the examples to the new structure, and mixing with SPL, turned out to be a breeze so far... :-)
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/gicking/STM8_templates/issues/1#issuecomment-354024833 , or mute the thread https://github.com/notifications/unsubscribe-auth/AAlOm_J4F_4UlTLXHeufIhEyk1YqBqYFks5tEYH1gaJpZM4RLKoc .  https://github.com/notifications/beacon/AAlOm7l6QpSI7InKMAZ9D0nWHc5NYkB-ks5tEYH1gaJpZM4RLKoc.gif

Georg,
Here’s the offending section, of the required header file. ???, looks correct to me???
// Cosmic compiler
#if defined(CSMC)
#define reg(addr,type,name)    extern volatile type name @addr    ///< syntax for variables at absolute addresses
#define ASM(mnem)    _asm(mnem)                                   ///< single line inline assembler
#define ASM_START    #asm                                         ///< start multi-line inline assembler
#define ASM_END        #endasm                                  ///< end multi-line inline assembler
Have a great day.
Best,
Randy
From: Georg Icking-Konert<EMAIL_ADDRESS>Sent: Tuesday, December 26, 2017 6:32 PM
To: gicking/STM8_templates
Cc: RandoSY; Author
Subject: Re: [gicking/STM8_templates] Compilation error Cosmic (#1)
hi RandoSY,
I have just updated the repository with my current status. As you can see I have made up my mind to stick to my own headers (i.e. not use SPL), but to provide an example how to mix with SPL.
All examples have been tested with the latest SDCC and Cosmic-Eval compilers (latter with exception of "STM8_StdPeriphLib"). Please let me know what you think of the changes, and if it now works for you. Thanks a lot in advance!
Greetings, Georg
PS: with the new ISR structure, I believe things are now much more robust. At least updating the examples to the new structure, and mixing with SPL, turned out to be a breeze so far... :-)
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/gicking/STM8_templates/issues/1#issuecomment-354024833 , or mute the thread https://github.com/notifications/unsubscribe-auth/AAlOm_J4F_4UlTLXHeufIhEyk1YqBqYFks5tEYH1gaJpZM4RLKoc .  https://github.com/notifications/beacon/AAlOm7l6QpSI7InKMAZ9D0nWHc5NYkB-ks5tEYH1gaJpZM4RLKoc.gif

hi RandoSY,
looking at lines 195 and 196 I think that the error is triggered by the hashtags inside the #define. Seems like this is forbidden for ANSI-C. But what I don't understand is, that it did compile on my PC - very strange indeed! Give some more time to investigate this...
As a first try: can you please comment lines 195 and 196 in stm8as.h and see if this solves this issue? ASM_START and ASM_END are currently only used in eeprom.h, so it should be ok for a test. Thanks in advance!
Georg

just downloaded and installed "Cosmic Eval" fresh, and "General_Examples/Minimal-C" compiles here without warning or error (see below). Seems like the Cosmic preprocessor can handle hashtags within #defines.
Is there anything special about your installation, e.g. does your build-process use the gcc preprocessor before compiling with cxstm8...? Else I'm at a loss...
Georg
----------- Project STM8_Template - STM8 Cosmic - Configuration Release -------------
Removing intermediate files ... Done!
----------- Project STM8_Template - STM8 Cosmic - Configuration Release -------------
Compiling ........\library\base\src\gpio.c...
cxstm8 +mods0 -pp -i.. -i........\library\base\inc -i"C:\Program Files\COSMIC\CXSTM8_EVAL\Hstm8"  -clRelease\ -coRelease\ ........\library\base\src\gpio.c
........\library\base\src\gpio.c:
Compiling ........\library\base\src\stm8_interrupt_vector.c...
cxstm8 +mods0 -pp -i.. -i........\library\base\inc -i"C:\Program Files\COSMIC\CXSTM8_EVAL\Hstm8"  -clRelease\ -coRelease\ ........\library\base\src\stm8_interrupt_vector.c
........\library\base\src\stm8_interrupt_vector.c:
Compiling ..\main.c...
cxstm8 +mods0 -pp -i.. -i........\library\base\inc -i"C:\Program Files\COSMIC\CXSTM8_EVAL\Hstm8"  -clRelease\ -coRelease\ ..\main.c
..\main.c:
Running Linker
clnk -l"C:\Program Files\COSMIC\CXSTM8_EVAL\Lib"  -o Release\STM8_Template.sm8 Release\STM8_Template.lkf
cvdwarf Release\STM8_Template.sm8
Running Post-Build step
chex -o Release\STM8_Template.s19 Release\STM8_Template.sm8
STM8_Template.elf - 0 error(s), 0 warning(s)

Hi Georg,
I’m thinking we’re getting somewhere
Here’s where I’ve been downloading my free Cosmic Compiler … (after they send  me the license file)
http://cosmicsoftware.com/download_stm8_free.php
This is where my Cosmic STM8 compiler lives … C:\Program Files (x86)\COSMIC\FSE_Compilers\CXSTM8,  etc. …
Whereas I notice your compiler’s instance is installed to here …  C:\Program Files\COSMIC\CXSTM8_EVAL …  DIFFERENT!
So I bet that is at the heart of the matter!
Now that I’ve sobered up a little, it did occur to me to search throughout the code base and just substitute literally whatever the #define statement  was attempting to do, and see if that would correct the matter.
By way of doing something expeditious, commenting out the two offending  lines works just dandy.
I really want to use the ST Develop IDE with your code, the debugger works great, got an old IAR feel, and I mean in a good way.
Thanks again!!!
Randy
Ivy Tech
https://www.ivytech.edu/
From: Georg Icking-Konert<EMAIL_ADDRESS>Sent: Wednesday, December 27, 2017 5:06 PM
To: gicking/STM8_templates
Cc: RandoSY; Author
Subject: Re: [gicking/STM8_templates] Compilation error Cosmic (#1)
just downloaded and installed "Cosmic Eval" fresh, and "General_Examples/Minimal-C" compiles here without warning or error (see below). Seems like the Cosmic preprocessor can handle hashtags within #defines.
Is there anything special about your installation, e.g. does your build-process use the gcc preprocessor before compiling with cxstm8...? Else I'm at a loss...
Georg
----------- Project STM8_Template - STM8 Cosmic - Configuration Release -------------
Removing intermediate files ... Done!
----------- Project STM8_Template - STM8 Cosmic - Configuration Release -------------
Compiling ........\library\base\src\gpio.c...
cxstm8 +mods0 -pp -i.. -i........\library\base\inc -i"C:\Program Files\COSMIC\CXSTM8_EVAL\Hstm8" -clRelease\ -coRelease\ ........\library\base\src\gpio.c
........\library\base\src\gpio.c:
Compiling ........\library\base\src\stm8_interrupt_vector.c...
cxstm8 +mods0 -pp -i.. -i........\library\base\inc -i"C:\Program Files\COSMIC\CXSTM8_EVAL\Hstm8" -clRelease\ -coRelease\ ........\library\base\src\stm8_interrupt_vector.c
........\library\base\src\stm8_interrupt_vector.c:
Compiling ..\main.c...
cxstm8 +mods0 -pp -i.. -i........\library\base\inc -i"C:\Program Files\COSMIC\CXSTM8_EVAL\Hstm8" -clRelease\ -coRelease\ ..\main.c
..\main.c:
Running Linker
clnk -l"C:\Program Files\COSMIC\CXSTM8_EVAL\Lib" -o Release\STM8_Template.sm8 Release\STM8_Template.lkf
cvdwarf Release\STM8_Template.sm8
Running Post-Build step
chex -o Release\STM8_Template.s19 Release\STM8_Template.sm8
STM8_Template.elf - 0 error(s), 0 warning(s)
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/gicking/STM8_templates/issues/1#issuecomment-354184588 , or mute the thread https://github.com/notifications/unsubscribe-auth/AAlOm-4vKA9-VtdU6BPSM5ESozA1gkIRks5tEr9IgaJpZM4RLKoc .  https://github.com/notifications/beacon/AAlOmx77DFYDZK8S1PY--c-qvxMfna6Sks5tEr9IgaJpZM4RLKoc.gif

hi RandoSY,
just found this hint -> seems like I don't need the hashtag after all :-)   Can you please try with the updated stm8as.h on Github? Let me know if it works now. Good luck!
Regards, Georg

Hi Georg!,
Remarkably I found this link as well, and wondered if it wouldn’t apply.
In answer to your question, works like CHAMP!  Now I can get to understanding mapping the Arduino application architecture to this new chip.
I’ve taken a hard look at debugging Arduino applications inside Atmel Studio,  this works but it’s quite difficult to follow all the twists and turns, and esp. the obsufications of the C++ based libraries, this should be much simpler.

Kudos!
Here’s  point is the STM8 is so cheap relatively, it’s almost disposable.   Perfect for education, lot’s to learn, and much appreciated.
Best,
Randy
From: Georg Icking-Konert<EMAIL_ADDRESS>Sent: Thursday, December 28, 2017 3:04 AM
To: gicking/STM8_templates
Cc: RandoSY; Author
Subject: Re: [gicking/STM8_templates] Compilation error Cosmic (#1)
hi RandoSY,
just found this hint https://stackoverflow.com/questions/3030099/pragma-in-define-macro  -> seems like I don't need the hashtag after all :-) Can you please try with the updated stm8as.h on Github? Let me know if it works now. Good luck!
Regards, Georg
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/gicking/STM8_templates/issues/1#issuecomment-354247303 , or mute the thread https://github.com/notifications/unsubscribe-auth/AAlOm-Vt4PrS1vBZNqbJd7-8njj00Lrpks5tE0t2gaJpZM4RLKoc .  https://github.com/notifications/beacon/AAlOm7dAbHDWx-sMRPGBGtEG7vjqDcaVks5tE0t2gaJpZM4RLKoc.gif

hi again,
great to hear this works now!
The way you write it, it sounds like I the progression from simple to complex projects was made on purpose... However, I have to tell you that I'm just learning as I go along - and as a side-effect I get some (hopefully) useful code. I guess that's also why I had some major changes already, i.e. when I stumbled into a issue that required some structural change. Hope it won't happen too often, since it's rather dull work.
I totally agree that C++ support would be really nice indeed. Apparently there is a way to achieve this with SDCC via LLVM, but it seems to be rather incomplete.
One more thing: if you are looking for true Arduino look&feel with STM8, you may also want to take a look at the sduino project. I see my project rather as a means to learn low(er)-level embedded programming - and to provide useful code for myself...
Thanks again for pointing this issue out and have a happy New Year! :-)
Georg

Georg,
Working through your other Github folders, of course, what I’ve just learned applies across your other examples.
If you wouldn’t mind I have  a question. Whatever attracted you to the STM8 processor in the first place?
It seems you’ve created a lot of the open source infrastructure that allows the STM8 to be used by the maker community.
Best,
Randy
From: Georg Icking-Konert<EMAIL_ADDRESS>Sent: Thursday, December 28, 2017 11:14 AM
To: gicking/STM8_templates
Cc: RandoSY; Author
Subject: Re: [gicking/STM8_templates] Compilation error Cosmic (#1)
hi again,
great to hear this works now!
The way you write it, it sounds like I the progression from simple to complex projects was made on purpose... However, I have to tell you that I'm just learning as I go along - and as a side-effect I get some (hopefully) useful code. I guess that's also why I had some major changes already, i.e. when I stumbled into a issue that required some structural change. Hope it won't happen too often, since it's rather dull work.
I totally agree that C++ support would be really nice indeed. Apparently there is a way to achieve this https://sourceforge.net/p/sdcc/mailman/message/36061704/  with SDCC via LLVM, but it seems to be rather incomplete.
One more thing: if you are looking for true Arduino look&feel with STM8, you may also want to take a look at the sduino project https://github.com/tenbaht/sduino . I see my project rather as a means to learn low(er)-level embedded programming - and to provide useful code for myself...
Thanks again for pointing this issue out and have a happy New Year! :-)
Georg
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/gicking/STM8_templates/issues/1#issuecomment-354313133 , or mute the thread https://github.com/notifications/unsubscribe-auth/AAlOm2whHBTgw1gErtXA1x8PzFEPADw6ks5tE75lgaJpZM4RLKoc .  https://github.com/notifications/beacon/AAlOmw8CDBNKgf1Z_D1FusXdGV-Ff73Aks5tE75lgaJpZM4RLKoc.gif

hi Randy,
not at all! I am working for an automotive supplier, and we've been using the STM8 for quite a while as our bred & butter controller. Reason is - as you correctly pointed out - an unparalleled features vs. price ratio.
So when I developed an IC validation tool ("muBoard") years ago, of course it was based on STM8. Unfortunately I overlooked the fact that no OS C-compiler was available (and Cosmic didn't offer eval versions). So use of the muBoard was restricted to the lucky holders of a Dongle :-(  The "solution" for this was a new language with corresponding byte code interpreter. But while that was a very interesting endeavor, and works surprisingly well, it also has some major drawbacks. So, when SDCC became available, I started porting (and extending) the muBoard SW-functions to SDCC, and tried to make the toolchain as easy to use as muBatch. The result is what you see on Github.
Why stick to STM8 and not switch to Arduino? Well, recently a colleague developed a muBoard successor, which is Arduino based -> in principle I could (and probably should) stop putting effort into STM8. However, while I use Arduinos a lot nowadays and appreciate the great community support, the universe of available libraries, and the ease of use, I sometimes miss the dirty, low-level programming. Besides, I still prefer the STM8 for my more serious projects, because it's very lean on HW, extremely robust, and has very good power-saving features (especially STM8L). Especially the latter makes it perfect for battery powered devices - and is the reason for the Minimal-C projects without any overhead.
So what's your reason to look into STM8, if I may ask?
Regards, Georg
PS: how about switching to PM<EMAIL_ADDRESS>instead of using the issues page on Github?

Hi Georg!
I did respond and forward to the<EMAIL_ADDRESS>email address you posted.
Best,
Randy
From: Georg Icking-Konert<EMAIL_ADDRESS>Sent: Friday, December 29, 2017 2:40 AM
To: gicking/STM8_templates
Cc: RandoSY; Author
Subject: Re: [gicking/STM8_templates] Compilation error Cosmic (#1)
hi Randy,
not at all! I am working for an automotive supplier, and we've been using the STM8 for quite a while as our bred & butter controller. Reason is - as you correctly pointed out - an unparalleled features vs. price ratio.
So when I developed an IC validation tool ("muBoard") years ago, of course it was based on STM8. Unfortunately I overlooked the fact that no OS C-compiler was available (and Cosmic didn't offer eval versions). So use of the muBoard was restricted to the lucky holders of a Dongle :-( The "solution" for this was a new language with corresponding byte code interpreter https://frosch.piandmore.de/en/pam9/call/public-media/event_media/160611_Vortrag_Interpreter.pdf . But while that was a very interesting endeavor, and works surprisingly well, it also has some major drawbacks. So, when SDCC became available, I started porting (and extending) the muBoard SW-functions to SDCC, and tried to make the toolchain as easy to use as muBatch. The result is what you see on Github.
Why stick to STM8 and not switch to Arduino? Well, recently a colleague developed a muBoard successor, which is Arduino based -> in principle I could (and probably should) stop putting effort into STM8. However, while I use Arduinos a lot nowadays and appreciate the great community support, the universe of available libraries, and the ease of use, I sometimes miss the dirty, low-level programming. Besides, I still prefer the STM8 for my more serious projects, because it's very lean on HW, extremely robust, and has very good power-saving features (especially STM8L). Especially the latter makes it perfect for battery powered devices - and is the reason for the Minimal-C projects without any overhead.
So what's your reason to look into STM8, if I may ask?
Regards, Georg
PS: how about switching to PM<EMAIL_ADDRESS>instead of using the issues page on Github?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/gicking/STM8_templates/issues/1#issuecomment-354410546 , or mute the thread https://github.com/notifications/unsubscribe-auth/AAlOm2jTK-WTVvIuQmWNyLxsj2xaf4Fvks5tFJc5gaJpZM4RLKoc .  https://github.com/notifications/beacon/AAlOm5LAOpiEjGKJdIb80KDNNhJ-jlSuks5tFJc5gaJpZM4RLKoc.gif


Open browser #31 issue

#31 issus에 대한 pull request

add more functions and re-pull request on this issue


eMRTD EF.COM gives corrupted stream - high tag number < 31 found

Hi,
I found a problem with the ASN1InputStream. It fails to read an EF.COM data group from an eMRTD (defined in https://www.icao.int/publications/Documents/9303_p10_cons_en.pdf, section 4.6.1).
This structure uses a tag ‘5F01’. This is a high tag number, however with a tag number 1, which obviously is < 31. There has been a commit to ASN1InputStream on 16th of June 2021 that throws the exception stated above.
I'm no expert on ASN.1 specs, but it appears that ICAO didn't find anything in the specs that prevented them from requiring the tag '5F01' on billions of passports. I think this Exception should not be thrown.

Probably there is some confusion here between the identifier octets encoding and the tag number itself.
Firstly, AFAIK it is correct that in BER low tag numbers cannot use the high tag number format. See X.690 8.1.2, in particular:

<IP_ADDRESS> For tags with a number ranging from zero to 30 (inclusive), the identifier octets shall comprise a single octet [...].

However that's probably not what is happening. I don't know anything about this specification (or how BER-TLV stuff is generally presented), but a straightforward reading would suggest that '5F01' is actually the tag number (24321 in decimal) and not the identifier octets themselves. e.g. if the ASN.1 field has type:
[24321] IMPLICIT PrintableString
then an example encoding would be:
9F81BE01 04 31353939
which is shown split into the identifier octets, length octet(s), and contents octets. 9F marks it as a CONTEXT class high tag number. The actual tag number (0x5F01) encoding then follows: 81BE01 (encoded per X.690 <IP_ADDRESS>).
I guess another possibility is that you are trying to read a Simple-TLV encoding as if it were BER-TLV or something like that. There is some possibly useful discussion in this stackoverflow answer: https://stackoverflow.com/a/18932655/264294 . In particular there is an example of an invalid BER-TLV encoding (5F0F 05 48656C6C6F) similar to the one involved here.

Thanks for your quick response!
I think the ICAO spec made a mistake in their choice of high tags. They describe the EF.COM structure to use the following tags:
'60' (= app. spec & constructed)
'5F01' (= app. spec & high tag number)
'5F36' (idem)
'5C' (= app. spec)
This is an actual EF.COM read from a passport:
60 17
5f 01   04    30 31 30 37
5f 36   06    30 34 30 30 30 30
5c        05    61 75 6f 63 6e
Apart from the illegal tag 5F01 this looks BER-TLVish to me.

The more I read, the stranger it looks, yes. Bearing in mind I don't have access to the complete ISO standards, it appears ISO/IEC 7816 references ISO/IEC 8825 to define BER (and this latter spec. looks like it simply copies X.690; there could be minor differences, but at least I have found it says the same thing about tag number encoding as X.690).
However, based on this: https://cardwerk.com/iso7816-4-annex-d-use-of-basic-encoding-rules-asn-1/ , the description in Annex D.2 gives a different description for tag numbers where it talks about (paraphrasing) "this encoding means that tag" instead of the X.690 text "if the tag is this, it shall be encoded thus", and in particular that D.2 text would not restrict using a "high tag" of 1.
That is not the only difference from actual BER though. Note in the D.3 section where it says that the "indefinite length" form is not to be used in BER-TLV, and worse the D.1 section where it says "Before, between or after BER-TLV data objects, ’00’ or ‘FF’ bytes without any meaning may occur".
Taken together my impression is you are going to need a specifically BER-TLV parser to handle this.

Yes, it seems ISO 7816 BER-TLV does a few things different (I'll BER this in mind :-). This is quite confusing because ICAO also uses proper ASN.1 definitions in other places of the specs (e.g. the stuff supported in package org.bouncycastle.asn1.icao.*).
Thanks for your help. I will follow your advice and find a BER-TLV parser to read this object.

Hi,
So reading up a bit on X.690, I think <IP_ADDRESS>.2 bullet c only requires the first subsequent tag octet of a high tag number to be > 0, but not > 30. I think where the surrounding text refers to 'tag numbers greater than 30', it conceptually also includes the 5 bits (11111) from the first octet into the tag number.
So, I still need to use a different parser for the ISO 7816 BER stuff (for the superfluous 00's and FF's), but I think the Exception 'high tag number < 31' is based on an (imho) incorrect interpretation of X.690.
(excerpt from https://www.itu.int/ITU-T/studygroups/com17/languages/X.690-0207.pdf)
<IP_ADDRESS>.2
The subsequent octets shall encode the number of the tag as follows:
a) bit 8 of each octet shall be set to one unless it is the last octet of the identifier octets;
ISO/IEC 8825-1:2003 (E)
ITU-T Rec. X.690 (07/2002) 5
b) bits 7 to 1 of the first subsequent octet, followed by bits 7 to 1 of the second subsequent octet, followed in
turn by bits 7 to 1 of each further octet, up to and including the last subsequent octet in the identifier
octets shall be the encoding of an unsigned binary integer equal to the tag number, with bit 7 of the first
subsequent octet as the most significant bit;
c) bits 7 to 1 of the first subsequent octet shall not all be zero.

So reading up a bit on X.690, I think <IP_ADDRESS>.2 bullet c only requires the first subsequent tag octet of a high tag number to be > 0, but not > 30. I think where the surrounding text refers to 'tag numbers greater than 30', it conceptually also includes the 5 bits (11111) from the first octet into the tag number.

No, the 1F is definitely just a marker. The > 0 for the first subsequent octet (<IP_ADDRESS>.2c) is to ensure a minimal length encoding for the high tag number. <IP_ADDRESS>.2b is clear that the tag number is made up of only (the lower bits of) the subsequent octets. The minimal length aspect also reinforces that high tag number shouldn't be used when the tag would fit into the first identifier octet (i.e < 31).
Our parser returns these tag numbers to code that matches them with tag numbers declared in proper ASN.1 types. It's very clear-cut. Of course it's quite possible that whoever chose the tag '5F01' also thought that the tag number "conceptually also includes the 5 bits", but it does not (in X.690, which we are following).
IMO it is the text of Annex D.2 in ISO7816-4 (which I mentioned in my previous comment) that distorted the original X.690 rule, and I am unsure if that was an intentional weakening or whether the restriction got lost in translation somehow.

I agree 1F is just a marker and not part of the tag. What I intended to say is that I think that 'tag numbers greater than 30' is just informal language, because it is necessary to recruit extra octets if a tag greater than 30 is required (which imho does not imply the inverse).

The > 0 for the first subsequent octet (<IP_ADDRESS>.2c) is to ensure a minimal length encoding for the high tag number.

I'm not sure if I agree on this one or understand you correctly. It is only the lower 7 bits of the first subsequent octet that must be > 0. The rule does not influence the value of bit 8 which acts as the marker for the last subsequent octet.

<IP_ADDRESS> For tags with a number ranging from zero to 30 (inclusive), the identifier octets shall comprise a single octet [...].

"shall" means MUST (not trying to shout at you, but it has to be read as though the spec is shouting at you). It's not optional.

I'm not sure if I agree on this one or understand you correctly. It is only the lower 7 bits of the first subsequent octet that must be > 0. The rule does not influence the value of bit 8 which acts as the marker for the last subsequent octet.

Yes I wrote lazily, so I will restate it: bits 7 to 1 of the first subsequent octet shall not all be zero. is there to prevent arbitrarily long representations of a given tag i.e. informally: you have to use the minimal length encoding for high tags, and indeed per the other rules, for all tags (because for tags <= 30 the identifier octets shall comprise a single octet).

Thanks for the clarification, fwiw, based on your arguments I agree with your conclusion. I appreciate <IP_ADDRESS>.2 c is there to prevent the construction of tags with "leading zero's".

ISO/IEC 7816-4:2005 seems to contradict itself in <IP_ADDRESS> BER-TLV tag fields:
On p. 13 (bottom):

Otherwise (bits 5 to 1 all set to 1), the tag field continues on one or more subsequent bytes.

Bit 8 of each subsequent byte shall be set to 1, unless it is the last subsequent byte.
Bits 7 to 1 of the first subsequent byte shall not be all set to 0.
Bits 7 to 1 of the first subsequent byte, followed by bits 7 to 1 of each further subsequent byte, up to and
including bits 7 to 1 of the last subsequent byte encode a tag number.

On p. 14 (just before <IP_ADDRESS>)

In tag fields of two or more bytes, the values '00' to '1E' and '80' are invalid for the second byte.

In two-byte tag fields, the second byte consists of bit 8 set to 0 and bits 7 to 1 encoding a number greater
than thirty. The second byte is valued from '1F' to '7F; the tag number is from 31 to 127.
In three-byte tag fields, the second byte consists of bit 8 set to 1 and bits 7 to 1 not all set to 0; the third
byte consists of bit 8 set to 0 and bits 7 to 1 with any value. The second byte is valued from '81' to 'FF'
and the third byte from '00' to '7F'; the tag number is from 128 to 16 383.

AFAICT this stems from the fact that bits 7 to 1 can be set to values between '01' to '1E' when bit 8 is set to 1 (i.e. '81' through '9E' for first subsequent byte).
Therefore it appears that ICAO is indeed incorrectly using the encoding for EF.COM objects, at least according to ISO 7816-4.


electron-packager  packaging    on windows.

When packaging .exe in windows error:The version parameter is deprecated, use electronVersion (or --electron-version in the CLI) instead
how to solve?

See #586  Please do not raise duplicate issues

@MarshallOfSound Didn't find a solution.


MvpNFT

A simple and easily human readable contract for a 1/1 NFT that stores a string at the top of the contract.

don't we want this to be a branch?


[BUG] Segfaults on "select count(*) from test" with tables on top of cuDF DataFrames

test.py:
if __name__ == "__main__":
    from dask.distributed import Client
    from dask_cuda import LocalCUDACluster

    cluster = LocalCUDACluster(protocol="tcp")
    client = Client(cluster)
    print(client)

    from dask_sql import Context
    import cudf

    c = Context()

    test_df = cudf.DataFrame({'id': [0, 1, 2]})
    c.create_table("test", test_df)

    # segfault
    print(c.sql("select count(*) from test").compute())

EDIT: Leaving the below UCX snippet and trace for historical purposes, but the issue seems entirely unrelated to UCX.
from dask.distributed import Client
from dask_cuda import LocalCUDACluster
from dask_sql import Context
import pandas as pd

cluster = LocalCUDACluster(protocol="ucx")
client = Client(cluster)

c = Context()

test_df = pd.DataFrame({'id': [0, 1, 2]})
c.create_table("test", test_df)

# segfault
c.sql("select count(*) from test")

trace:
/home/rgelhausen/conda/envs/dsql-3-07/lib/python3.9/site-packages/distributed-2022.2.1+8.g39c5e885-py3.9.egg/distributed/comm/ucx.py:83: UserWarning: A CUDA context for device 0 already exists on process ID 1251168. This is often the result of a CUDA-enabled library calling a CUDA runtime function before Dask-CUDA can spawn worker processes. Please make sure any such function calls don't happen at import time or in the global scope of a program.
  warnings.warn(
distributed.preloading - INFO - Import preload module: dask_cuda.initialize
...
[rl-dgx2-r13-u7-rapids-dgx201:1232380:0:1232380] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x8)
==== backtrace (tid:1232380) ====
 0  /home/rgelhausen/conda/envs/dsql-3-07/lib/python3.9/site-packages/ucp/_libs/../../../../libucs.so.0(ucs_handle_error+0x155) [0x7f921c5883f5]
 1  /home/rgelhausen/conda/envs/dsql-3-07/lib/python3.9/site-packages/ucp/_libs/../../../../libucs.so.0(+0x2d791) [0x7f921c588791]
 2  /home/rgelhausen/conda/envs/dsql-3-07/lib/python3.9/site-packages/ucp/_libs/../../../../libucs.so.0(+0x2d962) [0x7f921c588962]
 3  /lib/x86_64-linux-gnu/libc.so.6(+0x430c0) [0x7f976d27b0c0]
 4  [0x7f93a78e6b58]
=================================
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007f93a78e6b58, pid=1232380, tid=1232380
#
# JRE version: OpenJDK Runtime Environment (11.0.1+13) (build 11.0.1+13-LTS)
# Java VM: OpenJDK 64-Bit Server VM (11.0.1+13-LTS, mixed mode, tiered, compressed oops, g1 gc, linux-amd64)
# Problematic frame:
# J 1791 c2 java.util.Arrays.hashCode([Ljava/lang/Object;)I<EMAIL_ADDRESS>(56 bytes) @ 0x00007f93a78e6b58 [0x00007f93a78e6b20+0x0000000000000038]
#
# Core dump will be written. Default location: Core dumps may be processed with "/usr/share/apport/apport %p %s %c %d %P %E" (or dumping to /home/nfs/rgelhausen/notebooks/core.1232380)
#
# An error report file with more information is saved as:
# /home/nfs/rgelhausen/notebooks/hs_err_pid1232380.log
Compiled method (c2)   17616 1791       4       java.util.Arrays::hashCode (56 bytes)
 total in heap  [0x00007f93a78e6990,0x00007f93a78e6d80] = 1008
 relocation     [0x00007f93a78e6b08,0x00007f93a78e6b20] = 24
 main code      [0x00007f93a78e6b20,0x00007f93a78e6c60] = 320
 stub code      [0x00007f93a78e6c60,0x00007f93a78e6c78] = 24
 metadata       [0x00007f93a78e6c78,0x00007f93a78e6c80] = 8
 scopes data    [0x00007f93a78e6c80,0x00007f93a78e6ce8] = 104
 scopes pcs     [0x00007f93a78e6ce8,0x00007f93a78e6d48] = 96
 dependencies   [0x00007f93a78e6d48,0x00007f93a78e6d50] = 8
 handler table  [0x00007f93a78e6d50,0x00007f93a78e6d68] = 24
 nul chk table  [0x00007f93a78e6d68,0x00007f93a78e6d80] = 24
Could not load hsdis-amd64.so; library not loadable; PrintAssembly is disabled
#
# If you would like to submit a bug report, please visit:

cc @jdye64, @pentschev

Conda environment to reproduce
conda create -n dsql -c rapidsai-nightly -c nvidia -c conda-forge cudatoolkit=11.5 cudf=22.04 dask-cuda dask-cudf dask/label/dev::dask-sql dask-ml ipykernel pyngrok plotly s3fs requests nbformat cuml

Seems like the same issue from https://github.com/dask-contrib/dask-sql/issues/297 which was fixed by https://github.com/dask-contrib/dask-sql/pull/294 , any chance that fix has been reverted?

@pentschev I went back and checked the offending files from issue #297 and they have not been accidentally reverted but that was a good thing to check.

The point here seems nevertheless to be that JVM produces segmentation faults as part of its garbage collection which are posteriorly caught by the UCX error handling. As per my https://github.com/dask-contrib/dask-sql/issues/297#issuecomment-962263794 , I don't see any way around it but to disable UCX error handling, and as noted in that comment we may lose the ability to debug UCX more easily, but we can't really do anything else from outside of JVM's signal handler as the behavior of raising segmentation faults is inherent to the way garbage collection is done by JVM.

fyi, the second snippet above fails regardless of UCX. Updating the issue title to reflect.
Also, I notice that the only conda environments in which these don't fail are ones in which I'm building dask-sql, dask, and distributed from source. I'm going to keep peeling back env customizations to further narrow.

I was able to triage this to environments with cuml resolving to older dask versions. This issue only comes up when we use the newer nightlies of cudf/dask-cudf with an older version of dask (2021.11.2).
Here is a minimal reproducer outside dask-sql:
import cudf
import dask_cudf
df = cudf.DataFrame({"id":[1,2,3]})
ddf = df
ddf = dask_cudf.from_cudf(df,1)
ddf["tmp"] = 1
ddf.groupby(["id"]).agg({"tmp":"count"}).compute()

We should be good to close on this repo.
Side note: The reproducer outside dask-sql shows the actual error (python recursion depth exceeded), but with the jvm spun up it gets captured as a segfault with no additional information

Thank you for digging into this, Ayush. This explains my working environments (which were removing older versions of Dask, then building/installing from Dask & Distributed main).


Add docker support

This code was derived from sdegler's pull request #91.

LGTM

Will make another PR to address cosmetic concerns


provide Keydata.get_idx_version

Hi.  I really like this crate.  Thanks.
I had one small issue that I think would be best addressed by adding a new getter on KeyData.  My MR leaks the abstraction a bit more (about as much as the Debug impl does) but in my situation it would be quite useful.  To motivate, I will explain:
I'm writing a game system. Players in a game may come and go but there will be few at any one time.  I'm already using SlotMap for my record of the players.
I want to have something like a "player number" visible to the users: there's a player nickname but in one circumstance (a graphical representation) I really want something much terser.  A small integer would be ideal.  (Player 1 etc.)
Slotmap has that internally of course.  But right now it's hidden.  My options are 1. make a new data structure very similar to SlotMap 2. abuse KeyData.as_ffi 3. Add a method to get the slot index.
It seems to me that 3 is the right answer so here is my MR.  I hope you like it.  I wasn't quite sure what name and precise API you would like but this one seems OK to me.  Returning the version as well is potentially a tiny bit wasteful and/or irrelevant, but it does make the doc exposition a lot easier.
I have noticed that DenseSlotMap doesn't seem to use index 0.  I didn't add a guarantee that the index would be nonzero for a real entry.  Should I add that to the doc?

I'm sorry, I really, explicitly do not want to do this.

That's a shame.
I found a way to do what I needed using only things which seem to me to be part of slotmap's stable API - specifically, the serde format.  This saves me vendoring slotmap.  I'm pasting the code here in case you would like to suggest a better way, and of course in case anyone else would like this.  Astonishingly the magic of serde and rust means this monstrosity is only 10-15% slower than the method I provided in my MR.
//! Provides a [`get_idx_key`](trait.KeyDataExt.html#tymethod.get_idx_version) method on
//! `slotmap::KeyData`.  See [KeyDataExt::get_idx_version].

/// Extension trait for `slotmap::KeyData`, providing `get_idx_version`.
///
/// No-one is expected to implement this trait for anything else.
pub trait KeyDataExt {
  /// Returns the slot index and version.  This is useful only in
  /// unusual situations.
  ///
  /// At any one time, a slotmap has at most one entry with each
  /// index.  The combination of index and version are unique across
  /// time.  Indices are generally not significantly bigger than thw
  /// maximum ever occupancy.  No other guarantees are made.
  ///
  /// For serialisation, use `serde` or `as_ffi`.
  ///
  /// ### panics: ###
  ///
  /// This function panics if the `slotmap::KeyData` `serde::ser::Serialize`
  /// representation has changed too much.  This ought to be caught
  /// by the tests, and would probably be a breaking change in the
  /// underlying `slotmap` crate in any case.
  ///
  /// If you prefer to receive an error rather than panicing,
  /// see [keydata_extract].
  fn get_idx_version(self) -> (u32, u32);
}

impl KeyDataExt for slotmap::KeyData {
  fn get_idx_version(self) -> (u32, u32) {
    keydata_extract(self).expect(
      "slotmap KeyData Serialize representation changed!"
    )
  }
}

/// Underlying extraction function.  Fails rather than panicing.
///
/// Fails if the `slotmap::KeyData` `serde::ser::Serialize`
/// representation has changed too much.  Should not be able to fail
/// otherwise.
pub fn keydata_extract(key : slotmap::KeyData) -> Result<(u32, u32), Error> {
  let mut m : MainExtractor = std::default::Default::default();
  key.serialize(&mut m)?;
  Ok(( m.idx    .ok_or(error(line!()))?,
       m.version.ok_or(error(line!()))? ))
}

#[derive(Debug)]
/// Problem with the `slotmap::KeyData` `Serialize` implementation.
///
/// Not really helpful.  `Unexpected` gives the source line number
/// in `slotmap-slot-idx.rs`.  `WasCustomSerialize` threw the
/// actual error away.
pub enum Error {
  WasCustomSerialize,
  Unexpected(std::num::NonZeroU32),
}
impl std::error::Error for Error { }

//---------- implementation.  avert your eyes ----------

use serde::ser::{self,*};
use std::line;
use std::convert::TryFrom;
use std::fmt;

#[derive(Default)]
struct MainExtractor {
  idx: Option<u32>,
  version: Option<u32>,
}

struct ValueExtractor;

type R<Return> = Result<Return,Error>;
type ROk = R<()>;
use self::Error::*;

fn error(line: u32) -> Error { Unexpected(TryFrom::try_from(line).unwrap()) }
fn u<T>(line: u32) -> R<T> { Err(error(line)) }

type Imp = Impossible<(),Error>;
type RI = R<Imp>;

impl Serializer for &mut MainExtractor {
  type Ok = ();
  type Error = Error;

  type SerializeStruct = Self;

  type SerializeMap           = Imp;
  type SerializeSeq           = Imp;
  type SerializeTuple         = Imp;
  type SerializeTupleStruct   = Imp;
  type SerializeTupleVariant  = Imp;
  type SerializeStructVariant = Imp;

  fn serialize_struct(self, _:&str, _: usize) -> R<Self> { Ok(self) }

  fn serialize_bool (self, _: bool )  -> ROk { u(line!()) }
  fn serialize_i8   (self, _: i8   )  -> ROk { u(line!()) }
  fn serialize_i16  (self, _: i16  )  -> ROk { u(line!()) }
  fn serialize_i32  (self, _: i32  )  -> ROk { u(line!()) }
  fn serialize_i64  (self, _: i64  )  -> ROk { u(line!()) }
  fn serialize_u8   (self, _: u8   )  -> ROk { u(line!()) }
  fn serialize_u16  (self, _: u16  )  -> ROk { u(line!()) }
  fn serialize_u32  (self, _: u32  )  -> ROk { u(line!()) }
  fn serialize_u64  (self, _: u64  )  -> ROk { u(line!()) }
  fn serialize_f32  (self, _: f32  )  -> ROk { u(line!()) }
  fn serialize_f64  (self, _: f64  )  -> ROk { u(line!()) }
  fn serialize_char (self, _: char )  -> ROk { u(line!()) }
  fn serialize_str  (self, _: &str )  -> ROk { u(line!()) }
  fn serialize_bytes(self, _: &[u8 ]) -> ROk { u(line!()) }
  fn serialize_none (self)            -> ROk { u(line!()) }
  fn serialize_unit (self)            -> ROk { u(line!()) }

  fn serialize_some<T>(self,                                   _: &T) -> ROk
  where T : Serialize + ?Sized  { u(line!()) }
  fn serialize_newtype_struct <T>(self, _:&str,                _: &T) -> ROk
  where T : Serialize + ?Sized  { u(line!()) }
  fn serialize_newtype_variant<T>(self, _:&str, _:u32, _:&str, _: &T) -> ROk
  where T : Serialize + ?Sized  { u(line!()) }

  fn serialize_unit_struct (self,_:&str             ) -> ROk { u(line!()) }
  fn serialize_unit_variant(self,_:&str,_:u32,_:&str) -> ROk { u(line!()) }
  fn serialize_seq         (self,_:Option<usize>    ) -> RI  { u(line!()) }
  fn serialize_tuple       (self,_:usize            ) -> RI  { u(line!()) }
  fn serialize_tuple_struct(self,_:&str, _:usize    ) -> RI  { u(line!()) }
  fn serialize_map         (self,_:Option<usize>    ) -> RI  { u(line!()) }
 fn serialize_tuple_variant (self,_:&str,_:u32,_:&str,_:usize)->RI{u(line!())}
 fn serialize_struct_variant(self,_:&str,_:u32,_:&str,_:usize)->RI{u(line!())}
}

impl SerializeStruct for &mut MainExtractor {
  type Ok = ();
  type Error = Error;

  fn serialize_field<T: Serialize + ?Sized> (
    &mut self, key: &str, value: &T
  ) -> R<()> {
    if key == "idx"     { self.idx     = Some(value.serialize(ValueExtractor)?); }
    if key == "version" { self.version = Some(value.serialize(ValueExtractor)?); }
    Ok(())
  }

  fn end(self) -> ROk { Ok(()) }
}

type V = u32;
type ImpV = Impossible<V,Error>;
type RIV = R<ImpV>;

impl Serializer for ValueExtractor {
  type Ok = V;
  type Error = Error;

  fn serialize_u32(self, value: u32) -> R<V> { Ok(value) }

  type SerializeStruct        = ImpV;
  type SerializeMap           = ImpV;
  type SerializeSeq           = ImpV;
  type SerializeTuple         = ImpV;
  type SerializeTupleStruct   = ImpV;
  type SerializeTupleVariant  = ImpV;
  type SerializeStructVariant = ImpV;

  fn serialize_bool (self, _: bool )  -> R<V> { u(line!()) }
  fn serialize_i8   (self, _: i8   )  -> R<V> { u(line!()) }
  fn serialize_i16  (self, _: i16  )  -> R<V> { u(line!()) }
  fn serialize_i32  (self, _: i32  )  -> R<V> { u(line!()) }
  fn serialize_i64  (self, _: i64  )  -> R<V> { u(line!()) }
  fn serialize_u8   (self, _: u8   )  -> R<V> { u(line!()) }
  fn serialize_u16  (self, _: u16  )  -> R<V> { u(line!()) }
  fn serialize_u64  (self, _: u64  )  -> R<V> { u(line!()) }
  fn serialize_f32  (self, _: f32  )  -> R<V> { u(line!()) }
  fn serialize_f64  (self, _: f64  )  -> R<V> { u(line!()) }
  fn serialize_char (self, _: char )  -> R<V> { u(line!()) }
  fn serialize_str  (self, _: &str )  -> R<V> { u(line!()) }
  fn serialize_bytes(self, _: &[u8 ]) -> R<V> { u(line!()) }
  fn serialize_none (self)            -> R<V> { u(line!()) }
  fn serialize_unit (self)            -> R<V> { u(line!()) }

  fn serialize_some           <T>(self,                        _: &T) -> R<V>
  where T : Serialize + ?Sized  { u(line!()) }
  fn serialize_newtype_struct <T>(self, _:&str,                _: &T) -> R<V>
  where T : Serialize + ?Sized  { u(line!()) }
  fn serialize_newtype_variant<T>(self, _:&str, _:u32, _:&str, _: &T) -> R<V>
  where T : Serialize + ?Sized  { u(line!()) }

  fn serialize_unit_struct   (self,_:&str             ) -> R<V>    { u(line!()) }
  fn serialize_unit_variant  (self,_:&str,_:u32,_:&str) -> R<V>    { u(line!()) }
  fn serialize_seq           (self,_:Option<usize>    ) -> RIV { u(line!()) }
  fn serialize_tuple         (self,_:usize            ) -> RIV { u(line!()) }
  fn serialize_tuple_struct  (self,_:&str, _: usize   ) -> RIV { u(line!()) }
  fn serialize_struct        (self,_:&str, _: usize   ) -> RIV { u(line!()) }
  fn serialize_map           (self,_:Option<usize>    ) -> RIV { u(line!()) }
 fn serialize_tuple_variant (self,_:&str,_:u32,_:&str,_:usize)->RIV{u(line!())}
 fn serialize_struct_variant(self,_:&str,_:u32,_:&str,_:usize)->RIV{u(line!())}
}

impl ser::Error for Error {
  fn custom<T>(_msg: T) -> Self { return Error::WasCustomSerialize; }
}

impl fmt::Display for Error {
  fn fmt(&self, f: &mut fmt::Formatter) -> Result<(), fmt::Error> {
    fmt::Debug::fmt(self,f)
  }
}

#[test]
fn check(){
  fn t(v: u64) {
    let kd = slotmap::KeyData::from_ffi(v);
    let v = kd.as_ffi(); // KeyData is not transparent for all u64
    let (idx,vsn) = self::KeyDataExt::get_idx_version(kd);
    eprintln!("KeyData={:?} v=0x{:x?} idx={}=0x{:x} vsn={}=0x{:x}",
              &kd, &v, idx,idx, vsn,vsn);
    assert_eq!(v, ((vsn as u64) << 32) | (idx as u64));
  }
  t(0x0123456789abcdef);
  t(0xfedcba9876543210);
  t(0);
  t(0xffffffff12345678);
  t(0x12345678ffffffff);
  t(0xffffffff00000000);
  t(0x00000000ffffffff);
  t(0x0000000012345678);
  t(0x1234567800000000);
}


[release-v1.11] Backport kncloudevents package improvements

This will allow an easier backport for the trust-manager integration, see individual commits and references for details.

/assign @creydr

/cherry-pick release-v1.12


Multiple deploy scripts

How do I separate deploy scripts in separate recipes?
In my case I'm using the "common" recipe. In that file there's a mandatory "stage" argument like so:
argument('stage', \Symfony\Component\Console\Input\InputArgument::OPTIONAL, 'Run tasks only on this server or group of servers.');

I'm also using deploy scripts to start and stop Docker containers. In these scripts I'm not interested in the "stage" argument and everything should run on the local environment. But still, my deploy.php needs the common recipe and forces the need of the stage argument.
It would be nice if the stage argument really was optional and not forced as now.
Or perhaps possible to create a file "docker.php" and use dep docker as command. When the "docker" command is missed in "deploy.php" then PHP-Deployer should look for tasks in "docker.php".

You can specify file for dep commend. -f, --file[=FILE]

Ok, to bad :-(

How come the InputArgument::OPTIONAL isn't optional?


question about vocab.make_features

hey @hpzhao, I'm a little confused about how the vocab.make_features method works. I'm trying to write an individual document predict method.
It looks like sents_list combines sentences across the batch of documents and feeds it to the net at once? Is this different from how it previously worked? how does the net know when one document ends and the next begins? I remember seeing this old prepare_data method which made me thing it was going one document at a time:
def prepare_data(doc, word2id):
    data = deepcopy(doc.content)
    max_len = -1
    for sent in data:
        words = sent.strip().split()
        max_len = max(max_len, len(words))
    sent_list = []

    for sent in data:
        words = sent.strip().split()
        sent = [word2id[word] if word in word2id else 1 for word in words]
        sent += [0 for _ in range(max_len - len(sent))]
        sent_list.append(sent)

    sent_array = numpy.array(sent_list)
    label_array = numpy.array(doc.label)

    return sent_array, label_array

Thanks!

I have cal doc_len in vocab.make_features() , so I can infer the start index and end index of each document in a batch. If you wanna predict a doc, just set batch_size=1. Or I'll complete the predict() in main.py @AlJohri


updates to calico/vpp doc

…d for
specifying a pod network CIDR while creating the cluster otherwise the
operator complains about:
kubeadm configuration is missing required podSubnet field

Description
Add a note to the calico/vpp Getting Started doc highlighting the need for
specifying a pod network CIDR while creating the cluster otherwise the
operator complains about:
kubeadm configuration is missing required podSubnet field

Related issues/PRs
Todos

[ ] Documentation

Release Note
TBD

Reminder for the reviewer
Make sure that this PR has the correct labels and milestone set.
Every PR needs one docs-* label.

docs-pr-required: This change requires a change to the documentation that has not been completed yet.
docs-completed: This change has all necessary documentation completed.
docs-not-required: This change has no user-facing impact and requires no docs.

Every PR needs one release-note-* label.

release-note-required: This PR has user-facing changes. Most PRs should have this label.
release-note-not-required: This PR has no user-facing changes.

Other optional labels:

cherry-pick-candidate: This PR should be cherry-picked to an earlier release. For bug fixes only.
needs-operator-pr: This PR is related to install and requires a corresponding change to the operator.

Looks like this one needs a rebase

/sem-approve


BUG: Authenticated navbar unavailable if not associated with any admin/receiver/sender groups

The auth navbar isn't rendered unless a user is part of the three current group types (admin, receiver, sender). This prevents the logout button from being available. The base bar should always be available after login (with logout button), with aspects of it hidden depending on permissions (aka group).
How to recreate

Log in as a user that is not part of any group in Okta
The Auth nav bar does not appear

Acceptance Criteria

[ ] Auth navbar with user display and logout button available to all logged in users
[ ] group-based navigation links/sections available to the user if they are associated

[ ] If user is not part of any group, they should only see email and logout in the Auth Nav Bar

[ ] Create user that is not part of any Okta group for E2E tests
[ ] Update E2E

@chris-kuryak do we need design for this?

Joseph has identified that the fix is a one-line code change.
Bulk of work will be getting with DevOps to check on the change.


ORC-1518: Remove findbugs folders

What changes were proposed in this pull request?
This PR is aimed to remove findbugs folders since findbugs has been removed in ORC-1269.
Why are the changes needed?
Remove redundant files.
How was this patch tested?
N/A

cc @dongjoon-hyun @williamhyun

Also, cc @williamhyun


🩹(frontend) use browser language if no localStorage for detection

Previously, language detection was failing to read browser settings correctly. Added explicit detection order to ensure  localStorage preferences are checked before falling back to browser language.

Related to #211


weight here will lead to zero loss in the dice loss

https://github.com/open-mmlab/mmdetection/blob/db256a14bb7018ad36eed104ea0ce178a0d4050c/mmdet/models/losses/utils.py#L43
In the mmdet.models.roi_heads.mask_heads.fcn_mask_head.py#L173, the weight was initialized to zeros using torch.zeros_like(labels).
It will lead to zero loss when call loss = weight_reduce_loss(loss, weight, reduction, avg_factor). in the mmdet.models.losses.dice_loss.py#L44
I suggest that add
    if weight.sum() == 0:
        weight = torch.ones_like(weight)
    else:
        weight = weight/weight.sum()

before call mmdet.models.losses.dice_loss.py#L44

@Boyden Thank you for your feedback, there is indeed a potential problem. It seems more appropriate to modify fcn_mask_head directly, because dice loss is not responsible for this logic.  Are you interested to create PR?

Directly modifying the fcn_mask_head will lead to inconsistency when calling other loss functions such as CrossEntropyLoss. The more appropriate way I think is to modify the dice_loss code.


Questions on large datasets and best settings and scaling

I switched my code from Python to Julia. (To generate the datasets). I'm starting with a "small" dataset of 1,257,984 items. So the inputX is Matrix{Float32}(10, 1257984) and outputY is Vector{Float32}(1257984). (The actual data is integers, but I cast them to float). outputY is either 0 or 1, so it's a binary problem. Essentially finding an equation that divides two sets of vectors?
http://sirisian.com/randomfiles/symbolicregression/inputX.bin
http://sirisian.com/randomfiles/symbolicregression/outputY.bin
using SymbolicRegression
using SymbolicUtils
using Serialization

inputX = deserialize("inputX.bin")
outputY = deserialize("outputY.bin")
options = SymbolicRegression.Options(
	binary_operators=(+, *, /, -, ^),
	unary_operators=(sqrt,),
	constraints=((^)=>(-1, 3),),
	# target == 0
	# We want the predicted value to be < 0
	# target == 1
	# We want the predicted value to be >= 0
	loss=(t,p) -> t == 0 ? (p < 0 ? 0 : p^2 + 0.0001) : (p >= 0 ? 0 : p^2 + 0.0001),
	maxsize=48,
	maxdepth=8,
	npopulations=20,
	batching=true,
	batchSize=1000,
	progress=true
)
hallOfFame = EquationSearch(inputX, outputY, niterations=1000, options=options, numprocs=0, multithreading=true)
dominating = calculateParetoFrontier(inputX, outputY, hallOfFame, options)
eqn = node_to_symbolic(dominating[end].tree, options)
println(simplify(eqn))
for member in dominating
	size = countNodes(member.tree)
	score = member.score
	string = stringTree(member.tree, options)

	println("$(size)\t$(score)\t$(string)")
end

I have a lot of questions. I don't know what's feasible, so I think it's easier to ask them in one place.

What settings should I be using for best results given my problem. Say I have an i9-12900k, so 24 threads to work with. I'm having problems reasoning about the size of batching or if I need to use other settings to speed things up.

The loss function I have defined does not appear to work. My thinking was to drive the system to return values that are negative when the output is 0 and positive when the output is 1. For some reason when I let the above run it returns things like:
1           0.000e+00  Inf       -0.7851169
So I'm fundamentally not understanding something. Surely such a constant equation like -0.5 would return a large loss? Say the target was 1 for an element being evaluated. Then 1 == 0 ? (-0.5 < 0 ? 0 : (-0.5)^2 + 0.0001) : (-0.5 >= 0 ? 0 : (-0.5)^2 + 0.0001) = 0.2501
I noticed there are loss functions for binary classification problems. I see cross entropy and hinge loss mentioned a lot. I don't see these in the listed loss functions. I noticed in other places the target is expected to be -1 or 1 not 0 or 1. Any tips for using such loss functions with this library?

Is it possible to force the algorithm to include a minimum complexity of say including the 10 variables + 9 operators, so 19. Like the simplest possible equation I would ever expect to work is x1 + x2 + x3 + x4 + x5 + x6 + x7 + x8 + x9 + x10. I can write a cubic function that gets close to the result.

Is it possible to define custom expression subtrees that use variables with lower complexity? Ideally I'd expect it to use varMap? For example say I have varMap=["x0", "y0", "x1", "y1", "x2", "y2", "x3", "y3", "cx", "cy"]. While I'm not sure (x0 - x1) is definitely going to be in my equation the likelihood is much higher than other subtrees. So I might have a list like ["x0 - x1", "x1 - x0", "y0 - y1", "y1 - y1", ...] or however best it would be stored and instead of complexity 3 it's more like complexity 2. In my example this happens because they're control points in a curve where (x0, y0), (x1, y1), (x2, y2), (x3, y3) are sequential and vector subtraction is expected to show up.

My complete dataset is ~20,000,000 trillion elements. Same structure with 10 integers and a 0 or 1 binary result, but at different resolutions. Obviously I can't compute perfect equations or even loop over this within a lifetime. It is a fixed number though that I can randomly sample (aka Monte Carlo) to verify the quality of the result. My question is if it's possible to support a kind of randomized window dataset. So rather than supplying a fixed dataset one would supply a function (or generator) that returns a random record. Let's call this getRandomRecord. Calling this method might be slightly expensive, so you wouldn't want to just call it and throw away the result after using it once.
I assume this would be an extension of the batching system. The system could generate data (probably multithreaded) calling getRandomRecord() populating a few GBs then batching would create a random array view on this data for each of its batches. The algorithm wouldn't be able to ever loop over all of the real data, but it could create another random view and calculate a fuzzy loss. Periodically the whole random dataset would be regenerated.
Even with my 1257984 record example above I don't think it's statistically significant to iterate over the whole dataset at the beginning. It would definitely be useful to quickly sample them. Maybe if the total loss is low then it would sample more data. The closer the total loss (what's the term for this, sum of all losses? error?) is to 0 the more of the whole dataset it would attempt to sample. So starting out it might be fine with using 1K then slowly ramp up to 100K for the view size?

In addition to the above, could this be scaled over hundreds of computers in the cloud? I couldn't get the numprocs to work without it crashing, so I'm not sure how stable it is. The concept would be that each computer generates its own randomized dataset and they only talks to ensure they don't process the same equations. My experience with distributed computing is relegated to C++ MPI stuff years ago, so I'm not familiar with Julia's features yet. I don't know your algorithm, but is it in your scope to support such distributed cloud setup in an easy way?

(Also as a side note I wouldn't be against sponsoring some of these changes for beer money, like 500 dollars. I'm interested in the potential of these kind of projects even if they don't solve my specific problems).

Hi @sirisian, sorry for the delay in responding. Am on travel for the week so won't be able to answer all of these now.
The first thing I want to note is that symbolic regression works for very small datasets. I usually use ~3,000 datapoints max. More datapoints generally won't help find a better solution; it will just slow the search down significantly.
In addition, you should limit the number of features. 10 is pushing it, I would try to find out beforehand what the ~5 most important features are, and only input those. These combinatorial searches scale with O(n!) where n is the number of features, so the fewer features the better.
After training with 5 features, you could try to fit the error with the additional 5 features? You could also check https://github.com/MilesCranmer/symbolic_deep_learning/ for how to do symbolic regression on high-dim datasets, though it's not a robust method and will require some work to adapt to your problem.

With 24 threads available, I would try using 24 procs, or 48 if you use multithreading=true.
So what has printed out shows "Inf" for your loss, so I think it hasn't completed a search yet. If you slim the dataset down to 3000, it should be much easier to get some results out. But you are right that your prediction problem might be closer to a classification problem, so do check out the losses page: https://astroautomata.com/SymbolicRegression.jl/dev/losses/.
Minimum complexity no (complexity is built on iteratively, so would be hard to search when starting from a large equation), but maximum complexity yes. Your current maxsize=48 is a bit high and might hurt the search - maybe start with 30? You could also try the warmupMaxsizeBy option (https://github.com/MilesCranmer/SymbolicRegression.jl/blob/8eceff3807456d7547122b40171747a4c128377b/src/Options.jl#L126), which warms up the maxsize slowly over time up to the given fraction of training time (eg 0.5 would reach max size by 50% of the way through).
This is an interesting area. Your suggestion is what I would start with - feed in the features you think will matter most. The other option is again to try the symbolic deep learning method, where you learn a hierarchy of features via deep learning, then fit each level in the hierarchy with symbolic regression. Again, it's not a robust method, but for very complex equations, it might be the way to go.
This is very interesting. I'm not sure how fast it would be; the equation evaluation part is extremely fast because the data is fixed size, maybe loading or slicing data from a large array would be expensive. Maybe one could do a samples data, and do convergence testing until an accurate loss is found? Certainly one wouldn't need the entire dataset to get a reasonable estimate.
Yes, I've used it over a cluster before. Instead of passing numprocs (which probably crashed due to the dataset size - it duplicates the data for every proc), you could define a set of processors outside of EquationSearch, and then feed them in with procs=myprocs. Depending on what sort of computing you are using, check out https://github.com/JuliaParallel/ClusterManagers.jl for some options. These would let you create a set of processors, which you can feed into EquationSearch. To use this, you will need to declare things with the@everywhere macro, which will let you set up a search over a cluster: see e.g., https://docs.julialang.org/en/v1/manual/distributed-computing/#ClusterManagers.

And thanks for the offer! I can't take honorariums/sponsorship because of my current US visa and unfortunately must turn it down, but I appreciate the thoughtfulness! If anything I'd recommend sponsoring the Julia project, they're awesome (and the better funding they have, the faster Julia will get!).
Cheers,
Miles

With 24 threads available, I would try using 24 procs, or 48 if you use multithreading=true.

When I try to set multithreading=true and settings either procs or numprocs results in:
ERROR: LoadError: AssertionError: procs == nothing && numprocs in [0, nothing]
https://github.com/MilesCranmer/SymbolicRegression.jl/blob/d4458bd8137105c8210fa8a65dfb0bdcf91bd461/src/SymbolicRegression.jl#L173
Doesn't that mean that if multithreading then procs must be nothing and numprocs must be 0 or nothing? How do I set procs to 24 or 48? I have to be reading that wrong or using the wrong approach.
I have a smaller dataset now I'm testing with. It has 6 symbols and 1 output and 39,168 data points.
http://sirisian.com/randomfiles/symbolicregression/inputX2.bin
http://sirisian.com/randomfiles/symbolicregression/outputY2.bin

Hi @sirisian,
Yes, sorry, the error message should be clearer. distributed (procs) and multithreaded modes cannot be used at the same time. The number of threads is determined at runtime of the Julia binary (e.g., julia --threads=24 ... would set up 24 threads), so you cannot set them in calling EquationSearch (I would if I could, but its not possible yet). Further, you cannot pass procs, since that is for distributed (not multithreaded) mode.
40k points is actually still a bit large. Can you try 1k? You should only move to the larger dataset if it starts to overfit to the 1k points (which would presumably be hard to do). 6 symbols is great!
Cheers,
Miles

Okay I've lowered it to 480 points with 334 outputs that are 1 and 146 that are -1.
http://sirisian.com/randomfiles/symbolicregression/inputX3.bin
http://sirisian.com/randomfiles/symbolicregression/outputY3.bin
using SymbolicUtils
using Serialization
using SymbolicRegression

function Program()
	inputX = deserialize("inputX3.bin")
	outputY = deserialize("outputY3.bin")

	#display(view(inputX, :, 1:10))
	#println()
	#display(view(outputY, 1:50))
	#display(typeof(outputY))

	options = SymbolicRegression.Options(
		binary_operators=(+, *, /, -, ^),
		unary_operators=(sqrt,),
		constraints=((^)=>(-1, 3),),
		loss=L1HingeLoss(),
		npopulations=20,
		progress=true
	)
	hallOfFame = EquationSearch(inputX, outputY, niterations=1000, options=options, multithreading=true)
	dominating = calculateParetoFrontier(inputX, outputY, hallOfFame, options)
	eqn = node_to_symbolic(dominating[end].tree, options)
	println(simplify(eqn))
	for member in dominating
		size = countNodes(member.tree)
		score = member.score
		string = stringTree(member.tree, options)

		println("$(size)\t$(score)\t$(string)")
	end
end

println()
println(@allocated Program())

Running with  julia --threads=16 problem.jl
Using ZeroOneLoss() I see basically:
Complexity  Loss       Score     Equation
1           0.000e+00  Inf       0.0

With L1HingeLoss() I see basically:
Complexity  Loss       Score     Equation
1           6.083e-01  3.305e-01  1.0000001

I think I'm fundamentally not understanding how this should be working. With an equation 0.0 and ZeroOneLoss every data point would need to be >= 0. My data is filled with 1 and -1 though so the loss should be exactly 146 with that equation right since that's how many data points are -1?

Hey @sirisian,
Sorry I forgot about this thread. The issue you were seeing should be fixed. There is an internal loss that is normalized by the standard deviation of the y-data - that is what used to be printed out. Now the internal loss is re-scaled before printing, so you shouldn't see this.
Cheers,
Miles


feat: add chrome-ai polyfill when browser not support

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 100.00%. Comparing base (5cff6cc) to head (9e69a65).

Additional details and impacted files
@@            Coverage Diff            @@
##              main       #12   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files            5         5
  Lines          413       414    +1
  Branches        61        61
=========================================
+ Hits           413       414    +1

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Comment field presence on submit suggestion screen

It strikes me as odd to have the comment field appear before the submit button. It seems to imply that the author of the "suggestion" needs to comment before they can submit, which wouldn't make much sense. Earlier today there was an error message that appeared asking for that field to be completed before the suggestion could be submitted (although it was referred to as the "body", even though the body field had been written in). I can't seem to make it happen again.
Feedback system screenshots.pdf

@katebanner - perhaps I'm not understanding you, but from the looks of things you're talking about replying to a suggestion rather than creating one - is that correct? If so, it makes sense to me to have the existing comment section before the form to post an additional comment.
@meshulam - could we perhaps call this section "discussion"?

@jthrilly "Discussion" makes sense to me. And yeah, the "submit" button is for adding a comment, once you're on that page the main post already exists. So I think the error is just when trying to add a comment without typing anything in.

@jthrilly @meshulam - sorry for the delay. To clarify my initial comment at the top:
The workflow seemed strange to me in that if you, a user/author, create a suggestion, it appears as though you are also required to comment on that suggestion you just created based on the layout of the page.
I ran into an error several times where the system would not let me publish my suggestion without also writing a comment to that suggestion. In the second page of the pdf screenshot I included, a message appears stating that "body is required" even when there was already text in the original "body" of the suggestion.

@katebanner There's a "Post Created" flash when landing on the post page, if you still think it's unclear that you're done submitting when landing on the Post page, can you suggest a UI change to address that?
Otherwise let's close this.

I agree regarding the general flow, but @meshulam do you have any input on the apparent bug Kate mentions below?

I ran into an error several times where the system would not let me publish my suggestion without also writing a comment to that suggestion. In the second page of the pdf screenshot I included, a message appears stating that "body is required" even when there was already text in the original "body" of the suggestion.

@katebanner can you describe the steps to reproduce this? It still looks/sounds like you're just clicking "Submit" on the comment form after you've already published your suggestions.

@meshulam I tried to reproduce the bug, but no luck. The pop-up window to submit a new post followed by the "post created" tag appearing at the top of the page seem to address the confusion.

Cool, let me know if you see anything weird. Closing this for now.


x/tools: Language Server Protocol Support

Originally opened by @rudolph9 in https://github.com/cuelang/cue/issues/142
Here is an example of a YAML file LSP https://github.com/redhat-developer/yaml-language-server
I've never created one so I can't say for certian how much work it would be but it would be great for integrations in to vim, emacs, VSCode, etc.
https://langserver.org/

@myitcv Do you know if this is actively being worked on? @TomChv is looking to start a university project on LSP for Cue, with several students ready to code. But to avoid work duplication, this is on hold.

@samalba as discussed, whilst my time on this largely got derailed by the migration in #1078, the gopls team have been working to refactor parts of that code base so that it can be more easily reused as an "LSP in a box". The plan is now to create a CUE instance using the core building blocks, on top of which we can start to flesh out CUE-based implementations of the LSP methods. I suspect that some of those methods will be discrete pieces of work where we would very much welcome community contributions (albeit coordinated via an issue first).

@myitcv Could you please provide some pointers to "LSP in a box" refactoring work mentioned above? Would love to learn some context here.

@myitcv is there any progress comcerning this issue?
I know the golang implementation of the language server and from my point of view it is overcomplicated for this project (i have used the go language server for a language i've created). Only the very basic core and the protocol bindings seems to be useful.
The document symbols and the completion items can be relatively easy parsed with an ast implementation.

We’re approaching the one year anniversary of this issue!

Sorry, I don't have time for developing a language-server-protocol.
But I could support by donating some money to a crowd-funding compaign. Related: https://github.com/cue-lang/cue/discussions/2358


Add: RubyInstallerTeam.Ruby version 3.0.2-1

[ ] Have you signed the Contributor License Agreement?
[ ] Have you checked that there aren't other open pull requests for the same manifest update/change?
[ ] Have you validated your manifest locally with winget validate --manifest <path>?
[ ] Have you tested your manifest locally with winget install --manifest <path>?
[ ] Does your manifest conform to the 1.0 schema?

Note: <path> is the name of the directory containing the manifest you're submitting.

Microsoft Reviewers: Open in CodeFlow

Duplicate of #20910

Duplicate of #20817

It's not. You're adding the DevKit.


ZEPPELIN-3517 Remove zeppelin.pyspark.python in PySparkInterpreter

What is this PR for?
zeppelin.pyspark.python should be removed as it is zeppelin specific property, and only affect the driver, but not on executor. So we should use spark property instead.
What type of PR is it?
[Bug Fix]
Todos

[ ] - Task

What is the Jira issue?

https://issues.apache.org/jira/browse/ZEPPELIN-3517

How should this be tested?

CI pass

Screenshots (if appropriate)
Questions:

Does the licenses files need update? No
Is there breaking changes for older versions? No
Does this needs documentation? No

LGTM

@felixcheung The only place to do mapping is in zeppelin server side, and I it is not to put any interpreter specific login in zeppelin server side. And I think it's better for zeppelin to use the same configuration as spark instead of introducing zeppelin specific configuration.

Thanks @felixcheung . I still think it is better to remove it. As it is difficult to deprecate this and let users to aware of that. Zeppelin is a notebook instead of library. If we don't remove it and just deprecate it in documentation, I believe most of users won't be know that and would continue to use it and may ask question in user mail list why they hit python version mismatch issue between driver and executors.


[WIP] Update runtime programmatically through generators

This is a first step to complete #2193
Added a function that loads a service metadata (the one that Meraki sends to the generator itself) giving in input the runtime root path and the service name.
Refs: #2193

Ci is failing on windows

@leorossi CI is failing https://github.com/platformatic/platformatic/actions/runs/8329464193/job/22791875666?pr=2218#step:6:1500


ExpansionPanel header is cut out when font size is scaled up

On Android, you can configure different font sizes, which are followed by Flutter.
This creates a problem with ExpansionPanel because it has a fixed header height. When  the font size is set to "Huge", the bottom part of the header text is cut out.

/cc @paolosoares

Underlying issue is here: https://github.com/flutter/flutter/issues/12311

@robertoscaramuzzi Could we see what the native app behavior is for this as a reference? I don't think Material docs cover this. This most likely manifests in the appbar.

The native app seems to handle this just fine. The app header is tall enough to contain the largest font size.

We can address this in Flutter.
Unfortunately there's still not guidance from the Material Design spec  about how to change internal component layouts based the text scale factor. Whatever solution we provide now will have to be considered provisional until the general problem is sorted out.
cc @willlarche


Add input files to /jobs response

We would like to update the jobs table to include the name of the input file. This will allow users to know exactly which jobs they might be looking for when looking to see if their file has been processed. To make this more efficient, add the input_files information to the serializer information returned by the /jobs endpoint.
This supports Scale UI https://github.com/ngageoint/scale-ui/issues/414

closed via #1860


integ-tests: disable awsbatch tests in cn regions

There are sporadic failures due to slow networking when building docker images.
Suppressing tests for now only for version 2.6.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Codecov Report

Merging #1680 into release-2.6 will not change coverage.
The diff coverage is n/a.

@@             Coverage Diff              @@
##           release-2.6    #1680   +/-   ##
============================================
  Coverage        51.95%   51.95%
============================================
  Files               24       24
  Lines             3948     3948
============================================
  Hits              2051     2051
  Misses            1897     1897

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5fff8b7...be91c86. Read the comment docs.


Contributing

Hi!
I'm interested in contributing to this project.To give you some of my background I finished my Master's Degree in Math nearly two years ago and have been working with software consultancy since then. My master's thesis was about combinatorial algorithms related to discrete group theory and Riemann surfaces and since then I have gotten interested in Haskell and category theory as well as machine learning. My experience in Haskell and machine learning is not very big, but I'm very eager to learn on the way.
Please get in touch if you think that some sort of cooperation would be interesting.
Best regards,
David

I am thinking about implementing a monoid-friendly decision tree


The wrong http settings may be used if there are two git repo configs for the same url

While investigating issue Error: "More than one git repo URL template matched URL", I spotted another possible defect in the HttpClientConfigurableHttpConnectionFactory implementation.
If someone has multiple git repo configs for the same url, but with different http settings, only one of the http settings will be used, regardless of how the request coming in.
Note: This does not impact my team, but since I spotted it, I thought I would relay this.
If someone has a configuration such as:
spring:
  cloud:
    config:
      server:
        git:
          uri: https://blahblah/v1/repos/vvv_configs-{application}
          proxy:
            https:
              host: ${proxyuri}
              port: '8080'

          repos:

           # No proxy in dev environment
            WebApp-Dev:
              pattern: webapp/dev
              searchPaths: SharedDomains/*, {application}
              basedir: ${basedirParent}\\baseWebApp
              uri: https://blahblah/v1/repos/vvv_configs-webapp

           # Proxy needed for all environments except dev
            WebApp:
              pattern: webapp
              searchPaths: SharedDomains/*, {application}
              basedir: ${basedirParent}\\baseWebApp
              uri: https://blahblah/v1/repos/vvv_configs-webapp
              proxy:
                https:
                  host: ${proxyuri}
                  port: '8080'

The HttpClientConfigurableHttpConnectionFactory.addConfiguration method puts http clients into its internal map with the url as key. Since url is the same, that means that there will only be one entry for the vvv_configs-webapp URL in the map, even though there are two different configs for that URL. Whether the http client will be the one using the proxy or the one without is indeterminate (second one added to the map wins, obviously).
I did a quick test, and confirmed this behavior. Whether I call with "dev" profile or not, it will always use the same http client object. I am using Greenwich SR3.

I spoke too soon about this not impacting us. We have a new requirement where we don't want to use a proxy when client app is running in Cloud. So, we'd like it to pass a "noproxy" profile and have a separate repo definition that doesn't contain proxy info. (I don't see any other way to disable proxy dynamically.)
So, if you just replaced "pattern: webapp/dev" with "pattern: webapp/noproxy" in the config I referenced in the comment above, that is what we basically want to do.
I will attach a project that demonstrates this tomorrow.

I created a sample project for the issue: https://github.com/marnee01/vts-spring-config-server-1482.
Instructions are in the README file at the root of the project.

A workaround or way to avoid this problem that is working for us right now is to switch to using the https.proxyHost/Port system properties, rather than adding proxy settings for each git repo. (And for the Cloud ConfigServer instance, we just won't add those properties.) That turns out to be a better approach for us, anyway.
That might not be feasible for some scenarios, but wanted to mention it for anyone else reading this who is looking for a workaround.

When I apply the fix from #1480 that does not seem to address this issue, there is still only 2 entries in the map.  I don't see how that would address the uniqueness of the key.

If there are two connections that would match an incoming URL, the original code would find that there are two matches, log an error, and not use either one.
What the workaround does is that it chooses the first matching connection that doesn't have a placeholder, if any. If there is one and only one such connection, that is the one used. It ignores any connection containing a placeholder that would match. By contrast, the original code would match against all connections, including those with placeholders, and if there was more than one match, it would log an error and use neither.
The relevant code is the following. The first line looks up matches against connections with no placeholders (since second argument is false):
		Map<String, HttpClientBuilder> builderMap = lookupHttpClientBuilders(url, false);

		// Only if no match, try for URLs with placeholders.
		if (builderMap.isEmpty()) {
			// This is the original logic:
			builderMap = this.lookupHttpClientBuilders(url, true);
		}

I really don't see this as a "fix", just a workaround.

OK got it so since one is selected we instead of 2 being returned that would solve the problem.  Thanks!

Closed via  c7ab0f3

Please re-visit and possibly re-open this. This was marked as fixed, but I don't see how it could be.
Ryan Baxter had said:
When I apply the fix from #1480 that does not seem to address this issue, there is still only 2 entries in the map. I don't see how that would address the uniqueness of the key.
I replied to that, but I actually made a mistake and thought I was replying on the issue documented in #1480. (So, my response starting with "If there are two connections that would..." from Nov 15th is not applicable.) Ryan was in fact correct in saying that the fix from 1480 won't fix this. I am very sorry for that confusion.
In fact, it appears to me that there is an underlying design flaw in this area (which is why I kept insisting the workaround for 1480 was nothing but a workaround). The code in question does not take into account that you can have the same URL but with different properties provided in the spring.cloud.config.server.git properties. The code knows nothing about the spring.cloud.config.server.git configuration.


Add ability to distinguish contextual code from what the user needs to type in a code block

Sometimes, you may need to draw attention to a particular line in a code block and show it with different formatting. This came up with respect to tutorials where you include some lines in a code block for context but the user does not need to type all the code in the block.
Kicking around the idea of something like this to :
    <!-- insert this -->
    <link rel="stylesheet" href="leaflet.routing.valhalla.css"/>
    <!-- end insert this -->

This tutorial has an example: http://maptimeboston.github.io/leaflet-intro/ The code you actually type in for a given step is shown with a gray background.

Going to archive this for now. It's not possible as long as we have the need to make docs readable on GitHub, too. I put a request in to GitHub staff to consider support for code highlights.


How to change the words and language.

Hello there! I would like to make my own wordlist for this, is there any way i can do that? Thanks.

Currently not, as all words are managed by the taboo-data library.

Currently not, as all words are managed by the taboo-data library.

I also cannot edit that in some way?

Not really. I mean it would be possible by building the game on your own after making significant changes, but I cannot guarantee anything will work after that.


Separate all segments?

This is not an error, more a question; is it somehow possible to segment any object in the image + fill the hole and save the segmentations as separate pngs?

Yes, it's possible. You can refer to the "Everything" feature in SAM demo website.

This is not an error, more a question; is it somehow possible to segment any object in the image + fill the hole and save the segmentations as separate pngs?

Yes, it's possible. You can refer to the "Everything" feature on SAM demo website.


nvim-treesitter depencency check to prevent errors

As the modules depend on nvim-treesitter. I think it could be nice to add a little dependency check if it is installed.
It could be as simple as a:
if not pcall(require, "nvim-treesitter") then return end
inside lua/module-template.lua.
If preferred, it could include a little print to inform the end user of the module about the missing dependency:
print "module-template requires nvim-treesitter to be installed"
Next to following a good practice of checking for dependencies, for me this would mostly aim to prevent errors when pulling a config and doing a first setup, when treesitter is not yet fully installed. E.g.,

Would a PR on this have a chance?

Hey, it would definitely have a change, do you want to make it ?

Thanks for the reply @vigoux!
Coming back to this issue I notice that it had some spelling and semantic errors that also needed a fix to make the proposal understandable.
And of course, I'll be back at my desk on Sunday to submit the PR 👍

Awesome then ! Thanks !


Additional command needed in Step 4 to reduce ambiguity

In Step 4, the $ckt variable is defined, but it is not explicitly stated that this same variable should be passed back to the shell to pull the properties of the ExpressRoute circuit. Suggest adding a second line to the Azure PowerShell command box to eliminate any ambiguity.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 16283d77-b9b4-c687-e0dc-d32eb7cc38e8
Version Independent ID: 4ee72023-58e0-6d22-9905-99eedd70fa4a
Content: Azure ExpressRoute: Reset circuit peering
Content Source: articles/expressroute/expressroute-howto-reset-peering.md
Service: expressroute
GitHub Login: @charwen
Microsoft Alias: charwen

@brkrall, Thanks for your feedback! We will investigate and update as appropriate.

@brkrall , Appreciate your patience. The doc has been edited accordingly and it would take some time for the changes to get reflect over the doc.
We are closing this issue for now. If there are further questions regarding this matter, please reply and we will gladly continue the discussion.


Documentation and example for "Using the API" throw exceptions when run

Describe the bug
The sample code contained in the "Using the API" documentation page (https://docs.pyinfra.com/en/2.x/api/index.html) and the api_deploy.py example throw exceptions when run – specifically, line 122, which calls pyinfra.api.facts.get_facts:
https://github.com/Fizzadar/pyinfra/blob/2d4792eba3233c77a298d86e9638434688e3a1c5/examples/api_deploy.py#L122
To Reproduce
1. Create a Vagrantfile:

Vagrantfile contents
Vagrant.configure("2") do |config|
    config.vm.box = "generic/ubuntu2004"
    config.vm.provider "virtualbox" do |vb|
      vb.memory = "500"
    end
end

2. Bring up a Vagrant instance:
$ vagrant up

3. Create motd file needed for the example:
$ mkdir -p files && echo "foo" > files/motd

4. Copy api_deploy.py file into current directory
5. Execute api_deploy.py:
$ python3 api_deploy.py

Expected behavior
The script should run without error.
Actual behavior
A TypeError is thrown:
$ python3 api_deploy.py
Loading Vagrant config...
Connecting...
Host connected: @vagrant/default
Generating operations...
Start operation: 784a97bf1955d5f7a2b9dd6c1e371e17b73c42bc
End operation: 784a97bf1955d5f7a2b9dd6c1e371e17b73c42bc
Start operation: 9c1c01dc3ac1445a500251fc34a15d3e75a849df
End operation: 9c1c01dc3ac1445a500251fc34a15d3e75a849df
Start operation: f4f59e822581d785ba910fbf3f268eca79db8204
End operation: f4f59e822581d785ba910fbf3f268eca79db8204
Start operation:<PHONE_NUMBER>6b52abe1182f5a5a3e12b457ba28b8
End operation:<PHONE_NUMBER>6b52abe1182f5a5a3e12b457ba28b8
Traceback (most recent call last):
  File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run
  File "/mnt/data/dev/vouch-demo/env/lib/python3.8/site-packages/pyinfra/api/facts.py", line 127, in get_fact
    if issubclass(cls, ShortFactBase):
TypeError: issubclass() arg 1 must be a class
2022-07-09T11:00:01Z <Greenlet at 0x7f3d863c8ae0: get_fact(<pyinfra.api.state.State object at 0x7f3d8631b070>, Host(@vagrant/default), 'os')> failed with TypeError

Traceback (most recent call last):
  File "api_deploy.py", line 122, in <module>
    facts = get_facts(state, "os")
  File "/mnt/data/dev/vouch-demo/env/lib/python3.8/site-packages/pyinfra/api/facts.py", line 110, in get_facts
    results[host] = greenlet.get()
  File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get
  File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception
  File "/mnt/data/dev/vouch-demo/env/lib/python3.8/site-packages/gevent/_compat.py", line 65, in reraise
    raise value.with_traceback(tb)
  File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run
  File "/mnt/data/dev/vouch-demo/env/lib/python3.8/site-packages/pyinfra/api/facts.py", line 127, in get_fact
    if issubclass(cls, ShortFactBase):
TypeError: issubclass() arg 1 must be a class

Meta
output of pyinfra --support
--> Support information:

    If you are having issues with pyinfra or wish to make feature requests, please
    check out the GitHub issues at https://github.com/Fizzadar/pyinfra/issues .
    When adding an issue, be sure to include the following:

    System: Linux
      Platform: Linux-5.13.0-48-generic-x86_64-with-glibc2.29
      Release: 5.13.0-48-generic
      Machine: x86_64
    pyinfra: v2.3
    Executable: /mnt/data/dev/vouch-demo//env/bin/pyinfra
    Python: 3.8.10 (CPython, GCC 9.4.0)

How was pyinfra installed (source/pip)?
Using pip install pyinfra.
output with -vv and --debug
Not applicable

This should finally be fixed in https://github.com/Fizzadar/pyinfra/commit/976359cb1c9672830595f999f9fce1195f01402b!


White Screen on IOS

i faced white screen on my ios simulator while using
await AwesomeNotifications().requestPermissionToSendNotifications()

the weird thing is that i tried to put the code on try catch and it doesn't show any error its like the code only stops there and that is why it block the code and show white screen
*** update**
I tried to remove await and everything worked fine
// remove await
AwesomeNotifications().requestPermissionToSendNotifications();

more info:

flutter version: 3.10.6
iam using ios simulator
i already gave the permission to show notification so the problem happen after the first run

Can you paste here your full initialization?

This is my full initialization code and as i said when i removed await from line number 21 everything worked just fine,


[WIP] Testable Integrations

Description

[x] Add Simple Dummy :tm: Integration
[x] Test integration in Ocean Core (sync kind commands)

[x] Simple sync example
[x] Async example
[x] Http consuming example (mocked)

[x] Add test helpers to aid in testing integrations
[ ] Test full flow including entity creation
[ ] Add more coverage to core functions
[ ] MOAR

Type of change
Please leave one option from the following and delete the rest:

[x] New feature (non-breaking change which adds functionality)


luci-base: update and improve Russian translation

Add missing translations and update existing not quite correct translations.
Replaced hyphens on em dashes where it is required by the Russian rules.
Signed-off-by: Anton Kikin<EMAIL_ADDRESS>

Merged - thank you!


Update part3-run-node-in-background-tmux.md

recommended skipping this step

I wonder if I should just make the tmux part 3 optional and move it out. Make systemctl the new part 3.

That’s kinda what I was thinking – I was fortunate to have you advise me that tmux wasn’t necessary, to skip to next step.
Jim Edington
Owner
Western Pacific Insurancehttp://wpinsure.com/
301 N. Pecos | Suite D
Henderson, NV 89074
Cell:<PHONE_NUMBER>
@.***
From: Christophe Garant @.>
Sent: Wednesday, December 27, 2023 4:10 AM
To: ccgarant/ergo-full-node-raspi @.>
Cc: Jim Edington @.>; Author @.>
Subject: Re: [ccgarant/ergo-full-node-raspi] Update part3-run-node-in-background-tmux.md (PR #3)
I wonder if I should just make the tmux part 3 optional and move it out. Make systemctl the new part 3.
—
Reply to this email directly, view it on GitHubhttps://github.com/ccgarant/ergo-full-node-raspi/pull/3#issuecomment-1870249539, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ALGTV75MPIR4WKUO2V4CXIDYLQFZRAVCNFSM6AAAAABBD5JWRSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNZQGI2DSNJTHE.
You are receiving this because you authored the thread.Message ID<EMAIL_ADDRESS>

Alright, nice thanks for the input I'll make that update.