Some basic questions about usage

@Tyriar Wow this lib looks really nice, grats for getting it done that sophisticated, must have been alot of work. I def. will look into it.
First I have a few questions about proper usage:

In api.d.ts you state for decodePng:

Note that it's best to drop references to both metadata and rawChunks as soon as they are no
longer needed as they may take up a significant amount of memory depending on the image.

Does that mean, that the decoder carries the raw data along, thus should be removed manually from the returned IDecodedPng to free some memory?

For terminal output I'm only interested in pixel data (rgba) + dimension notion. I see that is contained IImage32 and IImage64. Is the pixel data there always fully RGBA colored, or is there a catch with some image modes (e.g. paletted contains idx, thus needs another indirection to resolve to real colors)?
The pixel being RGBA8888 - is that RGBA32 or ABGR32 aligned on LE? Not sure yet, if I'll need 32bit access pattern at all, but I found that being way faster than single channel handling in the sixel lib.
Whats the deal with 32 vs. 64, what exactly determines the bit depth of channels and can I somehow tell the decoder to always output IImage32 or would I have to do the color reduction afterwards? Dont want to go the 16bit channel route for now, as it just puts the memory under even more stress.
Later on I want to use the encoder for serialization. Are there any hints/pitfalls with PNG encoding in general (see me as a PNG noob here)?
About those promises - at which level is encoding/decoding async? Is it timer based/cooperative, or some real worker concurrency?
Is it possible to feed the decoder from arbitrary chunks (stream decoding)?

Sorry for so many questions, plz dont feel bugged by that, I dont need detailled answers but just some yes/no or a code pointer here and there.
Btw #15 would be a killer feature - imho APNG is a very nice, but sadly totally underrated format.

Thanks, it definitely did take a while 🙂

Does that mean, that the decoder carries the raw data along, thus should be removed manually from the returned IDecodedPng to free some memory?

Yeah, it's just calling out that IDecodedPng contains references to potentially some big buffers, for example:
https://github.com/lunapaint/png-codec/blob/60576e5f39123afb7d949d18be1b5e04668606d4/typings/api.d.ts#L572-L587

For terminal output I'm only interested in pixel data (rgba) + dimension notion. I see that is contained IImage32 and IImage64. Is the pixel data there always fully RGBA colored, or is there a catch with some image modes (e.g. paletted contains idx, thus needs another indirection to resolve to real colors)?

It's always rgba, the palette is used to decode and pass out the image in a standard. The palette is also available in the API as a potentially useful feature for image editors. Luna Paint lets you inspect an image for example:

The pixel being RGBA8888 - is that RGBA32 or ABGR32 aligned on LE? Not sure yet, if I'll need 32bit access pattern at all, but I found that being way faster than single channel handling in the sixel lib.

The decoded pixel data gets transferred to a regular ArrayBuffer via UInt8Array initialized here:
https://github.com/lunapaint/png-codec/blob/60576e5f39123afb7d949d18be1b5e04668606d4/src/decode/chunks/chunk_IDAT.ts#L246
I think JS uses LE by default?

Whats the deal with 32 vs. 64, what exactly determines the bit depth of channels and can I somehow tell the decoder to always output IImage32 or would I have to do the color reduction afterwards? Dont want to go the 16bit channel route for now, as it just puts the memory under even more stress.

PNGs can be encoded with 64 bits per pixel, if you don't handle 64 bit images, you can use force32:
https://github.com/lunapaint/png-codec/blob/60576e5f39123afb7d949d18be1b5e04668606d4/typings/api.d.ts#L21
I wouldn't expect you to want to do this for terminal images.

Later on I want to use the encoder for serialization. Are there any hints/pitfalls with PNG encoding in general (see me as a PNG noob here)?

I tried to make the encode API as simple as possible to a noob 😉, you will probably just want to encode without any options and some light analysis will happen to pick the right bit depth, color type, etc.

About those promises - at which level is encoding/decoding async? Is it timer based/cooperative, or some real worker concurrency?

The only reason the promises exist is for code splitting:

Importing the decoder will not load the encoder into the engine
There are a lot of chunks, the decoder will lazily load the optional ones only when needed as based on parseChunkTypes:
https://github.com/lunapaint/png-codec/blob/60576e5f39123afb7d949d18be1b5e04668606d4/typings/api.d.ts#L216-L221

Is it possible to feed the decoder from arbitrary chunks (stream decoding)?

It only works off a full UInt8Array currently, I'm not sure stream decoding would give much since pngs are typically tiny?
I considered progressive decoding but it seems really overkill considering the small size of pngs and the large bandwidth of most internet connections (plus my use case works off local or remote without streaming in vscode).

Btw https://github.com/lunapaint/png-codec/issues/15 would be a killer feature - imho APNG is a very nice, but sadly totally underrated format.

I'm guessing you mean https://github.com/lunapaint/png-codec/issues/10? Yeah I'll find the motivation to do this at some point. I actually don't expect it to be that tough to decode APNGs, the hard parts are making a nice multi-image API that I'll want to use for future libs (gif?) and the UX side in Luna Paint would need a bit of work.

It's always rgba, the palette is used to decode and pass out the image in a standard. The palette is also available in the API as a potentially useful feature for image editors.

Ah right, I did it likewise in the sixel lib and exposed the palette for whatever usage. Only issue there - sixels are colored immediately (printer mode), which might be misused by hi-res encoders to shift palette colors to get more colors in.  Which makes the palette unstable during decoding and limits its usage afterwards (was not keen to create a history preserving data structure just for that). Thats also one of the reasons, why I dont want to re-encode a former sixel image back to sixel format for serialization (others being bad compression/quality ratio and high CPU consumption).

I think JS uses LE by default?

Well webassembly is always LE, JS uses native endianess. Learnt that the hard way during wasm adoption for the sixel lib, and basically removed all BE stuff, as the wasm-LE <--> js-BE bridging became a code nuisance. Imho not addressing BE arch is a safe bet for xterm.js, as there is not a single desktop grade BE system anymore. I think ibms POWER arch is the last serious BE domain. Though I find it weird, that LE won the race - BE is for example alot easier in dynamic memory handling for vector processing.

PNGs can be encoded with 64 bits per pixel, if you don't handle 64 bit images, you can use force32:

Ah right, overlooked that in the api file.

I tried to make the encode API as simple as possible to a noob, you will probably just want to encode without any options and some light analysis will happen to pick the right bit depth, color type, etc.

Sweet, thats all I need I guess.
@stream decoder
Yes thats no biggie. The stream decoding approach came to as a logical consequence in the sixel lib, because the raw data is already chunked. But sixel images can grow really big (easily into several MBs) compared to PNGs.
@decoding/promise
Ah ic, you are awaiting module imports for different chunk decoders. I think I gonna test first, how things perform on the mainthread, but most likely I'll have to move it over to the worker (have not perf tested your decoder yet). My early tests with browser -API PNG decoding months ago was not as fast as I hoped for - in fact it is slower than the sixel path for the same reduced pixel data, but thats a summation effect of png decoding and the needed base64 decoding on top. (Already started a much faster lib with a base64 variant in C/wasm, which eliminates atob as ugly perf showstopper.)
So thanks for your detailed response.

@Tyriar I did some first perf tests in a dummy project just testing decoding of a 6000x4000 pixel image 10 times with these results (numbers in msec):

png-codec: [2070, 1980, 3114, 3124, 3104, 3082, 3286, 3093, 3101, 3264]
upng: [1478, 1382, 1227, 1206, 1234, 1215, 1231, 1215, 1260, 1219]
browser: [395, 388, 387, 392, 389, 392, 386, 395, 389, 381]
sixel: [210, 198, 201, 242, 191, 191, 186, 184, 187, 185]

The browser method uses the createImageBitmap function on recent chrome (not available on all browser engines for my purpose). The sixel image contains the same pixel information (in fact I created the png from the sixel one to ensure that).
To get abit more insight some profiling data:
Statistical profiling result from isolate-luna.log, (28788 ticks, 225 unaccounted, 0 excluded).

 [JavaScript]:
   ticks  total  nonlib   name
   6849   23.8%   26.7%  LazyCompile: *defilter /home/jerch/test-upng/node_modules/@lunapaint/png-codec/out-dist/decode/chunks/chunk_IDAT.js:39:18
   6475   22.5%   25.2%  LazyCompile: *<anonymous> /home/jerch/test-upng/node_modules/@lunapaint/png-codec/out-dist/decode/chunks/chunk_IDAT.js:119:24
   6137   21.3%   23.9%  LazyCompile: *inflate_fast /home/jerch/test-upng/node_modules/@lunapaint/png-codec/node_modules/pako/lib/zlib/inffast.js:61:39
   3020   10.5%   11.8%  LazyCompile: *mapPackedDataToRgba /home/jerch/test-upng/node_modules/@lunapaint/png-codec/out-dist/decode/chunks/chunk_IDAT.js:204:29
   1894    6.6%    7.4%  LazyCompile: *inflate /home/jerch/test-upng/node_modules/@lunapaint/png-codec/node_modules/pako/lib/zlib/inflate.js:363:17
    774    2.7%    3.0%  LazyCompile: *readChunk /home/jerch/test-upng/node_modules/@lunapaint/png-codec/out-dist/decode/decoder.js:220:19
     91    0.3%    0.4%  LazyCompile: *updateCrc /home/jerch/test-upng/node_modules/@lunapaint/png-codec/out-dist/shared/crc32.js:25:19
     64    0.2%    0.2%  LazyCompile: *inflate_table /home/jerch/test-upng/node_modules/@lunapaint/png-codec/node_modules/pako/lib/zlib/inftrees.js:53:23
     32    0.1%    0.1%  LazyCompile: *adler32 /home/jerch/test-upng/node_modules/@lunapaint/png-codec/node_modules/pako/lib/zlib/adler32.js:26:17
     11    0.0%    0.0%  LazyCompile: *Inflate.push /home/jerch/test-upng/node_modules/@lunapaint/png-codec/node_modules/pako/lib/inflate.js:196:35
      2    0.0%    0.0%  LazyCompile: *readChunks /home/jerch/test-upng/node_modules/@lunapaint/png-codec/out-dist/decode/decoder.js:187:20
      1    0.0%    0.0%  LazyCompile: *normalizeString path.js:52:25
      1    0.0%    0.0%  LazyCompile: *module.exports.flattenChunks /home/jerch/test-upng/node_modules/@lunapaint/png-codec/node_modules/pako/lib/utils/common.js:30:32

vs. upng:
Statistical profiling result from isolate-upng.log, (12588 ticks, 0 unaccounted, 0 excluded).

 [JavaScript]:
   ticks  total  nonlib   name
   5597   44.5%   46.6%  LazyCompile: *H.H.N /home/jerch/test-upng/upng.js:229:58
   3838   30.5%   32.0%  LazyCompile: *UPNG.decode._filterZero /home/jerch/test-upng/upng.js:323:35
   1817   14.4%   15.1%  LazyCompile: *UPNG.toRGBA8.decodeImage /home/jerch/test-upng/upng.js:34:36
    616    4.9%    5.1%  LazyCompile: *UPNG.decode /home/jerch/test-upng/upng.js:99:23
     29    0.2%    0.2%  LazyCompile: *H.H.A /home/jerch/test-upng/upng.js:247:25
     17    0.1%    0.1%  LazyCompile: *H.H.R /home/jerch/test-upng/upng.js:241:15
     10    0.1%    0.1%  LazyCompile: *H.H.V /home/jerch/test-upng/upng.js:243:98
      8    0.1%    0.1%  LazyCompile: *H.H.n /home/jerch/test-upng/upng.js:245:15
      1    0.0%    0.0%  LazyCompile: *H.H.e /home/jerch/test-upng/upng.js:250:98
      1    0.0%    0.0%  LazyCompile: *H.H.b /home/jerch/test-upng/upng.js:251:15

vs. sixel:
Statistical profiling result from isolate-sixel.log, (2081 ticks, 0 unaccounted, 0 excluded).

 [JavaScript]:
   ticks  total  nonlib   name
   1297   62.3%   96.2%  Function: *wasm-function[9]
      2    0.1%    0.1%  Function: wasm-function[9]
      1    0.0%    0.1%  RegExp: ^((?:@[^/\\%]+\/)?[^./\\%][^/\\%]*)(\/.*)?$

For browser decoding I cannot show you more detailed profiling data, as devtools just shows createImageBitmap as a big block of ~360ms on 4 worker threads plus ~30ms for blob conversion on the mainthread.

Interesting, good to know! Tweaking perf hasn't been a big focus yet as it's been fast enough in my experience, 3 seconds seems like a lot but that's for 24 megapixels which a png would never be used for in practice, going down to 2.4 megapixels is still a crazy size for a png and that's down to ~300ms. I also have the option of doing this in a worker if needed.
Since I haven't needed to focus on perf yet I expect there's probably some easy wins, like pulling variable declarations to the top of a function to reduce GC here and here, bit math tricks, etc.
I also notice for upng most time was spent in UPNG.decode._filterZero which indicates it's probably not a particularly compressed png file, which would decode faster than when using the different filter types.

Well I created it in chrome from the canvas back by blob export. Is there a way to check its compression level? This is what imagemagick has to say about the image:
Image: big.png
  Format: PNG (Portable Network Graphics)
  Mime type: image/png
  Class: DirectClass
  Geometry: 6000x4000+0+0
  Units: Undefined
  Type: PaletteAlpha
  Endianess: Undefined
  Colorspace: sRGB
  Depth: 8-bit
  Channel depth:
    red: 8-bit
    green: 8-bit
    blue: 8-bit
    alpha: 1-bit
  Channel statistics:
    Pixels: 24000000
    Red:
      min: 5 (0.0196078)
      max: 245 (0.960784)
      mean: 62.4418 (0.24487)
      standard deviation: 51.2455 (0.200963)
      kurtosis: 0.485273
      skewness: 1.01213
      entropy: 0.859009
    Green:
      min: 0 (0)
      max: 247 (0.968627)
      mean: 62.2639 (0.244172)
      standard deviation: 53.3428 (0.209187)
      kurtosis: -0.652367
      skewness: 0.620157
      entropy: 0.81602
    Blue:
      min: 0 (0)
      max: 232 (0.909804)
      mean: 68.4463 (0.268417)
      standard deviation: 64.3633 (0.252405)
      kurtosis: -1.50305
      skewness: 0.410485
      entropy: 0.836668
    Alpha:
      min: 255 (1)
      max: 255 (1)
      mean: 255 (1)
      standard deviation: 0 (0)
      kurtosis: 0
      skewness: 0
      entropy: 0.065006
  Image statistics:
    Overall:
      min: 0 (0)
      max: 247 (0.968627)
      mean: 48.288 (0.189365)
      standard deviation: 49.026 (0.192259)
      kurtosis: 1.88159
      skewness: 1.53239
      entropy: 0.644176
  Colors: 239
  Histogram:
    673437: (  5,  8, 13,255) #05080DFF srgba(5,8,13,1)
   3025258: (  8,  8,  8,255) #080808FF grey3
    910398: ( 15,  8,  0,255) #0F0800FF srgba(15,8,0,1)
   1065458: ( 15,  8, 15,255) #0F080FFF srgba(15,8,15,1)
     18736: ( 15, 15,  0,255) #0F0F00FF srgba(15,15,0,1)
      2059: ( 15, 15,  8,255) #0F0F08FF srgba(15,15,8,1)
    368105: ( 15, 15, 15,255) #0F0F0FFF grey6
    142238: ( 20, 15, 23,255) #140F17FF srgba(20,15,23,1)
      6462: ( 23,  0,  8,255) #170008FF srgba(23,0,8,1)
    243219: ( 23,  8,  0,255) #170800FF srgba(23,8,0,1)
    151498: ( 23,  8,  8,255) #170808FF srgba(23,8,8,1)
       226: ( 23,  8, 15,255) #17080FFF srgba(23,8,15,1)
    429605: ( 23, 15,  8,255) #170F08FF srgba(23,15,8,1)
     12867: ( 23, 23,  8,255) #171708FF srgba(23,23,8,1)
    507183: ( 23, 23, 15,255) #17170FFF srgba(23,23,15,1)
     41486: ( 23, 28, 33,255) #171C21FF srgba(23,28,33,1)
     53682: ( 23, 41, 79,255) #17294FFF srgba(23,41,79,1)
      5103: ( 23, 43, 84,255) #172B54FF srgba(23,43,84,1)
     50225: ( 23, 48, 89,255) #173059FF srgba(23,48,89,1)
        82: ( 23, 56,105,255) #173869FF srgba(23,56,105,1)
       348: ( 23, 64,232,255) #1740E8FF srgba(23,64,232,1)
      2528: ( 28, 28, 15,255) #1C1C0FFF srgba(28,28,15,1)
      8453: ( 28, 28, 56,255) #1C1C38FF srgba(28,28,56,1)
     23960: ( 28, 36,  0,255) #1C2400FF srgba(28,36,0,1)
     20905: ( 28, 51, 89,255) #1C3359FF srgba(28,51,89,1)
    347957: ( 33,  8,  0,255) #210800FF srgba(33,8,0,1)
    205233: ( 33,  8, 13,255) #21080DFF srgba(33,8,13,1)
    277925: ( 33, 15,  8,255) #210F08FF srgba(33,15,8,1)
    174193: ( 33, 15, 15,255) #210F0FFF srgba(33,15,15,1)
    156484: ( 33, 15, 23,255) #210F17FF srgba(33,15,23,1)
     37470: ( 33, 23,  8,255) #211708FF srgba(33,23,8,1)
     96233: ( 33, 23, 23,255) #211717FF srgba(33,23,23,1)
      5280: ( 33, 41, 84,255) #212954FF srgba(33,41,84,1)
     30940: ( 33, 48, 97,255) #213061FF srgba(33,48,97,1)
    121952: ( 33, 56, 97,255) #213861FF srgba(33,56,97,1)
    161663: ( 33, 56,105,255) #213869FF srgba(33,56,105,1)
       108: ( 33, 56,112,255) #213870FF srgba(33,56,112,1)
     41682: ( 33, 64,107,255) #21406BFF srgba(33,64,107,1)
    104751: ( 36, 28, 13,255) #241C0DFF srgba(36,28,13,1)
     65499: ( 36, 33, 20,255) #242114FF srgba(36,33,20,1)
      9395: ( 36, 61,115,255) #243D73FF srgba(36,61,115,1)
      3139: ( 36, 71,115,255) #244773FF srgba(36,71,115,1)
        92: ( 41,  8,  8,255) #290808FF srgba(41,8,8,1)
    136334: ( 41, 15,  8,255) #290F08FF srgba(41,15,8,1)
    108467: ( 41, 20, 20,255) #291414FF srgba(41,20,20,1)
     13938: ( 41, 33, 13,255) #29210DFF srgba(41,33,13,1)
     37568: ( 41, 33, 36,255) #292124FF srgba(41,33,36,1)
     21913: ( 41, 48, 89,255) #293059FF srgba(41,48,89,1)
     85255: ( 41, 61, 99,255) #293D63FF srgba(41,61,99,1)
     25528: ( 41, 64,105,255) #294069FF srgba(41,64,105,1)
    182137: ( 41, 64,112,255) #294070FF srgba(41,64,112,1)
    256515: ( 41, 64,120,255) #294078FF srgba(41,64,120,1)
       108: ( 41, 64,128,255) #294080FF srgba(41,64,128,1)
    131052: ( 41, 71,120,255) #294778FF srgba(41,71,120,1)
    210256: ( 41, 71,128,255) #294780FF srgba(41,71,128,1)
      5482: ( 41, 79,128,255) #294F80FF srgba(41,79,128,1)
       678: ( 41, 79,135,255) #294F87FF srgba(41,79,135,1)
     28843: ( 48,  5,  8,255) #300508FF srgba(48,5,8,1)
     92039: ( 48, 15,  0,255) #300F00FF srgba(48,15,0,1)
     90452: ( 48, 23,  8,255) #301708FF srgba(48,23,8,1)
    227072: ( 48, 23, 15,255) #30170FFF srgba(48,23,15,1)
    142712: ( 48, 28, 28,255) #301C1CFF srgba(48,28,28,1)
     67866: ( 48, 36, 41,255) #302429FF srgba(48,36,41,1)
       291: ( 48, 64,120,255) #304078FF srgba(48,64,120,1)
     86219: ( 48, 69,112,255) #304570FF srgba(48,69,112,1)
    150246: ( 48, 69,128,255) #304580FF srgba(48,69,128,1)
    104470: ( 48, 71,120,255) #304778FF srgba(48,71,120,1)
    368401: ( 48, 79,128,255) #304F80FF srgba(48,79,128,1)
    116650: ( 48, 79,135,255) #304F87FF srgba(48,79,135,1)
     41461: ( 48, 89,135,255) #305987FF srgba(48,89,135,1)
     50071: ( 48, 92,143,255) #305C8FFF srgba(48,92,143,1)
       232: ( 48, 97,135,255) #306187FF srgba(48,97,135,1)
    106812: ( 51, 20,  8,255) #331408FF srgba(51,20,8,1)
     46347: ( 51, 23, 20,255) #331714FF srgba(51,23,20,1)
     52523: ( 51, 33,  5,255) #332105FF srgba(51,33,5,1)
     60131: ( 51, 77,120,255) #334D78FF srgba(51,77,120,1)
     14758: ( 56, 13,  5,255) #380D05FF srgba(56,13,5,1)
     54288: ( 56, 23,  8,255) #381708FF srgba(56,23,8,1)
    114419: ( 56, 36, 23,255) #382417FF srgba(56,36,23,1)
     88341: ( 56, 41, 41,255) #382929FF srgba(56,41,41,1)
     12755: ( 56, 41, 48,255) #382930FF srgba(56,41,48,1)
     32669: ( 56, 51,  8,255) #383308FF srgba(56,51,8,1)
     40459: ( 56, 56, 23,255) #383817FF srgba(56,56,23,1)
     10724: ( 56, 56, 36,255) #383824FF srgba(56,56,36,1)
     22349: ( 56, 56, 48,255) #383830FF srgba(56,56,48,1)
      7153: ( 56, 64, 77,255) #38404DFF srgba(56,64,77,1)
     68102: ( 56, 79,128,255) #384F80FF srgba(56,79,128,1)
     59798: ( 56, 89,115,255) #385973FF srgba(56,89,115,1)
     22214: ( 56, 89,128,255) #385980FF srgba(56,89,128,1)
    455734: ( 56, 89,135,255) #385987FF srgba(56,89,135,1)
    135385: ( 56, 89,143,255) #38598FFF srgba(56,89,143,1)
      6343: ( 56, 97,135,255) #386187FF srgba(56,97,135,1)
    255353: ( 56, 97,143,255) #38618FFF srgba(56,97,143,1)
     99940: ( 56,105,148,255) #386994FF srgba(56,105,148,1)
     58884: ( 61, 20,  5,255) #3D1405FF srgba(61,20,5,1)
     33663: ( 61, 23, 20,255) #3D1714FF srgba(61,23,20,1)
     34300: ( 61, 71,107,255) #3D476BFF srgba(61,71,107,1)
     48142: ( 61, 89,133,255) #3D5985FF srgba(61,89,133,1)
     67602: ( 61, 92,143,255) #3D5C8FFF srgba(61,92,143,1)
      9072: ( 64, 13,  0,255) #400D00FF srgba(64,13,0,1)
     57158: ( 64, 23,  8,255) #401708FF srgba(64,23,8,1)
    152985: ( 64, 33,  5,255) #402105FF srgba(64,33,5,1)
     88654: ( 64, 36, 33,255) #402421FF srgba(64,36,33,1)
    151721: ( 64, 48, 48,255) #403030FF srgba(64,48,48,1)
     73325: ( 64, 97,128,255) #406180FF srgba(64,97,128,1)
     41855: ( 64, 97,153,255) #406199FF srgba(64,97,153,1)
     40222: ( 64,105,143,255) #40698FFF srgba(64,105,143,1)
    296013: ( 64,105,153,255) #406999FF srgba(64,105,153,1)
     26595: ( 64,112,153,255) #407099FF srgba(64,112,153,1)
     30327: ( 69,107,135,255) #456B87FF srgba(69,107,135,1)
    163230: ( 69,115,161,255) #4573A1FF srgba(69,115,161,1)
     18288: ( 71,105,153,255) #476999FF srgba(71,105,153,1)
    103113: ( 71,107,143,255) #476B8FFF srgba(71,107,143,1)
    191610: ( 71,112,153,255) #477099FF srgba(71,112,153,1)
    106046: ( 71,120,161,255) #4778A1FF srgba(71,120,161,1)
     13870: ( 77, 20, 13,255) #4D140DFF srgba(77,20,13,1)
     65952: ( 77, 36,  0,255) #4D2400FF srgba(77,36,0,1)
    161622: ( 77, 56, 51,255) #4D3833FF srgba(77,56,51,1)
     13630: ( 77, 71, 51,255) #4D4733FF srgba(77,71,51,1)
     66546: ( 77, 84, 92,255) #4D545CFF srgba(77,84,92,1)
     89080: ( 77, 92,115,255) #4D5C73FF srgba(77,92,115,1)
     10527: ( 77,112,148,255) #4D7094FF srgba(77,112,148,1)
     45489: ( 79, 20,  5,255) #4F1405FF srgba(79,20,5,1)
     74590: ( 79, 36, 20,255) #4F2414FF srgba(79,36,20,1)
     65570: ( 79, 41, 23,255) #4F2917FF srgba(79,41,23,1)
     77579: ( 79, 43, 15,255) #4F2B0FFF srgba(79,43,15,1)
      4649: ( 79, 56, 56,255) #4F3838FF srgba(79,56,56,1)
     51704: ( 79, 61, 41,255) #4F3D29FF srgba(79,61,41,1)
     37031: ( 79, 64, 15,255) #4F400FFF srgba(79,64,15,1)
     27979: ( 79,112,161,255) #4F70A1FF srgba(79,112,161,1)
     58649: ( 79,115,148,255) #4F7394FF srgba(79,115,148,1)
    141164: ( 79,120,161,255) #4F78A1FF srgba(79,120,161,1)
     95220: ( 79,125,163,255) #4F7DA3FF srgba(79,125,163,1)
    159620: ( 84, 61, 61,255) #543D3DFF srgba(84,61,61,1)
     81047: ( 84,107,133,255) #546B85FF srgba(84,107,133,1)
     49945: ( 84,125,148,255) #547D94FF srgba(84,125,148,1)
     72519: ( 89, 36,  0,255) #592400FF srgba(89,36,0,1)
     55758: ( 89, 84, 64,255) #595440FF srgba(89,84,64,1)
     30253: ( 89,112,143,255) #59708FFF srgba(89,112,143,1)
     67905: ( 89,120,168,255) #5978A8FF srgba(89,120,168,1)
     63514: ( 89,128,161,255) #5980A1FF srgba(89,128,161,1)
    214032: ( 89,128,168,255) #5980A8FF srgba(89,128,168,1)
     83453: ( 92, 89, 99,255) #5C5963FF srgba(92,89,99,1)
     59335: ( 92,107,115,255) #5C6B73FF srgba(92,107,115,1)
      1127: ( 92,133,161,255) #5C85A1FF srgba(92,133,161,1)
     86574: ( 97,128,161,255) #6180A1FF srgba(97,128,161,1)
     77805: ( 97,128,168,255) #6180A8FF srgba(97,128,168,1)
     12501: ( 97,135,161,255) #6187A1FF srgba(97,135,161,1)
    104639: ( 97,135,168,255) #6187A8FF srgba(97,135,168,1)
     86741: ( 97,135,176,255) #6187B0FF srgba(97,135,176,1)
    116828: ( 99, 64, 61,255) #63403DFF srgba(99,64,61,1)
     32936: ( 99, 77, 43,255) #634D2BFF srgba(99,77,43,1)
     15986: ( 99,115, 99,255) #637363FF srgba(99,115,99,1)
     61151: ( 99,120,133,255) #637885FF srgba(99,120,133,1)
     42044: (105, 41, 23,255) #692917FF srgba(105,41,23,1)
     61648: (105, 71, 15,255) #69470FFF srgba(105,71,15,1)
     79891: (105,128,148,255) #698094FF srgba(105,128,148,1)
     57581: (105,133,156,255) #69859CFF srgba(105,133,156,1)
     70901: (105,135,168,255) #6987A8FF srgba(105,135,168,1)
    121233: (105,135,176,255) #6987B0FF srgba(105,135,176,1)
       163: (105,143,168,255) #698FA8FF srgba(105,143,168,1)
     55879: (105,143,176,255) #698FB0FF srgba(105,143,176,1)
     22555: (107, 28,  0,255) #6B1C00FF srgba(107,28,0,1)
    109215: (107, 43,  0,255) #6B2B00FF srgba(107,43,0,1)
     75612: (107, 69, 61,255) #6B453DFF srgba(107,69,61,1)
       287: (107,133,156,255) #6B859CFF srgba(107,133,156,1)
     63753: (112, 71, 64,255) #704740FF srgba(112,71,64,1)
     68177: (112, 84, 64,255) #705440FF srgba(112,84,64,1)
     59716: (112,115,112,255) #707370FF srgba(112,115,112,1)
     25925: (112,125, 51,255) #707D33FF srgba(112,125,51,1)
     23877: (112,135,156,255) #70879CFF srgba(112,135,156,1)
     22892: (112,143,156,255) #708F9CFF srgba(112,143,156,1)
     72838: (112,143,168,255) #708FA8FF srgba(112,143,168,1)
    135048: (112,143,176,255) #708FB0FF srgba(112,143,176,1)
     90973: (115,140,161,255) #738CA1FF srgba(115,140,161,1)
     32534: (115,140,171,255) #738CABFF srgba(115,140,171,1)
     60246: (115,148,176,255) #7394B0FF srgba(115,148,176,1)
     79896: (120,128,133,255) #788085FF srgba(120,128,133,1)
     79702: (120,153,176,255) #7899B0FF srgba(120,153,176,1)
     15770: (125, 69, 89,255) #7D4559FF srgba(125,69,89,1)
     65300: (125, 77, 61,255) #7D4D3DFF srgba(125,77,61,1)
     95285: (125, 79, 69,255) #7D4F45FF srgba(125,79,69,1)
     16086: (125, 92, 64,255) #7D5C40FF srgba(125,92,64,1)
    104694: (125,107, 84,255) #7D6B54FF srgba(125,107,84,1)
     38079: (125,120,112,255) #7D7870FF srgba(125,120,112,1)
    101682: (125,135,143,255) #7D878FFF srgba(125,135,143,1)
     54942: (125,148,161,255) #7D94A1FF srgba(125,148,161,1)
    212598: (125,156,181,255) #7D9CB5FF srgba(125,156,181,1)
     34351: (128, 48, 20,255) #803014FF srgba(128,48,20,1)
     28761: (128,120, 97,255) #807861FF srgba(128,120,97,1)
     39151: (128,143,156,255) #808F9CFF srgba(128,143,156,1)
     87599: (128,153,168,255) #8099A8FF srgba(128,153,168,1)
      3592: (128,161,168,255) #80A1A8FF srgba(128,161,168,1)
     78572: (128,161,176,255) #80A1B0FF srgba(128,161,176,1)
     70110: (133, 79, 23,255) #854F17FF srgba(133,79,23,1)
     45823: (133, 84, 64,255) #855440FF srgba(133,84,64,1)
     68925: (133, 92, 69,255) #855C45FF srgba(133,92,69,1)
     41493: (133,120,112,255) #857870FF srgba(133,120,112,1)
     62094: (135, 43,  0,255) #872B00FF srgba(135,43,0,1)
     74313: (135,156,176,255) #879CB0FF srgba(135,156,176,1)
     43442: (135,161,168,255) #87A1A8FF srgba(135,161,168,1)
     71106: (140,125,105,255) #8C7D69FF srgba(140,125,105,1)
     89642: (140,143,140,255) #8C8F8CFF srgba(140,143,140,1)
     48689: (140,148,156,255) #8C949CFF srgba(140,148,156,1)
     17402: (140,153,163,255) #8C99A3FF srgba(140,153,163,1)
     22736: (140,161,156,255) #8CA19CFF srgba(140,161,156,1)
     69745: (143,133,128,255) #8F8580FF srgba(143,133,128,1)
     29012: (143,161,168,255) #8FA1A8FF srgba(143,161,168,1)
    143737: (143,161,176,255) #8FA1B0FF srgba(143,161,176,1)
    121562: (148, 92, 69,255) #945C45FF srgba(148,92,69,1)
     43242: (148,148,140,255) #94948CFF srgba(148,148,140,1)
    119580: (153, 77, 20,255) #994D14FF srgba(153,77,20,1)
     73652: (153,112, 89,255) #997059FF srgba(153,112,89,1)
     41377: (153,115, 41,255) #997329FF srgba(153,115,41,1)
     30459: (153,115, 77,255) #99734DFF srgba(153,115,77,1)
     46101: (153,125,105,255) #997D69FF srgba(153,125,105,1)
     22151: (153,156,156,255) #999C9CFF srgba(153,156,156,1)
     56411: (153,161,143,255) #99A18FFF srgba(153,161,143,1)
     63733: (153,161,161,255) #99A1A1FF srgba(153,161,161,1)
    152660: (153,168,163,255) #99A8A3FF srgba(153,168,163,1)
     64071: (156,140,120,255) #9C8C78FF srgba(156,140,120,1)
     35262: (156,148,140,255) #9C948CFF srgba(156,148,140,1)
    108005: (161,115, 84,255) #A17354FF srgba(161,115,84,1)
     12536: (161,133,120,255) #A18578FF srgba(161,133,120,1)
     50710: (161,153,135,255) #A19987FF srgba(161,153,135,1)
    104509: (168,128, 99,255) #A88063FF srgba(168,128,99,1)
    152640: (168,148,140,255) #A8948CFF srgba(168,148,140,1)
    142578: (171,140,115,255) #AB8C73FF srgba(171,140,115,1)
     67266: (181, 77, 23,255) #B54D17FF srgba(181,77,23,1)
     22518: (181,148, 51,255) #B59433FF srgba(181,148,51,1)
     19509: (196,196,120,255) #C4C478FF srgba(196,196,120,1)
     38063: (204,115, 51,255) #CC7333FF srgba(204,115,51,1)
     17746: (204,196,199,255) #CCC4C7FF srgba(204,196,199,1)
     49788: (204,199,135,255) #CCC787FF srgba(204,199,135,1)
     36897: (204,204, 84,255) #CCCC54FF srgba(204,204,84,1)
    124757: (212,115,  5,255) #D47305FF srgba(212,115,5,1)
     41941: (227,209, 28,255) #E3D11CFF srgba(227,209,28,1)
     78305: (232,148, 41,255) #E89429FF srgba(232,148,41,1)
     94959: (245,247,199,255) #F5F7C7FF srgba(245,247,199,1)
  Rendering intent: Perceptual
  Gamma: 0.454545
  Chromaticity:
    red primary: (0.64,0.33)
    green primary: (0.3,0.6)
    blue primary: (0.15,0.06)
    white point: (0.3127,0.329)
  Background color: white
  Border color: srgba(223,223,223,1)
  Matte color: grey74
  Transparent color: none
  Interlace: None
  Intensity: Undefined
  Compose: Over
  Page geometry: 6000x4000+0+0
  Dispose: Undefined
  Iterations: 0
  Compression: Zip
  Orientation: Undefined
  Properties:
    date:create: 2021-03-26T17:18:52+01:00
    date:modify: 2021-03-26T15:43:37+01:00
    png:IHDR.bit-depth-orig: 8
    png:IHDR.bit_depth: 8
    png:IHDR.color-type-orig: 6
    png:IHDR.color_type: 6 (RGBA)
    png:IHDR.interlace_method: 0 (Not interlaced)
    png:IHDR.width,height: 6000, 4000
    png:sRGB: intent=0 (Perceptual Intent)
    signature: 839909d68908dc5b7809d46f5f6349af761bbddfffc7cfa34b0bcbd3588e8b67
  Artifacts:
    filename: big.png
    verbose: true
  Tainted: False
  Filesize: 19.19MB
  Number pixels: 24M
  Pixels per second: 32.88MB
  User time: 0.720u
  Elapsed time: 0:01.730
  Version: ImageMagick 6.9.7-4 Q16 x86_64 20170114 http://www.imagemagick.org

Thats alot of stats, dunno where the real compression hides here. And yes, the image is quite big with 19MB, the sixel original is 25MB.

Well imagemagick does not contain compression level, as stated here https://legacy.imagemagick.org/discourse-server/viewtopic.php?t=32574
pngcheck prints this:
...
    zlib: deflated, 32K window, fast compression
...

Does that mean, it is uncompressed? The image has highly different colors, could it be that the browser decided not to compress it because of high entropy?

A first optimization possibility I have found is convert16BitTo8BitData in decoder/array.ts. If rewritten with a rshift plus some loop unrolling, it runs way faster (only tested in javascript and no tail handling yet):
function convert16BitTo8BitData(data) {
  const result = new Uint8Array(data.length);
  for (let i = 0; i < result.length; i += 4) {
      result[i+0] = data[i+0] >> 8;
      result[i+1] = data[i+1] >> 8;
      result[i+2] = data[i+2] >> 8;
      result[i+3] = data[i+3] >> 8;
  }
  return result;
}

conversion throughput

current variant: 250 MB/s
shift+unroll: 360 MB/s

Furthermore this task would highly benefit from a wasm rewrite, as I'd expect it to boost throughput above 1GB/s (scalar), or with wasm-simd its a single _mm_shuffle_epi8 picking the high bytes of 8 elements at once (runs in one cycle), which should give >>6GB/s in wasm. Thats more than 20 times faster.
To make this more explicit - conversion of 100M channel values (== 25M pixels in RGBA) takes:

current: 410 ms
shift+unroll: 270 ms
wasm scalar: <100 ms
wasm-simd: <16 ms

The image magick output doesn't say anything about filters, the filter selection is the hardest part with optimizing pngs and if it's using filter none primarily (ie. just raw pixel data), it's probably not very optimized.

convert16BitTo8BitData

This is arguably not too important generally, I find 16-bit (ie. 64-bit per channel) png files are fairly rare. The TS change looks great though, want to make a PR? Testing is very comprehensive so if the tests pass I'm very confident nothing broke 🙂

Ok managed to get it properly converted with gimp by indexing it first - 8.3MB, png indexed. New runtime numbers:

Looks like it comes out to 8.7mb when saving with Luna Paint which seems pretty good, not sure how many colors it has as Luna Paint is struggling to inspect the file (https://github.com/lunapaint/vscode-luna-paint/issues/145). Do you mean png indexed as in you needed to reduce it to 256 colors?

some wasm numbers

I haven't done much with wasm but I worry about the complexity it will bring both in the implementation and in consuming the library (where do the wasm files go, do they place nicely with bundlers, etc.), I always had issues when I tried to setup emscripten for example.

The image magick output doesn't say anything about filters, the filter selection is the hardest part with optimizing pngs and if it's using filter none primarily (ie. just raw pixel data), it's probably not very optimized.

Ic. Well I used the browser to create it, and it was pretty fast compared to what gimp did (<2s vs. >20s), prolly it just dumped pixel data with little additional effort. With is nice later on for term serialization because its fast, but at the same time created pngs are overly big which is bad when loading a serialize state back into the terminal. Hmm.

This is arguably not too important generally, I find 16-bit (ie. 64-bit per channel) png files are fairly rare. The TS change looks great though, want to make a PR? Testing is very comprehensive so if the tests pass I'm very confident nothing broke

Oh I know, its just the first function I stumbled over, and was a quite low hanging fruit, too. I did not even bother yet to look at more involved code paths as from the profiling data (those take much more time than a sunday afternoon). Sure I can create a PR (but need to write the tailing first :smile_cat:).

Looks like it comes out to 8.7mb when saving with Luna Paint which seems pretty good, not sure how many colors it has as Luna Paint is struggling to inspect the file (https://github.com/lunapaint/vscode-luna-paint/issues/145). Do you mean png indexed as in you needed to reduce it to 256 colors?

Yes 8.7MB is okish, that what I got from image magick's convert as well. With indexed I mean - the image is already indexed from its sixel origin and contains only 239 colors (thats what the libsixel's quantisation algo made of the original RGB image). But when converting it to PNG in the browser, its was not created as indexedPNG but as sRGB. Gimp works likewise - loading that PNG in Gimp and just re-exporting it assumes RGB mode, thus I had to tell Gimp explicitly to do a palette reduction first. Since the PNG was built from a paletted sixel image in the first place, it contains exactly the same pixel information - Gimp just collects all colors and finds <256 colors, thus can directly create the palette from all image colors (though no color reduction/quantization is needed). The indexedPNG file takes only 8.3 MB from Gimp, and 8.7 MB from image magick.

I haven't done much with wasm but I worry about the complexity it will bring both in the implementation and in consuming the library (where do the wasm files go, do they place nicely with bundlers, etc.), I always had issues when I tried to setup emscripten for example.

Can set up a toy project with the wasm impl for convert16BitTo8BitData if you like. I kinda managed to do emscripten and npm bundling fully automatic - works under linux, prolly also macos, no clue about windows though. The gh action of node-sixel uses it (https://github.com/jerch/node-sixel/runs/6261370627?check_suite_focus=true).

Luna Paint uses the recommended quick filter selection per line as I think is recommended by the spec. More involved methods try to guess which filters would be better encoded by zlib by hopefully doing less than brute forcing every option

works under linux, prolly also macos, no clue about windows though

Yeah, Windows is always the tricky part 😄

Can set up a toy project with the wasm impl for convert16BitTo8BitData if you like.

Sure, with that I can answer those questions about setup/bundling.

Oh well, this might explain why gimp took so long maybe brute forcing things.
Early repo is up here: https://github.com/jerch/wasm-dummy (def. not yet working on windows, macos might work though).
Gonna close the issue, as I have enough info to get something rolling with the lib.

@Tyriar Added an early wasm generator, which should have proper TS typing and creates an inline wasm module. Works already for very simple source code. :smile_cat:

Never knew inline wasm was a thing. I'll have to play around with this some upcoming weekend, thanks 🙂

Oh well - real in-lining across all stages is not possible, as wasm needs (foreign) compilation. The generator is a source compiler, that has to run at a certain build step and replaces things with wasm bootstrap logic, which is fully embedded in the final bundle though.

Thought that's how it was working after having a little look. Still might be nice to keep the wasm source close if it's just a small function.


for me at least       {"Key": "export", "hide": "true"       }, is not hiding the Export button in the command bar at all.

[Enter feedback here]

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 22dffaf9-de6c-5ec3-bd61-23b995c256ce
Version Independent ID: 22dffaf9-de6c-5ec3-bd61-23b995c256ce
Content: Command bar customization syntax reference
Content Source: docs/declarative-customization/view-commandbar-formatting.md
Service: sharepoint-online
GitHub Login: @VesaJuvonen
Microsoft Alias: vesaj

Hi @VincentVanAgogo
Your "Key" should be entirely lowercase.
Would you try with the following instead?
{   "key": "export",   "hide": "true" }
This should fix your issue.


Resolve native assemblies from NuGet package

Is your feature request related to a problem? Please describe.
I have a plugin with a dependency on Microsoft.CognitiveServices.Speech, which has native dependencies in the runtimes folder that are resolved per platform. When the P/Invoke call occurs from inside Microsoft.CognitiveServices.Speech after resolving the plugin with PluginLoader.CreateFromAssemblyFile, I get a DllNotFoundException.
Describe the solution you'd like
I'd like the PluginLoader to also search the runtimes folder in the base path for the AssemblyLoadContext so that I do not get the DllNotFoundException for the native dependency.
Describe alternatives you've considered
Haven't started thinking about alternatives.
Additional context
No additional context.

This is a supported scenario, in fact, there are tests for this. Can you share steps to reproduce this error? What do your project files look like?

I'll send you a repro today.

Run the main Console app (PluginLoaderTest) on macOS from:
https://github.com/rozele/PluginLoaderTest

(Haven't tested on Windows yet)

FYI - I get the same exception for dlopen when I use the latest NuGet preview package (and I also noticed that the sharedTypes feature is not working as expected there. I.e., I changed to:
            var assembly = PluginLoader.CreateFromAssemblyFile(
                    assemblyFile,
                    sharedTypes: new[] { typeof(ITest) })
                .LoadDefaultAssembly();

And when I run:
        var test = new ContainerConfiguration()
                .WithAssembly(assembly)
                .CreateContainer()
                .GetExport<ITest>();

I am not able to resolve an ITest from that assembly with System.Composition.

Here's the exception for quick reference:
System.DllNotFoundException
Unable to load shared library 'Microsoft.CognitiveServices.Speech.core.dll' or one of its dependencies. In order to help diagnose loading problems, consider setting the DYLD_PRINT_LIBRARIES environment variable: dlopen(libMicrosoft.CognitiveServices.Speech.core.dll, 1): image not found

I see the problem. Make this change to your TestPlugin.csproj file:
 <Project Sdk="Microsoft.NET.Sdk">

   <PropertyGroup>
-    <TargetFramework>netstandard2.0</TargetFramework>
+    <TargetFramework>netcoreapp2.1</TargetFramework>
   </PropertyGroup>

   <ItemGroup>

Explanation
.NET Standard 2.0 is a platform-less, runtime-agnostic API surface. When your project compiles a plugin for .NET Standard 2.0, all native assemblies are excluded from build output and the dependencies metadata. This is by-design. I opened a bug about this years ago against the .NET SDK team to request changes that would have fixed your exact scenario, but there is no plan to make this work. See https://github.com/dotnet/core-setup/issues/635 and this comment https://github.com/dotnet/core-setup/issues/635#issuecomment-284516681
I can't think of anything this library can do to workaround it...except maybe make the documentation clearer. Thoughts?

Okay - so this library uses a .NET Core API to load the native dependencies? It's strange that it works as expected if take a direct package dependency (meaning .NET Core has some mechanism to resolve the dependency from .NET Standard). Is it possible to add a configuration to tell the PluginLoader to look for .NET Standard libraries?
Otherwise, for now, I can just change the target framework.

The reason it works as a direct package dependency is the same reason changing target frameworks works. The mechanism by which this is happening is the .deps.json file. The .NET Core SDK resolves native assemblies at build time for netcoreapp2.1 and puts a list of the .dylib/.dll/.so files into the .deps.json file. The SDK doesn't include this in the .deps.json file for a .NET Standard project. My library reads the .deps.json file to determine which dependencies should be loaded, so I can't really think of a why my library can solve this problem.

@natemcmaster Unrelated question - I got things working when running from VS, but I'm working on a .NET Core CLI tool extension so I pack the output before running it. When I use the  and  approach to build the project, the plugin folders are not included in the NuGet output. Have you used this approach for a NuGet package, and is there a quick way (without setting up a .nuspec file) to ensure the plugin libraries are included in the .nupkg?

Thanks! This works.

Hey @natemcmaster - quick question. When I switch to using:
  <Target Name="BuildPlugin" BeforeTargets="Build">
    <MSBuild Projects="..\MyPlugin\MyPlugin.csproj" Targets="Publish" Properties="Configuration=$(Configuration);PublishDir=$(OutputPath)/plugins/MyPlugin/" />
  </Target>

And try to create a NuGet package for the "host" project as a .NET Core CLI tool, the plugins folder is not included in the .nupkg. I can setup a .nuspec file to get the plugins folder, but I was wondering if you had any tips / ran into this issue and figured out how to inform MSBuild to include the plugins folder without creating a .nuspec.

as a .NET Core CLI tool

Are you using <PackAsTool>true</PackAsTool>?

Yup.

I can setup a .nuspec file to get the plugins folder, but I was wondering if you had any tips / ran into this issue and figured out how to inform MSBuild to include the plugins folder without creating a .nuspec.

Disclaimer first....if you're not familiar with MSBuild targets, properties, and item groups, it's probably better to use a nuspec file. I usually recommend nuspec for complex packing tasks because I think the "API" for customizing a package via csproj is awful.
Nuspec generation in csproj relies on MSBuild items to tell it what to pack. There are a few magic default settings which pick up build outputs for most simple class libraries and console tools. But the magic falls apart when you need to include generated content (such as a plugins folder). You can write your own magic by adding files to the TfmSpecificPackageFile item group as a part of a target that is listed in the TargetsForTfmSpecificContentInPackage property. I haven't tested this on my own, but I imagine the result would look something like this:
<PropertyGroup>
  <TargetsForTfmSpecificContentInPackage>BuildPlugin;$(TargetsForTfmSpecificContentInPackage)</TargetsForTfmSpecificContentInPackage>
</PropertyGroup>

<Target Name="BuildPlugin" BeforeTargets="CoreBuild">
  <PropertyGroup>
    <MyPluginOutputPath>$(OutputPath)/plugins/MyPlugin/</MyPluginOutputPath>
  </PropertyGroup>

  <MSBuild
    Projects="..\MyPlugin\MyPlugin.csproj"
    Targets="Publish"
    Properties="Configuration=$(Configuration);PublishDir=$(MyPluginOutputPath)" />

  <ItemGroup>
    <TfmSpecificPackageFile Include="$(MyPluginOutputPath)**/*">
      <PackagePath>tools/$(TargetFramework)/any/plugins/MyPlugin/</PackagePath>
    </TfmSpecificPackageFile>
  </ItemGroup>

</Target>

By the way, checkout https://github.com/dotnet/sdk/blob/master/src/Tasks/Microsoft.NET.Build.Tasks/targets/Microsoft.NET.PackTool.targets and https://github.com/NuGet/NuGet.Client/blob/dev/src/NuGet.Core/NuGet.Build.Tasks.Pack/NuGet.Build.Tasks.Pack.targets to see what the SDK is doing under the hood.

Thanks @natemcmaster - the <TargetsForTfmSpecificContentInPackage> did the trick


QPPSF-10873: Depreciate CPC Plus API

Information

Depreciates the CPC+ API and all look up functionality.

Checklist

[ ] All JUnit tests pass (mvn clean verify).
[ ] New unit tests written to cover new functionality.
[ ] Added and updated JavaDocs for non-test classes and methods.
[ ] No local design debt. Do you feel that something is "ugly" after your changes?
[ ] Updated documentation (README.md, etc.) depending if the changes require it.

SonarCloud coverage is showing a false positive currently. A new issue will be created to find the issue with why coverage is not being picked up.


Forcefield error

So, I was messing about with force fields in Direwolf20 on my server, and came upon an error with reading packet 53.
https://gyazo.com/35754e3b54b01a7847d359dc74bf4af3
This issue wasn't happening before until after I placed the and activated to forcefield doing a large area.
I'll have to go test in 1.10 and see if it reoccurs there.

yes, known problem in the 1.7.10 version of RFTools. If you make the shield too large you get this problem. In later versions this no longer occurs. Best would be to try to remove the shield projector using mcedit or something

Alright, so if I were to update RF tools would that work in theory? Or was it not fixed until 1.8?

Updating RFtools also works

Ah yes, if the shield is not too big then that's a solution as I partially solved that problem to allow some bigger shields. Forgot about that


Support new cairo-lang pre-release

Feature Request
.

https://github.com/software-mansion/starknet.py/pull/763


Bug: Top-Level library targets not being merged correctly

Describe the bug
Top-level library targets used as the dependency for an ios_application that also have a sibling test target/library without a test host end up being marked as a non-mergable target. This manifests downstream as our primary ios_application target inherits the DEBUG_INFORMATION_FORMAT build setting as DWARF with dSYM File from Xcode's iOS Default instead of being set as just DWARF from the library target.
When this build setting is configured, the generate dSYM step in Xcode also generates warnings for every swiftmodule it tries to link into the dSYM bundle. Seems like it just can't find the swiftmodule path and runs stat over a missing file? Hence the timestamp date being 12/31/1969 😅
/.../Example/Example.swiftmodule/arm64-apple-ios-simulator.swiftmodule: timestamp mismatch between swift interface file (1969-12-31 19:00:00.000000000) and debug map (1969-12-31 19:00:00.000000000)

Additionally, the malformed dYSM causes p / po break in lldb even with the remaps (v still works though, hence the missing swiftmodule ast paths)
error: Couldn't realize type of self
It appears there are 2 distinct but connected issues here:

Top-level targets used by an ios_application that are also a dependency for an ios_unit_test/swift_library duo end up in the non_mergable_targets because removing the test_host doesn't de-dupe the linked static libraries list. Why do we use this list as the input for determining mergeable targets?
Is dSYM generation by Xcode currently supported in rules_xcodeproj?

Reproduction steps
Removing the test host for ExampleTests in examples/ios_app reproduces this issue in the generated project.
Expected behavior
Skip Xcode dSYM generation when not requested by the Bazel Build.
Versions

Bazel: 6.0.0-pre.20220310.1
rules_xcodeproj: 0.5.0
rules_apple: 1.0.1
rules_swift: 1.0.0

Additional context

cc: @cgrindel

This is because a static library can't merge into two targets. Target merging is to replicate the Xcode setup of code being compiled as part of a top-level target. If that code is used in two places then it's actually a static library, not the binary of a top-level target.
That said, I'll look into the repro steps and see how this situation can be improved. The warning most likely is expected though.

dSYM generation should be supported, but we don't have a test for it, so it might have broke.

https://github.com/buildbuddy-io/rules_xcodeproj/pull/669 fixes the DEBUG_INFORMATION_FORMAT setting. I need to add a test for dSYM generation next.

When this build setting is configured, the generate dSYM step in Xcode also generates warnings for every swiftmodule it tries to link into the dSYM bundle. Seems like it just can't find the swiftmodule path and runs stat over a missing file? Hence the timestamp date being 12/31/1969

I only get that warning when using lld. Do you have --ld-path set? The ios_app example does.

After #669 and #670 are merged, I think we can close this issue. Target merging is working as expected, as the app and the test both want to link to the static library, so it can't merge into the app. In an Xcode-only world this common library would be a dependency of the app library. Also, an unmerged library isn't the end of the world, and I should probably adjust the warning criteria in the future.


Make OAuth 2.0 redirect URIs regexes.

Summary
This addresses https://github.com/Mashape/kong/issues/1397 by making the redirect URIs a regex. This is necessary for any application that allows a user to login on arbitrary pages within the application. While this could be handled by saving that information in the app locally, that creates additional development overhead and complexity. I'm hoping this change doesn't qualify as breaking, although it's not a simple bugfix and does change behavior. If so, I can rebase off of next instead.
Full Changelog

Add table_contains_regex that mirrors old table_contains and use it when checking if the supplied redirect URI is valid for a given client. This method simply uses ngx.re.match instead of a simple equality check.
Add migrations to prepend all redirect URIs with '^' and append '$' to ensure that existing exact match behavior is preserved on upgrade.
If no redirect URI is specified, the first entry in the list is still used but the '^' and '$' characters are removed. This is hacky, feedback is appreciated.
Add related tests.

Issues resolved
Fix #1397

Thanks for the comments, I really appreciated the input. Hopefully I've addressed some of them and I'll comment on the others.

Will this feature be merged into master, and when? I've encounter this situation which the redirect_uri is like https://example.com/xxx/xxx?token=xxx which the token parameter is dynamic.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Failed to use UYVU format in VPP

My NUC machine has Core i 7-8650U (Kaby Lake-Refresh).
It seems to support UYVY format as VPP's INPUT. It works well before merging ae8f993 commit.
This commit added MFX_CHECK(platform >= MFX_HW_ICL, MFX_ERR_INVALID_VIDEO_PARAM) in case MFX_FOURCC_UYVY.
Why does it restrict platforms supporting UYVY except for ICL?

I confirmed that this problem was fixed at b49b03b37b21f89575aa8dd956531928492a9835.


How to use eez-framework in Zephyr RTOS

Hi,
Using EEZ Studio I have designed a simple UI that uses flows. Before going too much further I wanted to try to incorporate this UI into my project that is based on Zephyr RTOS (nRF Connect SDK more accurately). What is the best way to do this ? I am not super familiar with Cmake, so I may be missing something obvious.
I first thought about using this repo as a reference : https://github.com/eez-open/zephyrproject-rtos-eez-lvgl/tree/main but it seemed outdated.
Then I tried to follow the instructions in this repo's main branch readme regarding modifications of CMakeLists.txt files, but I get errors when compiling eez-framework files.
Building app
C:\WINDOWS\system32\cmd.exe /d /s /c "west build --build-dir c:/Users/nicol/Documents/xiao-round-display/xiao-round-display.git/app/build_shield c:/Users/nicol/Documents/xiao-round-display/xiao-round-display.git/app --pristine --board xiao_ble_sense --no-sysbuild -- -DNCS_TOOLCHAIN_VERSION=NONE -DBOARD_ROOT=c:/users/nicol/documents/xiao-round-display/xiao-round-display.git -DCACHED_CONF_FILE=c:/Users/nicol/Documents/xiao-round-display/xiao-round-display.git/app/prj_shield.conf -DDTC_OVERLAY_FILE=c:/Users/nicol/Documents/xiao-round-display/xiao-round-display.git/app/boards/xiao_ble_sense.overlay -DSHIELD=seeed_xiao_round_display"

-- west build: generating a build system
Loading Zephyr default modules (Zephyr base).
-- Application: C:/Users/nicol/Documents/xiao-round-display/xiao-round-display.git/app
-- CMake version: 3.21.0
-- Found Python3: C:/ncs/toolchains/cf2149caf2/opt/bin/python.exe (found suitable version "3.9.13", minimum required is "3.8") found components: Interpreter
-- Cache files will be written to: C:/Users/nicol/Documents/xiao-round-display/zephyr/.cache
-- Zephyr version: 3.5.99 (C:/Users/nicol/Documents/xiao-round-display/zephyr)
-- Found west (found suitable version "1.2.0", minimum required is "0.14.0")
-- Board: xiao_ble_sense
-- Shield(s): seeed_xiao_round_display
-- Found host-tools: zephyr 0.16.5 (C:/ncs/toolchains/cf2149caf2/opt/zephyr-sdk)
-- Found toolchain: zephyr 0.16.5 (C:/ncs/toolchains/cf2149caf2/opt/zephyr-sdk)
-- Found Dtc: C:/ncs/toolchains/cf2149caf2/opt/bin/dtc.exe (found suitable version "1.4.7", minimum required is "1.4.6")
-- Found BOARD.dts: C:/Users/nicol/Documents/xiao-round-display/zephyr/boards/arm/xiao_ble/xiao_ble_sense.dts
-- Found devicetree overlay: C:/Users/nicol/Documents/xiao-round-display/zephyr/boards/shields/seeed_xiao_round_display/seeed_xiao_round_display.overlay
-- Found devicetree overlay: c:/Users/nicol/Documents/xiao-round-display/xiao-round-display.git/app/boards/xiao_ble_sense.overlay
node '/soc/spi@40023000/gc9a01@0' compatible 'galaxycore,gc9x01x' has unknown vendor prefix 'galaxycore'
-- Generated zephyr.dts: C:/Users/nicol/Documents/xiao-round-display/xiao-round-display.git/app/build_shield/zephyr/zephyr.dts
-- Generated devicetree_generated.h: C:/Users/nicol/Documents/xiao-round-display/xiao-round-display.git/app/build_shield/zephyr/include/generated/devicetree_generated.h
-- Including generated dts.cmake file: C:/Users/nicol/Documents/xiao-round-display/xiao-round-display.git/app/build_shield/zephyr/dts.cmake
Parsing C:/Users/nicol/Documents/xiao-round-display/xiao-round-display.git/app/Kconfig
Loaded configuration 'C:/Users/nicol/Documents/xiao-round-display/zephyr/boards/arm/xiao_ble/xiao_ble_sense_defconfig'
Merged configuration 'c:/Users/nicol/Documents/xiao-round-display/xiao-round-display.git/app/prj_shield.conf'
Configuration saved to 'C:/Users/nicol/Documents/xiao-round-display/xiao-round-display.git/app/build_shield/zephyr/.config'
Kconfig header saved to 'C:/Users/nicol/Documents/xiao-round-display/xiao-round-display.git/app/build_shield/zephyr/include/generated/autoconf.h'
-- Found GnuLd: c:/ncs/toolchains/cf2149caf2/opt/zephyr-sdk/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/../../../../arm-zephyr-eabi/bin/ld.bfd.exe (found version "2.38")
-- The C compiler identification is GNU 12.2.0
-- The CXX compiler identification is GNU 12.2.0
-- The ASM compiler identification is GNU
-- Found assembler: C:/ncs/toolchains/cf2149caf2/opt/zephyr-sdk/arm-zephyr-eabi/bin/arm-zephyr-eabi-gcc.exe
CMake Warning at C:/Users/nicol/Documents/xiao-round-display/zephyr/subsys/usb/device/CMakeLists.txt:22 (message):
  CONFIG_USB_DEVICE_VID has default value 0x2FE3.

  This value is only for testing and MUST be configured for USB products.

CMake Warning at C:/Users/nicol/Documents/xiao-round-display/zephyr/CMakeLists.txt:862 (message):
  No SOURCES given to Zephyr library: drivers__display

  Excluding target from build.

-- Configuring done
-- Generating done
-- Build files have been written to: C:/Users/nicol/Documents/xiao-round-display/xiao-round-display.git/app/build_shield
-- west build: building application
[1/637] Building CXX object src/display/ui/eez-framework/CMakeFiles/eez-framework.dir/src/eez/core/action.cpp.obj
FAILED: src/display/ui/eez-framework/CMakeFiles/eez-framework.dir/src/eez/core/action.cpp.obj
C:\ncs\toolchains\cf2149caf2\opt\zephyr-sdk\arm-zephyr-eabi\bin\arm-zephyr-eabi-g++.exe -DEEZ_FOR_LVGL -DLV_LVGL_H_INCLUDE_SIMPLE -isystem C:/Users/nicol/Documents/xiao-round-display/xiao-round-display.git/app/src/display/ui/eez-framework/./src -isystem C:/Users/nicol/Documents/xiao-round-display/xiao-round-display.git/app/src/display/ui/eez-framework/./src/eez/libs/agg  -MD -MT src/display/ui/eez-framework/CMakeFiles/eez-framework.dir/src/eez/core/action.cpp.obj -MF src\display\ui\eez-framework\CMakeFiles\eez-framework.dir\src\eez\core\action.cpp.obj.d -o src/display/ui/eez-framework/CMakeFiles/eez-framework.dir/src/eez/core/action.cpp.obj -c C:/Users/nicol/Documents/xiao-round-display/xiao-round-display.git/app/src/display/ui/eez-framework/src/eez/core/action.cpp
In file included from C:/Users/nicol/Documents/xiao-round-display/xiao-round-display.git/app/src/display/ui/eez-framework/src/eez/core/action.cpp:13:
c:\users\nicol\documents\xiao-round-display\xiao-round-display.git\app\src\display\ui\eez-framework\src\eez\core\action.h:15:10: fatal error: lvgl.h: No such file or directory
   15 | #include "lvgl.h"
      |          ^~~~~~~~
compilation terminated.
[2/637] Building CXX object src/display/ui/eez-framework/CMakeFiles/eez-framework.dir/src/eez/flow/components/label_in.cpp.obj
FAILED: src/display/ui/eez-framework/CMakeFiles/eez-framework.dir/src/eez/flow/components/label_in.cpp.obj
C:\ncs\toolchains\cf2149caf2\opt\zephyr-sdk\arm-zephyr-eabi\bin\arm-zephyr-eabi-g++.exe -DEEZ_FOR_LVGL -DLV_LVGL_H_INCLUDE_SIMPLE -isystem C:/Users/nicol/Documents/xiao-round-display/xiao-round-display.git/app/src/display/ui/eez-framework/./src -isystem C:/Users/nicol/Documents/xiao-round-display/xiao-round-display.git/app/src/display/ui/eez-framework/./src/eez/libs/agg  -MD -MT src/display/ui/eez-framework/CMakeFiles/eez-framework.dir/src/eez/flow/components/label_in.cpp.obj -MF src\display\ui\eez-framework\CMakeFiles\eez-framework.dir\src\eez\flow\components\label_in.cpp.obj.d -o src/display/ui/eez-framework/CMakeFiles/eez-framework.dir/src/eez/flow/components/label_in.cpp.obj -c C:/Users/nicol/Documents/xiao-round-display/xiao-round-display.git/app/src/display/ui/eez-framework/src/eez/flow/components/label_in.cpp
In file included from C:/Users/nicol/Documents/xiao-round-display/xiao-round-display.git/app/src/display/ui/eez-framework/src/eez/flow/components/label_in.cpp:15:
c:\users\nicol\documents\xiao-round-display\xiao-round-display.git\app\src\display\ui\eez-framework\src\eez\flow\hooks.h:21:10: fatal error: lvgl.h: No such file or directory
   21 | #include "lvgl.h"
      |          ^~~~~~~~
compilation terminated.
[3/637] Building CXX object src/display/ui/eez-framework/CMakeFiles/eez-framework.dir/src/eez/flow/components/label_out.cpp.obj
FAILED: src/display/ui/eez-framework/CMakeFiles/eez-framework.dir/src/eez/flow/components/label_out.cpp.obj
C:\ncs\toolchains\cf2149caf2\opt\zephyr-sdk\arm-zephyr-eabi\bin\arm-zephyr-eabi-g++.exe -DEEZ_FOR_LVGL -DLV_LVGL_H_INCLUDE_SIMPLE -isystem C:/Users/nicol/Documents/xiao-round-display/xiao-round-display.git/app/src/display/ui/eez-framework/./src -isystem C:/Users/nicol/Documents/xiao-round-display/xiao-round-display.git/app/src/display/ui/eez-framework/./src/eez/libs/agg  -MD -MT src/display/ui/eez-framework/CMakeFiles/eez-framework.dir/src/eez/flow/components/label_out.cpp.obj -MF src\display\ui\eez-framework\CMakeFiles\eez-framework.dir\src\eez\flow\components\label_out.cpp.obj.d -o src/display/ui/eez-framework/CMakeFiles/eez-framework.dir/src/eez/flow/components/label_out.cpp.obj -c C:/Users/nicol/Documents/xiao-round-display/xiao-round-display.git/app/src/display/ui/eez-framework/src/eez/flow/components/label_out.cpp
In file included from C:/Users/nicol/Documents/xiao-round-display/xiao-round-display.git/app/src/display/ui/eez-framework/src/eez/flow/components/label_out.cpp:15:
c:\users\nicol\documents\xiao-round-display\xiao-round-display.git\app\src\display\ui\eez-framework\src\eez\flow\hooks.h:21:10: fatal error: lvgl.h: No such file or directory
   21 | #include "lvgl.h"
      |          ^~~~~~~~
compilation terminated.
[4/637] Building CXX object src/display/ui/eez-framework/CMakeFiles/eez-framework.dir/src/eez/core/eeprom.cpp.obj
[5/637] Building CXX object src/display/ui/eez-framework/CMakeFiles/eez-framework.dir/src/eez/flow/components/line_chart_widget.cpp.obj
[6/637] Generating include/generated/app_version.h
[7/637] Generating ../../zephyr/include/generated/ncs_version.h
[8/637] Building CXX object src/display/ui/eez-framework/CMakeFiles/eez-framework.dir/src/eez/flow/components/log.cpp.obj
FAILED: src/display/ui/eez-framework/CMakeFiles/eez-framework.dir/src/eez/flow/components/log.cpp.obj
C:\ncs\toolchains\cf2149caf2\opt\zephyr-sdk\arm-zephyr-eabi\bin\arm-zephyr-eabi-g++.exe -DEEZ_FOR_LVGL -DLV_LVGL_H_INCLUDE_SIMPLE -isystem C:/Users/nicol/Documents/xiao-round-display/xiao-round-display.git/app/src/display/ui/eez-framework/./src -isystem C:/Users/nicol/Documents/xiao-round-display/xiao-round-display.git/app/src/display/ui/eez-framework/./src/eez/libs/agg  -MD -MT src/display/ui/eez-framework/CMakeFiles/eez-framework.dir/src/eez/flow/components/log.cpp.obj -MF src\display\ui\eez-framework\CMakeFiles\eez-framework.dir\src\eez\flow\components\log.cpp.obj.d -o src/display/ui/eez-framework/CMakeFiles/eez-framework.dir/src/eez/flow/components/log.cpp.obj -c C:/Users/nicol/Documents/xiao-round-display/xiao-round-display.git/app/src/display/ui/eez-framework/src/eez/flow/components/log.cpp
In file included from c:\users\nicol\documents\xiao-round-display\xiao-round-display.git\app\src\display\ui\eez-framework\src\eez\core\value.h:14,
                 from c:\users\nicol\documents\xiao-round-display\xiao-round-display.git\app\src\display\ui\eez-framework\src\eez\core\assets.h:16,
                 from c:\users\nicol\documents\xiao-round-display\xiao-round-display.git\app\src\display\ui\eez-framework\src\eez\flow\private.h:14,
                 from c:\users\nicol\documents\xiao-round-display\xiao-round-display.git\app\src\display\ui\eez-framework\src\eez\flow\components.h:13,
                 from C:/Users/nicol/Documents/xiao-round-display/xiao-round-display.git/app/src/display/ui/eez-framework/src/eez/flow/components/log.cpp:15:
c:\users\nicol\documents\xiao-round-display\xiao-round-display.git\app\src\display\ui\eez-framework\src\eez\core\value_types.h:57:9: error: expected identifier before ',' token
   57 |         VALUE_TYPES
      |         ^~~~~~~~~~~
[9/637] Building CXX object src/display/ui/eez-framework/CMakeFiles/eez-framework.dir/src/eez/flow/components/is_true.cpp.obj
FAILED: src/display/ui/eez-framework/CMakeFiles/eez-framework.dir/src/eez/flow/components/is_true.cpp.obj
C:\ncs\toolchains\cf2149caf2\opt\zephyr-sdk\arm-zephyr-eabi\bin\arm-zephyr-eabi-g++.exe -DEEZ_FOR_LVGL -DLV_LVGL_H_INCLUDE_SIMPLE -isystem C:/Users/nicol/Documents/xiao-round-display/xiao-round-display.git/app/src/display/ui/eez-framework/./src -isystem C:/Users/nicol/Documents/xiao-round-display/xiao-round-display.git/app/src/display/ui/eez-framework/./src/eez/libs/agg  -MD -MT src/display/ui/eez-framework/CMakeFiles/eez-framework.dir/src/eez/flow/components/is_true.cpp.obj -MF src\display\ui\eez-framework\CMakeFiles\eez-framework.dir\src\eez\flow\components\is_true.cpp.obj.d -o src/display/ui/eez-framework/CMakeFiles/eez-framework.dir/src/eez/flow/components/is_true.cpp.obj -c C:/Users/nicol/Documents/xiao-round-display/xiao-round-display.git/app/src/display/ui/eez-framework/src/eez/flow/components/is_true.cpp
In file included from c:\users\nicol\documents\xiao-round-display\xiao-round-display.git\app\src\display\ui\eez-framework\src\eez\core\value.h:14,
                 from c:\users\nicol\documents\xiao-round-display\xiao-round-display.git\app\src\display\ui\eez-framework\src\eez\core\assets.h:16,
                 from c:\users\nicol\documents\xiao-round-display\xiao-round-display.git\app\src\display\ui\eez-framework\src\eez\flow\private.h:14,
                 from c:\users\nicol\documents\xiao-round-display\xiao-round-display.git\app\src\display\ui\eez-framework\src\eez\flow\components.h:13,
                 from C:/Users/nicol/Documents/xiao-round-display/xiao-round-display.git/app/src/display/ui/eez-framework/src/eez/flow/components/is_true.cpp:13:
c:\users\nicol\documents\xiao-round-display\xiao-round-display.git\app\src\display\ui\eez-framework\src\eez\core\value_types.h:57:9: error: expected identifier before ',' token
   57 |         VALUE_TYPES
      |         ^~~~~~~~~~~
[10/637] Generating misc/generated/syscalls.json, misc/generated/struct_tags.json
ninja: build stopped: subcommand failed.
FATAL ERROR: command exited with status 1: 'C:\ncs\toolchains\cf2149caf2\opt\bin\cmake.EXE' --build 'c:\Users\nicol\Documents\xiao-round-display\xiao-round-display.git\app\build_shield'

My relevant CmakeLists.txt file is available on my project repo : https://github.com/nicogou/xiao-round-display/blob/eez-ui/app/src/display/ui/CMakeLists.txt
I am using a Xiao BLE Sense board, with a Round Display for Xiao shield. Note that the display driver and shield are not available in current nrf Connect SDK version, but they are available on the main zephyr RTOS branch. I manually copied them to my project's Zephyr folder.
Let me know if you need any more information.
Thanks for your help!
Best regards,
Nicolas Goualard

To fix these errors:
error: expected identifier before ',' token
   57 |         VALUE_TYPES
      |         ^~~~~~~~~~~

pull the latest version of eez-framework which I just committed - there was a syntax error which I committed recently, you are the first one who noticed this, thanks!
As for these errors:
fatal error: lvgl.h: No such file or directory
   15 | #include "lvgl.h"
      |          ^~~~~~~~
compilation terminated.

you can try to fix it by adding include_directories(<path to lvgl directory>) (for example include_directories(lvgl), or wherever lvgl directory is in your case) line in your CMakeLists.txt. Or, maybe Zephyr has some dependency management setup, in which case you should setup that eez-framework depends on lvgl.

Thanks for the quick reply and fix!
I made the changes you suggested, it seemed to work! However it looks like I don't have enough flash on my device in order to run the code.
[634/639] Linking CXX executable zephyr\zephyr_pre0.elf
FAILED: zephyr/zephyr_pre0.elf zephyr/zephyr_pre0.map C:/Users/nicol/Documents/xiao-round-display/xiao-round-display.git/app/build_shield/zephyr/zephyr_pre0.map
cmd.exe /C "cd . && C:\ncs\toolchains\cf2149caf2\opt\zephyr-sdk\arm-zephyr-eabi\bin\arm-zephyr-eabi-g++.exe  -gdwarf-4 @CMakeFiles\zephyr_pre0.rsp -o zephyr\zephyr_pre0.elf  && cmd.exe /C "cd /D C:\Users\nicol\Documents\xiao-round-display\xiao-round-display.git\app\build_shield\zephyr && C:\ncs\toolchains\cf2149caf2\opt\bin\cmake.exe -E true""
c:/ncs/toolchains/cf2149caf2/opt/zephyr-sdk/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/../../../../arm-zephyr-eabi/bin/ld.bfd.exe: zephyr\zephyr_pre0.elf section `rodata' will not fit in region `FLASH'
c:/ncs/toolchains/cf2149caf2/opt/zephyr-sdk/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/../../../../arm-zephyr-eabi/bin/ld.bfd.exe: region `FLASH' overflowed by 48052 bytes
c:/ncs/toolchains/cf2149caf2/opt/zephyr-sdk/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/../../../../arm-zephyr-eabi/bin/ld.bfd.exe: src/display/ui/eez-framework/libeez-framework.a(value.cpp.obj): in function `eez::getVar(short)':
value.cpp:(.text+0x46f0): undefined reference to `native_vars'
c:/ncs/toolchains/cf2149caf2/opt/zephyr-sdk/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/../../../../arm-zephyr-eabi/bin/ld.bfd.exe: src/display/ui/eez-framework/libeez-framework.a(value.cpp.obj): in function `eez::setVar(short, eez::Value const&)':
value.cpp:(.text+0x483c): undefined reference to `native_vars'
collect2.exe: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.
FATAL ERROR: command exited with status 1: 'C:\ncs\toolchains\cf2149caf2\opt\bin\cmake.EXE' --build 'c:\Users\nicol\Documents\xiao-round-display\xiao-round-display.git\app\build_shield'

I'll see if anything can be done about that, and if not I'll try to implement the UI without EEZ flows and still achieve what I want to do.

Oh I forgot to mention one thing though: there was an issue with the loadScreen function being declared twice, first in the ui.h file generated by EEZ Studio, and also in the lvgl_api.h file in eez-framework\src\eez\flow.
As these two were doing very similar things I removed the function from the generated ui.h file in order to keep compilation going, but I thought I'd mention it here.

Oh I forgot to mention one thing though: there was an issue with the loadScreen function being declared twice, first in the ui.h file generated by EEZ Studio, and also in the lvgl_api.h file in eez-framework\src\eez\flow.
As these two were doing very similar things I removed the function from the generated ui.h file in order to keep compilation going, but I thought I'd mention it here.

This shouldn't be issue anymore as all LVGL templates (https://github.com/eez-open/eez-project-templates) and examples (https://github.com/eez-open/eez-project-examples) are recently corrected.

@nicogou did you find a solution for the native_vars issue? I'm having the same. https://github.com/eez-open/eez-framework/issues/4


feat: reconnect immediately on first retry

Fixes #297
Is this what we want here @rolznz?

@im-adithya please re-read the issue and think through it. If you don't understand it, please do ask before writing code

@rolznz can you re-review?

@im-adithya can you address the feedback?

Ready now @rolznz!


Handle multiple GPUs, avoid NVCC/GCC compiler warnings

Descriptors now create a moderngpu context on the current device, instead of always using device 0.
Added GrB_NOT_IMPLEMENTED and added a number of missing return statements.

Thanks for the pull request!
Be aware that this will still results in a double cudaFree when a + b are aliased to A + B and their underlying GPU memory is freed, so I've added an issue to track the fix for that (#11).


Reduce Pokémon accelleration on indeterminate progress bar

Current behaviour (optional)
When the progress bar is in indeterminate mode, the position is updated using a velocity. We accelerate (increase velocity) by 1 every time it is repainted.
Description of the desired feature (required)
I think this is too fast of an acceleration, as the Pokémon seems to really zoom across the bar sometimes when it is smaller. However, if we don't accelerate at all, it takes a really long time to get across on larger progress bars. We should reduce the acceleration.
Why this feature would be valuable (optional)
How to implement this feature (optional)
We could change the velocity to a double, and increment by 0.5 (or some other number, dependent on how it looks) instead of by 1. Then when we add it to the pos, we just take the floor value. We should introduce constants for initialVelocity and acceleration to make it easier to tweak these values.
Additional information or context (optional)
See PokemonProgressBarUi:207.

Hey can i work on this issue, also i see the label "Hacktoberfest" so does this mean i can work on this issue from 1st october onwards?

Hi @VaibhaveS, you're very welcome to work on this issue! I will assign it to you. You can work on it whenever - feel free to start it now, or if you want to hold off until October so you get the Hacktoberfest, that's absolutely fine too.
I think the ticket here has a decent amount of  information in it, but don't hesitate to ask for guidance if you need it!

Hi @VaibhaveS have you had any luck with this? :)

@VaibhaveS I've unassigned you and assigned myself, however if you'd still like to work on this feel free to let me know and I'll reassign you :)


could not find function cnn_train_dag.m

When I run cnn_ucf101_fusion(), I got a warning "Warning: Could not find appropriate function on path loading function handle /home/nvidia/Dropbox/mybox/PhD/ActionRecognition/THUMOS14/action_devkits/cnn_train_dag.m>extractStats" when loading the pretrained models. I also got an error "Undefined function or variable 'cnn_train_dag'" when running the last sentence "[info] = cnn_train_dag(net, imdb, fn, opts.train) ;". It seems that there is a missing function "cnn_train_dag.m". I used the default function "cnn_train_dag.m" of matConvNet but it still didn't work. Could you update the function "cnn_train_dag.m" or tell me how to solve this error. Thanks a lot.

I've added that function - thanks!


Feature Request: please make the actual hosts file configurable

It would be great to have a way to configure the path of the actual hosts file that this plugin writes to.
For example it is much more comfortable (and works without sudo) to configure dnsmasq to read an additional file from a users directory which contains additional host definitions.
I am sure there are a few other ways to implement what /etc/hosts does without leaving user space, so it would be a good addition to this plugin to make the destination file configurable.
Thanks for your attention!

Pull requests welcomed! Thanks for the idea :+1:

I will have this added to the next release @COLABORATI


Build failure: keychain on 23.11 unstable

Steps To Reproduce
Steps to reproduce the behavior:

build a 23.11 unstable derivation (flake-enabled in configuration.nix, but not using any flakes yet)
nix-env -iA nixos.keychain

Edit: I updated both personal and root nix-channel to nixos-unstable and rebuilt a new derivation with keychain included in configuration.nix, and that completes without error.
Build log
> nix-env -iA nixos.keychain
warning: name collision in input Nix expressions, skipping '/home/bgibson/.nix-defexpr/channels_root/nixos'
suggestion: remove 'nixos' from either the root channels or the user channels
installing 'keychain-2.8.5'
these 4 paths will be fetched (1.86 MiB download, 10.93 MiB unpacked):
  /nix/store/4qrbfa42xyicny7vlgbbqxz07kcsjbx2-findutils-4.9.0
  /nix/store/r2p66d3sh6syxiq7gxv8l4h7d2y733jr-keychain-2.8.5
  /nix/store/s1xnsm86dy195fr4i4n984ivakrvjv9d-openssh-9.3p1
  /nix/store/pi05gf5p4p1p1j41p12sha27zgjfs767-procps-3.3.17
copying path '/nix/store/4qrbfa42xyicny7vlgbbqxz07kcsjbx2-findutils-4.9.0' from 'https://cache.nixos.org'...
copying path '/nix/store/s1xnsm86dy195fr4i4n984ivakrvjv9d-openssh-9.3p1' from 'https://cache.nixos.org'...
copying path '/nix/store/pi05gf5p4p1p1j41p12sha27zgjfs767-procps-3.3.17' from 'https://cache.nixos.org'...
copying path '/nix/store/r2p66d3sh6syxiq7gxv8l4h7d2y733jr-keychain-2.8.5' from 'https://cache.nixos.org'...
error: this derivation has bad 'meta.outputsToInstall'

Additional context
Installing in nix-shell works:
>   nix-shell -p keychain

this path will be fetched (0.02 MiB download, 0.06 MiB unpacked):
  /nix/store/vl6w8gzd2q1fvbkch9wpnjwwd370vq64-keychain-2.8.5
copying path '/nix/store/vl6w8gzd2q1fvbkch9wpnjwwd370vq64-keychain-2.8.5' from 'https://cache.nixos.org'...

[nix-shell:~]$ keychain --help

 * keychain 2.8.5 ~ http://www.funtoo.org

Notify maintainers
@SuperSandro2000
(@sigma listed on search.nixos.org as maintainer, but seems inactive on this repo)
Metadata
Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.
> nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 6.3.13, NixOS, 23.11 (Tapir), 23.11pre507929.c9cf0708f00`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.15.1`
 - channels(root): `"nixos"`
 - channels(bgibson): `"nixos-23.05"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

Not sure why root channel is showing nixos, but...
> nix-channel --list
nixos https://channels.nixos.org/nixos-23.05

> sudo nix-channel --list
[sudo] password:
nixos https://nixos.org/channels/nixos-unstable

Solution: Building a new derivation with both user and system nix-channels set to nixos-unstable and keychain included in systempackages works, think I just had my nix-channels configured incorrectly.


Expand getCoordinates of newClass to access arguments

What problem are you trying to solve?
I would like to switch one constructor call for an other with JavaTemplate. E.g. if a class A offers two constructors A(int foo), A(int foo, double ba) I would like to be able to insert a new argument after, or replace the list of arguments altogether.
Describe the solution you'd like
API offers suitable methods through getCoordinates.
Have you considered any alternatives or workarounds?
Replacing the entire constructor call should work in theory.
withArguments() works as well, but JavaTemplates seem to be the way to go.
Additional context
Currently a constructor call can be replaced entirely, but it would be nice to be able to operate it without putting "new A" into the template, which requires me to find out the class name at runtime.
J.MethodInvocation already has getCoordinates().replaceArguments(), that could just be implemented in newClass as well.

Are you interested in contributing this feature to OpenRewrite?

probably won't find the time

Thanks for the suggestion @timo-abele ; I imagine this to be very similar to how we handle J.MethodInvocation, which I think should make for a nice addition.
https://github.com/openrewrite/rewrite/blob/9bcac5a236978eaa1235b55defa2de896309e08f/rewrite-java/src/main/java/org/openrewrite/java/tree/CoordinateBuilder.java#L232-L247
Which would then need to be returned instead of CoordinateBuilder.Statement in
https://github.com/openrewrite/rewrite/blob/516c70b89ebbb43803d2dd6e0bc2cf31307a80d3/rewrite-java/src/main/java/org/openrewrite/java/tree/J.java#L4454-L4456


Enabled invalidation duration log in python

Enable invalidation log inside the invalidation service plpython trigger. The logging is done through syslog. It's a more natural way to do it if postgresql is already configured to log through syslog.
Note that the format is just a string representation of a python dict. Not sure how expensive can be converting a dict to json to string every time a invalidation occurs.
I've taken the chance to write some more info that may help to debug in case of problems like the number of retries and error string.
/cc @rochoa @javisantana @rafatower @zenitraM

Would it make sense to make this configurable? so by default syslog logging is disabled

Agree. That's a nice addition


mask out tests that are failing on Linux

Bug/issue #, if applicable: None
Summary
This PR adds a couple #if os(macOS) wrappers to some tests that are currently failing on Linux. Some offline discussion pointed to https://github.com/apple/swift-corelibs-foundation/issues/5028 as a cause for these failures, so until that is fixed and available in a nightly toolchain, we need to keep these tests from running.
Dependencies
None
Testing
Test-only change
Checklist
Make sure you check off the following items. If they cannot be completed, provide a reason.

[ n/a ] Added tests
[x] Ran the ./bin/test script and it succeeded
[x] Updated documentation if necessary

@swift-ci Please test


NeoStickyModal to tailwindcss

Thank you for your contribution to the KodaDot - One Stop Shop for Polkadot NFTs.
👇 __ Let's make a quick check before the contribution.
PR Type

[ ] Bugfix
[ ] Feature
[x] Refactoring

Context

[x] Closes #8222
[ ] Requires deployment <snek/rubick/worker>

Before submitting pull request, please make sure:

[x] My contribution builds clean without any errors or warnings
[x] I've merged recent default branch -- main and I've no conflicts
[x] I've tried to respect high code quality standards
[x] I've didn't break any original functionality

Optional

[x] I've tested it at </ksm/collection>
[x] I've tested PR on mobile
[ ] I've written unit tests 🧪
[ ] I've found edge cases

Did your issue had any of the "$" label on it?

[x] Fill up your DOT address: Payout

Community participation

[x] Are you at KodaDot Ecosystem Telegram?

Screenshot 📸

[ ] My fix has changed something on UI; a screenshot is best to understand changes for others.

Copilot Summary

🤖[deprecated] Generated by Copilot at 7d24412
Refactored the NeoStickyModal component to use Tailwind CSS and the useWindowSize hook. Simplified the SCSS file and updated the Tailwind CSS config file with custom padding values.

🤖[deprecated] Generated by Copilot at 7d24412

NeoStickyModal
Simplified with Tailwind
No more Bulma, ah!

almost identical

it is 10px short compare to canary

apart from what daiagi pointed looks good


runtime: TestStackBarrierProfiling failing on ppc64, ppc64le and mips64

On linux/mips64:
panic: fatal error: mallocgc called with gcphase == _GCmarktermination
http://build.golang.org/log/2f49c7ce157ca522bfd747254573ce47c7a6564c
On linux/ppc64 and linux/ppc64le:
--- FAIL: TestStackBarrierProfiling (0.61s)
	pprof_test.go:338: subprocess failed with signal: segmentation fault:
FAIL

http://build.golang.org/log/af07ab5aa4c14e100952d98de41012f28a302b00
http://build.golang.org/log/d8c0744f5e27b972a383bb3dae01834cb2d601ce
/cc: @aclements

I'm working on getting access to a ppc64 box I can debug this on (the one I had access to went away).
My completely unsubstantiated guess is that GODEBUG=gcstackbarrierall=1 just doesn't work on ppc64. It chews up a lot of stack space and it may just chew up too much on ppc64.

It looks like my guess was right, so I'll probably just disable this test on ppc64 (it's not worth making gcstackbarrierall work on ppc64).
The mips64 failure looks like the test is catching a genuine bug.

624d798 fixed the ppc64 bug and the mips64 bug is a dup of #12528.


[Feature] - Evaluate the new Nova theme when updating Primereact to version 8

Is your feature request related to a problem? Please describe.
When updating Primereact to version 8, the  nova-dark theme that we used, would be deprectated.
Describe the solution you'd like
We will try to evaluate the free available Prime NG themes that could replace nova-dark in the matter of usability and output.
Describe alternatives you've considered
If the free new themes are not sufficient, we would consider purschasing the premium Prime NG which contains premium themes similar to nova-dark.

It seem like the nova-alt theme could replace the nova-dark. nova-alt has closely similar output if not the same.
this is a comparison of the UI after updating the UI from PrimeReact 4 to PrimeReact 8:
*** The configuration Page:**

version 4 (nova-dark):

-version 8 (nova-alt):

*** Aggent Mapping Page:**

version 4:

- version 8:
-

*** Aggent Status Page:**
- version 4:

- version 8:
-

*** Configuration Promotion Page:**
- version 4:
-

- version 8:
-

PS: the change in the icon output is NOT due to the  Primereact update but its due to the Primeicons update.


Add "email" and "government" supertypes

These replace the "whitehall" super type and will be
used for emailing changes to whitehall content.

Looks good! Can you split the commits into one with the config change and another one for with the schema regeneration?

@tijmenb done


[BUG] When clicking the offline and online collector button, the pop-up window is abnormal.

Is there an existing issue for this?

[X] I have searched the existing issues

Current Behavior
No response
Expected Behavior
No response
Steps To Reproduce
master分支：

查看代码是前端调用这两个接口的时候多了个/

移除这两个下划线接口成功调用
Environment
HertzBeat version(s): master

Debug logs

Anything else?
No response

hi, can you submit a pr to fix it?


[PT Run][Wox.Plugin.Common] Create VirtualDesktopHelper

At the moment I am working on a VirtualDesktopHelper class which will be available via Wox.Plugin.Common.
You can find an early proof of concept here.
This helper class enables us to fix/implement the following issues: #15887, #1987
Technical features
It will provide all necessary informations and functions to work with VDs. (Including methods to get a list if desktops, the desktop name, the desktop order, moving windows, detecting the desktop a window is assigned to and more.)
Remarks
As we can only use public apis/Interfaces I only use the official VirtualDesktopManager interface and some informations from Registry.

Work is in progres!

Thanks, @franky920920 .

This is fixed with 0.57.0. Please head over to https://aka.ms/installpowertoys


manifests cannot be opened in SAS

our manifests (such as https://iiifpres.bdrc.io/wvo:bdr:MW2CZ7888_B1B2C6::bdr:I1KG2937/manifest ) cannot be opened in Simple Annotation Server (SAS), producing the following error:

thank you for looking into the issue! It might well be a bug. Once I have a little less on my plate, I will try different annotation storages and report back.


"Winget is not installed"

Describe the bug
I think this may be a bug? It only started happening after I removed all onedrive folders and renamed my user folder in Windows 11. Problem persisted after redownloading winutil again. Every time I click "Get Installed" it says "winget is not installed", but it is installed. I even went to the config tab and clicked "reinstall winget". I fully let it reinstall, but get the same response when clicking "Get Installed" on the install tab.
Included screenshot of the 'error'. I'm not sure it's actually a bug or a result of what I changed, but I have no idea why it persists, or how to remedy it. First time making a report here, apologies if anything is wrong.
Screenshots

Could you please run the command "winget --version" in the Win11 Terminal or Powershell and comment the output of the command so we can see which winget version (if any) is installed on your system :)

Could you please run the command "winget --version" in the Win11 Terminal or Powershell and comment the output of the command so we can see which winget version (if any) is installed on your system :)

So I ran a few winget commands even before I made this post, and unfortunately I guess my problem must be a much larger issue. I ran it again for the screenshot. I also downloaded the winget installer from the Microsoft Github, and took a screenshot of it saying I have it installed already. I'm beginning to believe this might just be a corrupted user profile? I guess it's not a winutil issue.

Yes having the same issue I installed via the microsoft store then check in powershell winget --version and the tool started to install perfectly but something is definitely up with the tool on how its grabbing winget and think choco install winget is not working problem either

To update anyone viewing this in the future, I'm pretty sure I fixed this issue, and I suppose it wasn't a winutil issue after all.
All I did was reset and restart windows indexing. It took waiting for a full index to recomplete, but now the winget commands are working, as is the "get installed" function in winutil. Closing the issue.


How to make thumbnails generate automatically?

How to make thumbnails of the size I need generated automatically when I upload an image file?

You can use the ImageResize addon or the ImageStyles addon.


Fixed Markercolor Not Working in Torch7

Fixed a bug with the markercolor property of any plot that prevented the
plot from properly displaying the colors that were supplied via Torch7.  This is
due to a difference between Python and Lua string formatting.
A simple test can be run to show this erroneous behavior:
print(string.format("#%x%x%x",5,3,0))

produces:
#530

which is not the correct color specified, whereas:
print(string.format("#%02x%02x%02x",5,3,0))

produces the correctly formatted color:
#050300

Good catch, thanks for fixing this!


chore(main): release 1.5.0

:robot: I have created a release beep boop
1.5.0 (2024-05-22)
Features

将扩展函数移至DartUtil (16ef3fe)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/luoyi58624/luoyi_dart_base/releases/tag/v1.5.0 :sunflower:


Falcosidekick does not send events to WebUI

Describe the bug
The chart is deployed successfully, the discord events are properly fired, but not the WebUI ones.
How to reproduce it
My helmfile:
repositories:
  - name: falcosecurity
    url: https://falcosecurity.github.io/charts

releases:
  - name: falco
    chart: falcosecurity/falco
    namespace: falco
    version: 3.8.5
    values:
      - driver:
          kind: "ebpf"
      - falcosidekick:
          enabled: "true"
          config:
            existingSecret: "falcosidekick-config"
            discord:
              minimumpriority: "notice"
          webui:
            enabled: "true"
            replicaCount: 1
            disableauth: "true"

Expected behaviour
Looking at the logs, I only see Discord events:
2023/11/28 10:38:36 [INFO]  : Falco Sidekick version: 2.28.0
2023/11/28 10:38:36 [INFO]  : Enabled Outputs : [Discord]
2023/11/28 10:38:36 [WARN] : tlsserver.deploy is false but tlsserver.notlspaths is not empty, change tlsserver.deploy to true to deploy two servers
2023/11/28 10:38:36 [INFO] : Falco Sidekick is up and listening on :2801
2023/11/28 10:41:51 [INFO]  : Discord - Post OK (204)
2023/11/28 10:51:51 [INFO]  : Discord - Post OK (204)

But I would have expected WebUI events too.
Screenshots

Environment

Falco version:

Tue Nov 28 10:57:30 2023: Falco version: 0.36.2 (x86_64)
Tue Nov 28 10:57:30 2023: Falco initialized with configuration file: /etc/falco/falco.yaml
{"default_driver_version":"6.0.1+driver","driver_api_version":"5.0.0","driver_schema_version":"2.0.0","engine_version":"26","falco_version":"0.36.2","libs_version":"0.13.4","plugin_api_version":"3.1.0"}

System info:

Tue Nov 28 10:57:57 2023: Falco version: 0.36.2 (x86_64)
Tue Nov 28 10:57:57 2023: Falco initialized with configuration file: /etc/falco/falco.yaml
Tue Nov 28 10:57:57 2023: Loading rules from file /etc/falco/falco_rules.yaml
{
  "machine": "x86_64",
  "nodename": "falco-zl4nw",
  "release": "5.15.0-84-generic",
  "sysname": "Linux",
  "version": "#93-Ubuntu SMP Tue Sep 5 17:16:10 UTC 2023"
}

Cloud provider or hardware configuration:
OS:
OVH Managed K8S
Kernel:
Linux falco-zl4nw 5.15.0-84-generic #93-Ubuntu SMP Tue Sep 5 17:16:10 UTC 2023 x86_64 GNU/Linux
Installation method:
helmfile

Additional context

Can you check if the redis pod is up and running please?

❯ kubectl get pods -n falco
NAME                                     READY   STATUS    RESTARTS        AGE
falco-falcosidekick-5954875598-dmkwm     1/1     Running   0               3h22m
falco-falcosidekick-5954875598-tdp2x     1/1     Running   0               3h22m
falco-falcosidekick-ui-9896c86df-tbrlz   1/1     Running   3 (3h22m ago)   3h22m
falco-falcosidekick-ui-redis-0           1/1     Running   0               3h22m
falco-zl4nw                              2/2     Running   0               3h22m

I was curious about the 3 restarts for falco-falcosidekick-ui-9896c86df-tbrlz:
2023/11/28 10:38:55 [WARN] : Index does not exist
2023/11/28 10:38:55 [WARN] : Create Index
2023/11/28 10:38:55 dial tcp <IP_ADDRESS>:6379: connect: connection refused

Looks like a transient error.

The restarts of the UI at the bootstrap are "normal", I will fix that with an initContainer asap.
In your case, I don't understand how you can have the deployment of the UI but not the enabling in Falcosidekick.
Can you check the secrets for Falcosidekick to see if the webui output is correctly set please.

There's no secret, and apparently that's because I use existingSecret: "falcosidekick-config" to set the discord webhook url.
Removing that line resolve the issue.
Is there a way to set the webhook url through a secret without overriding the generated secret? Thanks.

If you use your own secret file you have to set up the settings for the Webui too, not just Discord. The existingSecret overrides everything, can't mix for now.

Ok thanks, I'll copy paste the generated one in my secret, and I'll freeze the chart updates until we can mix configs.

Ok thanks, I'll copy paste the generated one in my secret, and I'll freeze the chart updates until we can mix configs.

I'm adding that in my todo then.

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/falcosecurity/community.
/lifecycle stale

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/falcosecurity/community.
/lifecycle rotten

/remove-lifecycle rotten

What are the variables I need to include in my own secret/env vars so the events make it to the UI?
Im doing the same thing for a slack webhook url (using existingSecret) and trying to pass in:
falcosidekick:
  enabled: true
  environment:
    - WEBUI_URL=http://myaddress:2802
    - REDIS_DATABASE=0
    - REDIS_KEY=falco
    - REDIS_STORAGETYPE=list

But no dice. A lot of the other vars dont seem relevant.

/assign @Issif

I explained here how to use the existingSecret setting


ObjectSelector's allow_None=None

I assume this refers to allow_None needing to be validated against the allowable objects in ObjectSelector.

Closing for lack of description.


Scanner is 'slow' at Android devices

I implemented the barcode scanner in my Ionic 6 app. The scanner works very well at iOs devices, but on multiple Android devices it doesn't. Scanning a QR code takes very long because you have to find a certain way to hold the device, despite the QR code is in the middle of the screen. Other QR scan apps from the Play store working very well and fast.. I have added QR_CODE as target format.
I'm stuck in this, do you maybe have a solution? Thanks in advance!

Hmm, I never heard about performance issues on QR codes so far... The android version is using zxing as a lot of other projects and at least for 2D codes this usually works well.
Do you have a repo to reproduce this?

Thanks for your fast reply! In the meantime I know what the problem is.
In my design the camera-view is at about 25% from the top. Not vertically aligned in the center. This is the design:

So when I point the camera to the QR code, the QR code is not in the scanning area. Is it possible to move the scanning area upwards?

This should technically not be an issue for the scanning itself, the QR code (other than 1D codes) should be readable in the complete viewport (which is by designs, since always the complete viewport is used). The only issue that might occur is that you are currently not able to specify the area for the scan, that means that a code could also be read if it is outside of your rectangle, for example under the "Fill code manually" button. If I rebuild a super simple example of your UI I am able to scan a code.
There is a PR open to achieve custom scan areas, unfortunately only for android at the moment and in my opinion we should only release features that are available on all devices (as far as possible at least). This is why I am trying to move this repo to mlkit which would allow us to use the "same" components and achieve the more or less same results on all devices.

Wow oke.. Strange. I created a new app with the camera in fullscreen view and it does the same behavior. So even in the top of the view the QR code is already visible, but it doesn't scan. When I move the phone to center the QR code in the view, it works..
The devices are a OnePlus 8 Pro and a Samsung A40.

Can you share the newly created app with me to have a look?

Sure, here it is.

Okay, I can see your issue now... The problem really seems to be related to the scanning area, if you are only working on Android you should be able to work around it with #140 .
As I previously said, this is not merged since the behaviour should be more or less the same on Android and iOS IMHO. It looks like this can be done with rectOfInterest in AVFoundation, but unfortunately I have currently no time to have a detailed look at this one.

You are right, the behaviour should be the same at both platforms. So it's fine by me to have a fullscreen scanning area, because that's already the case on iOS. But how is that possible on Android? Because in my new app the scanning area is a little square in the center of the screen. Or we don't fully understand each other :)

Hi @thegnuu , do you maybe have a solution? Or is it necessary to do a redesign (center the scanning area in the viewport)?
Thanks in advance.

@died89 unfortunately I have no working solution at the moment, one solution is only working on android and the other is only working on iOS so far. I have no time to work on either one in the upcoming weeks because of other projects, so the easiest way to get a working solution fast would be to use the center of the area. I am working on it, but unfortunately my Android development know-how is not that great and it might take me a while to get the newest ml-kit version far enough for a beta release...

Allright, I understand it! Thank you for your time and your honest answer. Closing this issue for now.

Hi, I have the same performance problems on an Android device when I scan QR-codes (2D DATA_MATRIX). Is there any progress on this issue?