Review comment:  Extension Inflation

This may be late to (re)consider, but it seems pretty artificial to consider this as five independent extensions.
We've already seen that silicon vendors will take advantage of any possible flexibility. It's easy to argue that's a feature, not a bug, but it means that operating systems, in particular, will have to deal with the possibility of there being not one extension, but 2^5 extensions to deal with every possible (and often nonsensical) combination of having and not having each of these. It seems the implementation cost of each of these is likely to be low for any chip large enough to support any of these. Having Smnpm without Sspm, as an example, but the fact that it's possible seems to make it a near certainty that someone, somewhere will build such a thing.
RISC-V is already suffering from the F word as implementations choose from the smorgasboard of scores of extensions.  Please don't contribute further to this. Consider making this one extension that says, for example, that bits 33:32 of m*cfg are reserved/undefined even if this extension isn't present or that they must be written as 0's even if these extensions aren't present/wanted. It has no practical cost (hopefully existing implementations aren't setting reserved bits) and it would allow treating this whole collection as one "pointer masking" extension instead of defining an extension for the presence or absence of them.
Note that my examples may be tripped up in the phrasing. Please don't fixate on them, but instead register my plea that we reduce the cognitive load of systems software/tool developers and prefer snowflakes of extensions over avalanches of individual, super light-weight "extensions' that may or may not be present. (This plea generally applies to all extensions...)
Thank you for listening.

The decision to split the decision was made by the overarching RISC-V architecture review committee. The reason for these different extensions is to bring them into a format that they can be included in the standardized profiles, which are intended to avoid the fragmentation problem that you mention:
https://github.com/riscv/riscv-profiles/blob/main/rva23-profile.adoc
While the specification defines 5 extensions, the way they are used in the profile ensures that the only optionality is whether or not to support S-mode pointer masking (Smnpm). M-mode pointer masking (Smmpm) is outside the scope of the profile since M-mode is under control of vendors and thus should be entirely separate. Supm and Sspm are "pseudo-extensions" needed to refer to functionality that crosses the privilege mode boundary within profiles; they have no functionality attached nor could anyone implement them directly.

This issue has been addressed in 654a5c4. We added language to the spec to explain the rationale behind these extensions, and why they are necessary.


I want to become a software developer using C++ , I just need some advice from experts. Thanks

Hi,
My name is Rezwan Rafid. I am 17 years old now. I have a dream of becoming a Software Developer. So, I choose C++ to start. I started learning to code, but for some reason, I felt I was lagging. I thought only using C++ I can't build good quality user-friendly softs. So, I am in darkness right now. If any expert can give any tips.

Hi @yesitsrafid
I don't think awesome-compose is the right place for your question
Start to learn algorithmic first on Mooc platform for example


Move reboot_pending? method to windows_helper

Move method reboot_pending? from windows_reboot_handler to windows_helper.
This allows thirdparty providers to use the detection code.
I also added another check to the reboot_pending? method to handle CBS required reboot.

I think you are right @btm, the Chef DSL is way better, so I don't need to move the method out of the Handler anymore.
Therefore I created a new pull request #236  to fix some bugs and use the DSL when possible.


Fix Markdown link normalization to suit to Marpit color shorthand

Improvement of VS Code engine compatibility (#27) causes regression of Marpit color shorthand. We are fixed this by checking color string in normalizeLink method.
Fixes #40.

This bug may have to fix in Marpit, not Marp for VS Code. Essentially the link normalization can customize in markdown-it, and Marpit's image parse logic should take care the side effect from customized method.

It will fix in marp-team/marpit#165.


Increase maintainability score & code coverage on CodeClimate

Based on the generated report by codeclimate, more details to be found here
Bring code base to:

Maintainability: A
Coverage >80%

Worth noting that CodeClimate estimates 3 weeks of full-time 1-person work to solve all issues for Maintenance score:

If we start fresh, we will start with all in the green from the start


AD: Stick addresses in addresses field

Related issue: https://github.com/forta-network/starter-kits/issues/372
Added all involved_addresses into finding's addresses property.
example finding:
{
  "name": "Attack detector identified an EOA with behavior consistent with an attack",
  "description": "0xd0ccf523d198a9e0ef6222ad3d3e3f1919898725 likely involved in an attack preparation (0xdd276b19b8c9242077077d6589556bc7989d2cc510b927346b951a02b0438694).  Anomaly score: 0.00010383680854515402",
  "alertId": "ATTACK-DETECTOR-PREPARATION",
  "protocol": "ethereum",
  "severity": "Critical",
  "type": "Exploit",
  "metadata": {
    "attacker_address": "0xd0ccf523d198a9e0ef6222ad3d3e3f1919898725",
    "anomaly_score_stage_Funding": 0.0009206863689960211,
    "anomaly_score_stage_Preparation": 0.11278195490000001,
    "anomaly_score": 0.00010383680854515402,
    "involved_addresses_1": "0xd0ccf523d198a9e0ef6222ad3d3e3f1919898725",
    "involved_addresses_2": "0x18d88d253ded3f2c52937a081536697697669025",
    "involved_addresses_3": "0x4e5b2e1dc63f6b91cb6cd759936495434c7e972f",
    "involved_addresses_4": "0x0a06aa7b55e1210793af3b2e89b205d1c8ebf054",
    "involved_addresses_5": "0xa0b86991c6218b36c1d19d4a2e9eb0ce3606eb48",
    "involved_addresses_6": "0xbebc44782c7db0a1a60cb6fe97d0b483032ff1c7",
    "involved_addresses_7": "0xdac17f958d2ee523a2206206994597c13d831ec7",
    "involved_alerts_0": "0xf496e3f522ec18ed9be97b815d94ef6a92215fc8e9a1a16338aee9603a5035fb,CEX-FUNDING-1,0xdd276b19b8c9242077077d6589556bc7989d2cc510b927346b951a02b0438694",
    "involved_alerts_1": "0x9aaa5cd64000e8ba4fa2718a467b90055b70815d60351914cc1cbe89fe1c404c,SUSPICIOUS-CONTRACT-CREATION,0x0e85926328d50d40b148cf7a0da37a433b066fb506d029b4e6d694dbc1907b25",
    "involved_alerts_2": "0x9aaa5cd64000e8ba4fa2718a467b90055b70815d60351914cc1cbe89fe1c404c,SUSPICIOUS-CONTRACT-CREATION,0xa8172b2071cf86dfe2a1b2a768717a5fd4226ddc74a354bd11c22f2ba1857edf",
    "involved_address_bloom_filter_0": "",
    "involved_address_bloom_filter_1": "",
    "involved_address_bloom_filter_2": "",
    "bot_source": "Forta Base Bots"
  },
  "addresses": [
    "0xd0ccf523d198a9e0ef6222ad3d3e3f1919898725",
    "0x18d88d253ded3f2c52937a081536697697669025",
    "0x4e5b2e1dc63f6b91cb6cd759936495434c7e972f",
    "0x0a06aa7b55e1210793af3b2e89b205d1c8ebf054",
    "0xa0b86991c6218b36c1d19d4a2e9eb0ce3606eb48",
    "0xbebc44782c7db0a1a60cb6fe97d0b483032ff1c7",
    "0xdac17f958d2ee523a2206206994597c13d831ec7"
  ],
  "labels": [
    {
      "entityType": "Address",
      "entity": "0xd0ccf523d198a9e0ef6222ad3d3e3f1919898725",
      "label": "attacker-eoa",
      "confidence": 0.2,
      "remove": false,
      "metadata": {
        "alert_id": "ATTACK-DETECTOR-PREPARATION",
        "chain_id": 1,
        "threat_description_url": "https://forta.org/attacks/"
      }
    },
    {
      "entityType": "Address",
      "entity": "0x18d88d253ded3f2c52937a081536697697669025",
      "label": "attacker-contract",
      "confidence": 0.2,
      "remove": false,
      "metadata": {
        "alert_ids": "ATTACK-DETECTOR-PREPARATION",
        "chain_id": 1,
        "threat_description_url": "https://forta.org/attacks/"
      }
    }
  ],
  "uniqueKey": "512b72b0e3e0643a5a1e4aeb76191886fc4ba469f959c0d3d75efbdd84954675",
  "source": {},
  "timestamp": "2023-11-29 17:11:22.021101"
}

pls put to AD_3_52_MEV_Consensus


There is not way to have a unique ID if the record is not persisted.

This can be problematic for lists needed keys.
For now a workaround could be using the object ID if the record is not persisted.

closed - replaced with: https://github.com/ruby-hyperloop/hyper-mesh/issues/61


Add linux build automation

Description

Add Dockerfile / circleci to build phantomjs
This will permit to get a linux binary and verify the compilation is working.

We can download the binary for each build https://circleci.com/gh/ifeelgoods/phantomjs/3#artifacts.

I wonder if CircleCI can handle our build. I see that they offer Build Minutes per Month 1,500. I don't think this is enough for PhantomJS.

Indeed this would lead to only 16 builds possible by month...

Can they be triggered manually? And can we do builds for all supported platforms?

To build for windows, we need dedicated CI like http://www.appveyor.com/ for OS X, travis could still handle it.
Here, the interesting thing is just the Dockerfile so anyone can build the image for linux.
I'm looking in travis if we could keep the binary available to download and build for Mac OS too.

For Windows we have a VM in Azure. But I really want to have a service that can handle all platforms.

@mtparet Do you mind bringing this discussion to phantomjs-dev mailing-list. We've been looking for a proper CI system for PhantomJS and so far any off-the-shelf systems did not fulfill our requirements.

I'm also working on a Docker-based build but it doesn't quite work yet, see zackw/phantomjs/docker.  I'm about to go out of town for Christmas and may not have time to do anything more with it till January.

@zackw @mtparet Let's continue the Docker-based build discussion on #13822.

@mtparet Never mind the mailing-list, let's continue the CI build system on #13828.


fix(polka): typo

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (937e867) 100.00% compared to head (1ed0d1c) 100.00%.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff            @@
##              main      #203   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files            4         4
  Lines          104       104
=========================================
  Hits           104       104

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Spike: Download PDF button

What is the context of this PR?
This spike was to investigate adding the functionality to download the view-submission page as a PDF using flask-weasyprint (which is already being used in v2).
Unfortunately there were issues rendering the PDF correctly using flask-weasyprint. Having done some investigation , although we were not able to find the direct cause, we found that there were issues relating to css media queries which are used frequently in our design system that could potentially be causing issues, see the issue here. flask-weasyprint also requires system libraries pango and libffi to be installed in order to be able to run, which would need to be installed with our docker image.
As a result for this Spike I looked at alternative libraries for the PDF download and was able to get the download working with the PDFKit library.
Caveats
Unfortunately this library is also dependant on system libraries being installed, in order to run the code you will need to install wkhtmltopdf with brew ( brew install wkhtmltopdf). This will need to be added to the dockerfile.
As there is currently a design system PR open to add a download button to the Design system the styling for the Download pdf button is incorrect.
As this is a spike, I have not yet added tests, and am not currently installing wkhtmltopdf in the dockerfile.
There is a warning for the PDFKit pypi package that states that Version in debian/ubuntu repos have reduced functionality, so we would need to test this to see if we are impacted.
How to review
To run locally, ensure that wkhtmltopdf is installed via brew install wkhtmltopdf, and then pipenv install --dev to install the PDFKit library,
Use test_view_submitted_response to see the added pdf download button and generate the download.
Is this library appropriate - should we investigate further alternatives?
Checklist

[ ] New static content marked up for translation
[ ] Newly defined schema content included in eq-translations repo

 Spike Flask-WeasyPrint (1 day)

Given the statement Version in debian/ubuntu repos have reduced functionality, the Dockerfile should be updated so that we can test the spike under those conditions.

The output locally looks differently to the output under our deployed image. The deployed image looks slight better and closer to the print stylesheet, however the style sheets needs tweaks as the cookie banner is rendered as well but should be hidden. The margin is also a bit narrow on the deployed version. Can we pass additional CSS or can it only be provided upfront or only read from a file?. Not sure if it is within the scope of the spike to look at those or just address them when implementing for real.
In general, the PDF created works mostly as expected, just a bit slow, unfortunately (I can see users clicking it multiple times due to the slowness). We should disable the logging outputs it generates.
As Mark surfaced, the docker image size does increase by over 50%, maybe we can look to slim down our image also.

The output locally looks differently to the output under our deployed image

@MebinAbraham do you mean running Runner (e.g pipenv run) rather than a Docker container running locally?

The output locally looks differently to the output under our deployed image

@MebinAbraham do you mean running Runner (e.g pipenv run) rather than a Docker container running locally?

Ya, i.e. diff between MacOS vs Debian.

The output locally looks differently to the output under our deployed image

@MebinAbraham do you mean running Runner (e.g pipenv run) rather than a Docker container running locally?

I've added example outputs to the trello card now


[WIP] Testing Generic Linux Job GPU

Failure is as of the commit "change job name" (https://github.com/pytorch/torchrec/pull/796/commits/a1f6fb4680ac33cdb01911038869caa2b6d1d505). Jobs from that commit are here: https://github.com/pytorch/torchrec/actions/runs/3415494661/jobs/5684821612.

This was just for testing purposes. Closing.


Add Config command

The logchimp config command will be generate the logchimp configuration file just by passing the flags.

Closed by #16 and https://github.com/logchimp/logchimp-cli/commit/dfae5b3b72876f1b5d6ea9ff09d2d05bb416fe9d


Add mar_action to the Medication admin tables

Agree; would advocate for mapping to common categories ('stopped', 'hold', 'dose change', 'start')?

we need mar_action_name and mar_action_category. @rbaccile

@08wparker @kaveriC I know we have our gsheet CLIF2.0 with the changes to be made... are things like this being constantly added (or for example when we decide to change the discharge_category so that people making CLIF 2.0 changes can go back to it.  Similarly... it would be nice to have a place for sites to check off when they have done something.
Essentially you need a running to do list and once an "issue" is resolved (if it pertains to CLIF_2.0 framework), the person resolves that issue should immediately ADD a new line to this "running task list" so that everyone can keep track of what they need to do next.
It can be that first tab on the CLIF2.0 sheet but we need think about adding check boxes or something.     If its only me that is having trouble tracking things then please ignore this :)

@ingra107 If this is required for the manuscript, and not for the ATS submission, we can handle it Post-ATS


@ missing in AccessDenied page

Original issue: https://github.com/aspnet/Identity/issues/1511

dotnet new mvc --auth Individual -o Test
Views/Account/AccessDenied.cshtml

@{
 ViewData["Title"] = "Access denied";
}

ViewData["Title"]

You do not have access to this resource.

Should have been

@ViewData["Title"]

@jbagga could you take this one for the dev branch?

https://github.com/aspnet/templating/blob/dev/src/Microsoft.DotNet.Web.ProjectTemplates/content/StarterWeb-CSharp/Views/Account/AccessDenied.cshtml#L6
It doesn't seem like an issue in dev

This should've been fixed in https://github.com/aspnet/templating/pull/40.

@scottaddie It was still missing from one of the Views in Identity Area. It's merged in now as part of #184


No module named 'ckeditor'

I have problem with error ImportError: No module named 'ckeditor'. The error rises when I use django-ckeditor locally, and when I use global installation there is no such problem. Django 1.11, python 3.5.2, windows 7 64b. I've spend about 5 hours but could not solve this problem. Please help me. Also it is strange that on the pythonanywhere everything is fine (there is python 3.6) and django-ckeditor is installed locally.

I'm sorry, this was my mistake, I ran django test server without activation virtual environment)


Manually loaded vs. automatically loaded

Is it possible to load the local index.html first and then trigger when I want to load a choosen manifest file?

So you can do this , but it will take a bit of work on your end.  Basically, you'll want to generate the apps, then update the start page to point to your internal page.  The the internal page can do a redirect to your web page.  I've done this before when I knew my site took a while to laod, and I wanted to provide somewhat of an "instant" experience for users.


Error: Request logging is disabled

I'm running this example app and, at the moment I submit the login request the process is running ember s stops with the following error:
/Users/bruno/workspace/ember-hoodie-example-app/node_modules/hapi/node_modules/hoek/lib/index.js:736
    throw new Error(msgs.join(' ') || 'Unknown error');
    ^

Error: Request logging is disabled
    at Object.exports.unique.exports.contain.exports.reachTemplate.exports.assert.condition [as assert] (/Users/bruno/workspace/ember-hoodie-example-app/node_modules/hapi/node_modules/hoek/lib/index.js:736:11)
    at internals.Request.internals.Request.getLog (/Users/bruno/workspace/ember-hoodie-example-app/node_modules/hapi/lib/request.js:252:10)
    at new exports.GreatResponse (/Users/bruno/workspace/ember-hoodie-example-app/node_modules/good/lib/utils.js:142:24)
    at Object.exports.GreatResponse (/Users/bruno/workspace/ember-hoodie-example-app/node_modules/good/lib/utils.js:83:16)
    at internals.Monitor._responseHandler (/Users/bruno/workspace/ember-hoodie-example-app/node_modules/good/lib/monitor.js:244:33)
    at invoke (/Users/bruno/workspace/ember-hoodie-example-app/node_modules/hapi/node_modules/podium/lib/index.js:239:30)
    at invoke (/Users/bruno/workspace/ember-hoodie-example-app/node_modules/hapi/node_modules/podium/lib/index.js:243:13)
    at Object.exports.parallel (/Users/bruno/workspace/ember-hoodie-example-app/node_modules/hapi/node_modules/items/lib/index.js:70:13)
    at Object.internals.Podium.registerEvent.internals.Podium.registerPodium.internals.emit.each [as emit] (/Users/bruno/workspace/ember-hoodie-example-app/node_modules/hapi/node_modules/podium/lib/index.js:260:18)
    at internals.Podium.registerEvent.internals.Podium.registerPodium.internals.Podium._emit (/Users/bruno/workspace/ember-hoodie-example-app/node_modules/hapi/node_modules/podium/lib/index.js:140:15)
    at each (/Users/bruno/workspace/ember-hoodie-example-app/node_modules/hapi/node_modules/podium/lib/index.js:181:47)
    at Object.exports.parallel (/Users/bruno/workspace/ember-hoodie-example-app/node_modules/hapi/node_modules/items/lib/index.js:70:13)
    at relay (/Users/bruno/workspace/ember-hoodie-example-app/node_modules/hapi/node_modules/podium/lib/index.js:182:15)
    at Object.internals.Podium.registerEvent.internals.Podium.registerPodium.internals.emit.each [as emit] (/Users/bruno/workspace/ember-hoodie-example-app/node_modules/hapi/node_modules/podium/lib/index.js:186:16)
    at Immediate._onImmediate (/Users/bruno/workspace/ember-hoodie-example-app/node_modules/hapi/node_modules/podium/lib/index.js:169:38)
    at processImmediate [as _immediateCallback] (timers.js:383:17)

I researched a little and found out that this error is thrown by this assert.
I changed these lines to include routes: { log: true } just like someone proposed here, and the error stopped.

I think this was resolved via https://github.com/hoodiehq/hoodie/pull/600, released in https://github.com/hoodiehq/hoodie/releases/tag/v24.2.1, can you check please?

Yeah, it seem to solve the problem.
Maybe we should update ember-hoodie as well?

Actually, that does not solve the problem. I'm already on hoodie 24.2.2...
It looks like these lines do not use getHapiOptions()...

you are right! Hmm ... We kinda have a lot of code duplication here. I wonder if we should use some internal code here from the hoodie package, like the getHapiOptions method, like so
var getHapiOptions = require('hoodie/cli/hapi-options')

it’s usually bad practice, but still better than duplicating the code and risking it gets out of sync like it did now.
Another option is to remove the server component entirely from ember-hoodie and document how to set it up yourself, to make it more explicit ...
What do you think?

I must say I feel, as an app dev, it's very inviting to install an ember addon, type ember s and already have hoodie ready for development! I wont remove this...
Maybe the hoodie module could expose a configured hapi server instance?

it's very inviting to install an ember addon, type ember s and already have hoodie ready for development!

Yeah I agree

Maybe the hoodie module could expose a configured hapi server instance?

I’m not sure what you mean by that? We can either use hoodie which comes with the whole package, or we use @hoodie/server which only has the backend components for store & account

I mean, if ember-hoodie must require an internal file inside  hoodie module, then I think it would be more meaningful to me to import a configured instance than to import a getHapiOptions method...  something similar to what is done here when it imports the whole run function.
just an idea, btw...

actually, the run function does configre other things... not just the connection options, but also a a server options and who know what else eventually...


Support message[@xml:lang] as child of assert|report

for internationalization purposes.
(As suggested by @dmj and Renzo Kottman at the 2023 Schematron Users Meetup.)

diagnostics is available for this already, but this was felt to be perhaps too indirect.

I also see a problems with using diagnostics for localization on a conceptual level: Assertions and diagnostics are defined as separate items. One is "an assertion made about the context nodes", the other "a natural-language message giving more specific details concerning a failed assertion".
The example in Annex G uses a trick and repeats the englisch assertion as a diagnostic. I would say it is an example for localized diagnostics, but not localized assertions.


can I  make  nodes with out handles and connect them with edges?

You can hide them visually but you need them to connect the nodes.


Blurry images on Chrome and Safari

I don't know if it's just me but zoomed images look kinda blurry on Chrome and Safari.
Chrome

Firefox

Never mind, it was caused by a grid system I was using.


feat: Introduce Clerk chrome extension

Type of change

[ ] 🐛 Bug fix
[x] 🌟 New feature
[ ] 🔨 Breaking change
[ ] 📖 Refactoring / dependency upgrade / documentation
[ ] other:

Packages affected

[ ] @clerk/clerk-js
[ ] @clerk/clerk-react
[ ] @clerk/nextjs
[ ] @clerk/remix
[ ] @clerk/types
[ ] @clerk/themes
[ ] @clerk/localizations
[ ] @clerk/clerk-expo
[ ] @clerk/backend
[ ] @clerk/clerk-sdk-node
[ ] @clerk/shared
[ ] @clerk/fastify
[x] @clerk/chrome-extension
[ ] gatsby-plugin-clerk
[ ] build/tooling/chore

Description

[ ] npm test runs as expected.
[ ] npm run build runs as expected.

Introduce @clerk/chrome-extension package.
Pending tasks

[ ] manual testing
[ ] unit tests

#jit_ignore_fp

#jit_bypass_commit

#jit_ignore


Upgrade NullAway 0.10.24 -> 0.10.25

This PR contains the following updates:

Package
Type
Update
Change

NullAway

patch
0.10.24 -> 0.10.25

NullAway
compile
patch
0.10.24 -> 0.10.25

Release Notes

uber/NullAway (NullAway)
v0.10.25
Compare Source

JSpecify: Handle @nullable assignments to @nonnull arrays (#929)
Handle JDK 21 case operands in type refinement (#928)
Track access paths of the form Foo.this.bar (#937)
Add test for boolean contract with extra message argument (#945)
Collectors.toMap handling for streams (#938)
Handle methods that fail unconditionally in ContractHandler (#946)
Build / CI tooling upgrades:

Update various dependencies (#939)

[ ] If you want to rebase/retry this PR, check this box

Suggested commit message:
Upgrade NullAway 0.10.24 -> 0.10.25

See:
- https://github.com/uber/NullAway/blob/master/CHANGELOG.md
- https://github.com/uber/NullAway/releases/tag/v0.10.25
- https://github.com/uber/NullAway/compare/v0.10.24...v0.10.25


sc4s writing data into terminal screen when it is started

We have installed the lates version of sc4s -  2.43.1 on RHEL8 Virtual machine and as soon as we start the sc4s, it is writing the messages into the terminal screen. any one has any idea on how to fix this.
2022 Dec 14 14:55:25 hostxxxx matched='filter(riverbed_syslog_stealconnect)=0', not_matched='filter(riverbed_syslog_stealconnect)=0', processed='source(s_startup_out)=1', processed='center(received)=25', matched='filter(syslog-vmware_vsphere-sdata)=0', not_matched='filter(syslog-vmware_vsphere-sdata)=0', processed='source(s_NW)=0', discarded='parser(p_cef_kv)=0', discarded='parser(p_leef_kv)=0', discarded='parser(p_cef_ts_rt)=0', discarded='parser(app-group-sc4s-raw)=0', discarded='parser(app-group-sc4s-syslog-pgm)=0', processed='src.file(s_startup_out#0,/var/lib/syslog-ng/log/syslog-ng.out)=1', stamp='src.file(s_startup_out#0,/var/lib/syslog-ng/log/syslog-ng.out)=1671029005', matched='filter(f_is_rfc3164)=0', not_matched='filter(f_is_rfc3164)=0', matched='filter(syslog-vmware_vsphere-esx-pgm)=0', not_matched='filter(syslog-vmware_vsphere-esx-pgm)=0', msg_size_max='src.file(s_startup_out#0,/var/lib/syslog-ng/log/syslog-ng.out)=19', discarded='parser(p_leef_devTime)=0', discarded='parser(vendor_product_by_source)=0', discarded='parser(app-group-sc4s-almost-syslog)=0', discarded='parser(p_acs_event_time)=0', discarded='parser(p_add_context_host)=0', discarded='parser(sb_alertext_kv)=0', matched='filter(f_is_rfc)=0', not_matched='filter(f_is_rfc)=0', matched='filter(f_is_rfc5424)=0', not_matched='filter(f_is_rfc5424)=0', matched='filter(f_cisco_esa_error_logs)=0', not_matched='filter(f_cisco_esa_error_logs)=0', matched='filter(f_cisco_esa_textmail)=0', not_matched='filter(f_cisco_esa_textmail)=0', matched='filter(f_cisco_esa_authentication)=0', not_matched='filter(f_cisco_esa_authentication)=0', matched='filter(f_is_source_identified)=0', not_matched='filter(f_is_source_identified)=0', discarded='parser(app-group-sc4s-syslog)=0', matched='filter(f_cisco_esa_antispam)=0', not_matched='filter(f_cisco_esa_antispam)=0', matched='filter(f_cisco_esa_http)=0', not_matched='filter(f_cisco_esa_http)=0', matched='filter(f_cisco_esa_system_logs)=0', not_matched='filter(f_cisco_esa_system_logs)=0', matched='filter(f_host_is_nil_or_ip)=0', not_matched='filter(f_host_is_nil_or_ip)=50', matched='filter(syslog-vmware_vsphere-nsx-pgm)=0', not_matched='filter(syslog-vmware_vsphere-nsx-pgm)=0', matched='filter(syslog-vmware_vsphere-vsan-pgm)=0', not_matched='filter(syslog-vmware_vsphere-vsan-pgm)=0', processed='src.network(s_DEFAULT#3,<IP_ADDRESS>)=0', stamp='src.network(s_DEFAULT#3,<IP_ADDRESS>)=0', eps_since_start='src.network(s_DEFAULT#1,<IP_ADDRESS>)=0', eps_since_start='src.network(s_DEFAULT#2,<IP_ADDRESS>)=0', processed='src.network(s_DEFAULT#0,<IP_ADDRESS>)=0', stamp='src.network(s_DEFAULT#0,<IP_ADDRESS>)=0', processed='src.network(s_DEFAULT#2,<IP_ADDRESS>)=0', stamp='src.network(s_DEFAULT#2,<IP_ADDRESS>)=0', msg_size_max='src.network(s_DEFAULT#3,<IP_ADDRESS>)=0', processed='src.syslog(s_DEFAULT#5,<IP_ADDRESS>)=0', stamp='src.syslog(s_DEFAULT#5,<IP_ADDRESS>)=0', matched='filter(syslog-netsource-vmware_vsphere-vcenter-pgm)=0', not_matched='filter(syslog-netsource-vmware_vsphere-vcenter-pgm)=0', processed='src.file(s_startup_err#0,/var/lib/syslog-ng/log/syslog-ng.err)=0', stamp='src.file(s_startup_err#0,/var/lib/syslog-ng/log/syslog-ng.err)=0', processed='source(s_DEFAULT)=0', msg_size_max='src.network(s_DEFAULT#0,<IP_ADDRESS>)=0', processed='src.network(s_DEFAULT#1,<IP_ADDRESS>)=0', stamp='src.network(s_DEFAULT#1,<IP_ADDRESS>)=0', matched='filter(f_cisco_esa_amp)=0', not_matched='filter(f_cisco_esa_amp)=0', msg_size_avg='src.network(s_DEFAULT#3,<IP_ADDRESS>)=0', processed='source(s_internal)=24', discarded='parser(p_add_context_splunk)=0', msg_size_max='src.file(s_startup_err#0,/var/lib/syslog-ng/log/syslog-ng.err)=0', processed='src.internal(s_internal#0)=24', stamp='src.internal(s_internal#0)=1671029695', discarded='parser(p_ise_event_time_multi)=0', processed='source(s_startup_err)=0', processed='destination(d_stdout)=25', msg_size_avg='src.network(s_DEFAULT#0,<IP_ADDRESS>)=0', msg_size_m

@harnhbn,
Could you please share the configuration steps you followed to install sc4s?

I followed the steps mentioned in the link - https://splunk.github.io/splunk-connect-for-syslog/main/gettingstarted/byoe-rhel8/ and we are using first option  to start the sc4s via the sc4s.service unit file

the entrypoint.sh script (identical to that used in the container) can be run directly via systemd

I confirm the issue existed when we spin it with BYOE CENTOS 8, we need to check more to fix it, for now we recommend to use any container based solution,we will post update soon on this.

:tada: This issue has been resolved in version 2.44.2 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Initial setup

Hello ,
This popped during initial set up last setup  step while executing Install-AzSKTenantSecuritySolution
Code    : InvalidTemplate
Message : Deployment template validation failed: 'The template reference '98d01f3f-4a45-4eb8-99ba-29637d9f7f27' is not valid: could not find template resource or resource copy with this
name. Please see https://aka.ms/arm-function-reference for usage details.'.

Hi DevanshuSyk,
We need additional information for further investigation.
Please help us with the following information.

What were the steps executed?
Which step resulted in error?
Could you please share the logs for all the steps executed.

Hi DevanshuSyk,
Thank you sharing the details. Could you please help to share the command lines and output logs.

Hi @DevanshuSyk,
Could you help us with some specifics (commands that were run/screenshots) to provide us more context of the issue?


fix shell_plus --command globals / locals error

When running a "more complex" command via --command shell_plus has historically passed an empty globals and the imports as locals. This will cause Python to act strangely for certain inputs such as list comprehensions. This was fixed in Django's shell command 1 and by passing the imports as the only arg Python will use them as both the locals and the globals which is as if the exec was happening at the module level.
See also: https://stackoverflow.com/questions/45132645/list-comprehension-in-exec-with-empty-locals-nameerror

Coverage decreased (-71.1%) to 0.0% when pulling c2e6191b09b8f353ae88cd52373e6ee798899ab8 on terencehonles:fix-shell_plus-command into f3357da0c66096d5bacf7ac6228c18001dabd7ef on django-extensions:master.

Coverage decreased (-71.1%) to 0.0% when pulling c2e6191b09b8f353ae88cd52373e6ee798899ab8 on terencehonles:fix-shell_plus-command into f3357da0c66096d5bacf7ac6228c18001dabd7ef on django-extensions:master.

@trbs not sure if you have a chance to review / merge this one too.

Thanks !


[BUG] DMLC XGBoost train FAILED against rapids-4-spark 22.10.0-SNAPSHOT FAILED

Describe the bug
DMLC XGBoost train FAILED againt rapids-4-spark 22.10.0-SNAPSHOT FAILED with below error,
It PASS before commit f2b6bd0f3724256831ace0041a64886824686553, and failed after 49e0a85e930ec98635269a703069f631f6f7b419

PASS : [2022-08-17T06:59:23.132Z] 22/08/17 06:58:31 INFO RapidsPluginUtils: RAPIDS Accelerator build: {version=22.10.0-SNAPSHOT, user=, url=https://github.com/NVIDIA/spark-rapids.git, date=2022-08-16T17:31:42Z, revision=f2b6bd0f3724256831ace0041a64886824686553, cudf_version=22.10.0-SNAPSHOT, branch=HEAD}
FAILED:  [2022-08-18T06:42:32.397Z] 22/08/18 06:41:50 INFO RapidsPluginUtils: RAPIDS Accelerator build: {version=22.10.0-SNAPSHOT, user=, url=https://github.com/NVIDIA/spark-rapids.git, date=2022-08-17T16:31:58Z, revision=49e0a85e930ec98635269a703069f631f6f7b419, cudf_version=22.10.0-SNAPSHOT, branch=HEAD} (edited)
org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 3.0 failed 4 times, most recent failure: Lost task 0.3 in stage 3.0 (TID 6) (<IP_ADDRESS> executor 0): java.lang.NoClassDefFoundError: com/nvidia/spark/rapids/GpuColumnVector
	at ml.dmlc.xgboost4j.scala.rapids.spark.GpuPreXGBoost$$anon$1.$anonfun$loadNextBatch$2(GpuPreXGBoost.scala:334)
	at ml.dmlc.xgboost4j.scala.rapids.spark.GpuPreXGBoost$.withResource(GpuPreXGBoost.scala:596)
	at ml.dmlc.xgboost4j.scala.rapids.spark.GpuPreXGBoost$$anon$1.loadNextBatch(GpuPreXGBoost.scala:319)
	at ml.dmlc.xgboost4j.scala.rapids.spark.GpuPreXGBoost$$anon$1.hasNext(GpuPreXGBoost.scala:357)
	at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:458)
	at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.processNext(Unknown Source)
	at org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
	at org.apache.spark.sql.execution.WholeStageCodegenExec$$anon$1.hasNext(WholeStageCodegenExec.scala:755)
	at org.apache.spark.sql.execution.columnar.DefaultCachedBatchSerializer$$anon$1.hasNext(InMemoryRelation.scala:118)
	at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:458)
	at org.apache.spark.storage.memory.MemoryStore.putIterator(MemoryStore.scala:221)
	at org.apache.spark.storage.memory.MemoryStore.putIteratorAsValues(MemoryStore.scala:299)
	at org.apache.spark.storage.BlockManager.$anonfun$doPutIterator$1(BlockManager.scala:1423)
	at org.apache.spark.storage.BlockManager.org$apache$spark$storage$BlockManager$$doPut(BlockManager.scala:1350)
	at org.apache.spark.storage.BlockManager.doPutIterator(BlockManager.scala:1414)
	at org.apache.spark.storage.BlockManager.getOrElseUpdate(BlockManager.scala:1237)
	at org.apache.spark.rdd.RDD.getOrCompute(RDD.scala:384)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:335)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
	at org.apache.spark.sql.execution.SQLExecutionRDD.compute(SQLExecutionRDD.scala:55)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
	at org.apache.spark.scheduler.Task.run(Task.scala:131)
	at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:497)
	at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1439)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:500)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

Driver stacktrace:
  at org.apache.spark.scheduler.DAGScheduler.failJobAndIndependentStages(DAGScheduler.scala:2258)
  at org.apache.spark.scheduler.DAGScheduler.$anonfun$abortStage$2(DAGScheduler.scala:2207)
  at org.apache.spark.scheduler.DAGScheduler.$anonfun$abortStage$2$adapted(DAGScheduler.scala:2206)
  at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
  at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
  at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
  at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:2206)
  at org.apache.spark.scheduler.DAGScheduler.$anonfun$handleTaskSetFailed$1(DAGScheduler.scala:1079)
  at org.apache.spark.scheduler.DAGScheduler.$anonfun$handleTaskSetFailed$1$adapted(DAGScheduler.scala:1079)
  at scala.Option.foreach(Option.scala:407)
  at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:1079)
  at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:2445)
  at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2387)
  at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2376)
  at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:49)
  at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:868)
  at org.apache.spark.SparkContext.runJob(SparkContext.scala:2196)
  at org.apache.spark.SparkContext.runJob(SparkContext.scala:2217)
  at org.apache.spark.SparkContext.runJob(SparkContext.scala:2236)
  at org.apache.spark.SparkContext.runJob(SparkContext.scala:2261)
  at org.apache.spark.rdd.RDD.$anonfun$foreachPartition$1(RDD.scala:1020)
  at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
  at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
  at org.apache.spark.rdd.RDD.withScope(RDD.scala:414)
  at org.apache.spark.rdd.RDD.foreachPartition(RDD.scala:1018)
  at org.apache.spark.sql.Dataset.$anonfun$foreachPartition$1(Dataset.scala:2906)
  at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
  at org.apache.spark.sql.Dataset.$anonfun$withNewRDDExecutionId$1(Dataset.scala:3676)
  at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$5(SQLExecution.scala:103)
  at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:163)
  at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$1(SQLExecution.scala:90)
  at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:775)
  at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:64)
  at org.apache.spark.sql.Dataset.withNewRDDExecutionId(Dataset.scala:3674)
  at org.apache.spark.sql.Dataset.foreachPartition(Dataset.scala:2906)
  at $anonfun$prediction$1(<console>:39)
  at Benchmark$.time(<console>:31)
  ... 37 elided
Caused by: java.lang.NoClassDefFoundError: com/nvidia/spark/rapids/GpuColumnVector
  at ml.dmlc.xgboost4j.scala.rapids.spark.GpuPreXGBoost$$anon$1.$anonfun$loadNextBatch$2(GpuPreXGBoost.scala:334)
  at ml.dmlc.xgboost4j.scala.rapids.spark.GpuPreXGBoost$.withResource(GpuPreXGBoost.scala:596)
  at ml.dmlc.xgboost4j.scala.rapids.spark.GpuPreXGBoost$$anon$1.loadNextBatch(GpuPreXGBoost.scala:319)
  at ml.dmlc.xgboost4j.scala.rapids.spark.GpuPreXGBoost$$anon$1.hasNext(GpuPreXGBoost.scala:357)
  at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:458)
  at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.processNext(Unknown Source)
  at org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
  at org.apache.spark.sql.execution.WholeStageCodegenExec$$anon$1.hasNext(WholeStageCodegenExec.scala:755)
  at org.apache.spark.sql.execution.columnar.DefaultCachedBatchSerializer$$anon$1.hasNext(InMemoryRelation.scala:118)
  at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:458)
  at org.apache.spark.storage.memory.MemoryStore.putIterator(MemoryStore.scala:221)
  at org.apache.spark.storage.memory.MemoryStore.putIteratorAsValues(MemoryStore.scala:299)
  at org.apache.spark.storage.BlockManager.$anonfun$doPutIterator$1(BlockManager.scala:1423)
  at org.apache.spark.storage.BlockManager.org$apache$spark$storage$BlockManager$$doPut(BlockManager.scala:1350)
  at org.apache.spark.storage.BlockManager.doPutIterator(BlockManager.scala:1414)
  at org.apache.spark.storage.BlockManager.getOrElseUpdate(BlockManager.scala:1237)
  at org.apache.spark.rdd.RDD.getOrCompute(RDD.scala:384)
  at org.apache.spark.rdd.RDD.iterator(RDD.scala:335)
  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
  at org.apache.spark.sql.execution.SQLExecutionRDD.compute(SQLExecutionRDD.scala:55)
  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
  at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
  at org.apache.spark.scheduler.Task.run(Task.scala:131)
  at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:497)
  at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1439)
  at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:500)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  ... 1 more

org.apache.spark.SparkException:  Job aborted due to stage failure: Task 0 in stage 3.0 failed 4 times, most recent failure: Lost task 0.3 in stage 3.0 (TID 6) (<IP_ADDRESS> executor 0): java.lang.NoClassDefFoundError: com/nvidia/spark/rapids/GpuColumnVector

@gerashegalov Looks like it related to PR https://github.com/NVIDIA/spark-rapids/pull/5646
Could you please help to take a look?Thanks!

Thanks for reporting the issue, @NvTimLiu. Please make sure to include steps to reproduce the failure.

please refer  https://github.com/NVIDIA/spark-rapids-examples/blob/branch-22.08/docs/get-started/xgboost-examples/notebook/toree.md to repro this issue

@gerashegalov
Sorry, I did not catch it. Could you please help to explain how branch-22.10 removed GpuColumnVector's scala access  from rapids-4-spark, then aggregator jar restore it by transitive dependency?
Additional context
https://github.com/NVIDIA/spark-rapids/issues/6404 : xgboost4j-spark-gpu should fail to compile when the rapids-4-spark dependency is updated from 21.08 to 22.10.0-SNAPSHOT because GpuColumnVector is no longer accessible to scalac. Yet it worked because it was transitively including the aggregator jar.

@wbo4958  please take a look at DMLC XGboost build part, thanks

XGBoost explicitly depends on spark-rapids 21.08 and it calls some APIs of GpuColumnVector

so it seems we need to bring the access of GpuColumnVector back to be back-compatible?

XGBoost explicitly depends on spark-rapids 21.08 and it calls some APIs of GpuColumnVector

@wbo4958 How is explicit dependency 21.08 supposed to work when we then use rapids-4-spark later versions at run time?

We no longer support Spark 3.0.1 in rapids-4-spark 22.04 although it's used as a dependency https://github.com/dmlc/xgboost/blob/d6e2013c5f3df0fff702cf3bfbdbdbc911e566dc/jvm-packages/pom.xml#L37
Source code relying on Spark are generally not binary-compatible,  it is to a certain extent source-compatible between versions https://github.com/NVIDIA/spark-rapids/blob/branch-22.10/docs/dev/shims.md

@gerashegalov Sorry, I did not catch it. Could you please help to explain how branch-22.10 removed GpuColumnVector's scala access from rapids-4-spark, then aggregator jar restore it by transitive dependency?

@NvTimLiu Take a look at https://github.com/NVIDIA/spark-rapids/blob/branch-22.10/docs/dev/shims.md
Our final (dist module) artifact's content is split in multiple areas.
1/
Public classes with stable ABI that do not depend on supported Spark versions. It's placed at conventional locations at the top of the jar. These classes can loaded with any classloader.
$ jar tvf dist/target/rapids-4-spark_2.12-22.10.0-SNAPSHOT-cuda11.jar | grep SQLPlugin
  4601 Mon Aug 29 14:39:26 PDT 2022 com/nvidia/spark/SQLPlugin.class

2/
And we have classes that we load only after figuring out the Spark version at run time. They are so-called "parallel worlds" that are not visible to a conventional classloader. It includes classes whose bytecode definitely differs across versions
 $ jar tvf dist/target/rapids-4-spark_2.12-22.10.0-SNAPSHOT-cuda11.jar | grep 'spark320/com/nvidia' | grep '.class' | head -1
 12634 Mon Aug 29 14:39:00 PDT 2022 spark320/com/nvidia/spark/rapids/GpuUnaryExpression.class

3/
And classes that produce the same bytecode under all supported Spark versions under spark3xx-common.  They may be identical for all versions but may reference another class e.g. via method signature that is in a version-specific parallel world.
We try to keep 1/ small and GpuColumnVector is in 3/ under spark3xx-common. It now contains some code that is version specific and probably is not fully unshimmable to 1/
But we can see what we can do @wbo4958 regarding https://github.com/NVIDIA/spark-rapids/issues/6404#issuecomment-1229982315
In our build, the aggregator jars and all its dependencies are jars in a normal layout accessible to regular classloaders including compilers. So if by accident it gets onto the classpath because we end up using the full original pom before it has been reduced of its dependencies after mvn install, the XGBoost compilation succeeds since the compiler is still able to load GpuColumnVector from aggregator.

@gerashegalov Sorry, I did not catch it. Could you please help to explain how branch-22.10 removed GpuColumnVector's scala access from rapids-4-spark, then aggregator jar restore it by transitive dependency?

@NvTimLiu Take a look at https://github.com/NVIDIA/spark-rapids/blob/branch-22.10/docs/dev/shims.md
Our final (dist module) artifact's content is split in multiple areas.
1/ Public classes with stable ABI that do not depend on supported Spark versions. It's placed at conventional locations at the top of the jar. These classes can loaded with any classloader.
$ jar tvf dist/target/rapids-4-spark_2.12-22.10.0-SNAPSHOT-cuda11.jar | grep SQLPlugin
  4601 Mon Aug 29 14:39:26 PDT 2022 com/nvidia/spark/SQLPlugin.class

2/ And we have classes that we load only after figuring out the Spark version at run time. They are so-called "parallel worlds" that are not visible to a conventional classloader. It includes classes whose bytecode definitely differs across versions
 $ jar tvf dist/target/rapids-4-spark_2.12-22.10.0-SNAPSHOT-cuda11.jar | grep 'spark320/com/nvidia' | grep '.class' | head -1
 12634 Mon Aug 29 14:39:00 PDT 2022 spark320/com/nvidia/spark/rapids/GpuUnaryExpression.class

3/ And classes that produce the same bytecode under all supported Spark versions under spark3xx-common. They may be identical for all versions but may reference another class e.g. via method signature that is in a version-specific parallel world.
We try to keep 1/ small and GpuColumnVector is in 3/ under spark3xx-common. It now contains some code that is version specific and probably is not fully unshimmable to 1/ But we can see what we can do @wbo4958 regarding #6404 (comment)
In our build, the aggregator jars and all its dependencies are jars in a normal layout accessible to regular classloaders including compilers. So if by accident it gets onto the classpath because we end up using the full original pom before it has been reduced of its dependencies after mvn install, the XGBoost compilation succeeds since the compiler is still able to load GpuColumnVector from aggregator.

@gerashegalov  That's make sense, thanks a lot for your detailed explanation!

Analysis with the repro in this notebook:
https://github.com/gerashegalov/spark-rapids-examples/blob/rero6404/examples/XGBoost-Examples/taxi/notebooks/scala/taxi-gpu.ipynb
Interestingly that xgboost4j does not seem to support the deployment option using --jars or --packages independently of the RAPIDS plugin issue.

Do you mean --jars does not work for the notebook env?
I copied the rapids-4-spark jars into the spark/jars dir, the same issue existed in notebook.

Do you mean --jars does not work for the notebook env ?

yes, --jars does not work for xgboost4j in the spylon kernel even with the old spark-rapids 21.08. See this paragraph in the notebook.
%%init_spark
import os

m2dir = os.path.expanduser('~/.m2/repository')

xgboost_v = '1.6.1'
spark_rapids_v = '21.08.0'
cudf_v = '21.08.2'

launcher.jars = [
    f"{m2dir}/ml/dmlc/xgboost4j-gpu_2.12/{xgboost_v}/xgboost4j-gpu_2.12-{xgboost_v}.jar",
    f"{m2dir}/ml/dmlc/xgboost4j-spark-gpu_2.12/{xgboost_v}/xgboost4j-spark-gpu_2.12-{xgboost_v}.jar",
    f"{m2dir}/com/nvidia/rapids-4-spark_2.12/{spark_rapids_v}/rapids-4-spark_2.12-{spark_rapids_v}.jar",
    f"{m2dir}/ai/rapids/cudf/{cudf_v}/cudf-{cudf_v}-cuda11.jar"
]
launcher.conf.spark.plugins = "com.nvidia.spark.SQLPlugin"

I copied the rapids-4-spark jars into the spark/jars dir, the same issue existed in notebook.

Copying the jar to $SPARK_ HOME/jars is merely a workaround of the issue, not a feature. It is the same as using extraClassPath in this paragraph (but it  is discouraged and should not be necessary to use extraClassPath).
%%init_spark
import os

m2dir = os.path.expanduser('~/.m2/repository')

xgboost_v = '1.6.1'
spark_rapids_v = '21.08.0'
cudf_v = '21.08.2'

launcher.conf.spark.driver.extraClassPath = [
    f"{m2dir}/ml/dmlc/xgboost4j-gpu_2.12/{xgboost_v}/xgboost4j-gpu_2.12-{xgboost_v}.jar",
    f"{m2dir}/ml/dmlc/xgboost4j-spark-gpu_2.12/{xgboost_v}/xgboost4j-spark-gpu_2.12-{xgboost_v}.jar",
    f"{m2dir}/com/nvidia/rapids-4-spark_2.12/{spark_rapids_v}/rapids-4-spark_2.12-{spark_rapids_v}.jar",
    f"{m2dir}/ai/rapids/cudf/{cudf_v}/cudf-{cudf_v}-cuda11.jar"
]
launcher.conf.spark.plugins = "com.nvidia.spark.SQLPlugin"


java.sql.Timestamp handled inconsistently by datetimeoffset

Driver version or jar name
6.4.0.jre8
SQL Server version
13.0.4422
Client operating system
Windows 10 Enterprise (1607)
Java/JVM version
jdk1.8.0_102 (Oracle)
Table schema
create table times (
id varchar(255) not null,
start_time datetimeoffset
)
Problem description
I want to store DateTime into the DB so that it can be used internationally. So I picked the datetimeoffset.
When I use the java.sql.Timestamp the zone information is removed, so when I store 'Mon Jul 23 16:09:14 EEST 2018' it is being simply stored as '2018-07-23 16:09:14.0000000 +00:00'.
But when I retrieve this column, then I get 'Mon Jul 23 19:09:14 EEST 2018'. So it treats the datetimeoffset as UTC offset. The problem is that when I store Timestamp I get another Timestamp back.
Using microsoft.sql.DateTimeOffset is not an option, I prefer using standard java types.
Expected behavior and actual behavior
I expect to store time information and get the same time back.
Please note - if you're in UTC time the code will work correctly. Please move to another timezone.
Repro code
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Timestamp;
import java.util.Date;
import java.util.NoSuchElementException;
import java.util.Random;

public class Main {

    public static void main(String args[]) throws Exception {
        Date now = new Date();
        Timestamp timestampToSave = new Timestamp(now.getTime());
        Timestamp timestampRetrieved = saveAndGet(timestampToSave);
        showDate(timestampToSave);
        showDate(timestampRetrieved);
        System.out.println("Times are the same: " +
                (timestampToSave.getTime() == timestampRetrieved.getTime()));
    }

    private static Timestamp saveAndGet(Timestamp timestamp) throws SQLException {
        String url = "jdbc:sqlserver://localhost:1433;databaseName=mssqldatetimes";
        try (Connection c = DriverManager.getConnection(url, "admin", "Qwerty1234")) {
            String uniqueId = "id" + new Random().nextInt();
            String saveSql = "insert into times(id, start_time) values (?, ?)";
            try (PreparedStatement save = c.prepareStatement(saveSql)) {
                save.setString(1, uniqueId);
                save.setTimestamp(2, timestamp);
                save.execute();
            }
            String getSql = "select start_time from times where id = ?";
            try (PreparedStatement get = c.prepareStatement(getSql)) {
                get.setString(1, uniqueId);
                ResultSet resultSet = get.executeQuery();
                if (resultSet.next()) {
                    return resultSet.getTimestamp(1);
                }
            }
        }
        throw new NoSuchElementException();
    }

    private static void showDate(Timestamp timestampToSave) {
        System.out.printf("Timestamp: %s, Date: %s, Milliseconds: %d%n",
                timestampToSave,
                new Date(timestampToSave.getTime()),
                timestampToSave.getTime());
    }

}

Hi @navpil ,
The behavior is by design.
Note that date and time parts of datetimeoffset  are relative to UTC and the driver explicitly uses UTC to construct Timestamp object when calling getTimestamp() on a datetimeoffset column.
You should call save.setTimestamp(2, timestamp, Calendar.getInstance(TimeZone.getTimeZone("UTC"))); instead of save.setTimestamp(2, timestamp); to notify the driver about underlying datetimeoffset column, if you decide not to use microsoft.sql.DateTimeOffset.

Hi @ulvii,
Thank you for information. I haven't seen this workaround before on the web.
Just in case anyone else reads this thread I will leave the (similar) workaround I used for Hibernate. I could not make it save Timestamp or Date correctly, so I used the java.util.Calendar for the temporal data. This is how the Calendar should be created to be stored correctly in datetimeoffset:
public Calendar createUtcCalendarFromDate(Date date) {
    if (date == null) {
        return null;
    }
    Calendar calendar = Calendar.getInstance(TimeZone.getTimeZone("UTC"));
    calendar.setTime(date);
    return calendar;
}

In case there is a better solution for Hibernate, please post it here.

Glad you could resolve it. I am closing the issue, please feel free to re-open it or create a new one if you have any other questions.

Hi!
Sorry, but it looks like a bug.

java.sql.timestamp on one hand represents local time but on the other time it represents ~timestamp/instant.

Please look at my code and the result:
        Instant date = LocalDateTime.of(2010, 1, 1, 0, 0).toInstant(ZoneOffset.UTC);

        try (Connection conn = DriverManager.getConnection(mssql.getJdbcUrl(), mssql.getUsername(), mssql.getPassword())) {
            conn.setAutoCommit(true);
            // DDL
            try (Statement statement = conn.createStatement()) {
                statement.execute("create table test_table (test_date datetimeoffset)");
            }
            // INSERT
            try (PreparedStatement preparedStatement = conn.prepareStatement("insert into test_table values (?)")) {
                preparedStatement.setTimestamp(1, Timestamp.from(date));
                preparedStatement.execute();
                System.out.println("timestamp represents LOCAL DATE on the one hand: " + Timestamp.from(date).toString());
                System.out.println("timestamp represents ~INSTANT on the other hand: " + Timestamp.from(date).getTime()
                        + " - " + date.toEpochMilli());
            }
            // FETCH
            try (Statement statement = conn.createStatement()) {
                try (ResultSet rs = statement.executeQuery("select * from test_table")) {
                    if (rs.next()) {
                        Timestamp received = rs.getTimestamp(1);
                        System.out.println("ZoneId: " + ZoneId.systemDefault());
                        System.out.println("ORIGIN: " + date);
                        System.out.println("FETCHED: " + received.toInstant());
                    } else {
                        throw new RuntimeException("TLD");
                    }
                }
            }
        }

Result:
timestamp represents LOCAL DATE on the one hand: 2010-01-01 03:00:00.0
timestamp represents ~INSTANT on the other hand:<PHONE_NUMBER>000 -<PHONE_NUMBER>000
ZoneId: Europe/Moscow
ORIGIN: 2010-01-01T00:00:00Z
FETCHED: 2010-01-01T03:00:00Z

SO, I expected to receive "2010-01-01T00:00:00Z". Postgres does it, H2 Database does it, Oracle so does...
Thanks in advance! :)


Bad Gateway

My playlist hasn't been updated for 3 days, and at https://discoverifymusic.com/ i reach a "502 Bad Gateway" error.
Will this be fixed? And if not, will my subscription be cancelled? 🙏

This was due to the global IT outage, the service should have recovered shortly after this was resolved


[TK-11] Add the "Add Author" page

Pull Request
Story ID: TK-11
Title: [TK-11] Add the "Add Author" page
Card URL: https://trello.com/c/R3wBUmQz
Author: @flukekd


Error on Amazon Linux 2 with 0.5.11 version

I got following syntax error during installion on Amazon Linux 2 with latest version
Recipe Compile Error in /var/opt/firezone/cache/cache/cookbooks/firezone/attributes/default.rb
2022-09-28T11:53:35+00:00] FATAL: Stacktrace dumped to /var/opt/firezone/cache/cache/chef-stacktrace.out
[2022-09-28T11:53:35+00:00] FATAL: ---------------------------------------------------------------------------------------
[2022-09-28T11:53:35+00:00] FATAL: PLEASE PROVIDE THE CONTENTS OF THE stacktrace.out FILE (above) IF YOU FILE A BUG REPORT
[2022-09-28T11:53:35+00:00] FATAL: ---------------------------------------------------------------------------------------
[2022-09-28T11:53:35+00:00] FATAL: SyntaxError: /var/opt/firezone/cache/cache/cookbooks/firezone/attributes/default.rb:37: invalid multibyte char (UTF-8)

Strange, that file hasn't changed in 2 months.
Looks like line 37 is related to the admin email. Are you installing from the automatic install script? Is the admin email entered correctly?

I tried both manual and automatic install. I entered email like<EMAIL_ADDRESS>I installed last week with the same email with 0.5.10 on test stage, but it did not raise any issue.

Could you zip and upload that file here? Looks like yours is possibly getting corrupted.

Sorry the vm is already terminated. I am not able to get the file. I will try it with ubuntu 20.


UInt instances?

Should we add some of these instances?
https://pursuit.purescript.org/packages/purescript-uint-instances/

Looks like there was some previous discussion in #7 on this.
From what I can tell, the outcome of that issue was the addition of an Arbitrary instance in d3a6424ca266d5c8e2388e8b81bf4195f9a0e303.
The Arbitrary instance was then removed in 6820b50a97c4ae993b6ed34dede182063cdf97e4 when the QuickCheck dependencies were moved into the test dependencies.
Looking at the purescript-uint-instances package, it adds the following instances that we do not currently have:

Arbitrary and Coarbitrary from purescript-quickcheck
EncodeJson and DecodeJson from purescript-argonaut-codecs
Hashable from purescript-unordered-collections

I came here because I was looking for an Arbitrary instance for UInt. Is there a reason why we would not want to expose the Arbitrary instance in the API?

I think providing something in terms of MonadGen would be preferable since that has a much lower dependency overhead.

I would love a MonadGen PR @chtenb

Right, so if MonadGen is preferred then we would have something like
chooseUInt :: forall gen. MonadGen gen => UInt -> UInt -> gen UInt
chooseUInt a b = fromInt $ chooseInt (toInt a) (toInt b)

but we're using the chooseInt from quickCheck here, so that doesn't solve the dependency overhead.
Wouldn't it perhaps be better to add this to the quickCheck library instead then? The quickcheck lib would gain uint as a dependency, but that's a very small library. Another advantage would be that we could add Arbitrary while we're at it.

You don't need the quickcheck dependency, you can just use the chooseInt provided by the MonadGen. See for example https://pursuit.purescript.org/packages/purescript-float32/2.0.0/docs/Data.Float32.Gen

You're right @jamesdbrock
So in that case I'd say we add the chooseUInt based on MonadGen to this package and then also provide the arbitrary instance in the quickcheck package, if the folks there are okay with that.

We seem to have overlooked that genUInt already exists in UInt.Gen 😄


Chore/embeddings docs

Describe Your Changes

Add documentation for embeddings

Fixes Issues

Closes #1102

Thank you @nguyenhoangthuan99! Nicely written


chipsalliance/Surelog#2541 Access to included files

Fixing a number of location issues
Updating IncludeFileInfo to include startColumn, endLine and endColumn
Adding access to IncludeFileInfo without opening up the guts of the library
Fixed a number of warnings
Few minor optimizations

about opening up the exported internal headers

Temporary change made to keep moving while we narrowed down on a longer term solution. The solution we discussed was to push the IncludeFileInfo like structure in the UHDM::design model. This would avoid opening the internals of the parsing and compiling to public API.

Looks OK in principle, as you said, please break it down in PRs of relevant changes.
I'll fill in the column info in the AnalyzeFile after this is pushed, please remind me after you push the relevant PR.
Basically, AnalyzeFile is a character based fast parser, one has to monitor carriage return to know which column you are in.
That should be a simple change.

Opened a narrowed down PR specific to this issue - https://github.com/chipsalliance/Surelog/pull/2586


sample data

Could you share a sample data to run the demo?

Hello, I apologize but I am unable to deploy dataset due to the DATASET LICENSE AGREEMENT.
I hope you can receive the dataset directly from AI City.

I understand, but it's hard to find a sample with many camera views as your demo. I just need a sample to test your code (like 5s video), not the entire dataset or even a sample outside the dataset. I hope you can help me (my email<EMAIL_ADDRESS>


[BUG] Possible floating-point bug, works in Py, fails in C++

In file bessel_j0.py
import numpy as np
from math import factorial

#pythran export bessel_asymp_p_coeffs(float64, int)
def bessel_asymp_p_coeffs(v, maxk):
    mu = 4*v*v
    termlist = np.zeros(maxk, dtype=np.float64)
    for k in range(1, maxk+1):
        numer = np.float(1.0)
        term = 0.0
        for i in range(1, 4*k - 1 + 1, 2):
            numer *= mu - float(i)**2
        term = (-1)**k * numer / float(factorial(2*k)*8**(2*k))
        termlist[k-1] = term
    return termlist

Compile to C++ using: pythran bessel_j0.py -o bessel_j0_pythran.so
In file bessel_j0_test.py:
import bessel_j0 as b0
import bessel_j0_pythran as b0p

print(b0.bessel_asymp_p_coeffs(0.0, 14)[:])
print(b0p.bessel_asymp_p_coeffs(0.0, 14)[:])

Run using python bessel_j0_test.py.
Output:
[-7.03125000e-02  1.12152100e-01 -5.72501421e-01  6.07404200e+00
 -1.10017140e+02  3.03809051e+03 -1.18838426e+05  6.25295149e+06
 -4.25939217e+08  3.64684008e+10 -3.83353466e+12  4.85401469e+14
 -7.28685735e+16  1.27972194e+19]
[-7.03125000e-02  1.12152100e-01 -5.72501421e-01  6.07404200e+00
 -1.10017140e+02 -2.51471836e+04  3.26365012e+10 -3.99259004e+15
            -inf             inf            -inf             inf
            -inf             inf]

Got it: it's because pythran does not support long ints, and factorial(2*k) grows too much :-/

That's actually a really good catch. I've never thought about that before. I've since fixed this algorithm, but is there any quick/easy way to fix this? Otherwise, I think we can close this, now that we've figured out the source of this problem.


Add typescript definition to tiletype

In index.js fixed minor JS Docs @returns
Added tests for types (optional). By running tsc test/types.ts without any errors is a pass.

@tmcw Small Typescript binding for this library, it's not available on DefinitelyTyped, so I figured it's best to place in the root of the library.

👍 awesome, merging. I hope someday TypeScript and Flow merge and become cross-compatible :)

@tmcw I also agree! I believe Flow will have to conform to Typescript (my personal opinion) since Typescript has been pushing very hard with Microsoft, both of the interfaces are very similar.
You can clearly see the difference between both Flow & Typescript to make a "central" type repository (DefinitelyTyped is crushing it with 3000 contributors & 24.8K+ commits):
https://github.com/flowtype/flow-typed
https://github.com/DefinitelyTyped/DefinitelyTyped
The issue I'm having with Flow is I can't really figure out how their interfaces work, they don't have much documentation or live examples of it.
Thanks for the merge! I hope you don't mind me pushing all of these Typescript interfaces, most of my projects involve using Typescript (or vanilla ES6).
👍


adding division sets

GetMyDivisions test still fails.
status_message" : "Processing exception (message): Cannot join division set, already joined.

+1 tests passe, much simpler. thx


Add Support for GLTF 3D Format

Tracking issue for customer request to support GLTF 3D format

GLTF Supported has been added since ViroCore 1.7, as such resolving.


geo raster accessor

Hi, I have put together a very simple package that provides a universal read function for reading various raster formats including of course netCDF but also any other format that gdal or rasterio can recognize. This read function can also handle merging several tiles into one dataset.
In addition, the package provides a .geo dataset accessor that currently adds trimming functionality to extract a geographical subset of the data.
I plan to also add reprojection and spatial resampling methods which will wrap either rasterio functionality or directly use gdal's api.
I hope this is of interest to the geosciences community and perhaps even a broader community.
Contributions and any other input from others is of course welcome.
Have a quick look at the Demo section in the readme file to get some ideas as to what this package can do for you.

Looks like the discussion is at https://github.com/corteva/rioxarray/issues/4 now


Can Support 2.3.x Range Search?

Through the latest documentation of 2.3.x, I found that milvus itself supports range search, but when I tried to use cpp sdk for range query, I found that it did not work.
My query code is as follows:
// index use HNSW
// metric type is IP
milvus::SearchArguments arguments{};
arguments.SetCollectionName(collection_name);
vectorq_vector {0.1, 0.8, 0.1, 0.1};
arguments.SetGuaranteeTimestamp(milvus::GuaranteeStrongTs());
arguments.AddTargetVector(faceField, std::move(q_vector));
arguments.SetMetricType(milvus::MetricType::IP);
arguments.AddExtraParam("radius", 0.2);
arguments.AddExtraParam("range_filter", 10.0);
arguments.AddExtraParam("ef", 10);
arguments.AddOutputField(trackIdField);
int64_t topk = arguments.TopK();
cout << "topk:" << topk << endl;
milvus::SearchResults search_results{};
status = client->Search(arguments, search_results);
I has two entities both distance within target vector is between 0.2 and 10.0,but the search result is always one,
topk output is 1,So I doubt if the default topk parameter will affect the results (according to the documentation for range queries, the topk parameter is not required for range queries)。

How about for having a workaround by set top to a large number, e.g arguments.SetTopK(...) before do search.
Sorry for long time not maintain this project, this sdk may even not tested with the latest milvus's version(2.3)

if set topk set to a larger number, 10, then the result count is always 10,and some of the distances do not meet the requirements of range search. It seems that only topk plays a role.

@regain0001 you could try latest master code, it should be fixed by #248

thanks


minor changes in links and translations

CONTRIBUTING.md : changing one link  in order to avoid the process of redirection from
https://learn.freecodecamp.org to https://www.freecodecamp.org/learn/
docs\i18n-languages\spanish\CONTRIBUTING.md : minor changes in text links in order to be more similar to the English text and removing some parts that doesn't work and doesn't exist in the English Version.

[x] I have read freeCodeCamp's contribution guidelines.
[x] My pull request has a descriptive title (not a vague title like Update index.md)
[x] My pull request targets the master branch of freeCodeCamp.
[x] None of my changes are plagiarized from another source without proper attribution.
[x] All the files I changed are in the same world language (for example: only English changes, or only Chinese changes, etc.)
[x] My changes do not use shortened URLs or affiliate links.

Closes #XXXXX

See: https://github.com/freeCodeCamp/freeCodeCamp/pull/37525#issuecomment-548345753

Duplicate of #37593

It seems that similar changes have already been created in another PR you created, so we are closing this PR.


General Issue:  Upversion stringi in renv.lock?

Background Information
I encountered the below error message after running renv::restore(). It seems stringi 1.5.3  can't be compiled on my end, 1.7.12 works. Would it be okay to upversion stringi to 1.7.12 in renv.lock?
** libs
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I.    -UDEBUG -DNDEBUG -DU_HAVE_ELF_H    -fpic -fpic  -g -O2 -ffile-prefix-map=/home/jranke/git/r-backports/bullseye/r-base-4.2.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2  -c stri_brkiter.cpp -o stri_brkiter.o
stri_brkiter.cpp: In member function ‘void StriBrkIterOptions::setType(SEXP, const char*)’:
stri_brkiter.cpp:60:9: error: ‘isNull’ was not declared in this scope; did you mean ‘Rf_isNull’?
   60 |     if (isNull(opts_brkiter)) {
      |         ^~~~~~
      |         Rf_isNull
stri_brkiter.cpp: In member function ‘void StriBrkIterOptions::setLocale(SEXP)’:
stri_brkiter.cpp:130:9: error: ‘isNull’ was not declared in this scope; did you mean ‘Rf_isNull’?
  130 |     if (isNull(opts_brkiter)) {
      |         ^~~~~~
      |         Rf_isNull
stri_brkiter.cpp: In member function ‘void StriBrkIterOptions::setSkipRuleStatus(SEXP)’:
stri_brkiter.cpp:171:9: error: ‘isNull’ was not declared in this scope; did you mean ‘Rf_isNull’?
  171 |     if (isNull(opts_brkiter)) {
      |         ^~~~~~
      |         Rf_isNull
make: *** [/usr/lib/R/etc/Makeconf:178: stri_brkiter.o] Error 1
ERROR: compilation failed for package ‘stringi’
* removing ‘/home/kaz/admiral/renv/staging/1/stringi’
Error: install of package 'stringi' failed [error code 1]

R session info
> sessionInfo()
R version 4.2.2 (2022-10-31)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Debian GNU/Linux 11 (bullseye)

Matrix products: default
BLAS/LAPACK: /usr/lib/x86_64-linux-gnu/libmkl_rt.so

locale:
 [1] LC_CTYPE=en_CA.UTF-8       LC_NUMERIC=C               LC_TIME=en_CA.UTF-8        LC_COLLATE=en_CA.UTF-8
 [5] LC_MONETARY=en_CA.UTF-8    LC_MESSAGES=en_CA.UTF-8    LC_PAPER=en_CA.UTF-8       LC_NAME=C
 [9] LC_ADDRESS=C               LC_TELEPHONE=C             LC_MEASUREMENT=en_CA.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] stats     graphics  grDevices datasets  utils     methods   base

loaded via a namespace (and not attached):
[1] compiler_4.2.2 tools_4.2.2    renv_0.16.0

Definition of Done
No response

I have set it on the agenda of the admiral developer meeting tomorrow.

This is because the R Version is not 4.0.5, which is specified in the lock file.  The stringi package version in lock file is not compatible with R version 4.2.
So if we upgrade stringi then we have to upgrade the R version as well...I think.

@kaz462 , we will update renv.lock soon such that it supports more than one R version. I.e., depending on the R version you are using you will get a set of working packages.
The renv.lock is maintained centrally in admiralci. Thus you should not change it in a PR.
We are also investigating to provide an environment for developers via containers.

good to know, thanks @bundfussr @bms63 !

I'm also getting this issue.
Installing stringi [1.5.3] ...
	FAILED
Error installing package 'stringi':
===================================

* installing *source* package ‘stringi’ ...
** package ‘stringi’ successfully unpacked and MD5 sums checked
** using staged installation
checking for R_HOME... /usr/lib64/R
checking for R... /usr/lib64/R/bin/R
checking for endianness... little
checking for R >= 3.1.0 for C++11 use... yes
checking for R < 3.4.0 for CXX1X flag use... no
checking for cat... /usr/bin/cat
checking for local ICUDT_DIR... icu61/data
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking how to run the C preprocessor... gcc -E
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking elf.h usability... yes
checking elf.h presence... yes
checking for elf.h... yes
checking whether we are using the GNU C++ compiler... yes
checking whether g++ -std=gnu++11 accepts -g... yes
checking whether the C++ compiler supports the long long type... yes
checking whether the compiler implements namespaces... yes
checking whether the compiler supports Standard Template Library... yes
checking whether std::map is available... yes
checking for pkg-config... /usr/bin/pkg-config
checking with pkg-config for the system ICU4C... 72.1
checking for ICU4C >= 55... yes
checking for additional required CPPFLAGS, LDFLAGS, and LIBS... done
checking whether we may build an ICU4C-based project... yes
checking programmatically for sufficient U_ICU_VERSION_MAJOR_NUM... yes
checking programmatically if U_CHARSET_IS_UTF8 is defined and set... yes
checking for available ICU data library (ucnv, uloc, utrans)... yes
checking for available ICU data library (ucol)... yes
configure: creating ./config.status
config.status: creating src/Makevars
config.status: creating src/uconfig_local.h
config.status: creating src/install.libs.R

*** stringi configure summary:
    ICU_FOUND=1
    STRINGI_CXXSTD=CXX_STD=CXX11
    STRINGI_CFLAGS=   -fpic
    STRINGI_CPPFLAGS=-I.    -UDEBUG -DNDEBUG -DU_HAVE_ELF_H
    STRINGI_CXXFLAGS=   -fpic
    STRINGI_LDFLAGS=-L/usr/lib
    STRINGI_LIBS=-licui18n -licuuc -licudata

*** compiler settings used:
    CC=gcc
    LD=g++ -std=gnu++14
    CFLAGS=-g -O2 -Wall -pedantic -fstack-protector-strong -D_FORTIFY_SOURCE=2 -fpic
    CPPFLAGS=-I/usr/local/include   -UDEBUG -DNDEBUG
    CXX=g++ -std=gnu++11
    CXXFLAGS=-g -O2 -Wall -pedantic -fstack-protector-strong -D_FORTIFY_SOURCE=2 -fpic
    LDFLAGS=
    LIBS=

** libs
g++ -std=gnu++11 -I"/usr/include/R/" -DNDEBUG -I.    -UDEBUG -DNDEBUG -DU_HAVE_ELF_H  -I/usr/local/include  -fpic -fpic  -g -O2 -Wall -pedantic -fstack-protector-strong -D_FORTIFY_SOURCE=2 -c stri_brkiter.cpp -o stri_brkiter.o
stri_brkiter.cpp: In member function ‘void StriBrkIterOptions::setType(SEXP, const char*)’:
stri_brkiter.cpp:60:9: error: ‘isNull’ was not declared in this scope
   60 |     if (isNull(opts_brkiter)) {
      |         ^~~~~~
stri_brkiter.cpp: In member function ‘void StriBrkIterOptions::setLocale(SEXP)’:
stri_brkiter.cpp:130:9: error: ‘isNull’ was not declared in this scope
  130 |     if (isNull(opts_brkiter)) {
      |         ^~~~~~
stri_brkiter.cpp: In member function ‘void StriBrkIterOptions::setSkipRuleStatus(SEXP)’:
stri_brkiter.cpp:171:9: error: ‘isNull’ was not declared in this scope
  171 |     if (isNull(opts_brkiter)) {
      |         ^~~~~~
make: *** [/usr/lib64/R/etc/Makeconf:178: stri_brkiter.o] Error 1
ERROR: compilation failed for package ‘stringi’
* removing ‘/home/rmagno/dwl/admiral/renv/staging/1/stringi’
Error: install of package 'stringi' failed [error code 1]

What version of R are you using @ramiromagno ?

My apologies for not providing more context in advance.
sessionInfo()
#> R version 4.2.2 (2022-10-31)
#> Platform: x86_64-pc-linux-gnu (64-bit)
#> Running under: Arch Linux
#>
#> Matrix products: default
#> BLAS:   /usr/lib/libopenblasp-r0.3.21.so
#> LAPACK: /usr/lib/liblapack.so.3.11.0
#>
#> locale:
#>  [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
#>  [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8
#>  [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8
#>  [7] LC_PAPER=en_US.UTF-8       LC_NAME=C
#>  [9] LC_ADDRESS=C               LC_TELEPHONE=C
#> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
#>
#> attached base packages:
#> [1] stats     graphics  grDevices utils     datasets  methods   base
#>
#> loaded via a namespace (and not attached):
#>  [1] rstudioapi_0.14   knitr_1.41        magrittr_2.0.3    R.cache_0.16.0
#>  [5] rlang_1.0.6       fastmap_1.1.0     stringr_1.5.0     styler_1.9.0
#>  [9] highr_0.10        tools_4.2.2       xfun_0.36         R.oo_1.25.0
#> [13] cli_3.6.0         withr_2.5.0       htmltools_0.5.4   yaml_2.3.6
#> [17] digest_0.6.31     lifecycle_1.0.3   purrr_1.0.1       vctrs_<IP_ADDRESS>00
#> [21] R.utils_2.12.2    fs_1.5.2          glue_1.6.2        evaluate_0.20
#> [25] rmarkdown_2.19    reprex_2.0.2      stringi_1.7.12    compiler_4.2.2
#> [29] R.methodsS3_1.8.2

Created on 2023-02-15 with reprex v2.0.2

Cheers!

Okay to close this issue @kaz462 and @bundfussr ?

I'm fine to close this issue, thanks @bms63 !


Install extension while browser is in Incognito mode, auto-enable in incognito mode?

One thing for your awareness though. It wouldn’t let me add the extension while I was in Chrome Incognito mode. I could hit the link/store where the extension was visible, but the button to add to Chrome was greyed out. It had a notice saying, it couldn’t be loaded while in incognito. I figured out a work around, by opening up a normal window, loading it, then going into the extensions and clicking the “allow in incognito”. Then it showed up.
I wonder if in the future, if there’s an option when you’re building the extension, to select to allow in incognito.

It looks like you have to explicitly go through the settings in chrome. Maybe we can add instructions in the readme or a wiki entry.


added a delay to the NewClimateClock.module.css line#19 for 2.5 seco…

issue #439
Incorrect information showing on the Countdown Counter before loading correct info #430
add this to NewClimateClock.module.css
.clock {
animation: openClock 2.5s;
}
@keyframes openClock {
0% {
display: none;
}
100% {
display: none;
}
}

Thanks @MyBaBB! I'm still seeing NaN for just a microsecond but I don't think it's a big deal, this is a big improvement as far as getting the numbers to show earlier.

I will make the delay just a little longer,  I just noticed that is not compatible with FireFox.  I will fix now .


REPL command history is broken on Ubuntu 14.04

When pressing the up key any more than once, the REPL throws an error:
The value must be greater than or equal to zero and less than the console's buffer size in that dimension. Parameter name: top Actual value was -1.
And then terminates.  I tried with a few different terminal dimensions and had the same results.

thanks, does it not work at all (breaks on first up/down) or does it only break in specific situations? could you provide some simple repro steps?
To be honest I only tested on Ubuntu 18.04 but seems to work fine there 😄

As far as I can tell, it breaks on the second up press in every situation.  Here's a simple repro:

> var x = "hi";
> x
"hi"
**press up**
> x
**press up again**
*crash with error*


Timeline entries should not end up in history

open timeline for something
select many to its entries to find a certain version
😞 the history is now full of timeline entries that I never use again

@jrieken is this still happening for you? I can't reproduce this.


python311Packages.mediafile: 0.10.1 -> 0.12.0

Diff: https://github.com/beetbox/mediafile/compare/v0.10.1...v0.12.0
Changelog: https://github.com/beetbox/mediafile/blob/v0.12.0/docs/index.rst#changelog
Description of changes

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 268747 run on x86_64-linux 1

14 packages failed to build:

beets
beets.dist
beets.doc
beets.man
beetsPackages.alternatives
beetsPackages.alternatives.dist
beetsPackages.beets-minimal
beetsPackages.beets-minimal.dist
beetsPackages.beets-minimal.doc
beetsPackages.beets-minimal.man
beetsPackages.copyartifacts
beetsPackages.copyartifacts.dist
beetsPackages.extrafiles
beetsPackages.extrafiles.dist

13 packages built:

beets-unstable
beets-unstable.dist
beets-unstable.doc
beets-unstable.man
home-assistant-component-tests.apple_tv
python310Packages.mediafile
python310Packages.mediafile.dist
python310Packages.pyatv
python310Packages.pyatv.dist
python311Packages.mediafile
python311Packages.mediafile.dist
python311Packages.pyatv
python311Packages.pyatv.dist

Result of nixpkgs-review pr 268747 run on x86_64-linux 1

6 packages failed to build:

beets
beets.dist
beets.doc
beets.man
python312Packages.pyatv
python312Packages.pyatv.dist

21 packages built:

beets-unstable
beets-unstable.dist
beets-unstable.doc
beets-unstable.man
beetsPackages.alternatives
beetsPackages.alternatives.dist
beetsPackages.beets-minimal
beetsPackages.beets-minimal.dist
beetsPackages.beets-minimal.doc
beetsPackages.beets-minimal.man
beetsPackages.copyartifacts
beetsPackages.copyartifacts.dist
beetsPackages.extrafiles
beetsPackages.extrafiles.dist
home-assistant-component-tests.apple_tv
python311Packages.mediafile
python311Packages.mediafile.dist
python311Packages.pyatv
python311Packages.pyatv.dist
python312Packages.mediafile
python312Packages.mediafile.dist

Superseded by #291184


[stable/kiam] update service port name

What this PR does / why we need it:
update service port name
Which issue this PR fixes

fixes https://github.com/helm/charts/issues/8002

Special notes for your reviewer:
Checklist

[x] DCO signed
[x] Chart Version bumped

/lgtm

/assign @davidkarlsen

@goruha @mattfarina I saw we are using rc1 in the chart version and app version. for this PR what should be the chart version?
thanks

@cpanato I guess if this fix is not blocker and critical - let's use 2.0.0-rc-2

done @goruha

@davidkarlsen can you take a look at this one? thanks

/lgtm


api: GetLogs: use the correct type in LogsStreamWriter

Symptoms:

panic: interface conversion: []uint8 is not proto.Message: missing
method ProtoMessage

This patch fixes the panic, but there are other crashes that are not
related to LogsStreamWriter. I'll file a separate PR.
Helps with https://github.com/coreos/rkt/issues/2740

LGTM, but can we have tests for this?

LGTM, but can we have tests for this?

Can we add the tests in a separate PR? I filed https://github.com/coreos/rkt/issues/2746 for this.

@alban sure, thanks for filing the issue


Can't discover services after connecting 5 times without bonding

Hi all,
I'm implementing the nimble Bluetooth stack (esp-idf v4.1.1) into my ESP32-WROOM-32D project which has been set as BLE peripheral. After connecting 5 times using the NRF connect application (Android or IOS does not matter), I cannot discover services any longer. I have already tried several things but I can't figure out what goes wrong.
The settings:
void CEspNimbleBluetoothDevice::Init()
{
  esp_err_t result;
  result = esp_nimble_hci_and_controller_init();
  Assert(result == ESP_OK);

  nimble_port_init();

  ble_hs_cfg.reset_cb = PeripheralOnResetCallback;
  ble_hs_cfg.sync_cb = PeripheralOnSyncCallback;
  ble_hs_cfg.gatts_register_cb = GattRegisterEventCallback;
  ble_hs_cfg.gatts_register_arg = reinterpret_cast<void*>(this);
  ble_hs_cfg.store_status_cb = ble_store_util_status_rr;

  ble_hs_cfg.sm_io_cap = BLE_SM_IO_CAP_NO_IO; // No input/output capabilities (just works)
  ble_hs_cfg.sm_bonding = 1; // Enable bonding
  ble_hs_cfg.sm_sc = 1; // Use secure connection
  ble_hs_cfg.sm_our_key_dist = BLE_SM_PAIR_KEY_DIST_ENC | BLE_SM_PAIR_KEY_DIST_ID; // Enable LTK + IRK
  ble_hs_cfg.sm_their_key_dist = BLE_SM_PAIR_KEY_DIST_ENC | BLE_SM_PAIR_KEY_DIST_ID; // Enable LTK + IRK

  ble_svc_gap_init();
  ble_svc_gatt_init();

  memset(m_serviceTable, 0, sizeof(m_serviceTable));
  memset(m_characteristicRegistrationTable, 0, sizeof(m_characteristicRegistrationTable));
}

The SDK config is set as follows:

Decreasing the number of concurrent connections or number of bonds in the SDK config does not change the steps to reproduce.
Steps that I execute:

Push pairing button, so that the ESP accepts new connections by advertising with its name.
Scan and connect with the NRF connect application.
See if services can be discovered under the client tab.
Disconnect.

After repeating this 5 times, step 3 will not display any services. Logging in the application shows that discovering resulted in a nill. And also in the ESP logging does not give me any information about discovering.
Hope someone can help, thanks in advance!

Hi @Teu97 , I tried to reproduce the issue using bleprph example app and I could not face any issue. Is it possible for you to list out key differences between bleprph example and your custom example ? Can you please share sdkconfig file ? Can you please also share console logs with DEBUG prints enabled (menuconfg --> Components --> Log Component --> Log verbosity --> DEBUG) ?

Hi @Teu97 , I tried to reproduce the issue using bleprph example app and I could not face any issue. Is it possible for you to list out key differences between bleprph example and your custom example ? Can you please share sdkconfig file ? Can you please also share console logs with DEBUG prints enabled (menuconfg --> Components --> Log Component --> Log verbosity --> DEBUG) ?

Hi @prasad-alatkar, thanks for picking up this so quickly. I hope to answer your questions as best as possible:

Is it possible for you to list out key differences between bleprph example and your custom example ?

It is hard to tell the exact differences between both examples. The key differences in my opinion is the initialize as shared above. With the fact that I only want to allow a maximum number of 6 devices to bond with the ESP32. But the case is that I only connect my device and never execute a bond. I will try to minimize the number of differences to give you a better answer to this question.

Can you please share sdkconfig file ?

sdkconfig.txt

Can you please also share console logs with DEBUG prints enabled?

logging.txt
The most interesting part starts from 86424ms. At that moment I pressed the pairing button for the 6th time and connected the application with the ESP32. Then discovering seems to fail.

Hi @prasad-alatkar, I cleared all my events from the GapEventCallback and added them one by one. I noticed that the connect event was the problem. What happened was that we had a characteristic which keeps track if pairing was allowed. So if a connect event occurred pairing is no longer allowed and this could be notified if someone subscribed to the characteristic.
In our NotifyValue function we had:
os_mbuf* pBuffer = ble_hs_mbuf_from_flat(rMessage.data(), size);
for (uint16_t connectionHandle : m_notificationConnections[rNotifyCharacteristicUuid])
{
  ble_gattc_notify_custom(connectionHandle, m_notificationHandles[rNotifyCharacteristicUuid], pBuffer);
}

But it had to be:
for (uint16_t connectionHandle : m_notificationConnections[rNotifyCharacteristicUuid])
{
  os_mbuf* pBuffer = ble_hs_mbuf_from_flat(rMessage.data(), size);
  ble_gattc_notify_custom(connectionHandle, m_notificationHandles[rNotifyCharacteristicUuid], pBuffer);
}

In the code samples m_notificationConnections[rNotifyCharacteristicUuid] will be empty if no one is subscribed. So I ques that if no one is subscribed, notify is never called, and therefore the buffer is never freed. Discover will probably use the same buffer to allocate its messages, but it could not allocate any memory after repeating the sequence 5 times. Resulting in the fact that a discover would fail.
Do you know if I still have to check if pBuffer is not NULL or does it not matter for the ble_gattc_notify_custom?

Hi @prasad-alatkar, sorry that i'm not clear. Yes my issue has been solved, thanks for your help and support! My answers on your questions are below:

@Teu97 If I am not wrong bringing pBuffer inside the for loop resolved your issue of not able to discover services on 6th successful connection, right ?

Yes that is right.

Is it empty or zero valued ? I mean, 0 is a valid conn_handle (theoretically).

It is empty valued, if no one is subscribed it will never call ble_gattc_notify_custom. Which caused my issue because the allocated buffer was never freed.  m_notificationConnections[rNotifyCharacteristicUuid]) is a map containing a vector of subscribed handles for each uuid.

If I understand it correctly, I guess by moving pBuffer inside for loop, you have made sure that pBuffer is not wasted. If you keep outside of for loop, I think that should also be okay provided after sending notifications you check if pBuffer != NULL then call os_mbuf_free. This way you may not end up allocating os_mbuf for each conn_handle. You can also think of increasing CONFIG_BT_NIMBLE_MSYS1_BLOCK_COUNT, should you need more os_mbufs.

No I think that wont work because what I saw was that I had to call ble_hs_mbuf_from_flat each time I wanted to send a notification. I could not use pBuffer multiple times. Which can be right because ble_gattc_notify_custom also frees pBuffer.

To summarize my response, I am a little confused if your original issue of not able to discover services on 6th connection are solved or not. If we don't have sufficient number of os_mbufs then please increase MSYS1_BLOCK_COUNT as mentioned above. Please let me know if it helps you resolve the issue.

What made it so hard to found was that I did not expect the allocation for a notification to be the issue of discovering services not able to work.
Again, thanks for the support!

@Teu97 Glad that the issue is resolved, and yes ble_gattc_notify_custom does free the os_mbuf.


Mojito is not re-using existing translations for Android plurals

We found a situation where the old mojito (version 0.81) is generating the correct translations for plurals in an android app, but the latest mojito is not. (Version 0.84 and also the head of the master branch)
What is happening is that the latest mojito is generating all of the plural classes for various languages. This is a good thing because some of those classes were missing before. It is producing those strings with the English text because the translations are not there, which is also good. The part that is wrong is that the old mojito generated the strings for the "one" and "other" classes with the correct translations in them, but the new mojito only generates the string for the "one" class with the correct translation. The "other" string is in English, even though the translation is available.
I have distilled it down to a minimal set of files which I tarred up and put on my Box account publically: https://app.box.com/s/u9o7d0jg8hspe2p6h6yl4i1v02ose2na
The "bug.sh" script shows the commands I used to repro the problem. The source/res/values/strings.xml is the source file with just a single plural array in it. The tm/*.xliff are the translation memory files with the translations for Arabic, Polish, Russian, and Chinese. The localized/res/*/strings.xml files are the output files from running that script with mojito 0.84. You can see that the "other" classes are missing the translations that exist in the TMs for each of those locales.

@ehoogerbeets the xliff files don't look good. 'ar-SA' is missing some forms and 'zh' has too many though this shouldn't hurt.
If I remember well I didn't really look into the TM import/export when I implemented the plural support so it might have never worked.
Are the drops working fine with plural (is putting the right amount of forms for each language)? Can you confirm that plural support is ok in the workbench and generated files when not using the TM import/export?

I don't remember exactly the timeline of the releases and I thought you were on an higher version but there might be something with the PluralFormUpdater not ever being ran.
I made it optional and disabled by default it at some point and it is possible it was never run based on your version.
I'm reintroducing that as something to be run only once with the Quartz Scheduler that I'll push soon. but for now you can enable it with: l10n.PluralFormUpdater=true property.
If ran for the first time it does changes to the db so that plural support works fine. Could you check if that helps?

Sure, I'll check it and get back to you.

Actually, we are testing with HSQL so the PluralFormUpdater won't be run:
PluralFormUpdate.java:44
        } else if ("HSQL".equals(driver)) {
           logger.debug("Don't update (DB is HSQL)");

ok, plus disregard what i said. the PluralFormUpdater should keep running until all repositories are migrated. One time execution is not good, I need to fix the change made to Quartz and I'll enable it by default. To prevent it to run it will have to be explicitly defined in the config. It will make things safer.
I'll have to debug to see what's going with your issue. Can't say much more like that.

I think I know what's going on, it is a little messy:
<!-- Delete -->
    <plurals name="numberOfDeleteFolders">
        <item quantity="one" description="Confirmation to delete a single folder e.g. 'Are you sure you want to delete the folder 'Work' and all its contents?">Are you sure you want to delete the folder <b>\'%2$s\'</b> and all of its contents?</item>
        <item quantity="other" description="Confirmation to delete more than one folder" >Are you sure you want to delete these %1$d folders?</item>
    </plurals>

The Android filter accept 2 ways of giving translator directives:

XML comment before the text unit
"description" attribute in the XML element

"description" attribute doesn't seem to be extracted for plural forms, that probably only works for simple entries. So Delete is used a source comment. (This is probably not what you want in your case since the developer gave good comments in "description" but anyway that's another story)
So the problem is that in the TM export the "other" form doesn't have a source comments. only the "one" form has it ("Deleted"). Probably due to the older behavior of the filter.
The current filter make it so that the source comment is the same for all forms. So when pushing the source file, it creates text unit for all forms with "Deleted" source comment. The it finds an exact match for "one" but doesn't for "other".
If you look at unused translation you'll find the old string for "other" with the old translations actually imported. I would have expected the leveraging to kick in and copy the translations... That must be some edge case again.


Add asserts for combintorial output combiner.

This PR also wires the new tests into the CI system.  This does require https://github.com/SymbiFlow/prjxray-db/pull/7 because fasm2verilog needs the pin maps to generate the correct top-level signal names.

Opps, forgot this depends on https://github.com/YosysHQ/yosys/pull/928
I'll poke YosysHQ and see if they will merge soon, otherwise I'll merge on our fork.