[BUG] Batching using New-PnPBatch fails with Could not load file or assembly

Reporting an Issue or Missing Feature
Batching using New-PnPBatch fails with code that seems  to follow examples, old working code and even ChatGPT 😅 .
Powershell 7.3.2 and PnP.Powershell 1.12.0; uninstalling / reinstalling hasnt helped.
Expected behavior
expect batched PnPonline functions (e.g. Add-PnPListItem, Set-PnPListItem etc) to aggregate and execute
Actual behavior
Script fails immediately calling New-PnPBatch with error
Could not load file or assembly 'Microsoft.Extensions.DependencyInjection, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=adb9793829ddae60'. The system cannot find the file specified.
Steps to reproduce behavior
run anything calling New-PnPBatch, e.g.:
Connect-PnPOnline -Url $SiteURL -UseWebLogin

$batch = New-PnPBatch

foreach($i in 1..2)
{
Add-PnPListItem -List $ListName -Values @{
    "Title" = "batchtest doubles"
    } -Batch $batch
}
#run it
Invoke-PnPBatch -Batch $batch

What is the version of the Cmdlet module you are running?
1.12.0
Which operating system/environment are you running PnP PowerShell on?

[X ] Windows
[ ] Linux
[ ] MacOS
[ ] Azure Cloud Shell
[ ] Azure Functions
[ ] Other : please specify

@bGlasson - I have no issue running your script with 1.12.0. can you please try again, maybe it was at a point in time. Reopen if the problem persist 🙂

Sadly issue is definitely still live on my machine.
PowerShell 7.3.2 still behaves as above.
PowerShell 5.1.3 fails immediately with the connect:
Connect-PnPOnline : The type initializer for 'Microsoft.SharePoint.Client.ClientContextExtensions' threw an exception.

Connect-PnPOnline -Url $SiteURL -UseWebLogin

  + CategoryInfo          : NotSpecified: (:) [Connect-PnPOnline], TypeInitializationException
  + FullyQualifiedErrorId : System.TypeInitializationException,PnP.PowerShell.Commands.Base.ConnectOnline

For clarity - would rather a working solution for 7.. but I need a batched solution asap.... if I can solve 5.1x, that would at least get me working. Any help would be greatly appreciated.

Still no issue on PS5.
As for the connection, please try using -Interactive instead of -UseWebLogin which is legacy authentication).
https://pnp.github.io/powershell/cmdlets/Connect-PnPOnline.html#example-9

first - REALLY appreciate the help, this is killing me 😅
Would not have expected auth method impact execution.  Have used -UseWebLogin because it plays nice with the SSO-only world I'm in..... was working (with batching etc) until very recently. Tried with -Interactive, got the same message:
Line |
23 |  Connect-PnPOnline -Url $SiteURL -Interactive #-UseWebLogin
|  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| Could not load file or assembly 'Microsoft.Extensions.DependencyInjection, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=adb9793829ddae60'. The system cannot find the
| file specified.
If I comment out the batching with -Interactive in 7, I get the error, script finishes and list is not updated. If I return to -UseWebLogin without the batching, it works as expected, iterating through the updates.
full code with batching including comments etc below ($1,$2,$3 to protect modesty 😅), just incase there's something relevant:
$SiteURL = $1
$ListName = $2

#csv with changes you want to make
$csv1 = Import-CSV $3
$now = (get-date).ToString("yyyy-MM-ddThh:mm:ssZ")

#start the time-to-complete clock
$startTime = Get-Date

#Connect to PnP Online
Connect-PnPOnline -Url $SiteURL -UseWebLogin

# make a Batch of all changes to make from CSV file
$batch = New-PnPBatch
foreach($item in $csv1)
{
  # Make change to SPO
Set-PnPListItem -List $ListName -Identity $item."Id" -Values @{

    # sharepoint field name = $item."NAME OF FIELD IN CSV"
    # multichoice fields need ($item."name of field").split(",")
    # date fields work with dd/mm/yyyy format input
    # single people pickers take email addresses
    # lookup fields need the list item Id from the lookup list not the value (e.g. 5 , not "lookupTitle1")

    #fields commented out if they're not required for now

    #"Contract" = $item."cjContractId"
    "cjDocTypeV2" = $item."cjDocTypeV2";
   # "cjDocFlagsV2" = ($item."cjDocFlagsV2").split(",");
    "cjComments" = $item."cjComments";
    "cjStatus" = $item."cjStatus";
    "cjStatusChanged" = $now;

   }-Batch $batch

  }

# Execute the batch all at once
  Invoke-PnPBatch -Batch $batch -Details

#stop the clock, calc time-to-complete tell me about it in console
$endTime = Get-Date
$totalTime = $endTime - $startTime

write-host "Total script run time: $($totalTime.Hours) hours, $($totalTime.Minutes) minutes, $($totalTime.Seconds) seconds" -ForegroundColor Cyan

I've tried with -UseWebLogin, PS 7.3.2 and batching, and again no issue on my side. Maybe there's something on your system blocking somewhere, I'm not sure. If you can try to connect on another machine, or run your script with real values instead of a .csv / variables. But can't really help as everything is working on my side...

thanks for trying again
Have validated on 3 other machines and two other user accounts in my environment. Have also tried hardcoding everything (urls, list names, list items to change, list values to update) and restricting batch to a single item (they all fail the same). Am sure this was caused by a change on our side, but I don't know where to start challenging it.
I'd guess this is client side (windows patch? security config? m365 client patch?)... but I'm not certain it couldn't be a network change or even SPO config.
Was hoping someone could point out the embarrassing problem with my code or give me a list of a few things most likely to break it in this way .... that way I know which team to ask for help and I've got something more than "my code broke, was it you?" to start with 😅
Could not load file or assembly 'Microsoft.Extensions.DependencyInjection, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=adb9793829ddae60 <---- any comments / suggestions on this would be very very welcome 🥺

Wild shots in the dark... can you confirm if you're on same patch level? Most recent change for me were:
KB5022303 installed (I'm on Win 22h2)
and
.Net 4.8.09032 (release 533320) ... (check registry keys at HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\NET Framework Setup\NDP\v4\Full)
Some fixes for Microsoft.Extensions.DependencyInjection refernce underlying dependencies include .Net.... and only other lowish-fruit I can think of would be a windows patch :)

Yes, all the same - Win11 22H2

Thanks for confirming.... just to double check - you've got KB5022303 installed on 22H2 too?

No I don't seem to have it installed.

If I can get permission to roll it back, I'll see if that's the cause. Thanks!


[BUG] Balances: Allow hex currencies (RLJS-168)

Hex currencies were returning an error:
// GET https://api.ripple.com/v1/accounts/rQE5Z3FgVnRMbVfS6xiVQFgB4J3X162FVD/balances?currency=0000000000000000000000000000000000000000
{
  "success": false,
  "error_type": "invalid_request",
  "error": "restINVALID_PARAMETER",
  "message": "Parameter is not a valid currency: currency"
}

LGTM

LGTM


DBZ-1812

DBZ-1812 add e2e benchmark

Hey @zrlurb, thanks a lot for this. Just a quick heads-up that I'll get to reviewing this only next week, as I'm swamped with things this week. Sorry for the delay!

@zrlurb Hi, thanks a lot for the job. Generally it seems to be fine but it needs more work to make it generci tool from usbility point of view.
Also ae there any code formatters for Python? If yes please run the code through them to make it consistent in all places.

@jpechane, could you take another look? In general, how should we proceed with this one? Would it be usable for say the Postgres connector already?

@gunnarmorling There are still few minor unaddressed comments but I think it should be already usable.

Hey @zrlurb, any chance you could address those remaining remarks by Jiri? Thanks!

@jpechane, IIRC, you wanted to take care of this one, right?

Followed-up with https://github.com/debezium/debezium/pull/1604


Script to insert the minimal CSS in your HTML

First of all; minimalcss should be about generating that minimal CSS. That's the core and it's good to only have 1 core function.
I have my own hacky solutions (that work because the scope is tight) that change the HTML (e.g. build/index.html) so that it first loads the minimal CSS and then lazy loads in the CSS with JavaScript.
It would be nice to build such a script into the repo so it can be shared and improved on collectively.
Another option is to build this script into a separate repo and separate npmjs.com package.

Did any script to inject CSS into html get written?  Would be a huge help for something I'm working on!

@JohnAllen Nothing got added to this repo and I think that's good. It's just too many "it depends". For example, I have my own blog that uses minimalcss but the tooling that replaces the <link rel=stylesheet> tags with <style> (and lazy loading <link>) is done in Python.
If you're looking for inspiration you should definitely look at react-snap.

Cool.  I think what I need/will use is this: https://github.com/Automattic/juice. Want to extra CSS from all files (html, CSS + JS) and inline it.

Cool. I think what I need/will use is this: https://github.com/Automattic/juice. Want to extra CSS from all files (html, CSS + JS) and inline it.

No. That's something entirely different.
What tools like react-snap does is it converts...
<link rel=stylesheet href=/bloated.css>
</head>

...into this...
<style>
...minimal CSS extracted from bloated.css...
</style>
<link rel=preload as=style href=/bloated.css>
</head>

I get that.  The other thing is what I need though.


Adds Spring Boot 1.5 (webmvc4) example

Fixes #14

ShouldI wait for it merged into master to check it out?  Although  you hava commited the spring boot demo

if you can checkout the branch, this is best. unless you cannot check out a
branch


feat: add msgpack 3.0.0 release

update msgpack with release 3.0.0

:tada: This PR is included in version 1.367.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


config: Search for config files in default places

Fixes https://github.com/channable/vaultenv/issues/57
This is an enhancement to make vaultenv more useful on devmachines.
Currently users need to specify stuff like --host, --port, and --token manually (or create a wrapper script which does). This changes vaultenv to look for for files with the .env format in three places:

/etc/vaultenv.conf
$HOME/.vaultenv/config.conf
$(cwd)/.env

New config file precedence in case of duplicate settings:

/etc/vaultenv.conf
$HOME/.vaultenv/config.conf
$(cwd)/.env
Env vars which are set
CLI options

Could we use .config/vaultenv as specified in https://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html ? This makes it play nice with systemd (user) units


resource leak if exception in SerialImpl constructor

https://github.com/wjwwood/serial/issues/145

lgtm thanks


Google Cloud Storage support

manual QA passed.
now adding unit testing on it.

Hi Colin,
It also looks like you have some flake8 issues to clear up, if you didn't notice that build fail.
-Wes

I checked with PyFlake from my side, didn't see any error in the code I added, but found two additional errors in the beginning of tester classes, now fixed.
but from the message it seems like the error refers to the command-level error? if pep8 error it should point out which line.

oh wait, I checked the wrong files. let me fix it.

figured it out, I recently made my PyFlake vim plugin to ignore E501 errors (line too long exceed 80 chars per line), fixed all lines exceed the 80 chars.

now this left:
ERROR: InvocationError: '/home/travis/build/PolicyStat/terrarium/.tox/py26-venv-1.8.4/bin/nosetests -v'

Other than the typo, I think this needs:

A unittest like this one that verifies the error message/code if GCS isn't installed
GCS documentation that mirrors the s3 docs to let folks know they can use GCS.

It looks like your last failing travis build might have been just intermittent. I made a small change to master just to confirm that the travis build is passing there and it is.

@winhamwr : can you re-run the build#199 on Travis?

Hi Colin,
Maybe fix the typo and push? I'm on my mobile and don't see an obvious way to do that.
-Wes

docs added.

Excellent. Thanks, Colin!


Add date time to log

Is it pointed to the headless version?

Is it pointed to the headless version?

Is it pointed to the headless version?

both headless and headful, I have added date-time into log messages, just to keep consistency.

Code from this PR was manually added to the application, because there where a lot of changes (full refactoring)


Refactor spawn generation to use a queue

Currently the spawn generation pushes all the chunks to be generated into the generator at once, causing an overload if the generation distance is larger than 20.
Refactor the cWorldLoadProgress and cWorldLightProgress helper threads to actually feed the chunks to be loaded / generated / lit to the processors incrementally.

Done.


Bower walkthrough steps deprecated VS2017 v15.6.6

This tutorial maintains an outdated method of installing javascript packages as bower appears to no longer be supported as of Visual Studio 2017 (v15.6.6). The article should be updated to use current approaches to installing the packages (Be it npm, yarn, or webpack, or a combination).

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: c0d03b4f-4157-0d04-0f48-0a3e2fb966cb
Version Independent ID: 6b7bed89-eb53-105a-7971-476ddd4cf3b7
Content: Create a .NET application for Service Fabric
Content Source: articles/service-fabric/service-fabric-tutorial-create-dotnet-app.md
Service: service-fabric
Product: unspecified
GitHub Login: @rwike77
Microsoft Alias: ryanwi

@tlagore The ASP.NET team is recommending LibraryManager as a replacement for Bower.  We'll update this tutorial soon.  Thanks for the feedback.

Me too not able to follow bower configuration, is there an alternate i want to create the azure service, but stuck.

I see the bower instructions have been bumped... better, but still not complete.  you need a .bowerrc file... create json file at project level from wizard >> name it ".bowerrc" >> add {"directory":"wwwroot/lib"} | visual sutdios will put this in the correct spot (under your bower.json) and the script libraries will now resolve to the correct folder in your wwwroot.

@Justin-Mann, you're right.  I'm updating to include setting the .bowerrc file.
LibraryManager is still in preview, I don't think it's quite ready yet so we added info on manually installing using Bower.  We've received a number of comments that this tutorial is more complicated than it needs to be.  We're also looking at ways to make this tutorial (and the sample app) simpler.

Article  has been updated with info on creating the .bowerrc file.  I believe we have all the steps covered, now.  #please-close


[SPARK-38982][PYTHON] Skip categories setter test

What changes were proposed in this pull request?
Since https://github.com/pandas-dev/pandas/commit/126a19d038b65493729e21ca969fbb58dab9a408, pandas changes behavior, according to https://github.com/pandas-dev/pandas/issues/46820, categories setter is trying to modify an immutable object, it's not a recommend way.
Why are the changes needed?
Make CI passed with pandas 1.4.x
Does this PR introduce any user-facing change?
No
How was this patch tested?
test_categories_setter passed with 1.3.X and also 1.4.x

BTW, we added the categories setter since https://github.com/apache/spark/pull/33448 .

Seems like they replied https://github.com/pandas-dev/pandas/issues/46820#issuecomment-1113889315.
Maybe can we just close ?

@itholic Now we'd better to only fix test, becasue categories.setter will not be completed supported in Pandas, and also some inplace update methods like set_categories are deprecated.
So, I suggest only fix test and keep current behavior, then remove this setter support when we remove all deprecated methods.

@itholic I plus some note on this, and mark it read for review, thanks for reminder!

Got it! Looks good.

Merged to master.


Plugin for wordpress?

Is there a plugin for wordpress with Trumbowyg? I absolutely love it and would like to have it embedded as plugin and replace the wordpress fugly editor.

That's an interesting idea. Maybe a day :)

I thought about that and I don't this do that because it's sooo complex to support all WordPress Core possibilities : gallery, media, plugins, ...
Sorry :/


Update lucene version to 8.8.2

This PR updates the codebase to the latest released version of Lucene. API changes are included and tested.

start gradle check

start gradle check

start dco check
start wrapper validation
start gradle precommit

@rmuir do you want to give this another round of review?

start gradle check

start gradle check

start gradle check

start gradle check

start gradle check

start gradle check

start gradle check

@nknize this one fails with lots of version related test failures. Most interesting from the build log, are the ones that cause class initialization failures, e.g.:
org.opensearch.index.reindex.DeleteByQueryConcurrentTests > testConcurrentDeleteByQueriesOnDifferentDocs FAILED
    java.lang.ExceptionInInitializerError
        at org.opensearch.cluster.node.DiscoveryNodeRole.<clinit>(DiscoveryNodeRole.java:227)
        ...

        Caused by:
        java.lang.NullPointerException: Cannot read field "id" because "version" is null
            at org.opensearch.Version.<clinit>(Version.java:114)
            ... 49 more

    java.lang.NoClassDefFoundError: Could not initialize class org.opensearch.Version

I'm also seeing wacky racy stuff locally with NoClassDefFoundError for integ tests (in the main branch), and also a unit test failure (~50% of the time) related to versioning (which i have muted locally already).
I think part of the problem is that there is a circular clinit dependency between Version and LegacyESVersion. E.g. LegacyESVersion extends Version but Version.static {} refers to LegacyESVersion. I think this is causing classloading issues.
Should we revisit/solidify this version stuff before trying to upgrade lucene?

I think part of the problem is that there is a circular clinit dependency between Version and LegacyESVersion.

That was exactly the problem. I pushed a change so the LegacyESVersion fields are loaded before the static initialization block but this "fix" is still just another hack around the hack. It should fix the race conditions, though, so we could move forward and unblock a lucene upgrade (I'd like to get main on Lucene 9.0.0 ASAP to begin using all the cool Vector field stuff).
It doesn't fix the inherited tech debt though. IMHO the versioning logic needs a complete rewrite (interested? :) ) . The gradle implementation, for example, reads the Version.java as a text file and uses a regex to parse declared versions from the variable names! It's soo hacky.

start gradle check

IMO significantly changing this kind of logic almost always results in terrible bugs. So at the least we should reduce risk by pulling these changes away from this unrelated lucene upgrade PR, and get main branch stable?
Also maybe it is worth reconsidering how important is it to really reboot at version 1.0? I'm just throwing out the idea, because continuing with existing numbers means you could throw out LegacyESVersion completely, correct? It would reduce risk even more.

So at the least we should reduce risk by pulling these changes away from this unrelated lucene upgrade PR,

Yeah, I think splitting this into two PRs is important to track this tech debt. I just started doing that to avoid polluting the lucene changes PR w/ the versioning hacks. (looks like the CI infra is down though so we're getting that looked at)

continuing with existing numbers means you could throw out LegacyESVersion completely, correct?

Yes. And it would make things a LOT easier; but I think there are non-technical issues carrying forward w/ the versioning that lead to the desire to cleanly rebase the OpenSearch version to 1.0.0.

going to let the gradle check test complete before opening the split PR.

start gradle precommit

I can't tell, but it looks like the last one failed for several reasons:

version related test failures
thread blocked on entropy (seeding DRBG random)

Do CI machines have appropriate entropy (either newish kernel with random.trust_cpu=1 parameter or running rngd to seed entropy pools from RDRAND insn ?). I'm not sure all these tests and so on are careful about consuming precious entropy.

Do CI machines have appropriate entropy (either newish kernel with random.trust_cpu=1 parameter or running rngd to seed entropy pools from RDRAND insn ?)

/cc @bbarani @peterzhuamazon  (perhaps we should have an @infra team to ping the CI maintainers?)

continuing with existing numbers means you could throw out LegacyESVersion completely, correct?

Yes. And it would make things a LOT easier; but I think there are non-technical issues carrying forward w/ the versioning that lead to the desire to cleanly rebase the OpenSearch version to 1.0.0.

But current approach tries to take on too much: reversioning to 1.0, backwards compat with old 7.x versions (without tests). This is too complex and tests/build/code is failing everywhere, we should revert this reversioning!
Non-technical people can be informed that it didn't work out, at least this time. we can always try again at some later time. Gotta keep these PM-types on a short leash, or they will re-label and re-version you every day. They have no idea of e.g. the costs of backwards compatibility support.

backwards compat with old 7.x versions (without tests)

This!  I don't think the rebase to 1.0.0 is so much a concern as is the desire for backwards compatibility. It would be nice to have a clean break altogether and if the community wanted backward, sideways, cross, insideout compatibility it could be looked at in a separate plugin.

start gradle check

start gradle check

start gradle check

start gradle check

start gradle check

start gradle check

start gradle check

Unable to reproduce these flaky failing tests. I can't tell if these are CI induced or legitimate failures!
Example from last run:
REPRODUCE WITH: ./gradlew ':client:rest-high-level:asyncIntegTest' --tests "org.opensearch.client.ReindexIT.testDeleteByQueryTask" -Dtests.seed=4F5E74A42FAF0A0 -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1" -Dtests.locale=ar-BH -Dtests.timezone=America/Pangnirtung -Druntime.java=15

Dtests.timezone=America/Pangnirtung -Druntime.java=15
  2> java.lang.AssertionError
        at __randomizedtesting.SeedInfo.seed([4F5E74A42FAF0A0:C9242E500657469F]:0)
        at org.junit.Assert.fail(Assert.java:86)
        at org.junit.Assert.assertTrue(Assert.java:41)
        at org.junit.Assert.assertTrue(Assert.java:52)
        at org.opensearch.client.OpenSearchRestHighLevelClientTestCase.lambda$checkTaskCompletionStatus$2(OpenSearchRestHighLevelClientTestCase.java:360)
        at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1019)
        at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:992)
        at org.opensearch.client.ReindexIT.testDeleteByQueryTask(ReindexIT.java:327)


Fix ps__scrollbar-y-rail to top:0px;

perfect-scrollbar don't work with stickySectionHeaders. Can you add an option to fix a scollbar-rail to 0px? I think then both plugins should work together fine.
I have created an own setting fixScrollRail and added the following line after initializing yRailOffset in line 1355 of perfect-scrollbar.jquery.js:
if (i.settings.fixScrollRail) yRailOffset.top = 0;

You can just modify CSS to do this.
fyi: https://github.com/noraesae/perfect-scrollbar/blob/master/examples/scrollbars-on-the-opposite-side.html

Your scrollbar rail top property changes with inline-css while scrolling. In combination with sticky headers it moves out of area. Adding any styles won't fix that.

It sets a proper style values for top if top is set.

Oh, I may have misunderstood. You're using it with position: fixed?

Yes, sticky headers force position:relative; to the wrapper, to put the header on top. But then the slider moves to fast. Therefore I add a property to your plugin. I wrote it in my first post.

I have no idea of what's stickySectionHeaders. Not sure what it's doing, but if it's position: relative, not fixed, it may not have any problem. yRailOffset.top = 0; shouldn't be needed.

This Plugin http://polarblau.github.io/stickySectionHeaders/ uses overflow: hidden; and position: relative;.
You can test it hear:
https://jsfiddle.net/Adnan_Oezsarigoel/0cL4pf5a/

I agree with @Adnan-Oezsarigoel, I also have the problem, that I cannot set the container to position: relative;. Just keeping ps__rail-y top style at 0 would fix the problem. I think the best solution would be to give plugin users the ability to intercept the styles of the generated dom elements.
Something like
const container = document.getElementById('#container-with-position-other-than-relative');
new PerfectScrollbar(container, {
  interceptRailY: styles => ({ ...styles, top: 0 }),
});

FYI I solved it by just using a top: 0 !important in CSS. But anyway it would be nice to have more control over the scrollbar DOM elements if it's not a big performance deal

Wow, @janizde. Your fix is ways better then mine. Thank you.

@utatti are you sure you should have closed this? Having a relative wrapper around the scrollable content is required for my use case, and would cut down on the fancy logic mutating top and bottom on the scroll rails.
Having a relative wrapper would mean you only need to set top to 0 and never think about it again. (Since the rail would be absolute relative to the non-scrollable portion)  Would you consider re-opening this issue?

I had the same issue and it solved with top: 0px !important
Not sure what'st the reason why top changes for  .ps__rail-y


ROCANA-10183: NetConnState implements Stringer interface

Adds string values to NetConnState to make these easier to render.

Thanks @matthew-dailey! This is prep for another ticket.


Update log-export module.

It fixes a deprecated syntax which is now an error in Terraform 1.0.
https://github.com/terraform-google-modules/terraform-google-log-export/releases/tag/v6.0.1

/gcbrun


Add "Decorative?" option

Currently, if the AltText field is left empty, and the image is inserted directly into the template, e.g. $MyImage, the image's Title is used for the alt attribute. Meaning there is no way to purposely set the alt attribute as empty, to indicate a decorative image.
To resolve we could:

Add a "Decorative?" checkbox to explicitly set an image as decorative.
Add a config option to set it to NOT use the Title as the alt when AltText is left empty.

I had a look at this and found this is not trivial, because when you set $attributes['alt'] = ''; in the updateAttributes() hook, the alt attribute is stripped out completely. This is because getAttributesHTML() in SilverStripe\View\AttributesHTML is filtering out empty attributes.
As a work around, I have created a template in my theme that overrides the default DBFile_image.ss template:
<img $getAttributesHTML('alt') alt="$AltText" />


Bump version of golang.org/x/sys to 0.0.0-20220412211240-33da011f77ad

Need to bump version of golang.org/x/sys library to 0.0.0-20220412211240-33da011f77ad due to this CVE
Detected by Trivy scanner earlier today.
├──────────────────┼────────────────┼──────────┼────────────────────────────────────┼───────────────────────────────────┼───────────────────────────────────────────────┤
│ golang.org/x/sys │ CVE-2022-29526 │ MEDIUM   │ v0.0.0-20220408201424-a24fb2fb8a0f │ 0.0.0-20220412211240-33da011f77ad │ golang: syscall: faccessat checks wrong group │
│                  │                │          │                                    │                                   │ https://avd.aquasec.com/nvd/cve-2022-29526    │```

There are couple more discovered recently: CVE-2022-30580, CVE-2022-32189, CVE-2022-30635, CVE-2022-30633, CVE-2022-30632, CVE-2022-30631, CVE-2022-30630, CVE-2022-28131, CVE-2022-32148, CVE-2022-1705, and CVE-2022-1962. All are fixed in the go versions 1.17.13 and 1.18.5. @ahmetb could you please run the release pipeline?

@ahmetb friendly ping

Quite honestly, none of these tool-discovered security vulnerabilities are exploitable in a setting where you are running the probe on your container's loopback interface, against trusted server code that you own.
I am not going to be able to rebuild this project every week because Go open source has a security bug that doesn't impact this repo. If this bothers you a lot, I highly recommend you move off of this tool to use the new Kubernetes feature that replaces this tool (see README).
I just released v0.4.12 with updated Go version and Go library dependencies.


Training hyper-parameters for DCAE

Hi :wave:
I am having trouble training a DCAE from scratch. Could you provide the training details (optimizer, learning rate, batch size, weight decay, number of steps, ...) for each phase (1,2,3)?
Thank you very much for your work!

have you managed yet?


Clarification regarding definition of delta_hue, delta_sat, delta_lum, delta_edges in docs

Hi,
http://scenedetect.com/projects/Manual/en/latest/api/detectors.html states:
property delta_edges
Difference between calculated edges of adjacent frames.

Edge differences are typically larger than the other components, so the detection threshold may need to be adjusted accordingly.

property delta_hue
Difference between pixel hue values of adjacent frames.

property delta_lum
Difference between pixel luma (brightness) values of adjacent frames.

property delta_sat
Difference between pixel saturation values of adjacent frames.

I have noticed that these properties always have positive values, regardless of the type of video analyzed,
this would indicate that it is the absolute difference of the property between adjacent frames.
Assuming that I am correct, I am interested in a method to obtain  the property sign, for example the sign of delta_lum...I want to determine scenes with decreased luminance.
Regards

You are correct that the differences between frames referred to in the docs is just the absolute difference and does not contain directional information. For reference, here is where the absolute value is taken in the code. This is true for the calculations used in the detect-adaptive as well since it simply calls the same function.
If you want directional information for parameters like luminance, you could simply remove the absolute value from this line and manually install, but I don't think including a sign for a parameter like delta_edges really gives you any kind of additional information. So, you would probably want to apply a little bit of scrutiny to what parameters you want directional information and which you don't. I can imagine a scenario in which the change in direction for other parameters is used, but I would like to hear from you about how you are planning to use this data.
If there is demand for this to be a built-in feature, it wouldn't be too difficult to implement something like a command line flag to enable/disable the absolute difference method currently used. In the near-term, I can clarify this in the docs for future releases.

Thanks @wjs018:
I am currently actively using PySceneDetect, and I'm trying to take advantage of its functionality to detect problems or classify scenes.
a long video zone that does not contain changes of these variables (delta = 0) without any associated noise, we are undoubtedly in a video portion that does not provide any information, if it is not intentional then it is an error.
For example, In this video analysis, you can see how after two minutes there is an area where the delta variables are equal to 0. (the video at that moment has black frames).

zooming:

Just looking at the graph, you can't tell if it's black frames or a freeze or if it's actually a problem. However, if I had the value of the sign of delta_lum prior to the zone unchanged and that value was a significant drop, we would be facing a zone with black frames. if  lum rises abruptly before the constant zero area, probably it is a white frame area.
I usually do this task with ffmpeg and its blackdetect filter, but I have found a reliable alternative in this method.
I don't know if the detection of certain problems in the video is within the objectives of this project, however
PySceneDetect has all the information frame by frame of HSV colors, integrate this type of features to
PySceneDetect would reduce multiple processes on the same video, and solutions at this level are scarce.
Regards

If you want, you can add a custom calculation to ContentDetector in the source code, and add a new metric for it here:
here:
https://github.com/Breakthrough/PySceneDetect/blob/master/scenedetect/detectors/content_detector.py#L177-L181
If you also run with the ThresholdDetector, this will give you the average luma values per frame, which should allow you to determine if the frames are getting brighter or darker.  Does that resolve the issue?  I'm not sure if we want to include these calculations in by default, since it would lead to a performance hit.  Thanks!

Closing this due to lack of activity, but feel free to re-open if you would like us to revisit.  Hopefully you found an adequate workaround!


Updated tide module and Jupyter notebook!

It looks like the tide.py file in this PR is almost identical to the one in https://github.com/clawpack/geoclaw/pull/532, should that just be used instead?
Currently the Tide_Module_Examples.ipynb notebook imports tide at the top but then also has
    import clawpack.geoclaw.tide as tide
later on in the notebook, so it's using both versions.

Yes.

Updated jupyter notebook and deleted local tide.py to avoid issues.


Not null-teminated strings for included and open-file

https://github.com/SirWumpus/post4/blob/75a26404bd67d386cebd8014d90f576cc915948e/src/post4.p4#L2871
So, if we pass a string that is not null-terminated to included, it will not find the file?
If included (or, better, open-file, etc) needs a null-terminated string, it should check that and make a temporary copy by itself.

It actually does make the copy with a terminating NUL byte already.  If you follow the words it gets to
_string0_store which handles the copy to a temp. buffer and NUL.

I mean, that the following test fails:
s" test.fth-" 1- included

When the following works:
s" test.fth" included


"No YubiKey detected." on built-from-source Linux 4.3.0

Yubico Authenticator version: 4.3.0, built from source
Operating system and version: PureOS 8.0
YubiKey model and version: YubiKey 4
Bug description summary: "No YubiKey detected." on built-from-source Linux 4.3.0

Steps to reproduce

Open yubioath-desktop, either from the command line or through the application launcher.
See message "No YubiKey detected."
Insert YubiKey into a USB port.

Expected result
Some behavior involving the "No YubiKey detected." message disappearing.
Actual results
The message remains with no changes in the app whatsoever.
Other info
This YubiKey works with the corresponding app on my Mac, and is recognized by the personalization tool on this Linux machine. See #223 for more back story in case helpful.

The above happened with the 4.3.0 source download. I just reproduced with 96d11c2f9004640c3781477705b1fa07e85e1e8a.

Thanks for your quick reply. I get the exact same behavior and zero terminal output when I run that command.

That's strange. Do you have the YubiKey Manager python library (ykman) installed?

Doesn't look like it.
$ python -c "import ykman"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: No module named ykman
$ echo $?
1

Sorry, I forgot to mention it's a Python 3 library, so please check that too. If you don't have it, then that's at least one problem. It's still strange that you don't get any terminal output at all from yubioath-desktop, but I have no idea what might be causing that.
YubiKey Manager is included in the vendor/yubikey-manager directory in the source tarball, and those sources were enough for me to build the yubikey-manager and python3-yubikey-manager packages on Debian Buster:
$ cd vendor/yubikey-manager
$ debuild -us -uc
$ sudo dpkg -i ../python3-yubikey-manager_0.5.0_all.deb

After installing python3-yubikey-manager, the yubioath-desktop binary built by qmake && make successfully detects my YubiKey in Debian Buster.
Does any of that help?

Thank you for that information, which supplied the clues I needed. I now have a working Yubico Authenticator app after spending several hours on this project.
I needed to fulfill many further dependencies to get those commands to run. Here is what went down, as an experience report for the next person who has to slash through this jungle.
11:59 $ cd vendor/yubikey-manager/
12:12 $ debuild -us -uc
debuild: command not found
12:13 $ sudo apt-get install debuild
...
E: Unable to locate package debuild
12:13 $ # search the Internet for an incantation...
12:17 $ sudo apt-get install devscripts
...
Fetched 45.0 MB in 16s (2,667 kB/s)
...
12:18 $ debuild -us -uc
...
dpkg-checkbuilddeps: error: Unmet build dependencies: python-all python3-all python-setuptools python3-setuptools python-usb python3-usb python-pyscard python3-pyscard python3-cryptography python-click python3-click python3-openssl
...
12:18 $ sudo apt-get install python-all python3-all python-setuptools python3-setuptools python-usb python3-usb python-pyscard python3-pyscard python3-cryptography python-click python3-click python3-openssl
...
Fetched 1,555 kB in 1s (783 kB/s)
...
12:20 $ debuild -us -uc
...
12:22 $ sudo dpkg -i ../python3-yubikey-manager_0.5.0_all.deb
...
dpkg: dependency problems prevent configuration of python3-yubikey-manager:
 python3-yubikey-manager depends on libu2f-host0; however:
  Package libu2f-host0 is not installed.
...
12:23 $ sudo apt-get install libu2f-host0
...
You might want to run 'apt --fix-broken install' to correct these.
The following packages have unmet dependencies:
 libu2f-host0 : Depends: libhidapi-hidraw0 (>= 0.8.0~rc1+git20140201.3a66d4e+dfsg) but it is not going to be installed
                Recommends: libu2f-udev but it is not going to be installed
E: Unmet dependencies. Try 'apt --fix-broken install' with no packages (or specify a solution).
12:23 $ sudo apt-get install libhidapi-hidraw0
...
You might want to run 'apt --fix-broken install' to correct these.
The following packages have unmet dependencies:
 python3-yubikey-manager : Depends: libu2f-host0 but it is not going to be installed
...
12:23 $ sudo apt --fix-broken install
...
Fetched 46.7 kB in 0s (52.5 kB/s)
...
12:24 $ sudo dpkg -i ../python3-yubikey-manager_0.5.0_all.deb

So the defect here is at least the documentation bug of omitting the commands you provided in your last comment.
Thanks again for your assistance.

I was getting similar zero-output on ykman oath accounts list after an install of yubikey-manager 4.0.7 via ubuntu (apt install yubikey-manager). It worked via USB, but not USB-C. @replaid's notes above led me to try installing libu2f-host0 and that fixed it like a charm.


Link to streamlined FRS to DFS-R instructions is broken

The link in the document goes to a TechNet blog that no longer exists
https://blogs.technet.com/b/filecab/archive/2014/06/25/streamlined-migration-of-frs-to-dfsr-sysvol.aspx
should be replaced with
https://techcommunity.microsoft.com/t5/storage-at-microsoft/streamlined-migration-of-frs-to-dfsr-sysvol/ba-p/425405

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 418dc0a0-0312-9d6f-b2dc-2eb4841285ba
Version Independent ID: 600b0469-3c96-5737-c39b-2ce9021b0f5a
Content: Active Directory Domain Services Functional Levels in Windows Server
Content Source: WindowsServerDocs/identity/ad-ds/active-directory-functional-levels.md
Product: windows-server
Technology: ad-ds
GitHub Login: @dknappettmsft
Microsoft Alias: daveba

Hey mnooseman,
Thanks again for contacting Microsoft. We value your feedback.
A documentation change has been made in updating the blog link post and will be live after the next business day. We will now be closing this thread.
Thank you and have a most pleasant day.
#please-close


DimensionSparsityMaskCreator creates unexpected mask_type

Describe the bug
When using a recipe, using the parameter mask_type: unstructured works as expected:
    - !GMPruningModifier
        mask_type: unstructured

But according to the doc, we can choose to use "channel" or "filter" for structured pruning, and here is the problem.
When using either of those, we have an error saying we can't iterate on a string at the line reduced_dims = [idx for idx in range(n_dims) if idx not in self._dim]
self._dim is defined in the __init__ of DimensionSparsityMaskCreator.
But, it is overriden:
...
        if isinstance(dim, str):
            valid_dim_names = ["channel", "filter"]
            if dim in valid_dim_names:
                self._dim_name = dim
                self._dim = [1] if dim == "channel" else [0, 1]
...
        self._dim = dim  # List[int]

But dim is either "channel" or "filter", meaning that we can't define reduced_dims.
ALSO, something strange is that when using mask_type: 1 (I expected to trigger the following code)
        if isinstance(dim, int):
            dim = [dim]

I get the error telling me:
File "/gpfswork/rech/dod/uuj72kc/anaconda3/envs/SparseML/lib/python3.8/site-packages/sparseml/pytorch/optim/mask_creator_pruning.py", line 619, in load_mask_creator
    if ("[" in obj and "]" in obj) or ("(" in obj and ")" in obj):
TypeError: argument of type 'int' is not iterable

Expected behavior
As indicated in the docstrings, mask_type should accept channel, filter, int or List[int], in addition to "unstructured"
Environment
Include all relevant environment information:

OS [e.g. Ubuntu 18.04]: RedHat
Python version [e.g. 3.7]: 3.8
SparseML version or commit hash [e.g. 0.1.0, f7245c8]: 0.8.0
ML framework version(s) [e.g. torch 1.7.1]: Torch 1.9.0
Other Python package versions [e.g. SparseZoo, DeepSparse, numpy, ONNX]:
Other relevant environment information [e.g. hardware, CUDA version]:

To Reproduce
Exact steps to reproduce the behavior:

Define a model,
Try to train it with a recipe GMPruningModifier, mask_type: channel

Errors
If applicable, add a full print-out of any errors or exceptions that are raised or include screenshots to help explain your problem.
mask_type: channel
File "/gpfswork/rech/dod/uuj72kc/anaconda3/envs/SparseML/lib/python3.8/site-packages/torch/cuda/amp/grad_scaler.py", line 285, in _maybe_opt_step
    retval = optimizer.step(*args, **kwargs)
  File "/gpfswork/rech/dod/uuj72kc/anaconda3/envs/SparseML/lib/python3.8/site-packages/sparseml/pytorch/optim/manager.py", line 160, in step
    return self._perform_wrapped_step(*args, **kwargs)
  File "/gpfswork/rech/dod/uuj72kc/anaconda3/envs/SparseML/lib/python3.8/site-packages/sparseml/pytorch/optim/manager.py", line 198, in _perform_wrapped_step
    self._wrapped_manager.update(
  File "/gpfswork/rech/dod/uuj72kc/anaconda3/envs/SparseML/lib/python3.8/site-packages/sparseml/pytorch/optim/manager.py", line 448, in update
    mod.scheduled_update(module, optimizer, epoch, steps_per_epoch)
  File "/gpfswork/rech/dod/uuj72kc/anaconda3/envs/SparseML/lib/python3.8/site-packages/sparseml/pytorch/optim/modifier.py", line 519, in scheduled_update
    self.update(module, optimizer, epoch, steps_per_epoch)
  File "/gpfswork/rech/dod/uuj72kc/anaconda3/envs/SparseML/lib/python3.8/site-packages/sparseml/pytorch/optim/modifier_pruning.py", line 277, in update
    self._check_mask_update(module, epoch, steps_per_epoch)
  File "/gpfswork/rech/dod/uuj72kc/anaconda3/envs/SparseML/lib/python3.8/site-packages/sparseml/pytorch/optim/modifier_pruning.py", line 761, in _check_mask_update
    self._module_masks.set_param_masks_from_sparsity(self._applied_sparsity)
  File "/gpfswork/rech/dod/uuj72kc/anaconda3/envs/SparseML/lib/python3.8/site-packages/sparseml/pytorch/optim/mask_pruning.py", line 383, in set_param_masks_from_sparsity
    masks = self._mask_creator.create_sparsity_masks(
  File "/gpfswork/rech/dod/uuj72kc/anaconda3/envs/SparseML/lib/python3.8/site-packages/sparseml/pytorch/optim/mask_creator_pruning.py", line 443, in create_sparsity_masks
    return super().create_sparsity_masks(tensors, sparsity, global_sparsity=False)
  File "/gpfswork/rech/dod/uuj72kc/anaconda3/envs/SparseML/lib/python3.8/site-packages/sparseml/pytorch/optim/mask_creator_pruning.py", line 373, in create_sparsity_masks
    grouped_tensors = [self.group_tensor(tensor) for tensor in tensors]
  File "/gpfswork/rech/dod/uuj72kc/anaconda3/envs/SparseML/lib/python3.8/site-packages/sparseml/pytorch/optim/mask_creator_pruning.py", line 373, in <listcomp>
    grouped_tensors = [self.group_tensor(tensor) for tensor in tensors]
  File "/gpfswork/rech/dod/uuj72kc/anaconda3/envs/SparseML/lib/python3.8/site-packages/sparseml/pytorch/optim/mask_creator_pruning.py", line 456, in group_tensor
    reduced_dims = [idx for idx in range(n_dims) if idx not in self._dim]
  File "/gpfswork/rech/dod/uuj72kc/anaconda3/envs/SparseML/lib/python3.8/site-packages/sparseml/pytorch/optim/mask_creator_pruning.py", line 456, in <listcomp>
    reduced_dims = [idx for idx in range(n_dims) if idx not in self._dim]
TypeError: 'in <string>' requires string as left operand, not int

mask_type: 1:
Traceback (most recent call last):
  File "experiment_sparseml.py", line 176, in <module>
    main()
  File "/gpfswork/rech/dod/uuj72kc/anaconda3/envs/SparseML/lib/python3.8/site-packages/hydra/main.py", line 48, in decorated_main
    _run_hydra(
  File "/gpfswork/rech/dod/uuj72kc/anaconda3/envs/SparseML/lib/python3.8/site-packages/hydra/_internal/utils.py", line 377, in _run_hydra
    run_and_report(
  File "/gpfswork/rech/dod/uuj72kc/anaconda3/envs/SparseML/lib/python3.8/site-packages/hydra/_internal/utils.py", line 214, in run_and_report
    raise ex
  File "/gpfswork/rech/dod/uuj72kc/anaconda3/envs/SparseML/lib/python3.8/site-packages/hydra/_internal/utils.py", line 211, in run_and_report
    return func()
  File "/gpfswork/rech/dod/uuj72kc/anaconda3/envs/SparseML/lib/python3.8/site-packages/hydra/_internal/utils.py", line 378, in <lambda>
    lambda: hydra.run(
  File "/gpfswork/rech/dod/uuj72kc/anaconda3/envs/SparseML/lib/python3.8/site-packages/hydra/_internal/hydra.py", line 111, in run
    _ = ret.return_value
  File "/gpfswork/rech/dod/uuj72kc/anaconda3/envs/SparseML/lib/python3.8/site-packages/hydra/core/utils.py", line 233, in return_value
    raise self._return_value
  File "/gpfswork/rech/dod/uuj72kc/anaconda3/envs/SparseML/lib/python3.8/site-packages/hydra/core/utils.py", line 160, in run_job
    ret.return_value = task_function(task_cfg)
  File "experiment_sparseml.py", line 112, in main
    callbacks=[SparseMLCallback(recipe_path='/gpfswork/rech/dod/uuj72kc/bagginghsf/sparsify_70_qat_channel.yaml')],
  File "/gpfswork/rech/dod/uuj72kc/anaconda3/envs/SparseML/lib/python3.8/site-packages/pl_bolts/callbacks/sparseml.py", line 42, in __init__
    self.manager = ScheduledModifierManager.from_yaml(recipe_path)
  File "/gpfswork/rech/dod/uuj72kc/anaconda3/envs/SparseML/lib/python3.8/site-packages/sparseml/pytorch/optim/manager.py", line 269, in from_yaml
    modifiers = Modifier.load_list(yaml_str)
  File "/gpfswork/rech/dod/uuj72kc/anaconda3/envs/SparseML/lib/python3.8/site-packages/sparseml/pytorch/optim/modifier.py", line 90, in load_list
    return Modifier.load_framework_list(yaml_str, PYTORCH_FRAMEWORK)
  File "/gpfswork/rech/dod/uuj72kc/anaconda3/envs/SparseML/lib/python3.8/site-packages/sparseml/optim/modifier.py", line 294, in load_framework_list
    container = yaml.safe_load(yaml_str)
  File "/gpfswork/rech/dod/uuj72kc/anaconda3/envs/SparseML/lib/python3.8/site-packages/yaml/__init__.py", line 125, in safe_load
    return load(stream, SafeLoader)
  File "/gpfswork/rech/dod/uuj72kc/anaconda3/envs/SparseML/lib/python3.8/site-packages/yaml/__init__.py", line 81, in load
    return loader.get_single_data()
  File "/gpfswork/rech/dod/uuj72kc/anaconda3/envs/SparseML/lib/python3.8/site-packages/yaml/constructor.py", line 51, in get_single_data
    return self.construct_document(node)
  File "/gpfswork/rech/dod/uuj72kc/anaconda3/envs/SparseML/lib/python3.8/site-packages/yaml/constructor.py", line 60, in construct_document
    for dummy in generator:
  File "/gpfswork/rech/dod/uuj72kc/anaconda3/envs/SparseML/lib/python3.8/site-packages/sparseml/optim/modifier.py", line 760, in constructor
    instance.__init__(**state)
  File "/gpfswork/rech/dod/uuj72kc/anaconda3/envs/SparseML/lib/python3.8/site-packages/sparseml/pytorch/optim/modifier_pruning.py", line 545, in __init__
    else load_mask_creator(mask_type)
  File "/gpfswork/rech/dod/uuj72kc/anaconda3/envs/SparseML/lib/python3.8/site-packages/sparseml/pytorch/optim/mask_creator_pruning.py", line 619, in load_mask_creator
    if ("[" in obj and "]" in obj) or ("(" in obj and ")" in obj):
TypeError: argument of type 'int' is not iterable

Additional context
Add any other context about the problem here. Also include any relevant files.

Hi @clementpoiret thank you for finding and fixing this bug. I've approved #442 and will get that landed.
We're currently overhauling our structure pruning support including for filter pruning which is reducing the wrong dimensions (should be [0] instead of [0,1]) and adding support for pruning groups of parameters together (#440).

Thanks for the information @bfineran
As I am using sparseml to prune and quantize my models, does it mean I have to change (at least locally) [0,1] by [0] to have the expected filter pruning? Or is it better to wait until 0.9.0?

Hi @clementpoiret, we'd recommend waiting until 0.9 for better-structured pruning support. This should go out in the next two weeks and we'll keep you updated as this progresses.

Hi @clementpoiret
Release .9 has gone out the door; we hope you can try our latest version to address this issue; a summary of release highlights is also available - check them out here:
Best, Jeannie / Neural Magic

Hi @clementpoiret to follow up on the comment from @jeanniefinks, we have added a StructuredPruningModifier in the 0.9 release for PyTorch. Let us know if you have a chance to try it out as this enables proper filter pruning. Happy to help walkthrough an example as well!
Thanks,
Mark

I launched another training using StructuredPruningModifier to prune filters, but some questions are coming to my mind:
1/ In the doc, GMPruningModifier still have "filters" and "channels". Is it now equivalent to using the StructuredPruningModifier with the same arguments?
2/ How can I check if the model is correctly pruned? The final pruned ONNX file weights the same thing (around 140Mb) than its non-pruned counterpart, is this normal?
Thanks!


[FEATURE] Disable Dark Mode Route

Hello,
what's the cleanest way to disable POST adminlte/darkmode/toggle route?
I don't want to use the feature in the project, so i've added
'layout_dark_mode' => null
in my adminlte.php config file but the route is still available.

@MarioVillani I think currently there is no way to disable loading of darkmode routes. I will review this on the future.


[Suggestion] Provide Solutions

Would be nice to provide a solution/explanation for anyone who simply cannot figure one out ;-)

Hoping to add this feature eventually, but in the meantime, Programming Mentor has put together this nice guide:
http://programmingmentor.com/post/playing-css-grid-garden/

Hi, is there a chance I could work on this issue?
For starters, we can have a simple button that says 'Solution'  and on clicking that an alert pops up, asking the user if they really want to see the solution. If they say yes, we could display the solution in text format as an alert. If not, they can continue trying.
If this sounds like a good idea, I'd love to have a chance to work on it since I'm currently learning front-end web development and I'll learn a lot by contributing!


Error: TypeError: Failed to execute 'getFileHandle' on 'FileSystemDirectoryHandle': Name is not allowed.

As you can see. An error occurred while reading the file.

@Undertone0809 are you using Safari? Safari doesn't yet support the full file system access APIs. I would recommend using Chromium based browser as they have the API compatibility for file system access.
I need to add a message on website to recommend using Chromium.

I using Edge dev version on Windows10

@Undertone0809 did you get a dialog asking for permission and did you select a folder? Curious if this is happening because you may have accidentally disallowed when it asked for permission.

Once you press 'Give access' and select folder, you'll see two more dialogs

Sometimes I succeed, sometimes I fail. So it's confusing to me.

@Undertone0809 can you please share more on what's succeeding and what's failing? Is the file writing action is specifically failing due to the error you have in the initial screen shot or are there other failures you are running into?

Is it after I give the command, it says it needed to read the file and analyze my code, and then maybe it's because it took me a while to open the file? The above error occurs when the ta fails to read the file.

@Undertone0809 does the file already exist and is the action 'read_file'?
It's possible that in the scenario you ran into the LLM may be hallucinating and trying to read a file when one doesn't exist. In that case, you can either see if it fixes its error in next commands by generating the file or you can try to rephrase your prompt goals and see if that helps.

Thank you. I'll try again. Moreover, I would like to know why it doesn't answer the question after answering the question for a while, my question hasn't been solved yet.


Screen blacklist and repositioning API

API

[ ] Entrypoint for other mods

Defaults
Blacklist

[ ] ComputerCraft computers
[ ] Spectrum tool menu
[ ] Creative inventory

Reposition

[ ] Beacon
[ ] Create toolbox
[ ] Villagers

Ideally the blacklist would be configurable, but another default blacklisted screen should be Blabber's org.ladysnake.blabber.api.client.BlabberDialogueScreen


stdenv and base packages: fix with __structuredAttrs enabled

Description of changes
This fixes stdenv with __structuredAttrs enabled.
Test build:
nix-build -E 'with import ./. { config.structuredAttrsByDefault = true; }; hello'

makeFlags is a bash array and cannot be appended to as done in glibc's preConfigure.
env vars may only be strings and have to be in env to be exported
substituteAll only reads and substitutes exported variables

This is the first part of a number of PRs that intend to make nixpkgs forward-compatible with __structuredAttrs enabled globally.
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

I think this is a good point, where I'll leave this PR and continue with follow-ups based on this:

set of base packages upon stdenv works, including: systemd, nginx, postgresql, sway
rust/perl/python language infrastructure generally works for an adequate number of packages

Next steps will probably be around go/ruby language infrastructure and "leafier" packages.
I've tried to make sure to not break anything with structuredAttrs kept off, although there are some larger changes, if any are controversial I can exclude them for now, to get this merged more easily.

Found an issue with pytestFlagsArray which has been used different than all other *Array variables, which makes backward-compatibility difficult to achieve, might be a better option to break this option and fix it properly right away.

I think env should be an attrset of strings or lists of strings, same as with derivation arguments. This allows the env to be manipulated with list operations rather than string mangling. For example, I might want to override a package by filtering something out of nix.NIX_CFLAGS_COMPILE.


Telemetry events

Summary of the Pull Request
Fix trace events when turning modules on/off.

PR Checklist

[x] I've discussed this with core contributors already.

Detailed Description of the Pull Request / Additional comments
The runner was calling module->disable(); before calling module->destroy();, that was preventing a module to distinguish between being turned off and PowerToys exiting and therefore misfiring the off trace event.
Added the on/off trace events to ShortcutGuide.

Validation Steps Performed
Manual test.

@bzoz I pushed one more commit, PTAL. Thanks.


Improve Emergency SOS and also fix other js ERROR

Fixes:  #1685
Description
Improve the Emergency SOS for clear visibility  and also improve readability;
Type of PR

[x] Bug fix
[ ] Feature enhancement
[ ] Documentation update
[ ] Other (specify): _______________

Screenshots / videos (if applicable)

Checklist:

[x] I have made this change from my own.
[x] My code follows the style guidelines of this project.
[x] I have performed a self-review of my own code.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] My changes generate no new warnings.
[x] I have tested the changes thoroughly before submitting this pull request.
[x] I have provided relevant issue numbers and screenshots after making the changes.

@KrishChothani Please see the conflicts .

Done

Hey @PriyaGhosal
I resolved All Conflict . You can check it and assign this


Vite and jquery bootstrap4

Thanks for this PR @liamkeily but Bootstrap 4 is now deprecated so I don't see us adding this example. I'll let @mdo confirm it.

Sorry i thought i'd created this against my own fork, i was just creating this PR to demonstrate a bug.


Python 3 Ported. Fixes #151

Removed future imports
Made code PEP8 compliant
Changed indendation from tabs to spaces

#151

Will fix
On Sep 18, 2017 3:17 AM, "cclauss"<EMAIL_ADDRESS>wrote:

@cclauss commented on this pull request.
In app/scrapers/bing.py
https://github.com/fossasia/query-server/pull/152#discussion_r139322103:

     payload = {'q': query}

   response = requests.get('http://www.bing.com/search', params=payload, headers=header)

   response = requests.get('http://www.bing.com/search',\

       params = payload, headers = header)

Backslash is both undesirable and unnecessary inside parentheses. See PEP8.
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
https://github.com/fossasia/query-server/pull/152#pullrequestreview-63238148,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AM2tjKhKzqMzegUzauEgFc93NKnOTsnlks5sjZNagaJpZM4PaSVx
.

@cclauss Please review

Backslashes are undesirable.  See PEP8.
Why is it considered to be an advantage to remove __future__ imports?

@cclauss I read PEP8 and see why, sorry for the error. I thought since python2 would be reaching EOL soon, and also since the app was to be ported to Python 3, this would be unnecessary? I could add it back if thought necessary, however.

Python 2.7.14 was just released this weekend and Python 2's EOL is still a few years away so compatibility with both Python 2 and 3 will be the way to go until at least 2020.  The future imports help to ensure that code works the same on both but they are not mandatory.

@mariobehling Please review and merge accordingly.

@mariobehling Can you merge this ASAP? We can start adding tests and other features once this is merged.

@nikhilrayaprolu  Please review and merge accordingly.

@Pk13055 In addition to @AnshulMalik you can use atom-beautify if you use atom as your text-editor 😄

Again, can be closed considering #211

@mariobehling Please close this PR as recent merge fix the issue


:sparkles: Feature: add user defined environment variables

Category:
Feature
Overview
Add user defined environment vars parser to avoid leaking secrets
Code Quality Checklist (Please complete)

[x] All changes are backwards compatible
[x] All lint checks and tests are passing
[x] There are no (new) build warnings or errors
[x] Bumps version, if new feature added

Hi @Lissy93  Thank you so much for this great dashboard!
I added documentation and example for pihole widget, we can extend this to any other widget very easily. Hope you like it.

Hi again @Lissy93 Sorry I haven't seen your reaction to first message, had a traffic accident and still under recovery. Apparently you seemed fond of the feature.
When I can be back, would you still be interested in me fixing the conflicts? Please be aware, they seem to be caused by a strange spacing in your text files. I can either remove my documentation changes and keep your files, or "fix" them (but they will actually bring more spacing issues if the MR takes long to be merged).
Also, would you be interested in me applying the same approach in other widgets?

Actually, would you be able to switch the target branch for your PR into FEAT/Dashy-V3? That way I can get it released sooner.
Alternatively, I can just copy your changes over to that branch

Hi again @Lissy93
I´m recovering slower than I expected, but all good, thank you.
I just changed the target branch as you suggested

Also, I am happy if you change what you need, not sure I'll be able to follow up this thread soon-ish. A pity cause I think it was going to be quite easy to update the rest of the widgets, I am happy if you continue the job (if you want) or I can do it when I am fully recovered (it can be some weeks)

Well I'm wishing you all the best with your recovery 🌼🤗🧡
I've merged into the 3.0 branch, and will let you know once available in master- shouldn't be too long!


Replace JNI-based JSON collation with ICU4C

Original Ticket: https://github.com/couchbase/couchbase-lite-java-core/issues/801

JSONCollator used to call java.text.Collator. Instead of java.text.Collator, use ICU4C collator.

Note for reviewer:

All .h header files are just copied from ICU4C code base for compilation.
All *.a static libraries are compiled from https://github.com/couchbaselabs/icu4c-android

I updated my PR except README. Could you please review PR again?

About ICU4C library, I updated REAME.md .

@pasin,
Please review the code again?


AWS Request Signing Documentation enhancement

Can you please update the current documentation and explain how the client_id, client_secret, grant_type, refresh_token can be send using this filer? these are used to generate the token in amazon api first
The current documentation doesnt explain that
https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/aws_request_signing_filter#config-http-filters-aws-request-signing
also if you have an example using this filter will be very helpful

Currrently I'm seeing this issue, not sure where should I put the Credentials
[2020-11-04 21:33:13.833][13][debug][aws] [source/extensions/common/aws/credentials_provider_impl.cc:41] Getting AWS credentials from the environment
[2020-11-04 21:33:13.833][13][debug][aws] [source/extensions/common/aws/credentials_provider_impl.cc:71] Getting AWS credentials from the instance metadata
...
[2020-11-04 21:33:34.853][13][debug][misc] [source/extensions/common/aws/utility.cc:134] Could not fetch AWS metadata: Timeout was reached
[2020-11-04 21:33:35.853][13][error][aws] [source/extensions/common/aws/credentials_provider_impl.cc:77] Could not retrieve credentials listing from the instance metadata
[2020-11-04 21:33:35.853][13][debug][aws] [source/extensions/common/aws/credentials_provider_impl.cc:182] No AWS credentials found, using anonymous credentials

I was able to load the credentials
One question, the current authorization is appending x-amz-security-token is there a way to change it to use x-amz-access-token instead?
cause using x-amz-security-token its breaking for me

Hey @juanvasquezreyes the things you are referring to above:

client_id
client_secret
grant_type
refresh_token

Seem to be related to OAuth/WebIdentity credentials.
The credentials provider in Envoy doesn't support those credential sources right now. If you notice the logs you pasted, Envoy chose to use environment variables to get its credentials.
Perhaps those credentials were expired?

Envoy doesn't support those creden

They are not expired cause I'm using those using postman and their AWS Signature authorization
I think the issue I'm facing is related to headers I'm sending and some headers are being signed too
In this authorization I see content-length and x-request-id being signed, so my guess is that
'authorization', 'AWS4-HMAC-SHA256 Credential=AKIARYJ2SNXBEZAL526C/20201106/us-east-1/execute-api/aws4_request, SignedHeaders=content-length;host;x-amz-access-token;x-amz-content-sha256;x-amz-date;x-request-id, Si
Cause when I do the signing directly in postman to my service I dont see those headers in the authorization

Hey @juanvasquezreyes the things you are referring to above:

client_id
client_secret
grant_type
refresh_token

Seem to be related to OpenID/WebIdentity credentials.
The credentials provider in Envoy doesn't support those credential sources right now. It supports environment variables and the EC2 Instance or ECS Task Metadata endpoints to fetch credentials.
If you notice the logs you pasted, Envoy chose to use environment variables to get its credentials.
Perhaps those credentials were expired?

They are not expired cause I'm using those using postman and their AWS Signature authorization
I think the issue I'm facing is related to headers I'm sending and some headers are being signed too
In this authorization I see content-length and x-request-id being signed, so my guess is that
'authorization', 'AWS4-HMAC-SHA256 Credential=AKIARYJ2SNXBEZAL526C/20201106/us-east-1/execute-api/aws4_request, SignedHeaders=content-length;host;x-amz-access-token;x-amz-content-sha256;x-amz-date;x-request-id, Si
Cause when I do the signing directly in postman to my service I dont see those headers in the authorization

Oh I see. So the value
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

Is the hash of the empty string. In SigV4, if the x-amz-content-sha256 header is passed to the service as a signed header then it is implied that the client and server are using the request body to calculate the signature.
It is very common for this header to not be passed. So many SigV4 implementations have this value hard-coded, as it is expected to be used when the request body is not signed.
My guess is that this filter has a bug. It is signing the request as if the request body will be used as part of the signature but is not actually buffering the request body and calculating the hash.

Looking a bit closer, the empty hash string is used if the specified service is not one of  's3', 'glacier' or 'es'.
What does your filter config look like?  Specifically looking at the (service name](https://github.com/envoyproxy/envoy/blob/master/api/envoy/config/filter/http/aws_request_signing/v2alpha/aws_request_signing.proto#L26-L27)
Based on the request content, I would guess that the service is set to execute-api hence the filter not signing the body.  These are the lines I'm basing my statement on:
[2020-11-06 15:46:23.820][15][debug][http] [source/extensions/common/aws/signer_impl.cc:69] String to sign:
AWS4-HMAC-SHA256
20201106T154600Z
20201106/us-east-1/execute-api/aws4_request
...

Looking a bit closer, the empty hash string is used if the specified service is not one of 's3', 'glacier' or 'es'.
What does your filter config look like? Specifically looking at the service name
Based on the request content, I would guess that the service is set to execute-api hence the filter not signing the body. These are the lines I'm basing my statement on:
[2020-11-06 15:46:23.820][15][debug][http] [source/extensions/common/aws/signer_impl.cc:69] String to sign:
AWS4-HMAC-SHA256
20201106T154600Z
20201106/us-east-1/execute-api/aws4_request
...

      - name: envoy.filters.http.aws_request_signing
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.http.aws_request_signing.v3.AwsRequestSigning
          service_name: execute-api
          region: us-east-1
      - name: envoy.router

As @lavignes  mentioned, this is likely a bug.  The filter needs to be extended to include API Gateway, since requests to that service can have a body.

I'd vote to not add API Gateway to that list and instead just add a boolean option to the extension to enable/disable body signing. But even then it looks like the extension doesn't even support body signing: https://github.com/envoyproxy/envoy/blob/master/source/extensions/filters/http/aws_request_signing/aws_request_signing_filter.cc#L36
s3 should be the only service that requires special treatment as far as I'm aware. But it looks like those other services were  added here: https://github.com/envoyproxy/envoy/pull/12020. To me it seems like they were trying to fix the bug, but in the wrong way.

Hello @lavignes thank you for your reply
so, who should we talk to in order to do the changes required to make it work?

@lavignes @abaptiste: Sounds like this is not just a documentation issue, could you update the issue title and provide a summary of what the fix would involve?

Hey @antoniovicente I don't think I have permissions to update the title or the issue content.
The gist of this is that the AWS Request Signing filter always adds the header x-amz-content-sha256. The value of this header is always being set to HexEncode(SHA256("") that is, the SHA-256 of the empty string. The string in question being encoded is supposed to the be body of the http request. The server authorizing the request on the other end checks for this header and cross-references the signature by hashing the request body as well.
However, this header is optional for all AWS services with the exception of S3 where it is always defined to be the hash of the empty string. If the header is not present then the request body is not included in the signature calculation on the receiving service.
#12020 attempted to "fix" this by setting the value of this header to the hashed empty string for some other services. Presumably these were for requests with an empty or no request body.
The real and correct fix is the undo the changes from #12020 and instead only set x-amz-content-sha256 for S3 requests.

cc @azihsoyn @dio @rgs1 @derekargueta @mattklein123 @marcomagdy
cc'ing aws signing filter owners.  @lavignes  thanks for the explanation.  I don't see an issue associated with #12020, just some claims during the review that x-amz-content-sha256 is needed for some requests to es and glacier.

👋 Hi @azihsoyn @dio @rgs1 @derekargueta @mattklein123 @marcomagdy
Can we reopen this issue and add a no stalebot label? It precludes the use of POST (or any other method with with a body) because the x-amz-content-sha256 hash does not match the body.
For anyone looking for a workaround, I found that the filter works with GET requests, as long as the query parameters are already sorted. Of course, GET requests aren't going to cover a lot of use cases.
For example, I'm able to publish to SNS with a request like this:
✗ curl -G localhost:12345 \
--data-urlencode "Action=Publish" \
--data-urlencode "Message=Hello World" \
--data-urlencode "TopicArn=arn:aws:sns:us-east-1:1234567890:stew-test" \
--data-urlencode "Version=2010-03-31" \

Using the following route config:
  route_config:
    name: local_route
    virtual_hosts:
      - name: sns
        domains: ['*']
        routes:
        - name: sns
          match:
            prefix: /
          route:
            cluster: sns
  http_filters:
    - name: envoy.filters.http.aws_request_signing
      typed_config:
        "@type": type.googleapis.com/envoy.extensions.filters.http.aws_request_signing.v3.AwsRequestSigning
        service_name: sns
        region: us-east-1
        host_rewrite: sns.us-east-1.amazonaws.com
    - name: envoy.filters.http.router
      typed_config: {}

any working example for aws s3 with custom endpoint ?

any working example for aws s3 with custom endpoint ?

not that im aware of - but worth searching docs
minio sounds like something that would be pretty cool as a sandbox if we could figure out a working config

@phlax strongly needed! yes i meant Minio.