Fix tests on OSX.

Signed-off-by: dblock<EMAIL_ADDRESS>Description
The /tmp location is a symlink to /private/tmp. Using os.path.realpath we can get the actual location. Relative paths differ by one ../ because of this as well.
https://apple.stackexchange.com/questions/1043/why-is-tmp-a-symlink-to-private-tmp
https://stackoverflow.com/questions/58719364/tempfile-mkdtemp-difference-on-osx/69151922#69151922
Issues Resolved
Closes https://github.com/opensearch-project/opensearch-build/issues/452
Check List

[x] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Codecov Report

Merging #459 (5613724) into main (163a32a) will increase coverage by 0.02%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main     #459      +/-   ##
==========================================
+ Coverage   64.46%   64.48%   +0.02%
==========================================
  Files          48       48
  Lines        1269     1270       +1
==========================================
+ Hits          818      819       +1
  Misses        451      451

Impacted Files
Coverage Δ

bundle-workflow/src/build_workflow/builder.py
100.00% <100.00%> (ø)

bundle-workflow/src/git/git_repository.py
90.69% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 163a32a...5613724. Read the comment docs.


Ogm Filter for ver 3.0.1 isn't compatible with filter from earlier version (2.3.1)

In our base libraries we widely use Filters.
After upgrading neo4j (from 3.2.0 to 3.3.0) and ogm accordingly (from 2.1.3 to 3.0.1) we found out that constructor Filter(propertyName,propertyValu) doesn't exist anymore.
Now I got a compilation error:
Filter filter = new Filter(key, ob)-->
The constructor Filter(String, Object) is undefined
I would expected that upgrading ogm doesn't cause any compilation issues
in our old code.
Possible Solution
I suppose, class Filter must have follow constructor:
@Deprecated
public Filter(String propertyName, Object propertyValue){
this(propertyName, ComparisonOperator.DEFAULT, propertyValue);
}
Then we won't have any compilation issues in our base libraries, using Filters.
Possible workaround:
Probably, I could try to change the constructors, but I don't know which one of ComparisonOperator is default in the case.Which one should I use when I try "to find by property"?
Context

We can't upgrade neo4j to a last version (3.3.0), since it requires upgrading ogm to 3.0.0/3.0.1
and upgrading the  ogm causes compilation errors in our base libraries.
Your Environment

OGM Version used: 3.0.1
Java Version used: 8
Neo4J Version used:3.3.0
Ogm Bolt Driver:3.0.0
Operating System and Version:Windows10

In this case the replacement of
this(propertyName, propertyValue)
would be
this(propertyName, ComparisonOperator.EQUALS, propertyValue);
Is it something you can deal with, for example by bulk replacing with a regex like these ?
new Filter\(([^,]+)\s*,\s*([^\)^,]+)\) replaced by new Filter\($1, ComparisonOperator.EQUALS, $2)

Thanks
I would suppose that in order to save backward compatibility the follow constructor should be added:
public Filter(propertyName,propertyValue) {
this(propertyName, ComparisonOperator.EQUALS, propertyValue);
}

OGM 3 is a major version, do not expect it to be a drop in replacement.
Please check the migration checklists here : https://neo4j.com/docs/ogm-manual/current/migration/#appendix:migration:checklist

There is a mistake in title, I wrote( you can see in my post) about versions 3.3.0 and 3.2.0 - in both of them version 3 is major

There is no version 3.2 or 3.3 for OGM...

Just a summary - we aim to keep the backwards compatibility between minor versions (e.g. 3.0.x and future 3.1.x). Between major versions (2.x.x and 3.x.x) this compatibility is not ensured.
The constructor change happened between major versions -  2.1.x and 3.0.x.


Make QUIC availability check strict

Checking only TLS 1.3 support is not enough.

Since the official OpenSSL doesn't have APIs for QUIC yet, we don't have a way to do this now.

Closing this because official OpenSSL is going to have completely different API for QUIC, and we don't even know if we want to support it. Current check is enough for the fork (quittls) and BoringSSL.


Build Windows installer on AppVeyor

Still awaiting how we update the assembly info (hence update-assembly-info.sh is empty)
Parts of pack.sh is currently copied into build-windows-installer.sh, however we could unify more of that code when we know what is needed for the mac installer on the new CI (the zipping shouldn't be needed anymore i think, and also i think it's fine to assume you've built the solution before building any of the installers)

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Superseded by https://github.com/fuse-open/fuse-studio/pull/104.


Why is ACS Commons installed in /apps instead of /libs?

The whole Adobe framework makes creating overlays & extensions so nice.
However, ACS Commons is installed in /apps, so we cannot create overlays (unless we of course configure the search path to have an additional path).
It would make sense to me to move ACS to /libs instead of /apps.

...because /libs is reserved for the AEM product itself. Putting ACS AEM Commons's content under /libs would not be upgrade safe. I would expect that anything you want to overlay can be done by extending one of our components. If you have a specific use case which can't be supported currently, please create a specific issue for that.


[CONTROLLER/COMMON] refactors common dir

This PR is for:

Server

refactors common dir
Checklist

[ ] Added unit test.

Backport to branches

main

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.zhengya seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


Add infos on how to add tuya readings from datapoints

As per #377

Great improvement, thank you very much!


put some magic into [lfnoise~], [randpulse~], [randpulse2~], [accum~] and [inc~]

and pulsecount~ / pulsediv~


Sync with what improbable's using in prod

After this merges

I'll do a major-version bump since ansible 2.8 is possibly breaking for users.
I'll update the changelog, push that, and tag 2.0.0.

Sound good? @hectcastro I'd like to invite a couple of other people from improbable to review - are you and azavea ok with that?

Sound good? @hectcastro I'd like to invite a couple of other people from improbable to review - are you and azavea ok with that?

Yes. In all honesty, you all are contributing more to this role than we are. Perhaps we can work on transferring it to your org's GitHub account?

Yes. In all honesty, you all are contributing more to this role than we are. Perhaps we can work on transferring it to your org's GitHub account?

@hectcastro we'd be happy with that if you are :)

Cool. I will transfer ownership to you, then you can transfer it to a desired org (we usually run into issues trying to transfer from org to org).

Looks like you need to delete your fork (under petemounce) before I can transfer it to you.

Done


vorta: 0.8.12 -> 0.9.1

Description of changes
Diff: https://github.com/borgbase/vorta/compare/v0.8.12...v0.9.1
Changelog: https://github.com/borgbase/vorta/releases/tag/v0.9.1
fixes https://github.com/NixOS/nixpkgs/issues/284532

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

cc @nadir-ishiguro

Add a :+1: reaction to pull requests you find important.

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/prs-ready-for-review/3032/3385

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/prs-ready-for-review/3032/3586

I'll merge this and hope that someone figures out the tests eventually.


[Scope maps] Model top-level code via continuations

This PR starts modeling top-level code via continuations, which properly describes the scope of bindings introduced in top-level code, e.g., those variables introduced by a guard statement which can then be captured by a subsequent global function.

@swift-ci please smoke test and merge


Add support for Python 3.13.1, 3.12.8, 3.11.11, 3.10.16 and 3.9.21

Release announcement:
https://blog.python.org/2024/12/python-3131-3128-31111-31016-and-3921.html
Changelog:
https://docs.python.org/3.13/whatsnew/changelog.html#python-3-13-1-final
https://docs.python.org/3.12/whatsnew/changelog.html#python-3-12-8-final
https://docs.python.org/release/3.11.11/whatsnew/changelog.html#python-3-11-11-final
https://docs.python.org/release/3.10.16/whatsnew/changelog.html#python-3-10-16-final
https://docs.python.org/release/3.9.21/whatsnew/changelog.html#python-3-9-21-final
Binary builds:
https://github.com/heroku/heroku-buildpack-python/actions/runs/12159159015
https://github.com/heroku/heroku-buildpack-python/actions/runs/12159160169
https://github.com/heroku/heroku-buildpack-python/actions/runs/12159161685
https://github.com/heroku/heroku-buildpack-python/actions/runs/12162379421
https://github.com/heroku/heroku-buildpack-python/actions/runs/12159164423
GUS-W-14846804.
GUS-W-17350158.
GUS-W-17367727.

https://devcenter.heroku.com/changelog-items/3074


Missing queue name in Rails 5

After migration to Rails 5 job record is created with blank queue field (via ActiveJob) and worker doesn't pick it up. Jobs stay unprocessed. Why was explicit queue param removed https://github.com/rails/rails/commit/fdfca02c304d809aac53ebf8094b237d0ba6415e?  Does it require any additional ActiveJob config on top of queue_as :my_queue_name to get it working?

@chanks is it a correct fix for this issue https://github.com/dmkl/que/commit/a258459f15a09d0983e057bbd2779d3d964e49fb?

I originally removed named queue support from ActiveJob because people were using it for priorities, when Que already has an integer-based priority system that's more efficient for querying. Now I think this was a mistake on my part - people have brought up use cases where they need named queues that aren't priorities.
If that fix helps you work with Rails 5, that's great, but I don't think that abusing the args hash like that is a good idea in general. I think we should try to re-add support for named queues to the ActiveJob adapter instead, though I would encourage you to only use named queues if you really need them (you need different jobs to be processed by different codebases, or by application instances running with different resources, for example).

@chanks btw it would be super duper awesome if workers could specify 0, 1, or many queues to listen on.  In production I might want to have workers dedicated to each queue, but in dev or staging I might not. I believe sidekiq and resque support this.


ternary requires space

1?2:3 is valid js, but does not parse as civet, I don't see this mentioned in https://civet.dev/comparison, so assuming this is a bug
https://civet.dev/playground?code=MT8yOjM%3D

This is either a documentation bug or a compiler bug. 🙂
We have been discussing recently whether to make a?b shorthand for a?.b (an extension of #1327). In fact, I was inclined to implement this very soon. But it would mean that a?b:c wouldn't be a valid ternary. Which do we want?

Ah, fair, I think it's reasonable to require no spaces around the colon when there's no spaces around the question mark in a ternary, and this could probably be supported together with a?b.

I've gone ahead and documented the current behavior. Supporting tight ternary x?y:z is still interesting, but I found it difficult to add in my first attempt (given the current parser, which wants to greedily match an expression before looking for ternary suffix ?...:...).


[MM-28207] Preview post does not have a channel so get all group mention keys in this case

Summary

Fixes broken preview post where channelId is not passed as a parameter...

Ticket Link
https://mattermost.atlassian.net/browse/MM-28207

Could we pass the channel ID to the post preview instead? I think the CreatePost component and co should all be attached to a channel.

@hmhealey I modified a couple of component definitions to make channelId a required prop here
I think I got it all but let me know if you think I missed something...

/cherry-pick cloud-beta

Cherry pick is scheduled.


Improve performance

IMHO page refreshes are too slow for the amount of data being transmitted. Is this a limitation of Heroku? Is there any instrumentation for this?
One idea would be to run this on the same server as the wiki, see if it is any faster on AWS. The wiki needs a decent-sized server for memory reasons, but barely uses any CPU. I doubt comeals requires much in the way of memory.

Actually, this might be solved by a popover spinner while the next page is loading. That way it feels like the button press did something.

I've made calendar and meal navigation happen in the background now while maintaining url updates with client side routing. Things should quite a bit snappier now.

The app is now a full-blown PWA and Redis caches the API calls. At this point, page loads/transitions feel nearly instant.


Documentation for regenerating untrusted OTA certificates in tools/ota_e2e_tests/credentials

Is your feature request related to a problem? Please describe.
The documentation for recreating the untrusted certificate/key pairs for e2e testing either do not exist or the documentation is not easy to find.  Right now, the committed certificate/key pair fails to import with the following error at tag 202002.00.  The resulting ARN is required for e2e and IDT test configuration.
     #+begin_src bash :tangle /tmp/upload_untrusted.sh
SRC=/home/X/src/amazon-freertos-pic32mzef-ota
OTA_TOOLS=${SRC}//tools/ota_e2e_tests
KEY_UNTRUSTED=${OTA_TOOLS}/credentials/untrusted-ecdsa-sha256-signer.key.pem
CERT_UNTRUSTED=${OTA_TOOLS}/credentials/untrusted-ecdsa-sha256-signer.crt.pem

aws acm import-certificate \
    --certificate fileb://${CERT_UNTRUSTED} \
    --private-key fileb://${KEY_UNTRUSTED}
     #+end_src

     #+RESULTS:
An error occurred (ValidationException) when calling the ImportCertificate operation: The certificate is expired. You can Import a certificate only during its validity period.

Describe the solution you would like.
Documentation in README.md on how to recreate the key/certificate pair according to testing guidelines.

Hi @rpcme thanks for reporting the issue, I submitted a PR #1972 for this. Meanwhile, you can find the document for how to generate certificate here, https://docs.aws.amazon.com/freertos/latest/userguide/ota-code-sign-cert.html.

Hi @rpcme
Thank you for creating this issue. I will close it as PR #1972 has been merged.


Can't startup: FastlaneBuildRunner undefined method `user' for nil:NilClass

[2018-05-28 22:41:16] DEBUG FastlaneBuildRunner:  Setting build status to failure due to exception
[2018-05-28 22:41:16] ERROR FastlaneBuildRunner:  undefined method `user' for nil:NilClass
[2018-05-28 22:41:16] ERROR FastlaneBuildRunner:  /Users/jliebowitz/dev/ci/app/services/xcode_manager_service.rb:33:in `use_apple_id'
/Users/jliebowitz/dev/ci/app/services/xcode_manager_service.rb:22:in `initialize'
/Users/jliebowitz/dev/ci/app/services/services.rb:191:in `new'
/Users/jliebowitz/dev/ci/app/services/services.rb:191:in `xcode_manager_service'
/Users/jliebowitz/dev/ci/app/features/build_runner/fastlane_build_runner.rb:170:in `block in run'
/Users/jliebowitz/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/bundler-1.16.1/lib/bundler/shared_helpers.rb:65:in `chdir'
/Users/jliebowitz/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/bundler-1.16.1/lib/bundler/shared_helpers.rb:65:in `block in chdir'
/Users/jliebowitz/.rbenv/versions/2.3.3/lib/ruby/2.3.0/monitor.rb:214:in `mon_synchronize'
/Users/jliebowitz/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/bundler-1.16.1/lib/bundler/shared_helpers.rb:64:in `chdir'
/Users/jliebowitz/dev/ci/app/features/build_runner/fastlane_build_runner.rb:128:in `run'
/Users/jliebowitz/dev/ci/app/features/build_runner/build_runner.rb:289:in `run_action'
/Users/jliebowitz/dev/ci/app/features/build_runner/build_runner.rb:310:in `block (2 levels) in start'
/Users/jliebowitz/dev/ci/app/features/build_runner/build_runner.rb:178:in `block in pre_run_action'
/Users/jliebowitz/dev/ci/app/shared/models/git_repo.rb:507:in `block in checkout_commit'
/Users/jliebowitz/dev/ci/app/shared/models/git_repo.rb:446:in `block in perform_block'
/Users/jliebowitz/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/bundler/gems/TaskQueue-5b3f459be5c4/lib/queue_worker.rb:82:in `block in process'

This must be due to a env var or something, still catching up from vacation 🏖

Yup, old bug, missing env var. Will be fixed in #870


Create The Things Network doc

Summary

Create and link to documentation with TTN addresses
Screenshots

Checklist

[x] Scope: The referenced issue is addressed, there are no unrelated changes.
[x] Run Locally: Verified that the docs build using make server, posted screenshots, verified external links. Test with HUGO_PARAMS_SEARCH_ENABLED=true if style changes will affect the search bar.
[ ] New Features Marked: Documentation for new features is marked using the new-in-version shortcode, according to the guidelines in CONTRIBUTING.
[ ] Style Guidelines: Documentation obeys style guidelines in CONTRIBUTING.
[x] Commits: Commit messages follow guidelines in CONTRIBUTING, there are no fixup commits left.

@benolayinka please clear the code owners for /doc/layouts/shortcodes or assign the same as the content. Now it's blocked by frontend devs while that isn't necessary.


Way not zoomed to in search result

Sure I've seen this issue before, but can't find it - some weird search/zooming behaviour for ways:
If you put "n3191505807" into the search box and click on the result it zooms to the node.
If you put in "w5182746" it shows the tags but it does not zoom. However if you click the X then re-enter the search and click on w5182746 it zooms to it.

Yes it seems @willemarcel commented about this on issue #4276.
Better to report twice than never :+1:


Fix for CRAN Submission

This PR has some fix for the CRAN submission. The current version has an error from the R CMD check result on win-builder:
* using log directory 'd:/RCompile/CRANguest/R-release/xgboost.Rcheck'
* using R version 3.3.2 (2016-10-31)
* using platform: x86_64-w64-mingw32 (64-bit)
* using session charset: ISO8859-1
* checking for file 'xgboost/DESCRIPTION' ... OK
* checking extension type ... Package
* this is package 'xgboost' version '0.6-0'
* checking CRAN incoming feasibility ... NOTE
Maintainer: 'Tong He<EMAIL_ADDRESS>
Found the following (possibly) invalid URLs:
  URL: https://www.kaggle.com/wiki/MultiClassLogLoss
    From: man/xgb.train.Rd
    Status: 404
    Message: Not Found
* checking package namespace information ... OK

.......

* checking for unstated dependencies in 'tests' ... OK
* checking tests ...
** running tests for arch 'i386' ... ERROR
Running the tests in 'tests/testthat.R' failed.
Last 13 lines of output:
  [96]	train-rmse:0.998892
  [97]	train-rmse:0.998892
  [98]	train-rmse:0.998892
  [99]	train-rmse:0.998892
  [100]	train-rmse:0.998892
  testthat results ================================================================
  OK: 325 SKIPPED: 1 FAILED: 3
  1. Failure: training continuation works (@test_basic.R#151)
  2. Failure: training continuation works (@test_basic.R#157)
  3. Failure: training continuation works (@test_basic.R#162)

  Error: testthat unit tests failed
  Execution halted
** running tests for arch 'x64' ... [29s] OK
  Running 'testthat.R' [28s]
* checking for unstated dependencies in vignettes ... OK
* checking package vignettes in 'inst/doc' ... OK
* checking re-building of vignette outputs ... [24s] OK
* checking PDF version of manual ... OK
* DONE
Status: 1 ERROR, 1 NOTE

The most important issue is the error from the test_basic.R, on a 32-bit machine. All the failures are related to a call expect_equal(bst$raw, bst2$raw). However I don't have a 32-bit machine by hand for now. If someone could help with the test, please let me know the details.
I can visit the reported URL (https://www.kaggle.com/wiki/MultiClassLogLoss) from my web browser. Would be great if someone knows what is happening.

I will try to reanimate my old WinXP laptop tomorrow... But somehow I was assuming that 32 bit platforms were not actually supported by xgboost - were they?
And I don't see anything wrong with that url either.

I think the check for different archs always exists on CRAN. We might tweak the code a bit to fulfill CRAN's requirement, if necessary.
To reproduce the bug on a 64bit machine, can we somehow flag something in Makevars/Makevars.win, so that it simulate the 32bit environment? I am not sure how to do it. Besides, the environment for this check result is on Windows.

windows machine allows compilation in 32 bit when specified. XGBoost is supposed to be compatible in 32 mode. In case we need to run prediction on mobiles

I think I have met expect_equal problem on 32-bit machine before.
Not a big problem.

This seems to be related to the 32-bit compiler. I tried to compile xgboost using different optimization flags. No error occurred if the -O0 flag was used, but the check failed if -O1, -O2 etc. were used.

I compiled and tested with the 32bit R.exe on Windows, and compared with the bst$raw and bst2$raw generated by 64bit R.exe. Here are the results:
> # The mismatch from 32bit
> expect_equal(bst$raw,bst2$raw)
Error: bst$raw not equal to bst2$raw.
1 element mismatch

> # The normal result from 64bit
> expect_equal(bst3$raw,bst4$raw)

> # Comparing result from 64bit to 32bit
> expect_equal(bst3$raw,bst$raw)
Error: bst3$raw not equal to bst$raw.
72 element mismatches

> # Comparing result from 64bit to 32bit
> expect_equal(bst3$raw,bst2$raw)
Error: bst3$raw not equal to bst2$raw.
72 element mismatches

Note:

bst$raw and bst2$raw are generated from 32bit.
bst3$raw and bst3$raw are generated from 64bit.

Does anyone have any insight on this issue? I have no clue on the difference between the archs.

Could have something to do with filling of the padded elements in aligned structure, can you post a code? Which won't hurt loading but makes raw bits different.

It might be helpful to dump the model in text format, to see if it is problem of model format, or problem of tree construction difference, under 32 bit

I took a quick look at this test in 32 bits win too. I was also changing
the eta, depth and nrounds. A strange thing is that raw models were
identical for some configurations, but for some they were different, and
the number of different bytes usually grows with depth and nrounds in
continuation. The results didn't seem to depend on the seed value.
I also did some comparison of tree dumps (setting 17 digits stringstream
precision) for a few cases of not-same-raw models: sometimes the dumps were
identical (which means that something that not in the dump was different),
and when the differences were visible, those were tiny changes in either
gain or cover values.
On Thu, Dec 1, 2016 at 1:22 AM, Tianqi Chen<EMAIL_ADDRESS>wrote:

It might be helpful to dump the model in text format, to see if it is
problem of model format, or problem of tree construction difference, under
32 bit
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/dmlc/xgboost/pull/1826#issuecomment-264097091, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABie79uJaF2oaYpZ3AyrxmlF7t3uKQ0Tks5rDnW1gaJpZM4K_136
.

I have dumped the models. From the text format, the 32 bit and 64 bit have slight difference in the splitting value:  -9.53674e-007 against -1e-006. I believe this is what causes the 72 mismatches in the above result.
However, within each arch, the text files are exactly the same. I have attached the raw models (two R objects) generated by the i386 R.exe from the following code:
require(xgboost)
require(testthat)

data(agaricus.train, package='xgboost')
data(agaricus.test, package='xgboost')
train <- agaricus.train
test <- agaricus.test
set.seed(1994)

dtrain <- xgb.DMatrix(train$data, label = train$label)
watchlist = list(train=dtrain)
param <- list(objective = "binary:logistic", max_depth = 2, eta = 1, nthread = 2)

# for the reference, use 4 iterations at once:
set.seed(11)
bst <- xgb.train(param, dtrain, nrounds = 4, watchlist)
# first two iterations:
set.seed(11)
bst1 <- xgb.train(param, dtrain, nrounds = 2, watchlist)
# continue for two more:
bst2 <- xgb.train(param, dtrain, nrounds = 2, watchlist, xgb_model = bst1)
expect_equal(bst$raw, bst2$raw)

With i386 R.exe + latest xgboost version, the last line expect_equal is likely to trigger an error:
Error: bst$raw not equal to bst2$raw.
1 element mismatch

The difference lies at the 1718-th position. ce against bd. Since there's no difference in the model file, I am not sure if it is caused by floating error as well.
raw.zip

I will merge these changes for now, we can have another PR to fix the raw test issue


PCA updates

What changes were proposed in this pull request?
At the suggestion of @agibsonccc to go ahead and try to revamp the PCA routines, I tried to implement a PCA-type of method that takes data (records each with a fixed number of features) as rows (w/ features as columns) and calculates the covariance matrix, the mean values of each feature, the eigenvectors, and the eigenvalues, and then performs a dimensionality-reduction to a provided variance of the data.  Since it goes along with the existing PCA.java, I added it in there and started to think about everything.
The revamped PCA class has all the old static methods but can be instantiated as an object, and the reason is that since there is so much data reported, it helps to be able to access things like myPCA.getEigenvalues() and myPCA.getCovarianceMatrix().  In order to calculate the variance between data calculated with a limited basis set vs. the exact data represented in the full basis set, it was necessary to have those eigenvectors stored within the object.  The only other alternative was to write functions that return INDArray[4] which is not very readable and usable.
So the changes are meant to provide alternative usage of PCA and to maintain all existing routines.  To transform data from features into linear combinations of basis vectors, I multiply E_transpose*dx,  where E_transpose is the eigenvector matrix transposed (which is its inverse because it's orthonormal unit basis vector matrix) and dx is the difference between the features and their means for a given record.
How was this patch tested?
TestPCA.java has some of my tests.  I have some other ones in a little app that I can post as a gist.
One of the important tests was that the dimensionality reduction indeed works to a given variance.  To do that, I calculated the projection of the data from the mean onto the basis vectors and then calculated the Euclidean distance to the data point without the removed dimensions vs. the same value with all dimensions (which exactly lies at the point of interest) and calculated a ratio.  I did it for 100 random samples and it seems to provide 87% of the variance when I told it to reduce it to within 70% variance.  A similar test that intends to reduce to 90% variance was actually able to reproduce 99.5% of the variance for 100 random samples.  So I am just erring on the side of being safe here, since when this thing is fed with 10000 data records that have 1000 features, it may not be as great as when fed with 10000 data records that have 6 features.   But since randn was used in the generation procedure and some variables were pre-correlated to others by addition & subtraction, I could also check by eye that the results make sense.
I am definitely interested in someone providing large data (a lot of records each with many features) to compare the different methods here

I'm making a minor change, I was taking the sqrt(variance) so my estimates are more like 76% variance for the dimensionality reduction to 70%, and 99% variance for the dimensionality reduction to 90%.    Basically just squaring 0.87 and 0.995.  Pushed new minor update to PCA.java to remove Math.sqrt(of the sum of values squared) to just the sum of values squared.

I didn't check the math, but other than that looks good to me! Thanks

Yea for the math, this was the idea of the variance for each component in the reducedBasis method and the related static method.  But the variance of 100 random samples in TestPCA.java tests it directly, too.  If someone has real large data (w/ lots of features) I could give it a production test.

I think of the problem in terms of direction of force in physics.  The smallest eigenvalue is like the loosest spring that vibrates the most along the +-direction of its eigenvector (either in the direction or in the opposite direction of the vector from the mean), and the largest eigenvalue is like the stiffest spring that only vibrates a little along the +-direction of its eigenvector.  So if you have 16 features that picture extends into 16 dimensions.  But it turns out only 10 of those "modes" may be necessary to account for 97% of the variance in the data, because most of the fluctuations are in the direction of those vectors.   These modes do not lose any of the original features, but just couple them together into a reduced number of modes.
These things could literally be forces and you could use them to sample states that reproduce the same exact distribution if the data was purely bell-curved/gaussian.  It would basically produce fake data that appears to belong in the set.   I could add that to the code (sampling states using the Boltzmann distribution), @agibsonccc what do you think?  It's in my code already, which is (will be) open source, but I've been holding off on releasing it in Java form.

Dummy data generator has a big test in TestPCA.java, it reproduces the same quality of data (variance by feature, eigenvalues/vectors, and mean) as the actual input data records used to produce them, within the limits of your sample size.   10000 input data records (of 16 features) are analyzed and then 10000 dummy data records are created that then match the same principal components, eigenvalues, mean, and variances.

@lukeczapla FWIW, I think you have enough tests in there :) Thanks

ok, I'm going to clean it up in terms of documentation and add a routine to convert your data into the component vectors about the mean, and then you'd drop a certain number of columns at the end.  They're nearly always the smallest numbers.  tensorMmul ftw :)

great, let me know if it's acceptable.  @agibsonccc said he's away until next week, I can do more work on it and maintain it.  I'm a high school teacher so I'm busy after the summer but I want people's feedback and can add more stuff as necessary.  right now everything's chill and I'm writing a lot of code, between Java and JavaScript.  So I'm back and forth and will come back and improve it as I get new ideas about how people would use this.

Looks good to me, but let's try to get some feedback from @rcorbish. He's the last one to have updated code in that area. :) @rcorbish what do you think, good to merge?


Your discord link is invalid

Title says it all. No way to get to your discord server for updates, etc. Link is invalid.

https://discord.com/invite/gTgJRz7 doesn't work?

If you click on the link towards the bottom of the page, then you are clicking on an expired invite. If you click on chat at the top of the page then you are golden.

Yeah, the link at the bottom of the page is broken

ill fix that


Integrate changes to GetAsElementsFromEndpoint

GetAsElementsFromEndpoint was renamed to GetFromEndpoint after 1.8.0-RC1. Update the python shell entry.

Merged into develop

More work needs to be done to remove the element generator

Merged into develop


Add mTLS support

Hi,
The keycloak server can be protected with mutual TLS authentication done directly by Keycloak or with the help of  a reverse proxy.
The current pull request adds support for mTLS client authentication.
I have added 2 parameters instead of 1 single like some tools do (HAproxy for instance) where the client certificate and private key are provided as a file concatenation of PEM encoded assets. The main reason is that it allows them to be protected differently by the caller, the private key can be passed as a sensitive variable of a terraform module. It avoids parsing a file to extract them.
Also I think it is better to pass strings instead of file paths it gives more flexibility for the caller to store these assets.
But the drawback is that it adds 2 parameters and functions start to have too many parameters. I did not want to refactor that using option callbacks for instance.
No unit tests are provided.
I also go formatted the changed source files.
BR

Closing the pull reques.
I will re-create a new one following the module renaming (from mparkers) and other changes in the client. I had created the original from a repository that had already diverged from the source one.


Disable JWT authentication on console requests

Description
Previously the plugin would initialize itself on application load. This is fine, but Craft::$app may be either a craft\web\Application or a craft\console\Application, which are different applications with different use cases.
As seen in https://github.com/edenspiekermann/craft-jwt-auth/issues/11, this would cause all console requests to crash as the console application resolves a console request, and there is no craft\console\Request::headers.
This PR changes the class that the plugin initialzes on from Craft->$app to craft\web\Application, causing the plugin to not attempt to authenticate a user based on the current headers on console requests.
Definition of Done

[x] Ensure there are no formatting errors
[x] Ensure feature branch has latest development code integrated
[x] Version number in composer.json is updated
[x] New entry added in CHANGELOG.md documenting changes made

I'm running into this issue too.  Manually changing the code using this PR fixes the issue for me.


feat: Add Commit Message to Deployment Section

Showing the commit message in the deployment section would help in easy understanding of the version of current/past deployments

Fixed by https://github.com/swiftwave-org/swiftwave/pull/861
https://github.com/swiftwave-org/dashboard/issues/213
Thanks @AnupamKris for your contribution


Dialog - dataSource option

It would be nice to define/ set the content of the body container via ajax remote call/ html-string.
The dataSource option be also changeable via open method.

I will think about it and may add it in future versions.
Right now you can initiate an ajax call on open on initialized events in order to populate the body with remote call.

I'm consider me as beginner in javascript.
Can you give an example how I can load html content via ajax in the dialog body with ID as parameter?

I see at the moment no easy way to set the content of the body after the initialisation of the dialog.

Hi Stefan,
We added new method in the new development version called 'content'. You can use this method in a combination with initialized event in order to easily manage the dialog content via ajax calls.
Here is an example: http://gijgo.com/Temp/49ff3f7f-1353-47dd-81b3-d8ddc3860c4a.html
Here is the sample code:
<!DOCTYPE html>

    <title>jQuery Dialog</title>
    <meta charset="utf-8" />

    <link href="http://code.gijgo.com/1.6.0/css/gijgo.css" rel="stylesheet" type="text/css" />

okay, thanks...
This solution is universal.


PTS gaps in audio input

What version of Shaka Player are you using?
Shaka Player v2.0.5-debug

I would like your opinion and comments on the subject of having PTS gaps in input audio track which then propagate through the dash packager to the player.
Please take a look at this issue at shaka packager where we tried to address the problem in the packager itself.
https://github.com/google/shaka-packager/issues/196
A couple of basic questions I believe might help clear up my own understanding of the situation. I would appreciate comments...
Should this kind the gaps be handled by player?
Should this be handled/treated by transcoder or packager in any way?
Is this kind of input (with audio gaps) at all valid?
Observed behaviour
When no attempt to handle the gaps is done at packager, player stops after about 30-60 seconds.
Packager is correctly adding gaps in the SegmentTimeline by inserting S entries
When in the packager, I ignore the gaps and collect audio samples as they come, player works but audio/video slide out of sync after around 5-10 minutes. Packager is not adding any gaps in the SegmentTimeline. They repeat "perfectly" with the same duration
When in the packager, I "prolong" the duration of the audio samples that have gaps after them, the packager adds correctly entries in SegmentTimeline the player with different durations. The player works but audio/video slide out of sync after around 5-10 minutes

Gaps should usually be handled on the packager (or transcoder) side.  They are the ones producing the content and they should be the ones to produce valid, playable, content.  However, we are currently working on adding gap jumping to the player because we see a lot of content with gaps in it (especially with live streams).  See #180 and #555.
Having gaps is probably valid in terms of the spec, but it makes playing it in the browser difficult.  Because we use MSE, we just give data to the browser and it then it creates "buffered ranges" based on what data it has.  If there are gaps, the browser doesn't know whether that content will be added later.  When the browser plays to the gap, it will stop playback, waiting for the content.
But, since we have the manifest, we know that the content will never be added.  So we could see the gap and jump it automatically.  And that is what we are working on.

I should also note that small gaps, the browser will jump automatically.  What is small depends on the browser.  Firefox will jump a gap of 200ms while Safari will only jump 22ms.  Currently, we won't buffer past a gap of 20ms because that is the minimum amount that will be jumped.  This provides a consistent cross-browser experience and makes bad content more obvious.

I will try other browsers... a bit. I used so far only Chrome.
I have tried to create a fix in the shaka packager, specifying in the dash manifest that the segment duration is longer when there are gaps in any of it's contained audio samples. But, the pts gaps still do exist in the audio elementary stream itself.
In Chrome, would that explain the fact that on player reload (stream restart) the player successfully starts perfectly synced audio/video but they slide apart after a short while as those gaps present in elementary audio stream accumulate?

@TheModMaker, does Chrome skip gaps at all?
In my case, using Chrome I can see desync very soon.
Using Firefox there seems to be no desync at all.

This may be a bug in Chrome.  What I think may be happening is that Chrome is not using the times in the container (MP4) to determine the frame times.
What happens is we append the data to the media source.  The times for each frame is encoded both in the container and in the elementary stream.  However, in the stream, it only encodes the duration of the frame, not the absolute time.  So as Chrome is decoding frames, it is using the duration to determine when the next frame is and this leads to the audio lagging behind since the duration doesn't account for the gaps.
Can you generate an on-demand version of the content? If you generate an on-demand stream, the packager will generate a single MP4 file that also is playable by itself.  This will allow you to play the file by itself in Chrome to see if it get desync there too.

I do have a .ts recording of this live stream that I can convert to an on-demand version and could try that.
The .ts recording and the live transport stream as well are perfectly playable by itself in VLC for example

@hariszukanovic Any update on this?  Were you able to generate the on-demand content?

@hariszukanovic, since we haven't heard back from you in a few weeks, I'm closing this issue due to inactivity.  We can reopen it at any time if you wish.


Update Theta tutorial scripts for 1.0.0

Created a pull request here:
https://github.com/geopm/theta-tutorial/pull/3
The update was not super automatic, so please review.


Update: Relational: Potential optimization in MERGE SQL batching path?

(Based on feedback from  https://www.brentozar.com/archive/2017/05/case-entity-framework-cores-odd-sql)
Right now when using the MERGE batch update path, we perform a final ORDER BY on the server to retrieve the store gen'd values in the correct batch order. However, it could be that the values are already in the correct order, so we could perform the ordering on the client, lazily, by detecting out of order rows when they occur. NB. This would sometimes require client buffering of the reader.

cc @divega

Another comment from https://www.brentozar.com/archive/2017/05/case-entity-framework-cores-odd-sql/#comment-2398183:

Zac Torkelson     May 9, 2017 4:42 pm
Were Table Value Constructors (https://docs.microsoft.com/en-us/sql/t-sql/queries/table-value-constructor-transact-sql) considered instead of the temporary toInsert table variable? I have had good results with this technique in the past and would be interested in your benchmarking results.
Something like (apologies for the formatting and any syntax errors as I am presently mobile):

  MERGE [Weapons] USING (VALUES
  (@p0, @p1, 0),
  (@p2, @p3, 1),
  (@p4, @p5, 2)
  ) AS i ([IsPointy], [Name], [_Position]) ON 1=0
  WHEN NOT MATCHED THEN
  INSERT ([IsPointy], [Name])
  VALUES (i.[IsPointy], i.[Name])
  OUTPUT INSERTED.[WeaponId], i._Position
  INTO @inserted0;

@divega The Table Value Constructor pattern still allows for an explicit order sentinel and so it doesn't matter whether the order is deterministic. It looks to be a slightly cleaner syntax by omitting one TV.

My reading of #4080 is that non-deterministic output from the multi-row INSERT caused the client to incorrectly reconcile inserted identities with their respective entities. Is that correct?
If so, I think we'd be fine here, because the above construct still allows the explicit _Position column for deterministic ordering. Indeed, if client-side sorting were possible (as mentioned in the OP), then both variables could be eliminated, leaving only a single MERGE statement.

I think we'd be fine here, because the above construct still allows the explicit _Position column for deterministic ordering

@anpete @zac-torkelson you are right. BTW, thanks for the feedback!

@AndriySvyryd Thoughts on this for 2.0? Should it be punted for now? /cc @divega

I think it can go in. These aren't breaking changes.

I removed the ORDER BY and ran a microbenchmark inserting 100x10000 rows with identity and saw no measurable difference so this doesn't seem to be worth doing.

@AndriySvyryd Nice! Can you share the query plans for both?

The query plan shows that ordering takes up a big part of the execution time, but the actual duration is the same. I don't have enough expertise to interpret this.

Looks like the merge takes so long that the subsequent select duration is insignificant

Interesting, thanks for looking at this. Great to understand what's going on here.

@anpete @AndriySvyryd should we close this issue as fixed then?
I wonder whether we should still track offloading ORDER BY from the server. Perhaps some benefit would be observable under load, but we don't have time to test it now.

@divega Agree, it's hard for me to imagine that there is no scenario where this doesn't show an improvement. There is also some benefit in just having simpler SQL.

We should close this and track further optimization in separate issues when we find scenarios that need improvement.
While the SQL would be simpler client code would be much more complicated and could degrade overall performance in some scenarios.

Ok, created https://github.com/aspnet/EntityFramework/issues/8849 to track ORDER BY removal. Closing as fixed in preview2.


Create page "Contacts"

https://www.figma.com/file/IGERJuRUhZohuxq5b3O73w/Podkrepi.bg-Wireframes?node-id=278%3A0

@ani-kalpachka Draft of the integration has been started on branch contact-page
https://github.com/daritelska-platforma/frontend/compare/contact-page?expand=1
You can continue the effort if you'd like

@kachar Started working on the task.


[Enhancement]Some form of "Advanced" control for animations

I'm suggesting the ability to control the time between individual frames, as-well as the type of interpolation between 2 frames. (Please, it would be so useful for so many things)

Duplicate of #192


fit_generator does not expose a sample_weight parameter like fit

While the signature of fit is
fit(x=None, y=None, batch_size=None, epochs=1, verbose=1, callbacks=None, validation_split=0.0, validation_data=None, shuffle=True, class_weight=None, sample_weight=None, initial_epoch=0, steps_per_epoch=None, validation_steps=None)
the fit_generator is
fit_generator(generator, steps_per_epoch=None, epochs=1, verbose=1, callbacks=None, validation_data=None, validation_steps=None, class_weight=None, max_queue_size=10, workers=1, use_multiprocessing=False, shuffle=True, initial_epoch=0)
so class_weight is defined for both, the validation_data can have optionally sample weights in the form of tuple (x_val, y_val, val_sample_weights), that will work on validation set only, but the fit_generator has no sample_weight option, i.e. the array of weights for the training samples.
Why training the model for a given number of epochs (i.e. fit) support sample weighting while training the model on data generated batch-by-batch (i.e. fit_generator) does not?
At the end of the day fit_generator will use the model.train_on_batch api:
def fit_generator(model,
                  generator,
                  steps_per_epoch=None,
                  epochs=1,
                  verbose=1,
                  callbacks=None,
                  validation_data=None,
                  validation_steps=None,
                  class_weight=None,
                  max_queue_size=10,
                  workers=1,
                  use_multiprocessing=False,
                  shuffle=True,
                  initial_epoch=0):

https://github.com/keras-team/keras/blob/351e7a992d8f948e373e321252a35824da77ac35/keras/engine/training_generator.py#L215
Now the method train_on_batch support both class_weight than sample_weight parameters:
https://github.com/keras-team/keras/blob/351e7a992d8f948e373e321252a35824da77ac35/keras/engine/training.py#L1172
def train_on_batch(self, x, y,
                       sample_weight=None,
                       class_weight=None):

Now what happens internally here https://github.com/keras-team/keras/blob/351e7a992d8f948e373e321252a35824da77ac35/keras/engine/training_generator.py#L334 is like
if len(generator_output) == 2:
                x, y = generator_output
                sample_weight = None
            elif len(generator_output) == 3:
                x, y, sample_weight = generator_output
            else:
                raise ValueError('Output of generator should be a tuple '
                                 '(x, y, sample_weight) '
                                 'or (x, y). Found: ' +
                                 str(generator_output))

so a sample_weight is mapped from the generator output when your return the tuple  (x_val, y_val, val_sample_weights), so apparently it will be used on the training batch, depending on your generator.
That said, why the fit_generator is not exposing the sample_weight argument then?

I see you opened two issue related to the same problem. Would it be possible for you to close one and follow everything on one single thread? thank you.

@gabrieldemarmiesse correct, l have realized later that those issues could be related to the same problem. Which one do you suggest to close? Thank you.

Read the docs on fit_generator.
The output of the generator must be either

a tuple (inputs, targets)
a tuple (inputs, targets, sample_weights).

The generator for train or validation simply needs to
 yield X, y, w

@brge17 thanks, as for https://github.com/keras-team/keras/issues/11805#issuecomment-444565458 the fit_generator and fit will be soon merged into one taking in account the api signature of both methods.


Can't install

----- Installing 'pip install StringProgressBar' -----
Requirement already satisfied: pip in c:\program files (x86)\microsoft visual studio\shared\python39_64\lib\site-packages (22.0.4)
Collecting install
Using cached install-1.3.5-py3-none-any.whl (3.2 kB)
Collecting StringProgressBar
Using cached StringProgressBar-1.1.1.tar.gz (4.0 kB)
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'error'
error: subprocess-exited-with-error
python setup.py egg_info did not run successfully.
exit code: 1
[1 lines of output]
ERROR: Can not execute setup.py since setuptools is not available in the build environment.
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
Encountered error while generating package metadata.
See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
----- Failed to install 'pip install StringProgressBar' -----

Whats your python and pip version?
I done two tests on Windows 11 (pip 22.0.4 and python 3.8.8) and Raspberry OS (pip 20.3.4 and Python 3.9.2).
You can try to purge cache and install again

Whats your python and pip version?
I done two tests on Windows 11 (pip 22.0.4 and python 3.8.8) and Raspberry OS (pip 20.3.4 and Python 3.9.2).
You can try to purge cache and install again

Thanks, after updating setup tools, it worked :)


Rename wagtail.wagtaildocs to wagtail.documents

Part of #3918
This PR renames wagtail.wagtaildocs to wagtail.documents.

Merged in #4069


Update to latest version of Nimble and Quick

This commit addresses issue #22, by updating to the latest versions of the Nimble and Quick libraries, which compile successfully under Xcode 7.3.

Thanks!

@nuudles @maxmeyers should we put checkout files in the repo as well?
It's causing carthage to pickup Quick.xcproject and Nimble.xcproject and compile them too.

I've send out another Pull request here
https://github.com/IFTTT/RazzleDazzle/pull/26
Please let me know if I'm missing something.


Investigate auto migrating issues to new iteration with github actions

https://github.com/marketplace/actions/move-to-next-iteration

Needs specific timestamp and needs a secret to work.

Run it and will auto update all issues to current iteration.


invalid gradient at index 0 - expected shape [] but got [1]

Hi,
I am new to pytorch, and was trying to use your model for summarizing some text. I have been trying to run training with news data, and then added only a single new file, in both cases, train_full_rl fails in rl.py file with below error, I tried to switch to GPU and still get same problem, I was wondering if you have seen this error before. When I was using GPU, it would show error that input and target size does not match, that is why I am printing out sizes (torch.Size([700])):
{'net_args': {'extractor': {'net_args': {'conv_hidden': 100, 'emb_dim': 128, 'bi                                                                                directional': True, 'vocab_size': 30004, 'lstm_layer': 1, 'lstm_hidden': 256}, '                                                                                traing_params': {'batch_size': 32, 'lr_decay': 0.5, 'optimizer': ['adam', {'lr':                                                                                0.001}], 'clip_grad_norm': 2}, 'net': 'ml_rnn_extractor'}, 'abstractor': {'net_                                                                                args': {'emb_dim': 128, 'bidirectional': True, 'vocab_size': 30004, 'n_hidden':                                                                                256, 'n_layer': 1}, 'traing_params': {'batch_size': 32, 'lr_decay': 0.5, 'optimi                                                                                zer': ['adam', {'lr': 0.001}], 'clip_grad_norm': 2.0}, 'net': 'base_abstractor'}                                                                                }, 'train_params': {'gamma': 0.95, 'lr_decay': 0.5, 'stop_reward': 'rouge-1', 'c                                                                                lip_grad_norm': 2, 'batch_size': 32, 'reward': 'rouge-l', 'optimizer': ('adam',                                                                                {'lr': 0.0001}), 'stop_coeff': 1.0}, 'net': 'rnn-ext_abs_rl'}
Start training
torch.Size([700])
torch.Size([700])
Traceback (most recent call last):
File "Geneva_ABS.py", line 266, in
main(args)
File "Geneva_ABS.py", line 224, in main
train_full_rl().train(path=output_path+'/model/', abs_dir=output_path+'/abst                                                                                ractor/', ext_dir = output_path+'/extractor/')
File "/home/train_full_rl.py", line 195                                                                                , in train
trainer.train()
File "/home/training.py", line 211, in                                                                                train
log_dict = self.pipeline.train_step()
File "/home/rl.py", line 177, in train                                                                                step
self._stop_reward_fn, self._stop_coeff
File "/home/rl.py", line 108, in a2c_tr                                                                                ain_step
[torch.ones(1).to(critic_loss.device)]*(1+len(losses))
File "/home/venv/lib/python3.5/site-packages/torch/autograd/in                                                                                it.py", line 90, in backward
allow_unreachable=True)  # allow_unreachable flag
RuntimeError: invalid gradient at index 0 - expected shape [] but got [1]

Oh it seems another person had same problem probably:
https://github.com/ChenRocks/fast_abs_rl/issues/17

I have met this problem too, and that's how I fix it:
change
https://github.com/ChenRocks/fast_abs_rl/blob/aebf539107caba5be35720f5d1f9f98989a069e8/rl.py#L103
to
[critic_loss.unsqueeze(0)] + losses

this problem will happen when critic_loss is a number not an array.

@chenhuaizhen comment is the fix for the issue.

I solve it by this:
backprop and update
the critic_loss is a float but the losses is a list of tensor in shape [1]!!
they the critic_loss and the losses need different type of gradient one is a tensor(1.0)
the other is a ones(1)
autograd.backward(
[critic_loss] + losses,
[torch.tensor(1.0).to(critic_loss.device)] +
[torch.ones(1).to(critic_loss.device) for _ in losses]
)


Refactoring options input

Did a bit of refactoring on this, so it's easier to manage and maintain the various options, especially to make it easier to add options going forward.

I was only able to do this massive of a refactor due to the exhaustive unit tests we had on it


Improvements needed to handling of socket connection lost

Description
Currently, in the client, if the socket disconnects then it pops up Connection Lost alert and enters a sequence of repeated attempts to reconnect.  Initially it tries every 5 seconds, for a period of 60 seconds, then it drops back to trying every 30 seconds, After a total of 5 minutes it gives up entirely, leaving the Connection Lost, Trying to reconnect message on screen, which is not correct as it is no longer retrying.  Some other action needs to be taken at this time.  In addition there are some reasons for connection lost that will not recover (such as an authorisation token expiring), the retry sequence is not appropriate in these cases.
I am currently attempting to identify possible causes of disconnection and how to identify them. I will update this as I build the list, and will propose appropriate recovery action.
The first class of failure is for situations where attempting to reconnect for  period is entirely appropriate.  This includes network failures that prevent access to the server, and problems with node red itself that prevent the connection from succeeding, for example the server is stopped or is in the process of being restarted.
The retry algorithm described above has two problems:

If a laptop is suspended then the network may be shut down and the connection may fail before node red is suspended.  If it is left suspended for more than 5 minutes then the retry algorithm notices that more than 5 minutes has elapsed since the connection failed, so does not try again.  I believe a similar thing may happen on Android if the the OS decides to put the app to sleep when the screen is off or the app is put in the background.  To avoid this I propose to change the retry algorithm from time based to retry count based, so it will try 12 times at 5 second intervals, then another 8 times at 30 second intervals.
If the connection is not restored within the 5 minutes then the user is left with a notification saying the app is trying to reconnect, but in fact it is not trying.  I propose that after the 5 minutes, rather than just hanging, then we should force a page reload.  I will do some tests to see what the effect of this is when the connection still cannot be restored.

The second class of failures is for situations where it is permanently not possible to reconnect.  For these there is no point retrying, the solution is to immediately force a page reload. An examples of this is where an authorisation token has expired and it is necessary to reload in order to load the logon page. The only situation I have been able to test for this is a system that uses Cloudflare Zero Trust to connect to the server, and in that case, when the token expires, after a little while a socket.on('connect_error', (err) => { is triggered, where the error contains {"code": "parser error"}.  I propose to trap that, and immediately force a page reload for that situation.  There may well be other such permanent failure modes, but unless anyone can identify them now I don't see what we can do other than wait for them to be identified.
Have you provided an initial effort estimate for this issue?
I am no FlowFuse team member

I think since I use basic authentication directly with Node-Red, I don't think I have issues with token expiring. At least if I force a page refresh because some widgets stop working no login is required from the browser.

I have updated the Description with my findings and proposal.  I will generate a PR to implement them.

The {"code": "parser error} on connect_error seems to be a bit odd! Is there anything else on the error object other than the code key?

No, that is all.  Could it be that the server is again responding with the cloudflare logon page?  So effectively the same as the JSON parser error we were getting when trying to parse the _setup response in the same situation?

That is very unfortunate! That would be my best guess as well, expecting to receive a json or related type of response and receiving text, html or something unexpected.
We we would need some more info to go on. Could you add a global event listener and print out all the messages socket.io is brokering? Something similar to
socket.onAny((event, ...args) => {
    console.log(`Received event: ${event}`, args);
});

What would interest us is the event and args right before the parser error. I hope it will shed some light on what's happening because we're kind of shooting in the dark

I put that in, but it doesn't show anything useful.  The connection initially keeps going, even though the token has expired, presumably because it is an already open connection.  It is not until the connection is dropped for some reason that there is an issue.  The code attempts to reconnect and fails.  For example, if I trigger it by suspending the laptop briefly then I see
index-CYywKa1H.js:231

 GET https://nodered.clanlaw.org.uk/dashboard/socket.io/?EIO=4&transport=polling&t=O_oLHRK net::ERR_CONNECTION_CLOSED

index-CYywKa1H.js:239
 SIO connect error:
Error: timeout
    at index-CYywKa1H.js:231:55308
 err: {}

index-CYywKa1H.js:239
 SIO connect error:
Error: server error
    at lt.onPacket (index-CYywKa1H.js:231:36803)
    at Emitter.emit (index-CYywKa1H.js:231:20587)
    at Polling.onPacket (index-CYywKa1H.js:231:22781)
    at r (index-CYywKa1H.js:231:24971)
    at Array.forEach (<anonymous>)
    at Polling.onData (index-CYywKa1H.js:231:25024)
    at Emitter.emit (index-CYywKa1H.js:231:20587)
    at Request.onLoad (index-CYywKa1H.js:231:27661)
    at a.onreadystatechange (index-CYywKa1H.js:231:27088)
 err: {"code":"parser error"}
```
The final line there is shown because I have modified the connect_error log to

console.error('SIO connect error:', err, err: ${JSON.stringify(err)})
The only way I know to recover from this is to reload the page.

It wouldn't be enough to deduce what's happening. Is there anything logged on the NR side when these errors occur?
If we can't get anything out of NR logs I think the only solution left would be the initial option of hard reloading the page.

Nothing useful in the server log.  Even at Trace level in node red and NODE_ENV=development for the dashboard, all I get is
7 Jun 13:36:11 - [info] [ui-base:Dashboard] Disconnected rRwjJ9_VueSmXX22AAAS due to transport close

Would a better course of action be to force a page reload only in case of a disconnect ws message combined with authentication required.

The check on the error code, and potential page reload, would only happen if we had previously been connected, so this is a reconnect, not an initial connect, so that should stop any possibility of repeated load on startup.  I don't know whether it is possible for the server to know that authentication is required.  I guess that would be something in the socket connection code. If we could have an indication that the attempt to setup the socket has been prevented because of the redirection to the logon page that would be conclusive.

A happy scenario would be to have access to this information in the response on the _setup call.

The _setup call is not being invoked here, this is purely an attempt to reconnect to the socket.  The _setup call was before the disconnection.
In the meantime I will submit a PR for the the class 1 failures described in the description.  That is, how to reconnect after a disconnection where re-authorisation is not involved.

I have managed to capture a 'parser error' in the console and from the network activity in the browser tools I can see what is happening.  It is in fact very simple.
After the authorisation token has expired the dashboard carries on as normal, using the existing socket connection, until the connection is broken for some reason.  Then when the reconnect is attempted I can see that the socket.connect() call in function reconnect() does a GET from https://nodered.mydomain.org.uk/dashboard/socket.io/?EIO=4&transport=polling&t=O_ufrkf.
As the token has expired, Cloudflare responds with a 302 Found status, redirecting to https://mydomain.cloudflareaccess.com/cdn-cgi/access/login/nodered.mydomain.org.uk?kid=9df892976242106878f0a65fd2c4543695b45472395325aba3f8277a33c5e44c&redirect_url=%2Fdashboard%2Fsocket.io%2F%3FEIO%3D4%26transport%3Dpolling%26t%3DO_ufrkf&meta=eyJraWQiOiIyZmE3OTM1NWY2YzdlYjRmNTdhY....
Which is the cloudflare login web page, which is not what socket.io is expecting, so it returns the parser error failure.

That is an awesome result!
If we can detect that 302 redirect we can safely force a page reload!

I think that would have to be down in the socket code somewhere.
I don't see the argument against forcing a reload when parser error is seen.  Even if it isn't caused by exactly this problem, what else can we do?

You have a good point, I can't think of anything else at the moment.

I will do a PR for that. It is working well for me here. Anyone with any other ideas can chip in there.

Thanks @colinl - I'm playing catch up on this since returning today from holiday, can you reference this issue in any PR you open please sot hat it gets linked.

I have done so from the first one #950. I am waiting for that to be merged before doing the second as they are in the same bit of code.

You can open still open the PR, just put a comment in the first bit of the PR that it's dependent upon the other one first :)

Completed by PR #962


Feature/minikube

facilitates minikube deployment for local development environment.

Has merge conflicts.
Please fix ....

done

Monster commit. Better check CI is ok


After a transaction, ctx.message.transactionId is set to an empty string on XMTP

Following up on issue https://github.com/framesjs/frames.js/issues/440#issuecomment-2206083864
@michalkvasnicak tried with both the frame.js example and my project and both gave me an empty string (tried again today with the latest versions and same problem). What can I do to debug the issue and send you better info?

@stephancill I think this one is true even for farcaster right? The transactionId is not available in the message to post_url.

@stephancill scratch what I wrote. It is available in the post_url (as in example 😅 ).

@michalkvasnicak the transactionId should be posted to post_url when the transaction is completed

@pradel I tried the example we have, just added openframes middleware to support the XMTP and send the transaction to my wallet and I can access transactionId.
Could you try that flow as well?

@pradel are you using polygon chain as in this comment? https://github.com/framesjs/frames.js/issues/440#issuecomment-2206083864

Could you try that flow as well?

which flow exactly?

are you using polygon chain as in this comment?

yes I am doing all my txs on polygon

@michalkvasnicak Also just to note that the transaction is valid and sent onchain just the ID is empty in post_url

i am not able to reproduce this issue...
here is the frame i am testing with
// frames/route.tsx
/* eslint-disable react/jsx-key */
import { Button } from "frames.js/next";
import { frames } from "./frames";

const handleRequest = frames(async (ctx) => {
  if (ctx.message?.transactionId) {
    return {
      image: (

Transaction submitted! {ctx.message.transactionId}

      ),
      buttons: [
        <Button
          action="link"
          target={`https://www.onceupon.gg/tx/${ctx.message.transactionId}`}
        >
          View on block explorer
        </Button>,
      ],
    };
  }

  return {
    image:
Send transaction to self
,
    buttons: [
      <Button action="tx" target="/txdata" post_url="/">
        Send transaction
      </Button>,
    ],
  };
});

export const GET = handleRequest;
export const POST = handleRequest;

// frames/txdata/route.ts
/* eslint-disable react/jsx-key */
import { Button } from "frames.js/next";
import { frames } from "./frames";

const handleRequest = frames(async (ctx) => {
  if (ctx.message?.transactionId) {
    return {
      image: (

Transaction submitted! {ctx.message.transactionId}

      ),
      buttons: [
        <Button
          action="link"
          target={`https://www.onceupon.gg/tx/${ctx.message.transactionId}`}
        >
          View on block explorer
        </Button>,
      ],
    };
  }

  return {
    image:
Send transaction to self
,
    buttons: [
      <Button action="tx" target="/txdata" post_url="/">
        Send transaction
      </Button>,
    ],
  };
});

export const GET = handleRequest;
export const POST = handleRequest;

@stephancill I just tried your code, and I am wondering if you are using XMTP to do the transaction?

connectedAddress  is not set for me and I have to use verifiedWalletAddress in XMTP mode

I just noticed smth strange, I added console.log("ctx.message", ctx.message); in the txdata route and when I click on "send transaction" in the first step, the txdata route is called and the transactionId is already an empty string at this point (so before anything is signed)

ctx.message {
  frameUrl: 'http://localhost:3000/',
  buttonIndex: 1,
  timestamp: Long { low: -1486563908, high: 400, unsigned: true },
  opaqueConversationIdentifier: 'fVtwt/haSoKpwVoNp4oE4K7HCSsrsE+c1Fxr4ny1FGE=',
  unixTimestamp:<PHONE_NUMBER>,
  inputText: '',
  state: '',
  address: 'xxx',
  transactionId: '',
  verifiedWalletAddress: 'xxx'
}

@pradel are you on the latest version of the debugger?

@stephancill I am using<EMAIL_ADDRESS>"0.3.4", and "frames.js": "0.17.4",

also ensure the xmtp peer dependency is up to date
"@xmtp/frames-validator": "^0.6.2" is what we have in frames.js

@stephancill added the dependency to my project and based on the lockfile I was already using it. Same thing is happening the transactionId is still an empty string.
for the project I am using:

I am using pnpm v9.4.0
I am using next v14.2.5

transactionId empty string is expected in the txdata endpoint. you will receive the transactionId in post_url once the transaction is completed

@stephancill okay didn't know it was expected I thought it would be undefined rather than an empty string, but I am also getting the empty string in post_url

can you share steps to reproduce? maybe create a repo?

@stephancill this is the payload sent to post_url after signing the tx
URL: http://localhost:3010/frames?postType=post&postUrl=http%3A%2F%2Flocalhost%3A3000%2Fframes%3F__bi%3D1-p&specification=openframes
Request payload:
{
    "mockData": {
        "enabled": true,
        "requesterFollowsCaster": false,
        "casterFollowsRequester": false,
        "likedCast": false,
        "recastedCast": false
    },
    "clientProtocol": "xmtp@2024-02-09",
    "untrustedData": {
        "buttonIndex": 1,
        "opaqueConversationIdentifier": "xx",
        "walletAddress": "xx",
        "url": "http://localhost:3000/",
        "timestamp":<PHONE_NUMBER>993,
        "unixTimestamp":<PHONE_NUMBER>993,
        "address": "xx"
    },
    "trustedData": {
        "messageBytes": "xx"
    }
}

Response payload
{
    "status": "success",
    "frame": {
        "accepts": [
            {
                "id": "xmtp",
                "version": "2024-02-09"
            }
        ],
        "version": "vNext",
        "image": "http://localhost:3000/frames?jsx=%5B%7B%22type%22%3A%22div%22%2C%22props%22%3A%7B%22children%22%3A%5B%22Send+transaction+to+self%22%5D%7D%7D%5D&__framesjs_jsx=true",
        "ogImage": "http://localhost:3000/frames?jsx=%5B%7B%22type%22%3A%22div%22%2C%22props%22%3A%7B%22children%22%3A%5B%22Send+transaction+to+self%22%5D%7D%7D%5D&__framesjs_jsx=true",
        "imageAspectRatio": "1.91:1",
        "postUrl": "http://localhost:3000/",
        "state": "{}",
        "buttons": [
            {
                "action": "tx",
                "label": "Send transaction",
                "target": "http://localhost:3000/frames/txdata?__bi=1-tx",
                "post_url": "http://localhost:3000/frames?__bi=1-p"
            }
        ]
    },
    "reports": {},
    "framesVersion": "0.17.4"
}

can you share steps to reproduce? maybe create a repo?

I can create the repo but I am also getting this error in the frame.js example as mentioned before.

can you try launching the debugger with npx @frames.js/debugger@latest?

@stephancill just tried with npx @frames.js/debugger@latest and I can now see the transactionId 🎉 . Could that be an issue with pnpm? 🤔

nice! how were you launching the debugger before?

I have it as a devDependency in my project so I was launching with pnpm pnpm frames

hm okay, will have to look into that. happy you're sorted!

Thanks for the help!


⚠ SceneTime has degraded performance

In f1854bc, SceneTime ($ST) experienced degraded performance:

HTTP code: 200
Response time: 1691 ms

Resolved: SceneTime performance has improved in 52535b1 after 13 minutes.


Fix incorrect property name in docs

In toaster.mdx the property name theme is incorrect, it should be iconTheme.

Thank you!


$auth->forgotPassword creates fatal error

Hello Forum,
Great work but the password reset is throwing an error:
"PHP Fatal error: Uncaught Delight\Db\Throwable\IntegrityConstraintViolationException: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '0' for key 'PRIMARY' in //*/delight-im/db/src/ErrorHandler.php:66,"
Everything else works excellently.
Here is my code:
try {
    $auth->forgotPassword($_POST['email'], function ($selector, $token) {
    });

    echo 'Request has been generated';
}
catch (\Delight\Auth\InvalidEmailException $e) {
    die('Invalid email address');
}
catch (\Delight\Auth\EmailNotVerifiedException $e) {
    die('Email not verified');
}
catch (\Delight\Auth\ResetDisabledException $e) {
    die('Password reset is disabled');
}
catch (\Delight\Auth\TooManyRequestsException $e) {
    die('Too many requests');
}

Any help greatly appreciated.

I fixed the problem. There was no AI set on the id column of the users_resets table. Maybe a legacy issue. I set AI on the id column  to fix.

Thanks for sharing the cause of this problem for you. AI being AUTO_INCREMENT, of course. Perhaps make sure your database schema is consistent with the one from this library in all the remaining parts, to avoid future bugs in other areas.


Search Latency Tracking - Coordinator Slow Logs

Is your feature request related to a problem? Please describe.
As of today, we track search request latencies on a shard level via node stats. After every query/fetch phase is completed on a shard, we note down the time taken for each, keep accumulating those values and maintain an overall average value which is tracked under stats.
But we don’t have a mechanism to track search latencies around coordinator node. Coordinator node plays an important role in fanning out requests to individual shard/data-nodes, aggregating those responses and eventually sending response back to the client. We have seen multiple issues in the past where it becomes hard/impossible to reason latency related issues because of lack of insights into coordinator level stats and we ended up spending a lot of unnecessary time/bandwidth on figuring it out. Clients using search API only rely on overall took time(present as part of search response) which doesn’t offer much insights into time taken by different phases.
Parent RFC: https://github.com/opensearch-project/OpenSearch/issues/7334
Describe the solution you'd like
Slow logs at coordinator level: As of now, we only have the capability to enable slow logs at a shard level for desired search phase(query and fetch). See this. Setting this threshold is tricky when customer usually sees latency spikes at a request level. Plus shard level slow logs doesn't offer a holistic view. So as part of this, we will also add capabilities to capture slow logs at a request level along with different search phases from coordinator node perspective.
Additional context
Coordinator slow logs will be governed by cluster settings. We will offer for the following 3 intervals:

Overall request
Query phase
Fetch phase

// Setting on a whole request level
cluster.search.request.slowlog.threshold.warn: 10s
cluster.search.request.slowlog.threshold.info: 5s
cluster.search.request.slowlog.threshold.debug: 2s
cluster.search.request.slowlog.threshold.trace: 500ms

// Minimum level to print
cluster.search.request.slowlog.threshold.level: "trace"

Looking forward to seeing this! @dzane17 is this still slated for 2.11? Can you link a PR for this, if it is or remove the label if it isn't, please?

Thanks @macohen. I have opened the PR and we are now on track for 2.12


Which Nimbux card to choose?

In my nimbix account, I had to pick a card.  I used "PowerAI Notebooks" and then clicked on the TensorFlow button. Still busy downloading images so not sure if it is working, but would like instructions to say what to pick

I'm confused.  I tried again and didn't see the options.  Maybe there is no issue here.

trial process has changed. Closing this.


(Feature) Support CodeBuild Environment variables

Although undocumented, environment_variable can take a list of
environment variables - https://github.com/terraform-providers/terraform-provider-aws/issues/5563

Thanks for your contribution! I'm actually thinking about completely externalizing the CodeBuild configuration as a dependent resource. The user would be responsible for providing the ARN/ID of the aws_codebuild_project resource as an input. The possibility to provide a custom CodeBuild project is already implemented and I think switching to this approach would greatly improve flexibility and decrease the complexity of the interface of this project.
Any thoughts?

Currently it has a good default aws_codebuild_project, which builds what the module describes. I think removing it would add complexity for the user of the module.
Having worked on a few AWS terraform modules, the interface for the module can get complex as new features are added for resources. But I think providing good defaults to any option, to ensure minimal code maintenance for the users of the modules, is what they should be about.

The problem is that from previous experience I expect it to grow quite significantly effectively making everything inside that CodeBuild project configurable which completely eradicates the benefit of keeping a simple default project.

The CodeBuild project in this case is pretty standard, where only edge case uses will need to change the default CodeBuild configuration (eg. only needing to change one particular part of the CodeBuild project to suit the users need).
Although it does add more technical debt when it comes to maintaining the module for new CodeBuild features, I do feel it would be a detriment to the module if it was removed.

I need to think about this a little. Currently I'm considering merging this PR but I need some time to evaluate further potential future PRs and if it's worth. Including the default CodeBuild project into the documentation would probably also solve most of your concerns. I'll keep this open and let you know when I decided how we proceed. Anyway, thanks for your thoughts on this and your PR!

Thanks Np 👍


feat: automatically include env variables starting with NUXT_ENV_

If you don't want to add all env variables to your nuxt.config.js you now can prefix them with NUXT_ENV_ to get them automatically included in the process.env object.
Example:

Build and start your app. Include the variable in the build step:
NUXT_ENV_TEST=awesome nuxt build && nuxt start
Call it in your app process.env.NUXT_ENV_TEST //awesome

Types of changes

[x] New feature (non-breaking change which adds functionality)

Checklist:

[x] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly. (PR: #)
[x] I have added tests to cover my changes (if not applicable, please state why)
[x] All new and existing tests passed.

@pi0 That's why I haven't ticked the box yet :P
Will submit a PR to the docs and link it here!

Codecov Report

Merging #3862 into dev will increase coverage by <.01%.
The diff coverage is 100%.

@@           Coverage Diff            @@
##             dev   #3862      +/-   ##
========================================
+ Coverage   97.8%   97.8%   +<.01%
========================================
  Files         18      18
  Lines       1183    1185       +2
  Branches     325     325
========================================
+ Hits        1157    1159       +2
  Misses        25      25
  Partials       1       1

Impacted Files
Coverage Δ

lib/builder/webpack/base.js
98.36% <100%> (ø)
:arrow_up:

lib/common/options.js
100% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update baaf67d...b98466b. Read the comment docs.

Love this PR :)


Something is wrong with xz 5.0.5-1 from conda-forge

I am trying to release a new version of bob.io.image and its tests fail each time each time xz 5.0.5-1 is used: https://travis-ci.org/conda-forge/bob.io.image-feedstock/builds/135645811
I have reported this issue here too: https://github.com/bioidiap/bob.io.image/issues/13 but I think this problem is coming from xz.
Basically the tests pass when jpeg and python and xz and libtiff are pulled from the defaults channel.

@183amir I believe you already figure this out, right? Can we close this?

The problem is not fixed. I just ignore that the build fails in one of the conditions: https://travis-ci.org/conda-forge/bob.io.image-feedstock/builds/138908694 This happens only on Mac.
But, it might not be a problem with xz after all.


lmp: 1.1 -> 2.0

Motivation for this change
https://github.com/0xInfection/LogMePwn/releases/tag/v2.0
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
22.05 Release Notes (or backporting 21.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 152369 run on x86_64-linux 1

1 package built:

lmp

@ofborg build lmp


What's the use case of the Project?

Could you provide more insight into its intended use case and specific scenarios where it would be most beneficial? I’m particularly curious about where it shines the most in real-world applications.

Thank you for raising this, and it's a very relevant question! The Real-Time Log Processing API is designed to provide a scalable solution for handling and processing logs in real time, particularly in environments where system reliability and security are key priorities. Here’s a deeper dive into the intended use cases and where this project shines in real-world applications:
Primary Use Case:
Centralized Log Management: In large, distributed systems, logs are generated from multiple services running across different environments. This project allows for the collection and real-time processing of these logs in one place. Instead of waiting to analyze logs after the fact, it filters out sensitive information (like personal data) on the fly, reducing security risks and enhancing compliance with data privacy regulations like GDPR.
Real-World Scenarios:
Security Monitoring: The API can be integrated into a security operations center (SOC) to monitor application logs for patterns of suspicious activity. For example, the system could flag potential threats like brute-force attacks or unauthorized access attempts in real-time.
Fraud Detection Systems: Financial services or e-commerce platforms could use this project to monitor transactions and flag logs for potential fraudulent behavior. Any logs containing sensitive patterns (e.g., credit card info) are sanitized, ensuring security and privacy.
Compliance & Data Privacy: With regulations like GDPR and HIPAA, organizations are required to protect sensitive data. This API helps remove personally identifiable information (PII) from logs automatically before they are stored, ensuring compliance without manual intervention.
Performance Optimization & Scalability: By offloading log processing to a system like Celery with Redis, the API ensures that the main application remains performant, even under heavy loads. This is particularly beneficial for applications dealing with large volumes of user activity, such as high-traffic websites or mobile apps.
Incident Response and Debugging: DevOps teams can integrate this into their CI/CD pipeline to catch issues as they happen. The API provides real-time insight into what’s happening in production, enabling faster identification and resolution of issues.
Why It's Beneficial:
Real-Time Processing: Many systems batch process logs at intervals, which can delay insights. This project processes logs as they come in, giving you the ability to act on issues immediately.
Scalable Architecture: By using Celery and Redis, the system can handle spikes in log volume and is scalable across distributed environments.
Flexible: It’s designed to be integrated into larger ecosystems (e.g., microservices architectures) and doesn’t depend on any specific front end, making it suitable for both cloud and on-premise environments.
I hope this provides a clearer picture of the scope and potential applications for the project. It’s a foundational tool that can be customized and expanded based on the needs of different organizations and systems.
Feel free to reach out if you have any other questions or thoughts! 😊


spirv-val gives "block ... branches to the selection construct, but not to the selection header" error on seemingly valid SPIR-V binary.

Affected version of spirv-val:
spirv-val --version
SPIRV-Tools v2018.5-dev a3e3869
This archive has two SPIR-V binaries, ok.spv and nok.spv, with associated disassembled .asm files.
The binaries have pretty similar structure and describe semantically equivalent shaders, except that nok.spv uses "OpReturn" where ok.spv uses "OpBranch" in one place.
spirv-val says that ok.spv is valid (expected), but that nok.spv is invalid:

spirv-val nok.spv
error: line 22: block  15 branches to the selection construct, but not to the selection header  25
%15 = OpLabel

I believe both .spv files should be valid, and in particular I see no reason why nok.spv should be invalid if ok.spv is valid.
Issue found using GraphicsFuzz.

I do agree there's a bug here; however, I do think there is a problem with nok.spv. In nok.spv, %25 is unreachable. This means the selection construct for %25 is the set {%25}. That makes the branch to %29 an illegal exit from the selection. If the branch in %25 targeted %18 instead of %29 I think it would be ok.
When I make the change to remove %29 from the module, I still get that error so I'll fix it based on that modified case.

Thanks for clarifying this!  I need to study the rules for unreachable code in SPIR-V.  Our GraphicsFuzz tool is operating at the GLSL level, and sometimes it ends up producing unreachable GLSL snippets, as happened in this case, which glslangValidator then translates into SPIR-V.  I guess the issue of whether glslangValidator should take responsibility for generating legal SPIR-V in the case of unreachable code is a (low priority) grey area.


[509] Add visionOS to the list of platforms that can't start a new process from swift-parser-cli

Explanation: Process is not available on visionOS, causing a build failure of swift-syntax 509.x.x for that platform when building the swift-parser-cli command line test utility. We should use the same #if check that we have for iOS, tvOS and watchOS to also guard the process spawning code on visionOS.
Scope: Building the swift-parser-cli target on visionOS. Building commonly used products like SwiftSyntax and SwiftParser is fine
Risk: Very low, treat visionOS the same as iOS
Testing: Tested that the swift-syntax package now builds for visionOS
Issue: n/a
Reviewer:  @bnbarham

✅ Local testing succeeded


[Sprint: 49] XD-3092: Synchronous deployment/undeployments

REST API controller now blocks until deployment/undeployment requests have been fulfilled by the admin supervisor.

LGTM, rebased and merged as 9f2294ffee048bba87d76eb915ca77051e69a33e. Also, pushed the changes ea0f15e15b4ff641a10bb8f95aba0de6ce96617d that include:

Make sure to delete the children path when deleting the response node
Change the log level to debug at DeploymentMessageConsumer
Fix StreamPartitionTests to include redis and rabbit rule


Update SocialList.astro

Rearranged list and added LinkedIn

👷 Deploy Preview for famous-marshmallow-cfbd61 processing.

Name
Link

🔨 Latest commit
8f981006b867759ac51d8bdfc7374ee81a93a9d8

🔍 Latest deploy log
https://app.netlify.com/sites/famous-marshmallow-cfbd61/deploys/64a45e26f084a70008c6b959


Allow Kubernetes CRD Token to be specified as a file

Welcome!

[X] Yes, I've searched similar issues on GitHub and didn't find any.
[X] Yes, I've searched similar issues on the Traefik community forum and didn't find any.

What did you expect to see?
I am working some different scenarios for using the external Kubernetes provider. I'd like to be able to specify the token as a file so I can mount it in non-kubernetes environments. For example if I want to specify the token from a secret.

My current work around is to use a docker config mount a different entry point to grab the secret and append to the commands.

Hello @Enquier,
Thanks for the suggestion, we think it makes a lot of sense.
If you or another community member would like to build it, let us know and we will work with you to make sure you have all the information needed so that it can be merged.
Don’t forget to check out the contributor docs and link the PR to this issue.

@nmengin I would like to work for this issue


Contributing update

Fix: #139
-> Coding style guidelines & review process inferred from the repo and might not be well interpreted, all the referenced practices are open to discussion!

Alright, thanks for the review.
As I am not very familiar with pre-commit (especially the isort task), help would indeed be needed to describe it.

@SaboniAmine I've pushed a draft about code-carbon and a bit about testing and packaging, let me know what you think about it. It's a draft so we likely need to update the content form.

Looks fine to me, I guess the expected logs will be helpful for troubleshooting.
Thanks for the packaging description item, which raises questions to me (which can be treated in a separate issue) :

as code-carbon is tested against py38, should we reference a specific conda meta file ?
should we freeze dependencies versions ? Is there a setup where versions could be incoherent with the current components of the package ?

* as code-carbon is tested against py38, should we reference a specific conda meta file ?

We don't need a specific conda file for Python 3.8. We do have a specific Python 3.6 conda file because we need a backports of dataclasses for Python 3.6. The py37-plus conda file is working for any Python version greater or equals to Python 3.7 (See https://github.com/mlco2/codecarbon/blob/master/.conda/py37-plus/meta.yaml#L21 vs https://github.com/mlco2/codecarbon/blob/master/.conda/py36/meta.yaml#L21)

* should we freeze dependencies versions ? Is there a setup where versions could be incoherent with the current components of the package ?

Codecarbon is a library, not a product so we should be as flexible as possible when it comes to dependencies. We don't want to have users raising issues like codecarbon is requiring requests==2.25.0 but I have an environment where I'm stuck with version requests==2.24.0 because Tensorflow requires it for example. This being said, we can probably put some lower bounds in our dependencies as we probably won't work with requests==0.0.1 for example, but that requires checking the minimum viable version for each of our dependencies and manually test it.
Also when we extract the API / visualizer, we can probably be more precise in our requirements as the API / visualizer don't need to be in the same Python environment as other libraries so the risk of conflicts is much less likely to happen.

Thanks for the explanations, it would indeed be counterproductive to be less flexible on the client package.


New feature: block accounts after not logging in for x-period

In some cases it is useful that accounts are blocked after an x period not logging in.
For example when people are switching jobs and user management is not performed accurately.
Is it possible to add this to this plugin?

Waiting for fix on matomo-org/matomo-dev-environment#16

Waiting for fix on matomo-org/matomo-dev-environment#18


cmake: use export namespace for Drogon::Drogon

Same idea as in #254, but it actually works this time 😃. This PR also cleans up the inconsistent use of drogon and ${PROJECT_NAME}.

@mensinda Thank you very much, but I hope that you can achieve this without changing the project name (this will change the library from libdrogon.a to libDrogon.a, which will cause some applications that use the old version of CMakeLists.txt to fail) .
I think you can change some lines of DrogonConfig.cmake.in as follows:
if(NOT TARGET Drogon::Drogon)
  include("${DROGON_CMAKE_DIR}/DrogonTargets.cmake")
  set_target_properties(Drogon::drogon PROPERTIES IMPORTED_GLOBAL TRUE)
  add_library(Drogon::Drogon ALIAS Drogon::drogon)
endif()

Please check if this can work with meson, thanks.

Done. This also works, thanks.


lint: Fix template checking (code, a)

This won't build, as I think Tim has some HTML cleanups on a branch that need to hit master.

Automated message from Dropbox CLA bot
@showell, it looks like you've already signed the Dropbox CLA. Thanks!

The lint failures are actually just that I had missed a couple bad  tags in my audit.  I merged the fixes for those detected by this branch; it looks like you have some mypy issues as well so not merging this for now.

Merged (BTW that syntax for "fixes" doesn't work, and we generally never include it in the first line of the commit message; I fixed that up for you).


Refs: #9194 fix error nullptr check

Signed-off-by: jparisu<EMAIL_ADDRESS>Creation of object is checked as (obj != nullptr) //Error when the correct form is (obj == nullptr) //Error

Build status:


🐛 Identifiers cannot begin with the letter e.

I think this has something to do with the parser trying to parse this as a scientific notation number 1e2 or something.

OK after a bit more digging it seems like its function application that makes this break:
let a = e

is ok, but:
let a = f e

breaks.


User story: Multiple images/Image carousel

As a user,
I want to see multiple pictures of a product,
so that I can better decide if I want to buy a product.

Has just one image, due to time restriction.


The eol_at_end_of_file lint reads file content from disk and can slow down linting

Splitting this out of https://github.com/dart-lang/sdk/issues/55281#issuecomment-2519977056 so it doesn't get lost.
This lint reads the content in a way that appears that it can hit the disk and can be quite slow when analyzing a lot of files (or a large library):
https://github.com/dart-lang/sdk/blob/33bdacc604f09199d5b89685ea4c0de8a91833e7/pkg/linter/lib/src/rules/eol_at_end_of_file.dart#L38
One simple option suggested by @bwilkerson was:
    var unitContext =
        context.allUnits.firstWhereOrNull((unit) => unit.unit == node);
    var content = unitContext.content;

And @scheglov:

And yes, I think registerNodeProcessors() should have access to the current LintRuleUnitContext instance. Currently it knows all units of the library, and its defining unit, but not the current.

I had a quick look at this, but I am a little confused by LinterContext.definedUnit because it seems like there is code that passes each individual unit in for it. For example this call here happens inside a loop for all units within a library (called currentUnit):
https://github.com/dart-lang/sdk/blob/0d72db1b8866c647602a02064c8ce948b7744cc8/pkg/analysis_server/lib/src/services/correction/bulk_fix_processor.dart#L293
And then currentUnit is passed as definingUnit:
https://github.com/dart-lang/sdk/blob/0d72db1b8866c647602a02064c8ce948b7744cc8/pkg/analysis_server/lib/src/services/correction/bulk_fix_processor.dart#L557
So I wonder whether definingUnit could just be rename and repurposed as currentUnit or whether any lints really do need/want the defining unit?

I don't know whether any of the existing lints need the defining compilation unit. If there are any, then at least some of them are likely using it to get to the imports and that logic will need to be changed to support enhanced-parts. The only way for me to find the answer to that question would be to look at all uses of definingUnit and try to determine why it's being used.


Invalid issue test

Hey, I think swc has a bug

same issue

Closing as I'm going to test with a new issue