feat: add admission control configuration variable

Fixes the https://github.com/hcloud-k8s/terraform-hcloud-kubernetes/issues/20

Thanks for the contribution, @mgerasimchuk!
Could you consider making some changes? Disabling the admission controller entirely just to suppress restricted profile warnings (warn: restricted) is very risky as this would implicitly also disable the enforcement of the baseline security policy enforcement (enforce: baseline) and restricted audit log policy (audit: restricted).
If you only want to get rid of the warnings, it would be better to add a new variable called kube_api_admission_control (the admission controller is part of kube-apiserver binary) with default value [] that can overwrite cluster.apiServer.admissionControl default values here: https://github.com/hcloud-k8s/terraform-hcloud-kubernetes/blob/43cbb080a39da5d217f29e40797402276113a25a/talos_config.tf#L221-L227
Example talos_config.tf snippet:
apiServer = {
  admissionControl = var.kube_api_admission_control # the new variable
  certSANs = local.certificate_san
  extraArgs = merge(
    { "enable-aggregator-routing" = true },
    var.kube_api_extra_args
  )
}

In a kubernetes.tf it could be used like this for a more relaxed warning profile:
kube_api_admission_control = [
  {
    name = "PodSecurity"
    configuration = {
      defaults = {
        warn = "baseline" # or "privileged"
      }
    }
  }
]

See here for more infos:

https://www.talos.dev/v1.8/reference/configuration/v1alpha1/config/#Config.cluster.apiServer.admissionControl.
https://kubernetes.io/docs/concepts/security/pod-security-standards/

Hi @M4t7e ,
I made the changes in the way you suggested in this commit, and tested it:
1.1 Option with kube_api_admission_control. The use-case of changing existence PodSecurity configuration
main.tf:
module "kubernetes" {
  ...
  kube_api_admission_control = [
    {
      name = "PodSecurity"
      configuration = {
        defaults = {
          warn = "baseline"
        }
      }
    }
  ]
  ...
}

» ~  talosctl get machineconfig v1alpha1 -o yaml | yq '.spec.cluster.apiServer.admissionControl[].configuration.defaults'
audit: restricted
audit-version: latest
enforce: baseline
enforce-version: latest
warn: baseline
warn-version: latest

1.2 Option with kube_api_admission_control.  The use-case of deleting admissionControl
In this case I faced the problem with deleting the admissionControl..., unfortunatelly, I can't override the admissionControl with empty [] list..
That's why I've added one more commit, and tried to do it the same way as do it Talos module by providing config_patches fields.
Tested it too, all works as expected.
2.1 Option with ..._config_patches. The use-case of changing existence PodSecurity configuration
main.tf:
module "kubernetes" {
  ...
  control_plane_config_patches = [
    {
      name = "PodSecurity"
      configuration = {
        defaults = {
          warn = "baseline"
        }
      }
    }
  ]
  ...
}

» ~  talosctl get machineconfig v1alpha1 -o yaml | yq '.spec.cluster.apiServer.admissionControl[].configuration.defaults'
audit: restricted
audit-version: latest
enforce: baseline
enforce-version: latest
warn: baseline
warn-version: latest

2.2 Option with ..._config_patches. The use-case of deleting admissionControl
main.tf:
module "kubernetes" {
  ...
  control_plane_config_patches = [
    {
      op    = "remove"
      path  = "/cluster/apiServer/admissionControl"
    }
  ]
  ...
}

» ~  talosctl get machineconfig v1alpha1 -o yaml | yq '.spec.cluster.apiServer.admissionControl'
null

So, ..._config_patches can cover a lot of needs, but I have a not very good feeling that these variables represent the not very beautiful and comfortable configuration interface cos the typeless
*but it was not stop the siderolabs/talos cos guys have the same thing in talos_machine_configuration_controlplane_* :)
@M4t7e what do you think?

Thank you very much! That's really great work.
Talos merges list entries instead of overwriting the entire list. That's the reason I proposed to set [] as default for kube_api_admission_control variable as it would not change anything from Talos defaults. If you want to delete the entire list, Talos supports delete syntax in patches. Your 1.2 Option should work with this config:
module "kubernetes" {
  ...
  kube_api_admission_control = { "$patch" = "delete" }
  ...
}

Unfortunately this would require to change the var type to any as it is not a list anymore and I prefer to avoid that if we can.
In general I really like your second approach and I think we should add that general patch functionality regardless. I recently added a similar feature to all Helm based components to pass custom values that overwrite ("patch") the defaults from this module. It is not very comfortable, but provides all options for more advanced users or for quick debugging.
I'd propose let's do both. Adding your initial kube_api_admission_control from option 1 to just configure the admission control. It is also a dedicated Talos config item (see talosctl get admissioncontrolconfig -o yaml) and probably worth adding it. And for those who really want to get rid of it instead of re-configuring, option 2 provides all possibilities to do so and also covers all other use cases. Then it would be good to also provide that functionality to Autoscaler nodes. The config will be basically the same.
I think this could be a good compromise that fits to all needs. What's your opinion @mgerasimchuk?

kube_api_admission_control = { "$patch" = "delete" }

Oh, cool! I didn't know that was possible. I spent some time trying to find a similar solution, but all my attempts were based on the list)) I have tested it and confirmed that it works.
Regarding your other thoughts - yeah, totally agree, I've reverted the admission setting deletion and applied new changes for the autoscaler, here it is.

*test session info
» talosctl get machineconfig v1alpha1 -o json | jq -c '.spec.cluster.apiServer.admissionControl[].configuration.defaults'
{"audit":"restricted","audit-version":"latest","enforce":"baseline","enforce-version":"latest","warn":"restricted","warn-version":"latest"}
»
» cat main.tf | grep -m 1 -A 5 kube_api_admission_control
  kube_api_admission_control = [
    {
      name = "PodSecurity"
      configuration = { defaults = { warn = "baseline" } }
    }
  ]
»
» terraform apply -auto-approve > /dev/null                                                                                »
» talosctl get machineconfig v1alpha1 -o json | jq -c '.spec.cluster.apiServer.admissionControl[].configuration.defaults'
{"audit":"restricted","audit-version":"latest","enforce":"baseline","enforce-version":"latest","warn":"baseline","warn-version":"latest"}

» cat main.tf | grep -m 1 -A 6 control_plane_config_patches
  control_plane_config_patches = [
    {
      op    = "replace"
      path  = "/cluster/apiServer/admissionControl/0/configuration/defaults/warn"
      value = "restricted"
    },
  ]
»
» terraform apply -auto-approve > /dev/null
» talosctl get machineconfig v1alpha1 -o json | jq -c '.spec.cluster.apiServer.admissionControl[].configuration.defaults'
{"audit":"restricted","audit-version":"latest","enforce":"baseline","enforce-version":"latest","warn":"restricted","warn-version":"latest"}

» cat main.tf | grep -m 1 -A 5 control_plane_config_patches
  control_plane_config_patches = [
    {
      op    = "remove"
      path  = "/cluster/apiServer/admissionControl"
    }
  ]
»
» terraform apply -auto-approve > /dev/null
» talosctl get machineconfig v1alpha1 -o json | jq -c '.spec.cluster.apiServer.admissionControl'
null

» talosctl get machineconfig v1alpha1 -o yaml | yq '.spec.cluster.inlineManifests[-1].contents' | yq '. | select(.kind == "Deployment") | .spec.template.spec.containers[].env[] | select(.name == "HCLOUD_CLUSTER_CONFIG") | .value' | base64 -D | jq -r '.nodeConfigs[].cloudInit' | yq .machine.nodeAnnotations
null
»
» cat main.tf | grep -m 1 -A 6 cluster_autoscaler_config_patches
  cluster_autoscaler_config_patches = [
    {
      op    = "add"
      path  = "/machine/nodeAnnotations"
      value = {"test-annotation": "test-annotation-value"}
    }
  ]
»
» terraform apply -auto-approve > /dev/null
» talosctl get machineconfig v1alpha1 -o yaml | yq '.spec.cluster.inlineManifests[-1].contents' | yq '. | select(.kind == "Deployment") | .spec.template.spec.containers[].env[] | select(.name == "HCLOUD_CLUSTER_CONFIG") | .value' | base64 -D | jq -r '.nodeConfigs[].cloudInit' | yq .machine.nodeAnnotations
test-annotation: test-annotation-value

Thx @mgerasimchuk, that looks beautiful!
It is released in 0.7.0


fix previous commit

remove spurious '/', and use '--' in the lxc exec rm commands
to keep 'lxc' itself from interpreting the -f we are passing
to rm.
Signed-off-by: Serge Hallyn<EMAIL_ADDRESS>

I'm rejecting this one and will submit a new one fixing this for good.


WIP: add OCaml Dockerfile

This is a draft for #4 , we are reviewing the packages to install.
packages

alcotest
angstrom
base
batteries
cohttp
core
core_kernel
dune
fix
fmt
js_of_ocaml
lwt
menhir
ounit2
owl
ppx_deriving
re
react
reason
sexplib
utop
yojson

Looks good! Mandatory package suggestions: ocamlbuild, cppo. These are very popular build tools.
Also, the twitter account says "Time to sharpen your numerals!", so let's throw in zarith, just in case. :-)
Other suggestions: atd, atdgen, ppx_deriving_yojson, ppx_sexp_conv.

Very good to know, first timer here using OCaml so I've just thrown together the popular packages from opam

Ok, now we got to debug the dune build because this is not fun anymore
full log
interesting bits
#=== ERROR while compiling dune.2.6.1 =========================================#
# context     2.0.7 | linux/x86_64 | ocaml-base-compiler.4.10.0 | git+file:///home/opam/opam-repository
# path        ~/.opam/4.10/.opam-switch/build/dune.2.6.1
# command     ~/.opam/4.10/bin/ocaml bootstrap.ml -j 71
# exit-code   2
# env-file    ~/.opam/log/dune-69-93d75f.env
# output-file ~/.opam/log/dune-69-93d75f.out
### output ###
# ocamlc -output-complete-exe -w -24 -g -o .duneboot.exe -I boot unix.cma boot/libs.ml boot/duneboot.ml
# ./.duneboot.exe -j 71
# cd _boot && /home/opam/.opam/4.10/bin/ocamlopt.opt -c -g -no-alias-deps -w -49 -I +threads main__upgrade.mli
# File "bin/upgrade.mli", line 1, characters 19-34:
# Error: Unbound module Cmdliner
#

I see dune vendors some packages but I guess the command options are not what it would expect to find them?

Can't confirm this. For me, it works.

A build took 25 minutes on my laptop.
Packages to drop: bnfgen (something obscure), utop (interactive toplevel).

Rudi Grinberg (@rgrinberg), the omnipresent demon of Dune, we call thee, the Lord and Leader of build systems and heavenly docker containers (I hope so). For the glory of ICFP Programming Contest 2020, for OCaml (the weapon of humble, but smart), and for Bashkortostan (the superior geopolitical entity), could you please take a look at the compilation error above? Does it look familiar? It happens on opam install dune inside a fresh container derived from ocaml/opam2:4.10.

It suddenly struck me: OCaml is great at producing self-contained binaries, so why spend the effort at reproducible builds, if we can simply push binaries?
@beevee, would that be against rules?

@murmour this is not ideal for the performance of our submission system, but it is not against the rules. Just try not to overwhelm us by sending gigabyte-sized submissions every minute!

Just try not to overwhelm us by sending gigabyte-sized submissions every minute!

Nah. Typical OCaml binaries are quite small -- up to several megabytes with the runtime baked in, except if some crazy library is used (which shouldn't be the case in this contest). Building those binaries, on the other hand, can be complicated and slow. Also, git is great at compressing binaries.
Still, a specialized docker image would be of use -- for OCaml novices, and for poor Windows souls.

I'm not able to reproduce the issue. Installing dune individually in that image works. Executing the entire build from the docker file also works for me.

Is this still a work-in-progress? Is somebody going to finish it before the contest? Do you need any help?

@rgrinberg, receive my holy blessing for spending your time! Thanks! :^)
@edoput told me that he and his friend use Podman instead of Docker, so the culprit might be in that. Workaround: use Docker.
@beevee, the image seems good. Could you try building it? This might take some time and requires a few GBs of RAM.
I will prepare a starterkit tomorrow (unless something happens to me).

OK, here you go: https://github.com/icfpcontest2020/starterkit-ocaml
I'll merge this PR to test in real conditions.

@beevee, FROM icfpcontest2020/ocaml doesn't work yet. Did it build?

Sadly it didn't:
#=== ERROR while installing cryptokit.1.14 ====================================#
opam: "fork" failed: Cannot allocate memory

This base image won't build with the default Docker Hub limits (2 GB RAM).
This base image won't build on my local machine (4 GB RAM dedicated to Docker).
How much memory does it take?

Around 8GB.


Change robot ROB-E to ROBE

Robot names should be standardized - this should just mean making sure they match what is on the machine.

'ROB-E', 'RIK-E', and 'JER-E' changed to 'ROBE', 'RIKE', and 'JERE', respectively.


Move operator tests to logic tests

A datadriven logic testing framework was recently added in https://github.com/polarsignals/frostdb/pull/211. Slowly but surely, it would be nice to move our operator tests to this logic testing framework to promote readability and conciseness. #255 does this for the distinct operator. This issue can be closed once the following tests have been moved over:

[ ] aggregate_test.go
[ ] filter_test.go

@asubiotto Can you assign this to me?

Sure!

@asubiotto Thanks!!!

hi @bitethecode are you still working on this? if not then i would like to pick this up if that's ok with you

Hi @palash25, are you still interested in working on this? I can reassign to you given that we haven't heard back.

No problem! Thanks for giving it a shot. @palash25 assigned to you. Feel free to use https://github.com/polarsignals/frostdb/pull/225 as a guide.

thanks for assigning i will look into it this weekend

as i have already submitted 2 PRs, i would like to be unassigned so that someone else can try to convert the remaining go tests

I think there are still some tests to be moved (notably the projection tests in aggregate_test.go). I believe #302 will likely address this, but reopening to track.


[Question]-Apollo Auto can send and Receive data(like lidar and Radar) using WebSocket to Application via listen port- TCP/UPD & IP address?

The Apollo Auto simulator Can sends like car telemetry information the data specifications to the application using WebSocketi.e WebSocket(uWebSockets) Data-describes the JSON object send back from Application to  the simulator
For sample example i.e understanding:
https://github.com/shazraz/Extended-Kalman-Filter.
Here bins tested with Udacity SDC Term 2 Simulator (Extended-Kalman-Filter connected/communicat Udacity on Port TCP and data receive Json msg)
Similar Apollo Auto Can be used send/receive other from other Application
I think Apollo-Auto dream view connected via WebSocket. so, Apollo auto will be provided WebSocket interfaces to send/receive data to Application/Hardware-TCP/UDP port and Ip Address. ?

Yes, dreamview already use websocket to send data, you can also ref the modules/bridge module to send data to other software

@daohu527 could you please let know how to test module/bridge.?
root@in_dev_docker:/apollo/bazel-bin/modules/bridge# ./udp_bridge_sender_cpplint
Traceback (most recent call last):
File "./udp_bridge_sender_cpplint", line 178, in
Main()
File "./udp_bridge_sender_cpplint", line 145, in Main
'Cannot exec() %r: file not found.' % main_filename
AssertionError: Cannot exec() '/root/.cache/bazel/_bazel_root/540135163923dd7d5820f3ee4b306b32/execroot/apollo/bazel-out/local-dbg/bin/modules/bridge/udp_bridge_sender_cpplint.runfiles/apollo/../google_styleguide/cpplint/cpplint.py': file not found.
root@in_dev_docker:/apollo/bazel-bin/modules/bridge/test# ./bridge_sender_test
./bridge_sender_test: error while loading shared libraries: libgflags.so.2.2: cannot open shared object file: No such file or directory
root@in_dev_docker:/apollo/bazel-bin/modules/bridge/test#


Pushing in 2 terminals at the same time results in HTTP error

I tried pushing 2 versions of my game at the same time, each in their own terminal
http://i.imgur.com/u3qkGlD.png
The left terminal started first and resulted in HTTP 404
The right terminal started after that resulting in a HTTP 505
Whenever I retried uploading in that same terminal I got the same error code again.
When I opened a new terminal after that, my push was succesfull.
I used Gitbash in Windows as terminal which is based on MINGW64

Maybe it was a coinsidence. It seemed fixed at first when i tried one at the time, but i'm not certain anymore where the cause is.

this is unrelated, please see #71 - updating to butler 0.16.0 mitigates the issue, but until google fully resolves it, it's out of our hands (short of deploying a US proxy, which I'm planning to do)


Onboard Search to pipelines

Search still isn't building via unified pipelines. This issue is just tracking that work once the Search team merges their work.

Done. It was a rocky road. Some of the lessons learned were that trying to merge from a private fork, to the private repo caused problems with pipelines. So how we handle pipelines in private repo work is going to have to be something we take a look at in the future. But for now search is onboarded.


fix/nix flake mac

Hello, the nix flake was not working on my mac. After these fixes, it works.
I noticed the flake.nix was not formatted. I let my autoformatter do it. I am using the official nix formatter https://github.com/NixOS/nixfmt. I would be happy to alter the nix file to bundle the preferred formatter if you have one. This way, people will not be confused about how to format.
I am not a nix expert, but I do recommend trying out denvenv.nix (https://devenv.sh) instead of using a flake.nix. It works with direnv, it has better caching (basically a flake.nix needs to take in the whole repo, binaries and all to set up.. blegh), it is easier to set up and understand. Respectable if you want to keep the status quo, but devenv is awesome! :)

fix(flake): refactor for mac support
chore(flake.lock): update

I see. I can try and minimize the formatting changes, but the flake includes a lot of packages that just don't work on aarch64-darwin machines. Trying to use the flake will not work on darwin because packages like systemd are included unconditionally. I just add conditions to check if linux or not.
I can look into improving the bevy flake instead. Feel free to close, it should at least be searchable now.
If you know what formatter you use, I can try and use that to minimize the changes.

I'm not using nix these days [don't have the time].
But I do like it, and would like to see it supported, both in bevy and here!
Maybe we can rename it to flake.nix.example and then update the readme so we point nix users to it?
I'm happy to merge it and do that later, but as I'm on the phone [holiday no rig here ..] it'd be speedier if you can update, on the assumption you're happy to do so.
I never tested the nix stuff on mac when I had one, and even then I had an old-school Intel mac for which it's likely a lot of the packages are different again...


Encountered an unexpected exception

I was just walking around when the server crashed.  Every time I try to get back on it crashes again.
Here is the error I get in the console.
Here is the crash report.

Actual block type: ID # 52 (tile.mobSpawner // net.minecraft.block.BlockMobSpawner)
I'm wondering if it really was a mob spawner, because there should not be any.  It's a void world.

Nevermind.  Checking the world and location in MCEdit, it's a "lightningbolt" spawner minecart.  I changed that to "lightning_bolt" to see if that helped.
The spawner cart is being used just to show what it looks like if someone is using a resource pack

Could you post the full log?

I, uh, don't understand this bit;
"The spawner cart is being used just to show what it looks like if someone is using a resource pack"

Look up QMagnet's resource pack test world on planet minecraft
The mob spawner and spawner minecart were set to "lightningbolt".  Checking the entity ID on gamepedia, it's "lightning_bolt" now.  I just changed it accordingly and all is fine.

@EmanYm There, I've back ported the fix to both stable branches for 1.10.2 and 1.11.2 and to bleeding. Should apply for both SpongeForge and SpongeVanilla.

I will grab the update in the morning.  it's coming up on 2:00 am on my piece of the planet.

Update in #330

@EmanYm Is this fixed now?

This issue is.  I forgot to click close earlier.


2nd order swerve drive kinematics

see this post

https://github.com/cachemoney8096/2023-charged-up/blob/main/src/main/java/frc/robot/subsystems/drive/DriveSubsystem.java#L21
This seems to be an actual implementation of it (java though)

also this for their implementation of deriving a twist from the future pose (idk how this math works)


Legacy of Kain: Soul Reaver function doesn't match with MASPSX

Hi,
Here's a function that has got two memcpy() calls:
https://decomp.me/scratch/cKvGg
Currently it is a 100% match using gcc 2.8.1 + aspsx 2.79 (CCPSX), but changing to the current Legacy of Kain: Soul Reaver preset which uses maspsx causes it to match 99.78% instead, due to a diff between memcpy() on the left and bcopy() on the right. Using the -fno-builtin flag doesn't quite solve this issue like in most cases for a reason I currently don't know, since the first memcpy() call in the function produces very mismatching results that way.

This is not an maspsx issue, gcc is emitting the bcopy.
	lw	$4,4($18)
	lw	$6,0($18)
	.set	noreorder
	.set	nomacro
	jal	bcopy
	addu	$5,$16,40

Created https://github.com/decompals/old-gcc/issues/23 and closing this.

https://decomp.me/scratch/OBK59

Glad it works, could the game preset be updated to use gcc 2.8.1-psx + maspsx? That and -G65536 needs to be changed to -G4096 which is what I'm using in the Soul Reaverse project, currently.

Done. Per api:
{
    "id": 17,
    "name": "Legacy of Kain: Soul Reaver",
    "platform": "ps1",
    "compiler": "gcc2.8.1-psx",
    "assembler_flags": "",
    "compiler_flags": "-O2 -G4096 -fpeephole -ffunction-cse -fkeep-static-consts -fpcc-struct-return -fcommon -fgnu-linker -msplit-addresses -mgas -mgpOPT -mgpopt -msoft-float -gcoff -Wa,--aspsx-version=2.81",
    "diff_flags": [],
    "decompiler_flags": "",
    "libraries": [],
    "num_scratches": 2883,
    "owner": null
}```


Adjust AZP_AGENT_DOWNLOAD_TIMEOUT to maximum agent zip download size

Agent Version and Platform
Version of your agent? 2.153.1
OS of the machine running the agent? Windows
Azure DevOps Type and Version
Azure DevOps Server 2020
What's not working?
Upgrade agents to 2.175.2 fails due to download timeouts
It seems that when AZP_AGENT_DOWNLOAD_TIMEOUT was introduced in https://github.com/microsoft/azure-pipelines-agent/pull/2183 , the default timeout was set to 15 due to
// Allow a 15-minute package download timeout, which is good enough to update the agent from a 1 Mbit/s ADSL connection.`

Unfortunately, it seems vsts-agent-win-x64-2.175.2.zip (which AzureDevOps 2020 Server wants to upgrade to) has a size of 170.843.551 bytes, which would require ~22 minutes to download.
It would be good if the default download timeout has enough room to succeed the download (although I understand it can be difficult to "predict" the size of the agent zip upfront).

Hi @japj thanks for reporting this - we made a fix and will roll it out soon, will let you know once it's available.

Hi @japj we have rolled out the fix for this issue - please let us know if you have any other issues.


dependency check improvements

Some improvements to the dependency check parser:

Better / more descriptive title for findings
Fixes #4309
Better severity logic
Truncate breadcrumbs to avoid messed up layout
Add full title as title to finding in view_test (was already done elsewhere)
Align dedupe inside parser with new hashcode algo

fyi @Wadeck

This PR is actually removing the CVE and CWE from the title and tries to construct a more meaningful / descriptive title.


Reconstruction performance on KITTI

Hi @chaytonmin, thanks for releasing the code!
I am trying to train the Voxel-MAE model on KITTI dataset, and try to visualize the reconstruction results. Here is a example I got at epoch30:

From top to bottom, it shows the ground truth, the masked input and the model output. It seems that the model didn't reconstruct the input well. I am wondering if this is expected or not? If not, do you have idea which part could be wrong?
Moreover, may I ask which dataset are you using in Figure 3.

Hi @chaytonmin, thanks for releasing the code!
I am trying to train the Voxel-MAE model on KITTI dataset, and try to visualize the reconstruction results. Here is a example I got at epoch30:  From top to bottom, it shows the ground truth, the masked input and the model output. It seems that the model didn't reconstruct the input well. I am wondering if this is expected or not? If not, do you have idea which part could be wrong?
Moreover, may I ask which dataset are you using in Figure 3.

We use KITTI in Figure 3.
How do you visualize the reconstruction results?  What is the masking ratio?

Thanks for your reply! I found the issue: it's because I only plot the top-5000 points in the reconstruction image. If I increase the number it looks better.

Hi, xiaohui. @ZENGXH  Would you like to share your vis code ? If it still exists. :joy_cat: I'm confused about getting points from dense's features.


quantize: add max-mse-loss option

This option excludes parameters from quantization for which the MSE would be larger than the given value.

We should also update the spacy project to include the correct threshold for XLM-R (and a couple of comments clarifying the need).

Added to the project. I called the variable max_mse_qloss, to make it clearer that this is about the quantization loss (since it is a global variable in the configuration).


Issue #74 Ideal routing with an initial page DO NOT MERGE

DO NOT MERGE
@rasmus-storjohann-PG
Based on your feedback and also what I've learned about the redux-first-router, this is what I think would be the most elegant solution. The theory being that whenever SET_MAIN_PAGE is called (whether by the links in the footer or by the Back button) then it will set the mainPage to a value from the Page enum. If that were so, then we'd have the mapping between the string used in the route to the Page used in the Action defined by the toPath and fromPath functions in router.ts.
However, when I try to compile, I get this error message:
src/application/router.ts(39,60): error TS2345: Argument of type '{ [constants.SET_MAIN_PAGE]: { path: string; toPath: (page: Page) => string; fromPath: (pageRoute...' is not assignable to parameter of type 'RoutesMap<{ path
: string; toPath: (page: Page) => string; fromPath: (pageRoute: string) => Page; ...'.
  Property '[constants.SET_MAIN_PAGE]' is incompatible with index signature.
    Type '{ path: string; toPath: (page: Page) => string; fromPath: (pageRoute: string) => Page; }' is not assignable to type 'Route<{ path: string; toPath: (page: Page) => string; fromPath: (pageRoute: string) => Page; }, a
...'.
      Type '{ path: string; toPath: (page: Page) => string; fromPath: (pageRoute: string) => Page; }' is not assignable to type 'RouteObject<{ path: string; toPath: (page: Page) => string; fromPath: (pageRoute: string) => Pa
ge...'.
        Type '{ path: string; toPath: (page: Page) => string; fromPath: (pageRoute: string) => Page; }' is not assignable to type '{ capitalizedWords?: boolean; navKey?: string; path: string; thunk?: RouteThunk<any>; fromPat
h?(p...'.
          Types of property 'fromPath' are incompatible.
            Type '(pageRoute: string) => Page' is not assignable to type '(path: string, key?: string) => string'.
              Type 'Page' is not assignable to type 'string'.

I believe that it's telling me that the type signature of toPath and fromPath must be (string) => string whereas mine are (string) => Page and (Page) => string. Is there any way around this in Typescript?
If I don't use the toPath and fromPath methods, then the router just calls toString() on the enum values as they come into the path and then supplies them as string values when putting it back into the mainPage payload of the SET_MAIN_PAGE action when you use the Back button.
If you have time after sprint planning tomorrow, I'd love to talk this over.
(Note, since it's not compiling, I didn't bother to update the tests again to reflect these changes.)

We weren't happy with this version either because it relies on the string version of the Page enum


More backports

Summary of the changes / Why this improves CrateDB
See individual commits.
Note on https://github.com/crate/crate/pull/13189/commits/59a75ba1952c9bae11fe2d5e1b05d36e22654078:
I decided to not support of snapshots for mixed clusters and left out createSnapshotLegacy intentionally.
Checklist

[x] Added an entry in CHANGES.txt for user facing changes
[x] Updated documentation & sql_features table for user facing changes
[x] Touched code is covered by tests
[x] CLA is signed
[x] This does not contain breaking changes, or if it does:

It is released within a major release
It is recorded in CHANGES.txt
It was marked as deprecated in an earlier release if possible
You've thought about the consequences and other components are adapted
(E.g. AdminUI)

retest this please

retest this please
io.crate.integrationtests.ObjectColumnTest.updateToDynamicObject

As discussed, please add a note to the commit that createSnapshotLegacy is skipped by intentionally. Also, why didn't you mark as [s] those intermediate commits, I see a couple that are data stream related which we won't backport.

The note is added. I skipped a few backports to collect all the snapshot related backports together. The remaining backports will be updated/backported later.

retest this please

retest this please
io.crate.integrationtests.TransportSQLActionTest.test_types_with_storage_can_be_inserted_and_queried
Failing for the past 1 build (Since
#4431 )
Took 1.2 sec.
Error Message

java.lang.ArrayIndexOutOfBoundsException: Index 0 out of bounds for length 0

Stacktrace

java.lang.ArrayIndexOutOfBoundsException: Index 0 out of bounds for length 0
	at __randomizedtesting.SeedInfo.seed([C0C2400F806461FE:E5DCBD735E3F1340]:0)
	at io.crate.integrationtests.TransportSQLActionTest.test_types_with_storage_can_be_inserted_and_queried(TransportSQLActionTest.java:2018)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
	at java.base/java.lang.reflect.Method.invoke(Method.java:578)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
	at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54)
	at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
	at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
	at java.base/java.lang.Thread.run(Thread.java:1589)

retest this please


Infinite loop

Why this line called infinitely?
I have same code with other sagas and they are fine.
// api.ts
export getResults({ authParams }: AppRequest): Promise<ApiResponse<ResultsResponseModel>> {
  return superagent
          .get(withBackend('/results'))
          .set(headers(authParams));
}

// sagas/index.ts
export function* rootSaga() {
  yield takeEvery(FETCH_RESULTS, fetchResultsSaga)
}

// fetchResultsSaga.ts
export function* fetchResultsSaga() {
  const authHeaders = yield select(selectAuthHeaders);
  const request: AppRequest = { authParams: authHeaders };

  try {
    const response = yield call(getResults, request);   // CALLED INFINITELY

    yield put(fetchResultsSuccessAction(response.body));
  }
  catch (error) {
    console.error(error);
  }
}

The issue was caused because actual string values of two related actions were the same.
Another pitfal of redux. Let's hope that someday it will support integer as action type and we will be able to create unique actions without writing names manually, for example like this:
const action = {
  type: Crypto.randomUUID()
};


Added new patchcode for DefPolicyPatch

termsrv.6.2.9200.23509.zip
Original

Changed

Can you merge the PR?

I believe the last commit date would answer this question.

Can you merge the PR?

I believe the last commit date would answer this question.

That means you'll never merge?


Add IO hooks

Fixes https://github.com/runtimeverification/haskell-backend/issues/3296
This PR adds the following new hooks:

IO.open
IO.close
IO.getc
IO.read
IO.write

Well, this one was tricky. Thank you for implementing it!
Is it possible to add an integration test for this?

yes, will add one.


Refactor strategies to works for single address only

Hey @bonustrack, do you have more details on that one?

@zzuziak The idea is to make strategies resolve only a single address instead of multiple ones at once, but this is blocked by strategies with overriding which may be hard deprecate.

Ok, shall we maybe scope it out in a pitch to figure out a possible solution and have an overview of all affected strategies?


Support ember-modifier v4

ember-modifier v4 was just released and it’s v2 addon requiring ember-auto-import
it’s worth including in v1 as it would be a breaking change for this library so good time to do it now

Agreed

This was fixed in the 0.6.0 release I believe.


Support remaining pycodestyle rules

Would be great to have these:
E201 whitespace after '['
E202 whitespace before ')'
E203 whitespace before ','
E211 whitespace before '('
E221 multiple spaces before operator
E301 expected 1 blank line, found 0
E302 expected 2 blank lines, found 1
E303 too many blank lines (4)
E401 multiple imports on one line

Awesome!
Docs mention E201 3 times with (, [ and {. So I assume it's just whitespace after any of them.

@charliermarsh First time contributing and I've never worked with ASTs before. I think I have some code in place but would welcome support on this. I could open a PR to show what I have currently, if that is acceptable.
Basically stuck on the AST logic implementation and believe I have the rest in place.

@nanthony007 - Happy to help! Do you want to put up a draft PR with questions we can discuss in the context of the code? Or what would be most effective?

I can put up a PR and we can go through it there I think.
I was thinking that since the E3 series seems to be whitespace oriented it might be worth implementing them as a part of this issue as well.

Run into this as well when testing ruff on a relatively large, mostly legacy code base where it provides an immense speed boost over flake8; but we'll have to wait until basic stuff along the lines of E302 is supported.
Don't get me wrong, I do think ruff is fantastic, and I'm planning to adopt it soon both in various hobby projects and at work, but maybe "near-parity with Flake8" is a bit misleading at the time being?

@vytas7 - Appreciate it!
We prioritized the rules based on the assumption that Ruff would be used alongside Black -- so we just haven't focused on implementing rules like E302 that are made obsolete by an autoformatter. (It doesn't mean we won't implement them, just that they weren't / haven't been prioritized.) That's one of the key assumptions baked into any "parity with Flake8" claims. I hope it doesn't come across as misleading, it's all documented in the FAQ, definitely not my intent.

I was also looking for it, those messages are useful when teaching.
I will need to use ruff + flake8 for a while

Closing in favor of #2402, which is the same issue but includes the rule tabulation up top.


ApplyOE tutorial for the multisurface run

Quick tutorial outlining:

Surface classification and constructing the classification file
Apply OE
Some visualization

Accompanies: https://github.com/isofit/isofit/pull/617
The data to run the tutorial still has to be set up in a similar way as the NEON example. It uses the rdn, loc, obs files as well as a pre-built Modtran LUT.

The tutorial runs with the analytical_line, glint model on water, and topography mode on land. The analytical_line is not fully vetted for the new glint_model. @unbohn The glint model successfully executes with analytical line, but there is something incorrect (I think) with the solved magnitude of sky_glint. This is a conversation for a different place than this PR.

Thanks for confirming @evan-greenbrg! The incorrect magnitude of estimated sky glint was indeed one of the main issues with the analytical line solution for the new glint model. As you mention, let's discuss that in a different PR.

And thanks for uploading this tutorial. Looks great!


SC-2032 Create Cassandra table to store consent details

Codecov Report

Merging #959 into release-3.3.0 will not change coverage.
The diff coverage is n/a.

@@               Coverage Diff                @@
##             release-3.3.0     #959   +/-   ##
================================================
  Coverage            37.33%   37.33%
  Complexity             942      942
================================================
  Files                  152      152
  Lines                10234    10234
  Branches              1144     1144
================================================
  Hits                  3821     3821
  Misses                6028     6028
  Partials               385      385

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 514064e...d72bc44. Read the comment docs.


An example of how to do this in Yaml probably wouldn't go amiss.

My title pretty much says it. : )

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: a3b20c46-c5f1-3a2a-738d-8aa8e69cf10c
Version Independent ID: 7055e752-859b-6106-6802-70010d0305d9
Content: NuGet Package Restore
Content Source: docs/consume-packages/Package-Restore.md
Product: nuget
GitHub Login: @JonDouglas
Microsoft Alias: jodou

I've lost all context on my comment at this point given that this was apparently 'posted' a year and ~3 months ago. If you guys can trace it back to the page against which I'd made this posting/comment/suggestion, then, well, my suggestion stands.

The primary feedback here is around having an example for yml, which is something specific to Azure DevOps than just NuGet.


Release of PhantomJS 2.0.1

This is the first maintenance release after 2.0.0 designed to address immediate critical issues.

[ ] Update ChangeLog
[ ] Update web sites
[ ] Upload source tarball
[ ] Prepare OS X binary
[ ] Prepare Windows binary
[ ] Prepare Linux 64-bit binary
[ ] Prepare Linux 32-bit binary

I've not tested via Linux/OSX but has 2.0.0 taken on the scaling problems that Windows had prior to 2.0.0 on these platforms? See this example/issue with a detailed response from me on my own findings in this comment. You can see that with 2.0.0 on OSX example image, the original content has been scaled up by 1/3rd converting it from 72px per inch to 96px per inch(CSS inch), if I'm not mistaken this is due to OSX being 72dpi? I'm not sure what changes with 2.0.0 caused this, but Windows now renders PDF perfectly, hopefully you can fix this issue asap :)

@ariya any release date ? I'm thinking about prepare a release for CasperJS with phantomjs 2 support :)

I think a regression like #12685 cannot wait for 2.1 and should instead be part of 2.0.1.

I agree, #12685 is a dealbreaker for many of us. I'm excited to upgrade some projects to v2, but can't do it until this is resolved.

^ agreed. thanks for all the hard work.

Issues to fix before we release 2.0.1 version:

[ ] Integrate the latest version of Ghostdriver
[ ] #12506
[ ] #12948
[ ] #12900
[ ] #12685

@Vitallium Sorry to not have the skills for help (I don't know C++).
If I can do something to help you (like testing installation process on my linux mint 17) tell me :)

@Vitallium Sorry itou
I can help you to testing installation process & application on Windows 7 64 bits & 32bits.

FYI: I recently discovered that we have a Gitter chat for PhantomJS. You can catch me there.

Is this Gitter chat : https://gitter.im/ ?

@LeMoussel yes.

Any news on 2.0.1 release?

:+1:

:+1::+1::+1:

:-1: PDF's still not working on 2.x in Linux

@davidwindell issue?

@Vitallium #12685 (and it's associated dupes #13048 #12990, etc). Stuck on 1.x for now :(

@davidwindell this issue is in the block list for 2.0.1 :-)

A nice to have is IMHO a fix for https://github.com/ariya/phantomjs/issues/10465# and an opportunity to repeat the table headers after a page break.

Pardon if I've misunderstood but given only #12506 seems to be the only regression from 2.0.0 that breaks people's test cases outright and a fix is already known for a few months I'm not sure why that can't be released as 2.0.1 (source + Windows / OS X binaries) and the others worked on for a separate release. Releasing often is a good thing.

@alecl :+1:

@alecl Especially since I hear this update is requirement to get phantomjs working on 10.11 El Capitan.

I agree!

:+1:

I think I can publish nightly builds of PhantomJS 2.0.1 for all platforms. Since you guys want it (esp for El Capitan) :-)

That would be great :)!

:+1:

:+1:

Would be much appreciated!

As I understand it, building binaries for el capitan requires https://github.com/ariya/phantomjs/pull/13640 to be merged. It would be nice if that happened before 2.0.1 :)

@jeantil yes, 2.0.1 will support El Capitan.

Nice ! Let us know if we can help :)

With regard to #12506, it seems that the issue is reported upstream at https://bugreports.qt.io/browse/QTBUG-44237

Will 2.0.1 support TLSv1.2?

@Vitallium thx for the prerelease, but are you sure, it has all planned fixes for 2.0.1 included? I'm on El Capitan and running both into errors in Upload form and with strange "undefined is not an object" errors (which were there reason, we tried PhantomJS 2 in the first place)
The "inofficial" prerelease from July (found here https://github.com/skakri/phantomjs/releases/tag/2.0.1-regression-12506) solved these issues (on El Capitan, but not on Ubuntu).
I'm not sure if I should introduce a new bug report for this, because I fear it gets very confusion with different bugs for different "versions of 2.0.1"...

:+1:

@MartinKoerner not all. This prerelease doesn't have two fixes.

oh, It's important to me about fixed bug below;
1.Webpage.uploadFile not working in phantomjs 2.0 #12506
https://github.com/ariya/phantomjs/issues/12506
2. PhantomJS 2 does not build on OS X 10.10 (Yosemite)
https://github.com/ariya/phantomjs/issues/12928
when i build phantomjs from sourcecode, i got some errors:
a.error: "Feature _POSIX_TIMERS not available"
b.error: use of undeclared identifier
c.error: no member named 'posix_fallocate' in the global namespace
d.error: no member named 'mremap' in the global namespace
e.  error: use of undeclared identifier 'MREMAP_MAYMOVE'
f.fatal error: 'sys/inotify.h' file not found
g. fatal error: 'sys/eventfd.h' file not found
......
i am very hurry,  please help to to fix first bug. thank you.......

@zhagnsz we have fixes for both issues. So, don't worry. They are included in the upcoming 2.0.1 release.

@Vitallium  okay, thank you very much!  when are you going to update to the 2.0.1 version.....

+1 for a release of 2.0.1. We would really enjoy #12506 + El capitan support.

@Vitallium Any ETA for 2.0.1? Thanks.

+1

We're low on manpower so we won't do 2.0.1 at all. We'll skip to 2.1 (follow #12970).


[Suggestion] Music fade in and out on pause/unpause.

Checklist:

[x] I have checked that there are no issues with similar or the same content
YOU SHOULD CHECK CLOSED ISSUES ASWELL

[x] I have checked the FAQ (https://github.com/MarshallOfSound/Google-Play-Music-Desktop-Player-UNOFFICIAL-/wiki/FAQ) and the answer I am looking for is not there

I've read the other issues and I saw that the suggestion was closed because it would be complicated to hold the pause action while the music fades in. However I am suggesting to fade out the volume on pause, so save the current volume value and over 1 second fade the volume to 0. Then on unpause it would still be at 0, but then it can fade in to the saved volume value from before. The time it takes to fade in and out can be tweaked. This way you are not affecting the playing/pausing, but can still achieve the same effect.
OS:Windows 10
GPMDP Version: 4.6.1

What you describe is what was proposed in other issues and declined due to having to manage the pause logic.  In order to fade out on pause, you have to reject the pause and then pause later once you've faded out.  There are so many edge and race conditions to asyncronously pausing / playing music.  I don't want to go anywhere near that


how to install hmac on the Openwrt???

I am interested to install hmac in OpenWRT too. How can we do this job?

if you can do it install a tdma in openwrt with other software please report to me about it?? i can do it now

if you can do it install a tdma in openwrt with other software please report to me about it?? i can't find posible iisuse this

if you can do it install a tdma in openwrt with other software please report to me about it?? i can't find right method

@tiuman @ssssaaamm I have tried to use HMAC on OpenWrt a while ago (I think barrier breaker).
Here is the kernel patch file:
https://raw.githubusercontent.com/freifunk-graviton/graviton/patch/patches/openwrt/0081-add-hmac-support.patch
Unfortunately the whole zeromq thing does not make sense using OpenWrt. Another thing is using C++ needs much flash memory as you need all the standard libs. So you end up writing the user space program in C just to see, that you need to implement the scheduling in kernel space instead of using netlink to make it work reliable on commodity routers (and the timing is still an issue).
I'm thinking about implementing something like eMCCA in 802.11s as that could become part of the mainline kernel in a distant future. All the generic power save and timing hooks needed already exist. So if you know someone who is interested to participate or wants to support it in some way leave me a message.

@tiuman @ssssaaamm我曾尝试过在OpenWrt上使用HMAC（我认为是障碍突破者）。
这是内核补丁文件：https :
//raw.githubusercontent.com/freifunk-graviton/graviton/patch/patches/openwrt/0081-add-hmac-support.patch
不幸的是，使用OpenWrt整个Zeromq都没有意义。另一件事是使用C ++需要大量闪存，因为您需要所有标准库。因此，您最终使用C编写了用户空间程序，只是看到需要在内核空间中实现调度，而不是使用netlink使其在商用路由器上可靠运行（并且时间仍然是问题）。
我正在考虑在802.11s中实现类似eMCCA的功能，因为在不久的将来它可能成为主线内核的一部分。所需的所有通用节电和定时挂钩已经存在。因此，如果您知道有兴趣参加或希望以某种方式支持它的人，请给我留言。

Hi Do you have the latest progress？ Or do you have any good ideas？ I am most grateful for your selfless help.

@tiuman @ssssaaamm我曾尝试过在OpenWrt上使用HMAC（我认为是障碍突破者）。
这是内核补丁文件：https :
//raw.githubusercontent.com/freifunk-graviton/graviton/patch/patches/openwrt/0081-add-hmac-support.patch
不幸的是，使用OpenWrt整个Zeromq都没有意义。另一件事是使用C ++需要大量闪存，因为您需要所有标准库。因此，您最终使用C编写了用户空间程序，只是看到需要在内核空间中实现调度，而不是使用netlink使其在商用路由器上可靠运行（并且时间仍然是问题）。
我正在考虑在802.11s中实现类似eMCCA的功能，因为在不久的将来它可能成为主线内核的一部分。所需的所有通用节电和定时挂钩已经存在。因此，如果您知道有兴趣参加或希望以某种方式支持它的人，请给我留言。

Hi Do you have the latest progress？ Or do you have any good ideas？ I am most grateful for your selfless help.

@CodeFetch Hi Do you have any latest process?Or do you have any good idea? I am most grateful for your selfless help

@CodeFetch Hi Do you have any latest process?Or do you have any good idea? I am most grateful for your selfless help

@Tim1Wu I think MCCA should be implemented in ath11k. But for that one needs the source code for the firmware and Qualcomm is not willing to offer it for free. I guess it will cost $200k+ for getting it implemented. People will also have a look at Mediatek and researchers at https://github.com/open-sdr/openwifi for alternatives.
There is a "premium" feature in the ath10k firmware called QBoost which is some kind of polling-approach to TDMA and might be interesting to you. I have a patch for it, but I think there is some synchronization code missing as the performance is bad.
802.11ax gives us much better possibilities to do collision avoidance and has a better timing granularity. Extending and implementing the MCCA standard for 802.11ax is therefore what we should focus on. When we solve the "hidden node" and the "fastest node" problem with that, sophisticated routing (on top of WiFi 7) will make it perform better and after that only fractal coding together with agent chatting will come. That will possibly be another chip generation (WiFi 8) as it would profilt from AI acceleration to do the carrier switches for the different fractal patterns and doing guessing-work for predicting the choices of other nodes.
By the way... Intelligence including AI is always about predictions. My father says: "In the 80s we called it statistical analysis". What makes a human being intelligent is mimicry (mirror neurons etc.) and imagination. So what we could get from AI in collision avoidance is "imagining" the traffic flows of other devices and mimicing the behaviour of other nodes in a provision-phase. And it can get as mad and erroneous as every other intelligent organism... Concerning the additional power consumption needed for calculating these predictions, the limiting factor of "knowledge" which itself must either be transferred or extrapolated and the user expectation of comprehensible behavior I don't see this coming before 2040 which might bring us a not-so-much-AI WiFi 8 first in 2030.
It's forseeable where this journey will go until we reach "physical spectrum" throughput. I'm working on some of these things already, but getting funding is hard. So I play my role as a consultant at the moment.

@Tim1Wu I think MCCA should be implemented in ath11k. But for that one needs the source code for the firmware and Qualcomm is not willing to offer it for free. I guess it will cost $200k+ for getting it implemented. People will also have a look at Mediatek and researchers at https://github.com/open-sdr/openwifi for alternatives.
There is a "premium" feature in the ath10k firmware called QBoost which is some kind of polling-approach to TDMA and might be interesting to you. I have a patch for it, but I think there is some synchronization code missing as the performance is bad.
802.11ax gives us much better possibilities to do collision avoidance and has a better timing granularity. Extending and implementing the MCCA standard for 802.11ax is therefore what we should focus on. When we solve the "hidden node" and the "fastest node" problem with that, sophisticated routing (on top of WiFi 7) will make it perform better and after that only fractal coding together with agent chatting will come. That will possibly be another chip generation (WiFi 8) as it would profilt from AI acceleration to do the carrier switches for the different fractal patterns and doing guessing-work for predicting the choices of other nodes.
By the way... Intelligence including AI is always about predictions. My father says: "In the 80s we called it statistical analysis". What makes a human being intelligent is mimicry (mirror neurons etc.) and imagination. So what we could get from AI in collision avoidance is "imagining" the traffic flows of other devices and mimicing the behaviour of other nodes in a provision-phase. And it can get as mad and erroneous as every other intelligent organism... Concerning the additional power consumption needed for calculating these predictions, the limiting factor of "knowledge" which itself must either be transferred or extrapolated and the user expectation of comprehensible behavior I don't see this coming before 2040 which might bring us a not-so-much-AI WiFi 8 first in 2030.
It's forseeable where this journey will go until we reach "physical spectrum" throughput. I'm working on some of these things already, but getting funding is hard. So I play my role as a consultant at the moment.

@CodeFetch Thanks for your answer, but I am pessimistic about the research prospects of this project.By the way, could you please send me the QBoost patch as well as the related pictures, information or anything else you mentioned? I would appreciate it very much.My Email<EMAIL_ADDRESS>

@CodeFetch Thanks for your answer, but I am pessimistic about the research prospects of this project.By the way, could you please send me the QBoost patch as well as the related pictures, information or anything else you mentioned? I would appreciate it very much.My Email<EMAIL_ADDRESS>


Implement Stringer

Fixes #276
Review of colleague's PR #
Changes proposed in this PR:
-Create an executable go program in directory 05_stringer/brvishnu92
-Make the IPAddr type implement fmt.Stringer to print the address as a dotted quad.

Codecov Report

Merging #277 into master will not change coverage.
The diff coverage is 100%.

@@          Coverage Diff          @@
##           master   #277   +/-   ##
=====================================
  Coverage     100%   100%
=====================================
  Files          18     19    +1
  Lines         303    307    +4
=====================================
+ Hits          303    307    +4

Impacted Files
Coverage Δ

05_stringer/brvishnu92/main.go
100% <100%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 15b200b...ba09866. Read the comment docs.


findAndConnect not working properly on Android 8.0

Hello, i have tested on LG G6 on android 8.0, a Sony on android 8.0 and Asus zenpad on 7.0.
On the zenpad all working
On the LG and Sony, all working, but not the findAndConnect. The wifi network disconnect from the current network, and ( i have debug java ), the function connectTo return true, but the wifi dont connect to wanted network, and reconnect to previous network ...
I've test on an Open network, and a secure network, with and wihout internet on both
This seem arrive only on android 8.0, i can't find a fix ... An idea ?

This also doesn't work on my Galaxy S8 running android 8.0.  findAndConnect returns false for found.  Any help?

after many test i have seen that its because the networks was configured from another app, try to forgot the network and retry, normally it's working, i've modified java code in consequences

I tried using isRemoveWifiNetwork to forget the network first, but that didn't help either.  Even with a brand new network visible, it can't connect.  Am I missing a permission or some other step?

No, you can't update, remove  or create a wifi network that has already been configured by another app or by settings page of your phone.
The findAndConnect, try to update everytime network config if already exist, and return an error if it couldn't.
You must forget network from your phone settings and not from code before trying to connect from code.
For example, if you have an error of connection, write into a TextView at the user intention, asking him to forget the network himself and retry.
For permission you need one of them : ACCESS_COARSE_LOCATION or ACCESS_FINE_LOCATION
Another fix, try to remove double quote from ssid in the java code in version 8.0 of android
I hope i have help you

I tried with adding double qoutes for android version 8 and it worked.
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) {   conf.SSID = "\"" + ssid + "\""; }

I opened a PR (https://github.com/devstepbcn/react-native-android-wifi/pull/51) adding back double quote fpr SDK 26 and a change in behaviour to use phone stored config if exists. That eliminates the need for user to forget the network manually.

in android version <8 works fine. but in the latest versions 9, 10. it's not working.
Any solution?

Did anyone find a solution for verions of android >8?


Bump web3Modal version

Bump web3Modal to the latest version. This version provides Tally(open source community build browser extension wallet) support

tests failing b/c on forked repo. Passes locally. lgtm

thank you for the pr. closing this in favor of the clean https://github.com/OlympusDAO/olympus-frontend/pull/1679


[cleanup] Consolidate some query and controller/broker methods in integration tests

Remove unnecessary usage of static PostQuery calls.
Remove direct reference to _controllerBaseApiUrl, _controllerPort and _brokerBaseApiUrl in extended test classes.
Always add useMultistageEngine query option if possible.
Simplify MultiStageEngineCustomTenantIntegrationTest

Codecov Report

Merging #11064 (4b3fc85) into master (9b21eb9) will increase coverage by 0.00%.
The diff coverage is 0.00%.

@@            Coverage Diff            @@
##           master   #11064     +/-   ##
=========================================
  Coverage    0.11%    0.11%
=========================================
  Files        2200     2146     -54
  Lines      118802   116300   -2502
  Branches    17991    17689    -302
=========================================
  Hits          137      137
+ Misses     118645   116143   -2502
  Partials       20       20

Flag
Coverage Δ

integration1temurin11
?

integration1temurin17
?

integration2temurin11
?

integration2temurin17
?

integration2temurin20
?

unittests1temurin11
?

unittests1temurin20
?

unittests2temurin11
0.11% <0.00%> (ø)

unittests2temurin17
0.11% <0.00%> (?)

unittests2temurin20
0.11% <0.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

...spi/utils/builder/ControllerRequestURLBuilder.java
0.00% <0.00%> (ø)

... and 54 files with indirect coverage changes
:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

Please take a look at the failed test

Fixed.


Using switch with is cases instead of if with multiple as?.

Changes in this pull request
Using switch with is cases instead of if with multiple as?.
Pull request checklist

[x] All tests pass. Demo project builds and runs.
[ ] I added tests, an experiment, or detailed why my change isn't tested.
[x] I have reviewed the contributing guide

Thanks @valeriyvan ! 👍


Question: Can I record VoiceOver as text for comparison?

Hi, I'm trying to find a tool that I can use with react testing library to create a test for screenreader. I was wondering if VoiceOver can also return the readout as text for comparison.

hi Ben, yes you can initialize the VoiceOver object with logging to stdout with const voiceOver = new VoiceOver({ log: true });

Sorry for the late response @ckundo Thank you! I'll definitely try this out to make my project more accessible :D


KAFKA-10470: Zstd upgrade and buffering

Zstd-jni 1.4.5-6 allocates large internal buffers inside of ZstdInputStream and ZstdOutputStream. This caused a lot of allocation and GC activity when creating and closing the streams. It also does not buffer the reads or writes. This causes inefficiency when DefaultRecord.writeTo() does a series of small single bytes reads using various ByteUtils methods. The JNI is more efficient if the writes of uncompressed data were flushed in large pieces rather than for each byte. This is due to the the expense of context switching between the Java code and the native code. This is also the case when reading as well. Per https://github.com/luben/zstd-jni/issues/141 the maintainer of zstd-jni and I agreed to not buffer reads and writes in favor of having the caller do that, so here we are updating the caller. Here is part of a flame graph of  CPU time spent in the single byte writes from Kafka:

In this patch, I upgraded to the most recent zstd-jni version with the buffer reuse built-in. This was done in https://github.com/luben/zstd-jni/pull/143 and https://github.com/luben/zstd-jni/pull/146 Since we decided not to add additional buffering of input/output with zstd-jni, I added the BufferedInputStream and BufferedOutputStream to CompressionType.ZSTD just like we currently do for CompressionType.GZIP which also is inefficient for single byte reads and writes. I used the same buffer sizes as that existing implementation.
NOTE: if so desired we could pass a wrapped BufferSupplier into the Zstd*Stream classes to have Kafka decide how the buffer recycling occurs. This functionality was added in the latter PR linked above. I am holding off on this since based on jmh benchmarking the performance gains were not clear and personally I don't know if it worth the complexity of trying to hack around the reflection at this point in time. The zstd-jni uses a very similar default recycler as snappy does currently which seems to provide decent efficiency. While this PR fixes the defect, I feel that using BufferSupplier in both zstd-jni and snappy is outside of the scope of this bugfix and should be considered a separate improvement. I would prefer this change get merged in on its own since the performance gains here are very significant relative to the more incremental and minor optimizations which could be achieved by doing that separate work.
 NOTE: the RecordBatchIterationBenchmark looks basically the same since that code uses the BufferSupplier provided by the JMH harness, so recycling is actually occurring. The CompressedRecordBatchValidationBenchmark does not and fixing that is the crux of that other PR which is indeed correct.
Given the snappy recycler implementation is currently acceptable and that zstd-jni uses a very similar one, I propose we lean on zstd-jni's default recycler for the time being (to guarantee buffer recycling regardless of the type of BufferSupplier it gets) and only after https://github.com/apache/kafka/pull/9229 or something like it is merged then we revisit using the BufferSupplier in zstd. I agree in the long term that is the best case, but I don't want a half-fix when we can have a fix here that gets us 98% of the way there. I an eager to get a fast zstd implementation in since we have very promising results but had to stop using it because of the negative performance impact.
Additionally there are some open questions (mostly beyond the scope of this bugfix):

The wrapForOutput does not have a BufferSupplier, so where does it come from: static constant/singleton, somewhere else, break the API, do we just lean on the default zstd-jni recycler?
Creating the wrapper around BufferSupplier that implements zstd-jni's BufferPool means implementing BufferPool but the checkstyle prohbits this. Would we be allowed to import it? I see jpountz lz4 is allowed.
Do we want to do the recycling for snappy? I see see SnappyOutputStream seems to allow for one to pass in a BufferAllocatorFactory implementation which may be interoperable with BufferSupplier. However this may be moot due to my first point. The SnappyInputStream does not seem to accept any recyclers. There are SnappyFramedInputStream and SnappyFramedOutputStream which seem to accept another snappy-specific recycler "BufferPool". Maybe those could be promising?
Do we want to do the recycling for GZIP? GZIP seems like a lost cause since that stuff related is buffers is hidden within the standard library, unless it is ported over like the LZ4 streams.
Do we want to create versions of BufferedInputStream and BufferedOutputStream which use BufferSupplier?
Does any of this work interfere with any future work related to passing in compressor configuration parameters?

@yuzawa-san Thanks for great profile and explanation. I'm eager (too) to see this patch to be merged since it not only improves the memory usage but also brings better performance (especially for small message). The following story is just my two cents. The reason to use kafka pool instead of (zstd) recycling buffers is not only about performance but also other improvements. For example, synchronization issue (that was discussed in https://github.com/luben/zstd-jni/issues/131). Also, on our kafka (private branch), we introduces some sensors to (kafka) pool to get oom/duration metrics to trace compression status. In short, it would be great to introduce kafka pool to compression. (We don't use snappy since it is not best compression on our use cases)
At any rate, I'm +1 to merge this PR as it does bring significant improvement. We can address kafka pool in next PR.
@yuzawa-san thank you (again) that your effort on both zstd and kafka!

There is a newer zstd-jni version, so I'm going to push the newer one shortly after my local checks pass.

ok most recent version is pushed and checks pass now.


Cow: Watch<PHONE_NUMBER>

Cow requests the watch of the watch_number<PHONE_NUMBER>. See the MS search here and the Stack Exchange search in text, in URLs, and in code.

Approved by Cody Gray in Charcoal HQ


Add dark mode and light mode.

Description
Add dark mode and light mode for the user's good experience.
Screenshots
No response
Checklist

[X] I have checked the existing issues.
[X] I follow Contributing Guidelines of this project.
[X] I am a GSSoC'24 contributor.
[X] I want to work on this issue.

Code of Conduct

[X] I agree to follow this project's "Code of Conduct".

I would like to work on this issue. It would be great if you assign this to me.
Thank you
Your sincerely,
Sarthak Sharma


Smpl Joint rotations.

You can get the rotations from output[0]['theta']. Code for the refiner module is in the branch 'refine'.


Fix kustomize in the 'single' example #838

This is a bunch of changes I've made to get the example single working. I have it working with my custom hacked up docker image, once Cosmos SDK merges my fix it should be possible to have a working example again.
This is a brief summary of the changes

Fix things referencing akashctl & AKASH_CTL since those are obsolete
Update the run.sh scripts that are part of kustomize to set the correct flags
Add the ping command to the Makefile for this example
Allow uploading a custom image via Makefile commands
Remove some un-needed verbs in the Makefile
Update the Ingress declarations to be v1

I pushed up a change to the kube example since I couldn't communicate with the deployment after it was created.


no output what so ever

Expected Behavior
I expect some output when running the azqr scan command
Actual Behavior
azqr scan
2023/06/26 07:51:51 Generating Report: azqr_report_2023_06_26_T075150.xlsx
2023/06/26 07:51:51 Skipping Overview. No data to render
2023/06/26 07:51:52 Skipping Recommendations. No data to render
2023/06/26 07:51:52 Skipping Services. No data to render
2023/06/26 07:51:52 Skipping Defender. No data to render
2023/06/26 07:51:52 Skipping Advisor. No data to render
2023/06/26 07:51:52 Skipping Costs. No data to render
2023/06/26 07:51:52 Scan completed.
Steps to Reproduce the Problem
az login successful , account set also , full access to subscription , still no output ...

HI @t2kx,
Some questions:

what version are you running? Run azqr --version if not sure.
do you only have virtual machines deployed?

can you please run th efollowing:
$env:AZURE_SDK_GO_LOGGING="all"
azqr scan > azqr_results.txt

and share the azqr_results.txt file with me?

Closing since issue has not been reproduced and no feedback provided.


Support for ActiveAdmin 2.0

Hi, first of all: thanks for this component and the work you put into it :)
So, ActiveAdmin 2.0 is out (and soon 2.1), but this gem is stuck requiring ~> 1.0
Any plans for releasing a v2.x compatible version?
Thanks again!
EDIT: #2 already takes care of this. Any temptative merging date?

Hi @julitrows. Thank you for your feedback. I plan to make changes in the next few days

Sorry to keep you waiting. Just released a new version.


chore(aspenetcore): update aspnetcore projects for 1.0

Both aspnet core projects should work properly now. I had Mads Kristensen do a once over of everything. He suggested we get rid of Program.cs and move static void Main to Startup.cs.

I wouldn't know if it worked or not, but it looks pretty clean. @AshleyGrant Give me the final word and I'll merge it immediately. After that, maybe we need to update the readme files? Let me know.

This is good. We can update readmes next.

Ok. I'll merge momentarily.

Ok, it's merged :) Thanks for putting the effort into that. Now, let's make sure the readmes are clear for both of them.


Update Composer dependencies (2021-05-06-00-10)

Visual regression test passed!
View the visual regression test report


java.lang.NoClassDefFoundError: java.time.OffsetDateTime

Hello.
Every time when i call any method of library i get that error:
java.lang.ClassNotFoundException: java.time.OffsetDateTime
Can you help me with this problem please?

Hi @Melowetty, can you try with 1.2.0 and let me know?

closing due to no reply. Feel free to reopen if the issue is still happening


Sensor scales

Reference issue
Implement https://github.com/mne-tools/mne-python/issues/12791
What does this implement/fix?
Add option to specify sensors scales in _plot_sensors_3d and expose it as parameter in mne.viz.plot_alignment and mne.viz._brain._brain.py add_sensors.
The implementation is the same as for sensor_colors

larsoner regarding my github URL, note that I also have another open PR in collaboration with a colleague. I have added it to both, I hope it won't cause any redundancies.


python3Packages.tokenizers: 0.9.4 -> 0.10.0

Motivation for this change
Changelog:
https://github.com/huggingface/tokenizers/releases/tag/python-v0.10.0
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[ ] NixOS
[ ] macOS
[x] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

seems to fail for me locally:
https://github.com/NixOS/nixpkgs/pull/109532

4 packages failed to build:
python37Packages.tokenizers python37Packages.transformers python38Packages.tokenizers python38Packages.transformers

/nix/store/yi9a1fafijn0nf5m8qkmnl955vi2mncm-python3-3.8.6/lib/python3.8/os.py:213: in makedirs
    makedirs(head, exist_ok=exist_ok)
/nix/store/yi9a1fafijn0nf5m8qkmnl955vi2mncm-python3-3.8.6/lib/python3.8/os.py:223: in makedirs
    mkdir(name, mode)
E   PermissionError: [Errno 13] Permission denied: '/homeless-shelter'

seems to fail for me locally:
https://github.com/NixOS/nixpkgs/pull/109532

4 packages failed to build:
python37Packages.tokenizers python37Packages.transformers python38Packages.tokenizers python38Packages.transformers

/nix/store/yi9a1fafijn0nf5m8qkmnl955vi2mncm-python3-3.8.6/lib/python3.8/os.py:213: in makedirs
    makedirs(head, exist_ok=exist_ok)
/nix/store/yi9a1fafijn0nf5m8qkmnl955vi2mncm-python3-3.8.6/lib/python3.8/os.py:223: in makedirs
    mkdir(name, mode)
E   PermissionError: [Errno 13] Permission denied: '/homeless-shelter'

Result of nixpkgs-review pr 109532 run on x86_64-linux 1

4 packages failed to build:

python37Packages.tokenizers
python37Packages.transformers
python38Packages.tokenizers
python38Packages.transformers

Result of nixpkgs-review pr 109532 run on x86_64-linux 1

4 packages failed to build:

python37Packages.tokenizers
python37Packages.transformers
python38Packages.tokenizers
python38Packages.transformers

Result of nixpkgs-review pr 109532 run on x86_64-darwin 1

4 packages failed to build:

python37Packages.tokenizers
python37Packages.transformers
python38Packages.tokenizers
python38Packages.transformers

rootdir: /private/tmp/nix-build-tokenizers-0.10.0.drv-0/source/bindings/python
collected 117 items / 1 error / 116 selected

==================================== ERRORS ====================================
__ ERROR collecting tests/documentation/test_tutorial_train_from_iterators.py __
tests/documentation/test_tutorial_train_from_iterators.py:4: in <module>
    import datasets
/nix/store/janmd6006jr9kmxipkc22g0g09b9j0n7-python3.8-datasets-1.1.2/lib/python3.8/site-packages/datasets/__init__.py:27: in <module>
    from .arrow_dataset import Dataset, concatenate_datasets
/nix/store/janmd6006jr9kmxipkc22g0g09b9j0n7-python3.8-datasets-1.1.2/lib/python3.8/site-packages/datasets/arrow_dataset.py:40: in <module>
    from .arrow_reader import ArrowReader
/nix/store/janmd6006jr9kmxipkc22g0g09b9j0n7-python3.8-datasets-1.1.2/lib/python3.8/site-packages/datasets/arrow_reader.py:31: in <module>
    from .utils import cached_path, logging
/nix/store/janmd6006jr9kmxipkc22g0g09b9j0n7-python3.8-datasets-1.1.2/lib/python3.8/site-packages/datasets/utils/__init__.py:20: in <module>
    from .download_manager import DownloadManager, GenerateMode
/nix/store/janmd6006jr9kmxipkc22g0g09b9j0n7-python3.8-datasets-1.1.2/lib/python3.8/site-packages/datasets/utils/download_manager.py:25: in <module>
    from .file_utils import HF_DATASETS_CACHE, cached_path, get_from_cache, hash_url_to_filename
/nix/store/janmd6006jr9kmxipkc22g0g09b9j0n7-python3.8-datasets-1.1.2/lib/python3.8/site-packages/datasets/utils/file_utils.py:118: in <module>
    os.makedirs(HF_MODULES_CACHE, exist_ok=True)
/nix/store/wsv9iz7p87a72blxkj24828vm4ibb9in-python3-3.8.6/lib/python3.8/os.py:213: in makedirs
    makedirs(head, exist_ok=exist_ok)
/nix/store/wsv9iz7p87a72blxkj24828vm4ibb9in-python3-3.8.6/lib/python3.8/os.py:213: in makedirs
    makedirs(head, exist_ok=exist_ok)
/nix/store/wsv9iz7p87a72blxkj24828vm4ibb9in-python3-3.8.6/lib/python3.8/os.py:213: in makedirs
    makedirs(head, exist_ok=exist_ok)
/nix/store/wsv9iz7p87a72blxkj24828vm4ibb9in-python3-3.8.6/lib/python3.8/os.py:223: in makedirs
    mkdir(name, mode)
E   PermissionError: [Errno 1] Operation not permitted: '/homeless-shelter'
=============================== warnings summary ===============================
tests/bindings/test_processors.py:150
  /private/tmp/nix-build-tokenizers-0.10.0.drv-0/source/bindings/python/tests/bindings/test_processors.py:150: DeprecationWarning: invalid escape sequence \(
    with pytest.raises(Exception, match="Missing SpecialToken\(s\) with id\(s\)"):

-- Docs: https://docs.pytest.org/en/stable/warnings.html
=========================== short test summary info ============================
ERROR tests/documentation/test_tutorial_train_from_iterators.py - PermissionE...
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
========================= 1 warning, 1 error in 2.78s ==========================
builder for '/nix/store/hkbpqdw9f0p77rwy4g1l2yg8in82jp5v-tokenizers-0.10.0.drv' failed with exit code 2
cannot build derivation '/nix/store/zix4mw1ar77xjrjdr66jsx4d8s37vaw0-python3.8-transformers-4.1.1.drv': 1 dependencies couldn't be built
error: build of '/nix/store/zix4mw1ar77xjrjdr66jsx4d8s37vaw0-python3.8-transformers-4.1.1.drv' failed

Result of nixpkgs-review pr 109532 run on x86_64-darwin 1

4 packages failed to build:

python37Packages.tokenizers
python37Packages.transformers
python38Packages.tokenizers
python38Packages.transformers

rootdir: /private/tmp/nix-build-tokenizers-0.10.0.drv-0/source/bindings/python
collected 117 items / 1 error / 116 selected

==================================== ERRORS ====================================
__ ERROR collecting tests/documentation/test_tutorial_train_from_iterators.py __
tests/documentation/test_tutorial_train_from_iterators.py:4: in <module>
    import datasets
/nix/store/janmd6006jr9kmxipkc22g0g09b9j0n7-python3.8-datasets-1.1.2/lib/python3.8/site-packages/datasets/__init__.py:27: in <module>
    from .arrow_dataset import Dataset, concatenate_datasets
/nix/store/janmd6006jr9kmxipkc22g0g09b9j0n7-python3.8-datasets-1.1.2/lib/python3.8/site-packages/datasets/arrow_dataset.py:40: in <module>
    from .arrow_reader import ArrowReader
/nix/store/janmd6006jr9kmxipkc22g0g09b9j0n7-python3.8-datasets-1.1.2/lib/python3.8/site-packages/datasets/arrow_reader.py:31: in <module>
    from .utils import cached_path, logging
/nix/store/janmd6006jr9kmxipkc22g0g09b9j0n7-python3.8-datasets-1.1.2/lib/python3.8/site-packages/datasets/utils/__init__.py:20: in <module>
    from .download_manager import DownloadManager, GenerateMode
/nix/store/janmd6006jr9kmxipkc22g0g09b9j0n7-python3.8-datasets-1.1.2/lib/python3.8/site-packages/datasets/utils/download_manager.py:25: in <module>
    from .file_utils import HF_DATASETS_CACHE, cached_path, get_from_cache, hash_url_to_filename
/nix/store/janmd6006jr9kmxipkc22g0g09b9j0n7-python3.8-datasets-1.1.2/lib/python3.8/site-packages/datasets/utils/file_utils.py:118: in <module>
    os.makedirs(HF_MODULES_CACHE, exist_ok=True)
/nix/store/wsv9iz7p87a72blxkj24828vm4ibb9in-python3-3.8.6/lib/python3.8/os.py:213: in makedirs
    makedirs(head, exist_ok=exist_ok)
/nix/store/wsv9iz7p87a72blxkj24828vm4ibb9in-python3-3.8.6/lib/python3.8/os.py:213: in makedirs
    makedirs(head, exist_ok=exist_ok)
/nix/store/wsv9iz7p87a72blxkj24828vm4ibb9in-python3-3.8.6/lib/python3.8/os.py:213: in makedirs
    makedirs(head, exist_ok=exist_ok)
/nix/store/wsv9iz7p87a72blxkj24828vm4ibb9in-python3-3.8.6/lib/python3.8/os.py:223: in makedirs
    mkdir(name, mode)
E   PermissionError: [Errno 1] Operation not permitted: '/homeless-shelter'
=============================== warnings summary ===============================
tests/bindings/test_processors.py:150
  /private/tmp/nix-build-tokenizers-0.10.0.drv-0/source/bindings/python/tests/bindings/test_processors.py:150: DeprecationWarning: invalid escape sequence \(
    with pytest.raises(Exception, match="Missing SpecialToken\(s\) with id\(s\)"):

-- Docs: https://docs.pytest.org/en/stable/warnings.html
=========================== short test summary info ============================
ERROR tests/documentation/test_tutorial_train_from_iterators.py - PermissionE...
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
========================= 1 warning, 1 error in 2.78s ==========================
builder for '/nix/store/hkbpqdw9f0p77rwy4g1l2yg8in82jp5v-tokenizers-0.10.0.drv' failed with exit code 2
cannot build derivation '/nix/store/zix4mw1ar77xjrjdr66jsx4d8s37vaw0-python3.8-transformers-4.1.1.drv': 1 dependencies couldn't be built
error: build of '/nix/store/zix4mw1ar77xjrjdr66jsx4d8s37vaw0-python3.8-transformers-4.1.1.drv' failed

this breaks python3Packages.transformers. Presumably because of overly restrictive pinning, but as maintainer you probably know better.

this breaks python3Packages.transformers. Presumably because of overly restrictive pinning, but as maintainer you probably know better.

this breaks python3Packages.transformers. Presumably because of overly restrictive pinning, but as maintainer you probably know better.

Indeed, transformers pins tokenizers to a specific version. We do relax the bounds in transformers, but it was too specific. Now also fixed as part of #109505 . Thanks!

this breaks python3Packages.transformers. Presumably because of overly restrictive pinning, but as maintainer you probably know better.

Indeed, transformers pins tokenizers to a specific version. We do relax the bounds in transformers, but it was too specific. Now also fixed as part of #109505 . Thanks!


fix: update hash script data

Is there a source on the change? Is this the case for old scripts?

I guess this is related to the change in redeemer structure from always being a flat array to also being a map, does the node assume if it's empty it uses the map structure then? The cddl seems to allow it being an array still (at least until the next era after conway). What happens if in the tx you use the empty legacy flat map structure?
redeemer_key = [ tag: redeemer_tag, index: uint ]

redeemer_val = [ data: plutus_data, ex_units: ex_units ]

legacy_redeemer = [ tag: redeemer_tag, index: uint, data: plutus_data, ex_units: ex_units ]

; Flat Array support is included for backwards compatibility and will be removed in the next era.
; It is recommended for tools to adopt using a Map instead of Array going forward.
redeemers =
  ; @name LegacyFlatFormat @doc This format will be deprecated in the next era
  [ + legacy_redeemer ] /
  ; @name MapFormat @doc New map format. Use this when possibe.
  { + redeemer_key => redeemer_val }

Currently, I encounter a PPViewHashesDontMatch issue when submitting the transaction with asHash datum in preprod.

@hadelive  I can't seem to access that channel. You don't have access to any text channels, or there are none in this server.
My point was does this mismatch always happen? Does it matter if your datums/redeemers are encoded differently in the tx? You can have an empty list be either one of those (empty list or empty map), or are they just missing entirely? Our API for the script data hash calc requires a list for redeemers (unlike datums). In the previous cbor spec for the data hash it used to say how to handle this exactly when there were none, as the old comments said. Is there anywhere we can refer to for the new behavior?
Does this apply to spending old (v1/v2) scripts?

I guess from this change
https://github.com/IntersectMBO/cardano-ledger/pull/3960/files#diff-53fd908a27e168b35772865938c6945d8bc528544a490d648859b02cadecdcdeR198-R207

here's one example
{
  "body": {
    "inputs": [
      {
        "transaction_id": "8e3d9138777c56c6e3fc1ce9083b13f91f5ce45da8ad7e80d4215b524af2bf8a",
        "index": 1
      }
    ],
    "outputs": [
      {
        "AlonzoFormatTxOut": {
          "address": "addr_test1qz0ugv82ruadcg8whwqnkfjfapwfxn49hsfa0dlmu2eyu5zsvjm8zc6dzn9c64p7w8wcadph53l0k3askg5g9pnvggxsvy935c",
          "amount": {
            "coin": 1327480,
            "multiasset": {
              "ef6ed47a6917a3cbbeb46561e8853da969343794d66128598a34af2c": {
                "4275726e61626c65546f6b656e32": 1
              }
            }
          },
          "datum_hash": "923918e403bf43c34b4ef6b48eb2ee04babed17320d8d1b9ff9ad086e86f44ec"
        }
      },
      {
        "AlonzoFormatTxOut": {
          "address": "addr_test1qz0ugv82ruadcg8whwqnkfjfapwfxn49hsfa0dlmu2eyu5zsvjm8zc6dzn9c64p7w8wcadph53l0k3askg5g9pnvggxsvy935c",
          "amount": {
            "coin":<PHONE_NUMBER>,
            "multiasset": {
              "22691d3d969ecf5802226290c2fb98e2bc08522d5b726c1f5f400105": {
                "54657374": 21
              },
              "cac67dd80f706e084b2aac605288b2ff793475ea43b2313e1ed384ab": {
                "54657374": 13,
                "4275726e61626c65546f6b656e506c75747573": 1,
                "accbfb633f637e3bb1abee40c9539d1effd742cd2716b3b1db9de3aaf3f37794": 1
              },
              "ef6ed47a6917a3cbbeb46561e8853da969343794d66128598a34af2c": {
                "4275726e61626c65546f6b656e32": 3
              }
            }
          },
          "datum_hash": null
        }
      }
    ],
    "fee": 202461,
    "ttl": null,
    "certs": null,
    "withdrawals": null,
    "auxiliary_data_hash": null,
    "validity_interval_start": null,
    "mint": {
      "ef6ed47a6917a3cbbeb46561e8853da969343794d66128598a34af2c": {
        "4275726e61626c65546f6b656e32": 3
      }
    },
    "script_data_hash": "2f50ea2546f8ce020ca45bfcf2abeb02ff18af2283466f888ae489184b3d2d39",
    "collateral_inputs": [
      {
        "transaction_id": "8e3d9138777c56c6e3fc1ce9083b13f91f5ce45da8ad7e80d4215b524af2bf8a",
        "index": 1
      }
    ],
    "required_signers": null,
    "network_id": null,
    "collateral_return": {
      "AlonzoFormatTxOut": {
        "address": "addr_test1qz0ugv82ruadcg8whwqnkfjfapwfxn49hsfa0dlmu2eyu5zsvjm8zc6dzn9c64p7w8wcadph53l0k3askg5g9pnvggxsvy935c",
        "amount": {
          "coin":<PHONE_NUMBER>,
          "multiasset": {
            "22691d3d969ecf5802226290c2fb98e2bc08522d5b726c1f5f400105": {
              "54657374": 21
            },
            "cac67dd80f706e084b2aac605288b2ff793475ea43b2313e1ed384ab": {
              "54657374": 13,
              "4275726e61626c65546f6b656e506c75747573": 1,
              "accbfb633f637e3bb1abee40c9539d1effd742cd2716b3b1db9de3aaf3f37794": 1
            },
            "ef6ed47a6917a3cbbeb46561e8853da969343794d66128598a34af2c": {
              "4275726e61626c65546f6b656e32": 1
            }
          }
        },
        "datum_hash": null
      }
    },
    "total_collateral": 5000000,
    "reference_inputs": null,
    "voting_procedures": null,
    "proposal_procedures": null,
    "current_treasury_value": null,
    "donation": null
  },
  "witness_set": {
    "vkeywitnesses": null,
    "native_scripts": [
      {
        "ScriptAll": {
          "native_scripts": [
            {
              "ScriptPubkey": {
                "ed25519_key_hash": "9fc430ea1f3adc20eebb813b2649e85c934ea5bc13d7b7fbe2b24e50"
              }
            }
          ]
        }
      }
    ],
    "bootstrap_witnesses": null,
    "plutus_v1_scripts": null,
    "plutus_datums": [
      {
        "constructor": 0,
        "fields": []
      }
    ],
    "redeemers": null,
    "plutus_v2_scripts": null,
    "plutus_v3_scripts": null
  },
  "is_valid": true,
  "auxiliary_data": null
}

cbor
84a800818258208e3d9138777c56c6e3fc1ce9083b13f91f5ce45da8ad7e80d4215b524af2bf8a010182835839009fc430ea1f3adc20eebb813b2649e85c934ea5bc13d7b7fbe2b24e505064b671634d14cb8d543e71dd8eb437a47efb47b0b22882866c420d821a00144178a1581cef6ed47a6917a3cbbeb46561e8853da969343794d66128598a34af2ca14e4275726e61626c65546f6b656e32015820923918e403bf43c34b4ef6b48eb2ee04babed17320d8d1b9ff9ad086e86f44ec825839009fc430ea1f3adc20eebb813b2649e85c934ea5bc13d7b7fbe2b24e505064b671634d14cb8d543e71dd8eb437a47efb47b0b22882866c420d821b000000017124c896a3581c22691d3d969ecf5802226290c2fb98e2bc08522d5b726c1f5f400105a1445465737415581ccac67dd80f706e084b2aac605288b2ff793475ea43b2313e1ed384aba3534275726e61626c65546f6b656e506c757475730144546573740d5820accbfb633f637e3bb1abee40c9539d1effd742cd2716b3b1db9de3aaf3f3779401581cef6ed47a6917a3cbbeb46561e8853da969343794d66128598a34af2ca14e4275726e61626c65546f6b656e3203021a000316dd09a1581cef6ed47a6917a3cbbeb46561e8853da969343794d66128598a34af2ca14e4275726e61626c65546f6b656e32030b58202f50ea2546f8ce020ca45bfcf2abeb02ff18af2283466f888ae489184b3d2d390d818258208e3d9138777c56c6e3fc1ce9083b13f91f5ce45da8ad7e80d4215b524af2bf8a0110825839009fc430ea1f3adc20eebb813b2649e85c934ea5bc13d7b7fbe2b24e505064b671634d14cb8d543e71dd8eb437a47efb47b0b22882866c420d821b0000000170efd5aba3581c22691d3d969ecf5802226290c2fb98e2bc08522d5b726c1f5f400105a1445465737415581ccac67dd80f706e084b2aac605288b2ff793475ea43b2313e1ed384aba3534275726e61626c65546f6b656e506c757475730144546573740d5820accbfb633f637e3bb1abee40c9539d1effd742cd2716b3b1db9de3aaf3f3779401581cef6ed47a6917a3cbbeb46561e8853da969343794d66128598a34af2ca14e4275726e61626c65546f6b656e3201111a004c4b40a201818201818200581c9fc430ea1f3adc20eebb813b2649e85c934ea5bc13d7b7fbe2b24e500481d87980f5f6

The error we get when paying to an address with datumhash
TxSubmitError: Error: {\"contents\":{\"contents\":{\"contents\":{\"era\":\"ShelleyBasedEraConway\",\"error\":[\"ConwayUtxowFailure (PPViewHashesDontMatch (SJust (SafeHash \"2f50ea2546f8ce020ca45bfcf2abeb02ff18af2283466f888ae489184b3d2d39\")) (SJust (SafeHash \"244926529564c04ffdea89005076a6b6aac5e4a2f38182cd48bfbc734b3be296\")))\",\"ConwayUtxowFailure (UtxoFailure (ValueNotConservedUTxO (MaryValue (Coin 0) (MultiAsset (fromList [(PolicyID {policyID = ScriptHash \"ef6ed47a6917a3cbbeb46561e8853da969343794d66128598a34af2c\"},fromList [(\"4275726e61626c65546f6b656e32\",3)])]))) (MaryValue (Coin<PHONE_NUMBER>) (MultiAsset (fromList [(PolicyID {policyID = ScriptHash \"22691d3d969ecf5802226290c2fb98e2bc08522d5b726c1f5f400105\"},fromList [(\"54657374\",21)]),(PolicyID {policyID = ScriptHash \"cac67dd80f706e084b2aac605288b2ff793475ea43b2313e1ed384ab\"},fromList [(\"4275726e61626c65546f6b656e506c75747573\",1),(\"54657374\",13),(\"accbfb633f637e3bb1abee40c9539d1effd742cd2716b3b1db9de3aaf3f37794\",1)]),(PolicyID {policyID = ScriptHash \"ef6ed47a6917a3cbbeb46561e8853da969343794d66128598a34af2c\"},fromList [(\"4275726e61626c65546f6b656e32\",4)])])))))\",\"ConwayUtxowFailure (UtxoFailure (BadInputsUTxO (fromList [TxIn (TxId {unTxId = SafeHash \"8e3d9138777c56c6e3fc1ce9083b13f91f5ce45da8ad7e80d4215b524af2bf8a\"}) (TxIx {unTxIx = 1})])))\"],\"kind\":\"ShelleyTxValidationError\"},\"tag\":\"TxValidationErrorInCardanoMode\"},\"tag\":\"TxCmdTxSubmitValidationError\"},\"tag\":\"TxSubmitFail\"}

@rooooooooob btw, could you check the WASM output? It doesn't reflect the changes made in the Rust code.

+1 for this patch, it fixed ConwayUtxowFailure PPViewHashesDontMatch for us.

I run npm run js:publish-nodejs:prod
Could you give me your discord handle?

could you check the WASM output? It doesn't reflect the changes made in the Rust code.

I'm not sure what you mean. For hash_script_data()? Did you re-run the wasm build scripts? The WASM code directly calls the rust code.

Yes, it's quite strange. I also tried using set_script_data_hash, but the hash isn't updated

could you check the WASM output? It doesn't reflect the changes made in the Rust code.

I'm not sure what you mean. For hash_script_data()? Did you re-run the wasm build scripts? The WASM code directly calls the rust code.

The wasm library doesn’t work as the rust library

I just set up a local WASM project with my locally built CML as a dependency and using this PR's changes I was able to get the new hash from WASM.
npm run js:publish-browser:prod was what I used to build it for WASM.
This was directly calling calc_script_data_hash(). The TX builder shouldn't be any different though since that is a direct API over the rust code too. All our WASM code is usually just 1-liners to call the rust one, sometimes with some conversion when the types need it (lots of .into()s/.clone()s/.map()s)

Could you give me your discord handle?

rooooooooob_
I'm going to merge this PR.

rooooooooob_

thanks, friend request sent.


DW Enhancements

Enhancements to DirectWrite. Feel free to test and let me know if we need to make any adjustments.

Legacy font naming parser.
Requires pyglet.options["dw_legacy_naming"] = True
Attempt to parse files by the passed name, to best match legacy RBIZ naming. For instance asking for things like "Arial Narrow" rather than "Arial" with a condensed stretch or "Arial Black" instead of "Arial" with a weight of black. This may enhance naming compatibility cross-platform for select fonts as older font renderers went by this naming scheme.
Starts by parsing the string for any known style names (which we can expand), and searches all font collections for a matching RBIZ name. If found, will go through the faces attempting to find the closest match to the passed styles. Will find the second closest match if a perfect match is not found.
Due to the high variation of styles and limited capability of some fonts, there is no guarantee the second closest match will be exactly what the user wants which is why this is an option. Any suggestions on improvement are welcome. Enabling debug_font option can provide information on what settings are being selected.

Filename property to a font
Will return 'Not Available' if the path cannot be queried (such as from custom fonts).

Also expands the render width to account for italics.
There is no way to get the actual width of an italic from glyph properties. So we use an estimate at the cost of additional texture memory. The only solution I have found online was to render it to a larger texture, then scan every pixel trying to find the furthest pixel to the right to get the width, which would be a lot slower.

Looks good to me. We could deprecate that pyglet option at some point in the future if it proves unnecessary.


feat: optional SSLKEYLOGFILE support

Motivation
Using SSLKEYLOGFILE is helpful when you want to intercept TLS traffic for debugging and is generally supported by many libraries and browsers. Also see: https://wiki.wireshark.org/TLS#using-the-pre-master-secret
Solution
Add a use_key_log option to server and client TLS configs that -- when set -- will enable rustls's SSLKEYLOGFILE handling.

Since this is mostly config wiring and relies on the already existing rustls features, does this need a test? If so, where should this test be placed?

Seems to be related to #893.

The solution there seems to be: just bypass tonic::transport entirely and write 100+ lines of glue-code to wire up all the components (tower, hyper, rustls) yourself. That seems doable, but somewhat defeats the purpose of tonic, i.e. having an easy-to-use client&server toolkit.


About transpose processing in MultiHeadedAttention class.

Below is the forward function of the MultiHeadedAttention class:
    def forward(self, query, key, value, mask=None):
        "Implements Figure 2"
        if mask is not None:
            # Same mask applied to all h heads.
            mask = mask.unsqueeze(1)
        nbatches = query.size(0)

        # 1) Do all the linear projections in batch from d_model => h x d_k
        query, key, value = [
            lin(x).view(nbatches, -1, self.h, self.d_k).transpose(1, 2)
            for lin, x in zip(self.linears, (query, key, value))
        ]

        # 2) Apply attention on all the projected vectors in batch.
        x, self.attn = attention(
            query, key, value, mask=mask, dropout=self.dropout
        )

        # 3) "Concat" using a view and apply a final linear.
        x = (
            x.transpose(1, 2)
            .contiguous()
            .view(nbatches, -1, self.h * self.d_k)
        )
        del query
        del key
        del value
        return self.linears[-1](x)

I notice  that the query, key, value is transposed (lin(x).view(nbatches, -1, self.h, self.d_k).transpose(1, 2)') after passing through the linear layers. After calculating the attention, x' is then transposed back (`x.transpose(1, 2)').
May I know why we need such processing? Can we just use `lin(x).view(nbatches, -1, self.h, self.d_k)' and
x =x.contiguous().view(nbatches, -1, self.h * self.d_k)?
I delete all the transposing processing and the result is different. So I am wondering which one is correct, the original one with transpose, or the one without transpose.

I have the same confusion with this code.

$h*d_k$ represents the embedding dimension $d_{model}$. And $h$ represents the number of head, embedding was divided into $h$ parts, when multiheads calculate Attention separately, $h$ should be localed Index 1.

Note that -1 represents the length $d_{toke}$(#token) of the longest sentence in a batch and the shape of the Attention matrix for each head is the same, $d_{toke} /times d_{toke}$.

Can we just use `lin(x).view(nbatches, -1, self.h, self.d_k)' and
x =x.contiguous().view(nbatches, -1, self.h * self.d_k)?

I don't think so.
With your suggestion the resulting shape of query/key/value will be
(batch_size, seq_len, h, d_k)
We have
scores = torch.matmul(query, key.transpose(-2, -1)) / math.sqrt(d_k)
in the attention function.
torch.matmul(query, key.transpose(-2, -1)) will then be the matrix multiplication of
(batch_size, seq_len, h, d_k) and (batch_size, seq_len, d_k, h). This doesn't denote a score.


Empty files when decoding

When using the decode_cnn.sh script (on my own dataset), the script finishes running, but the output files (bpe-test.txt and formatted-test.txt) are both empty. When I check the log file, I get this error:
Traceback (most recent call last):
  File "/home/lewis.2799/miniconda3/envs/QP2/bin/fairseq-generate", line 33, in <module>
    sys.exit(load_entry_point('fairseq', 'console_scripts', 'fairseq-generate')())
  File "/home/lewis.2799/fairseq/fairseq_cli/generate.py", line 392, in cli_main
    main(args)
  File "/home/lewis.2799/fairseq/fairseq_cli/generate.py", line 48, in main
    return _main(cfg, h)
  File "/home/lewis.2799/fairseq/fairseq_cli/generate.py", line 97, in _main
    models, saved_cfg = checkpoint_utils.load_model_ensemble(
  File "/home/lewis.2799/fairseq/fairseq/checkpoint_utils.py", line 262, in load_model_ensemble
    ensemble, args, _task = load_model_ensemble_and_task(
  File "/home/lewis.2799/fairseq/fairseq/checkpoint_utils.py", line 320, in load_model_ensemble_and_task
    model.load_state_dict(state["model"], strict=strict, model_cfg=cfg.model)
  File "/home/lewis.2799/fairseq/fairseq/models/fairseq_model.py", line 113, in load_state_dict
    self.upgrade_state_dict(state_dict)
  File "/home/lewis.2799/fairseq/fairseq/models/fairseq_model.py", line 119, in upgrade_state_dict
    self.upgrade_state_dict_named(state_dict, "")
  File "/home/lewis.2799/fairseq/fairseq/models/bart/model.py", line 185, in upgrade_state_dict_named
    num_classes = state_dict[
KeyError: 'classification_heads.contrast.out_proj.weight'

This appears to be related to the error described in this issue (not the initial post but a few comments down), but the author said that this error shouldn't be a problem.
I also printed the keys of the state_dict dictionary and several that are similar to the error key, though somewhat different:
classification_heads.contrast.dense.weight
classification_heads.contrast.dense.bias
classification_heads.contrast.out_proj.bias
classification_heads.contrast.out_proj.weight_orig
classification_heads.contrast.out_proj.weight_u
classification_heads.contrast.out_proj.weight_v

Wondering if there are any insights to why the generated files are empty? Thanks!!!

How long does it take for the script to finish running?

It finishes within about a minute.

I've been comparing my files with the ones provided to see if my formatting is the problem, and I'm slightly confused by the train.combine_target and val.combine_target files. The train file appears to be the raw text followed by the binarized version, but the val.combine_target appears to be the valid.target and the synthetic data val.raw_target combined. According to the command given on this page it seems that it should be the two raw files combined, right?
I don't think that's my main problem, but wanted to clarify!