Allow Bazel to be distro-packaged (Part 1 of many)

I'm working through packaging Bazel for Debian.  There are a lot of things that still need work, but these are the first few necessary changes that I was able to pop off into discrete patches.
I'm looking forward to your feedback.
Cheers,
Kyle Moffett

I am a Google employee, and don't need to sign a CLA.
On Mon, Apr 20, 2015 at 8:21 PM, googlebot<EMAIL_ADDRESS>wrote:

Thanks for your pull request. It looks like this may be your first
contribution to a Google open source project, in which case you'll need to
sign a Contributor License Agreement (CLA).
[image: :memo:] Please visit https://cla.developers.google.com/
https://cla.developers.google.com/ to sign.
Once you've signed, please reply here (e.g. I signed it!) and we'll
verify. Thanks.

If you've already signed a CLA, it's possible we don't have your
GitHub username or you're using a different email address. Check your
existing CLA data https://cla.developers.google.com/clas and verify
that your email is set on your git commits
https://help.github.com/articles/setting-your-email-in-git/.
If you signed the CLA as a corporation, please let us know the
company's name.

—
Reply to this email directly or view it on GitHub
https://github.com/google/bazel/pull/149#issuecomment-94626346.

Also you might want to look at https://github.com/google/bazel/commit/a2c5425bc9ac7078d483ef63bcf575c8ef8d66b7.
I don't think we can accept the one that drops the requirements to drop the requirements of the server jar to be the first one (and the BUILD is actually fixed, there is no need for it).

Odd that googlebot doesn't handle this case. Who can we ask about that?

you can fetch the repo, and then import the commit locally. The reason that it doesn't work remotely, is that stock git (as opposed to gerrit) doesn't let you fetch commits indexed by SHA1.

damienmg: Can you clarify why you can't take the one that drops the requirement to have the server jar be the first entry in the ZIP?
I'm trying to be able to build a debian-compliant packaging, and so I've got another commit to run from a pre-extracted root-owned install_base directory (EG: Files in /usr/lib/${ARCH}/bazel/_embedded_binaries).
Unfortunately, without having an actual ZIP, there is no "first entry"...

More precisely I think we can but it is a bit more complicate because we
don't have the same name for the library internally and externally. That is
why I just said "I think" Maybe Ulf has more insight for that one.
If you have a commit for running a pre-extracted root, then the launcher
would change and could find the library differently in that case, can't it?
Anyway it's best to discuss those complex one over gerrit (of if you prefer
to use critique, they it's ok too + it will be faster)
On Thu, Apr 23, 2015 at 6:58 PM Kyle Moffett<EMAIL_ADDRESS>wrote:

damienmg: Can you clarify why you can't take the one that drops the
requirement to have the server jar be the first entry in the ZIP?
I'm trying to be able to build a debian-compliant packaging, and so I've
got another commit to run from a pre-extracted root-owned install_base
directory (EG: Files in /usr/lib/${ARCH}/bazel/_embedded_binaries).
Unfortunately, without having an actual ZIP, there is no "first entry"...
—
Reply to this email directly or view it on GitHub
https://github.com/google/bazel/pull/149#issuecomment-95653064.

We can rename it internally. However, I still don't quite follow what
you're doing, Kyle. Would you mind giving a summary of the end state you're
aiming for?
On Thu, Apr 23, 2015 at 7:07 PM, Damien Martin-Guillerez <
<EMAIL_ADDRESS>wrote:

More precisely I think we can but it is a bit more complicate because we
don't have the same name for the library internally and externally. That is
why I just said "I think" Maybe Ulf has more insight for that one.
If you have a commit for running a pre-extracted root, then the launcher
would change and could find the library differently in that case, can't it?
Anyway it's best to discuss those complex one over gerrit (of if you prefer
to use critique, they it's ok too + it will be faster)
On Thu, Apr 23, 2015 at 6:58 PM Kyle Moffett<EMAIL_ADDRESS>wrote:

damienmg: Can you clarify why you can't take the one that drops the
requirement to have the server jar be the first entry in the ZIP?
I'm trying to be able to build a debian-compliant packaging, and so I've
got another commit to run from a pre-extracted root-owned install_base
directory (EG: Files in /usr/lib/${ARCH}/bazel/_embedded_binaries).
Unfortunately, without having an actual ZIP, there is no "first entry"...
—
Reply to this email directly or view it on GitHub
https://github.com/google/bazel/pull/149#issuecomment-95653064.

—
Reply to this email directly or view it on GitHub
https://github.com/google/bazel/pull/149#issuecomment-95655828.

ulfjack: Basically, the installed file tree right now (after a bunch of hacky patches that aren't ready for upstream yet) looks like this:
/usr/bin/bazel (just the C++ binary, no ZIP file appended)
/usr/share/bazel/base_workspace/tools/BUILD
/usr/share/bazel/base_workspace/tools/...
/usr/lib/x86_64-linux-gnu/bazel/_embedded_binaries/libblaze-main_deploy.jar
/usr/lib/x86_64-linux-gnu/bazel/_embedded_binaries/...
With this package configuration, "bazel" just trusts the file tree under /usr/lib/x86_64-linux-gnu/bazel/ rather than trying to unpack it on every launch.
(With the hacks I made to get everything relevant to builtin rules out of //third_party, the only directory you need to symlink from the base_workspace is //tools, and there are no architecture-dependent files there.)
If file versioning during package upgrades is considered a hermetic-build problem, the package version could even be included in the path, but I haven't thought super hard about whether or not I should be worried about that.

Oh, I've also been mulling over one other vague idea for how to handle cross-platform toolchains and cross-compilers on Debian.  Debian has "Multiarch" tuples, which are a slightly modified subset of GNU architecture triplets, EG: x86_64-linux-gnu/powerpc-linux-gnu/etc.
The installed compilers are all available under the corresponding "Multiarch" tuple, which is available during Debian package builds as $(DEB_BUILD_MULTIARCH) (the machine doing the build, AKA "--host_cpu" in blaze) and $(DEB_HOST_MULTIARCH). (the target machine, AKA "--cpu" in blaze)
I'm planning to use those raw triplets as the actual --host_cpu and --cpu arguments, and configure blaze's CROSSTOOL file with all of the supported compilers, etc.
Since Debian also has per-architecture JDKs that can be co-installed, I think the same strategy could work for JAVA_HOME, --java_cpu, and //tools/defaults:jdk too, but that was going to take a lot more integration work with jdk.WORKSPACE and jdk.BUILD.

I don't think you need to have the jar files in the embedded directory for
that. Just symlink the jar from whatever place you are putting it to its
place in tools/jdk. + If it's pure jar file it doesn't have to live in your
x86_64 directory because it is not architecture dependent.
You don't need to tweak jdk.WORKSPACE but i think we need to make WORKSPACE
file that can be used from the package path (i.e. Package path workspace
would be cat'd instead of overwritten). I have encountered that issue from
my auto configuration prototype also.
Le ven. 24 avr. 2015 à 07:48, Kyle Moffett<EMAIL_ADDRESS>a
écrit :

ulfjack: Basically, the installed file tree right now (after a bunch of
hacky patches that aren't ready for upstream yet) looks like this:
/usr/bin/bazel (just the C++ binary, no ZIP file appended)
/usr/share/bazel/base_workspace/tools/BUILD
/usr/share/bazel/base_workspace/tools/...
/usr/lib/x86_64-linux-gnu/bazel/_embedded_binaries/libblaze-main_deploy.jar
/usr/lib/x86_64-linux-gnu/bazel/_embedded_binaries/...
With this package configuration, "bazel" just trusts the file tree under
/usr/lib/x86_64-linux-gnu/bazel/ rather than trying to unpack it on every
launch.
(With the hacks I made to get everything relevant to builtin rules out of
//third_party, the only directory you need to symlink from the
base_workspace is //tools, and there are no architecture-dependent files
there.)
If file versioning during package upgrades is considered a
hermetic-build problem, the package version could even be included in the
path, but I haven't thought super hard about whether or not I should be
worried about that.
—
Reply to this email directly or view it on GitHub
https://github.com/google/bazel/pull/149#issuecomment-95808058.

Instead of symlinking /usr/share/bazel/base_workspace/tools/ into your
client, we were thinking about using --package_path to overlay it onto your
client.
How are you going to setup the classpath for Bazel and how can we ensure
that the server is restarted when the jars on the classpath change (say
after a new package is installed, including dependencies)?
On Fri, Apr 24, 2015 at 9:22 AM, Damien Martin-Guillerez <
<EMAIL_ADDRESS>wrote:

I don't think you need to have the jar files in the embedded directory for
that. Just symlink the jar from whatever place you are putting it to its
place in tools/jdk. + If it's pure jar file it doesn't have to live in your
x86_64 directory because it is not architecture dependent.
You don't need to tweak jdk.WORKSPACE but i think we need to make WORKSPACE
file that can be used from the package path (i.e. Package path workspace
would be cat'd instead of overwritten). I have encountered that issue from
my auto configuration prototype also.
Le ven. 24 avr. 2015 à 07:48, Kyle Moffett<EMAIL_ADDRESS>a
écrit :

ulfjack: Basically, the installed file tree right now (after a bunch of
hacky patches that aren't ready for upstream yet) looks like this:
/usr/bin/bazel (just the C++ binary, no ZIP file appended)
/usr/share/bazel/base_workspace/tools/BUILD
/usr/share/bazel/base_workspace/tools/...

/usr/lib/x86_64-linux-gnu/bazel/_embedded_binaries/libblaze-main_deploy.jar

/usr/lib/x86_64-linux-gnu/bazel/_embedded_binaries/...
With this package configuration, "bazel" just trusts the file tree under
/usr/lib/x86_64-linux-gnu/bazel/ rather than trying to unpack it on every
launch.
(With the hacks I made to get everything relevant to builtin rules out of
//third_party, the only directory you need to symlink from the
base_workspace is //tools, and there are no architecture-dependent files
there.)
If file versioning during package upgrades is considered a
hermetic-build problem, the package version could even be included in the
path, but I haven't thought super hard about whether or not I should be
worried about that.
—
Reply to this email directly or view it on GitHub
https://github.com/google/bazel/pull/149#issuecomment-95808058.

—
Reply to this email directly or view it on GitHub
https://github.com/google/bazel/pull/149#issuecomment-95832510.

ulfjack: Use of --package_path seems very reasonable to me (especially if the contents of //tools are carefully placed in subdirectories so you can override one piece without having to override any others).
RE: The Bazel classpath, I hadn't actually thought about that much, but it certainly could be tricky.  Depending on exactly what should trigger a relaunch, we may also need to check the bootclasspath too.
As a first pass, at build-time I'm thinking I will symlink all loaded jar files (possibly including rt.jar) into a directory for the bazel launcher to checksum; if the checksum is different it would relaunch.  All the JAR files in Debian set "Class-Path: ..." in their manifest to reference dependencies, so limiting the classpath to ensure that unexpected dependencies don't creep in won't work without writing a custom ClassLoader.

On Sun, Apr 26, 2015 at 7:19 PM, Kyle Moffett<EMAIL_ADDRESS>wrote:

ulfjack: Use of --package_path seems very reasonable to me (especially if
the contents of //tools are carefully placed in subdirectories so you can
override one piece without having to override any others).
RE: The Bazel classpath, I hadn't actually thought about that much, but it
certainly could be tricky. Depending on exactly what should trigger a
relaunch, we may also need to check the bootclasspath too.
As a first pass, at build-time I'm thinking I will symlink all loaded jar
files (possibly including rt.jar) into a directory for the bazel launcher
to checksum; if the checksum is different it would relaunch. All the JAR
files in Debian set "Class-Path: ..." in their manifest to reference
dependencies, so limiting the classpath to ensure that unexpected
dependencies don't creep in won't work without writing a custom ClassLoader.

I'm not worried about the bootclasspath - we're not handling that right
now, either. I'm more worried about someone installing a new version of -
say - apache_httpclient, to fix a security issue, and it doesn't get picked
up. On second thought, as long as we cover the 'core' bazel code and make
sure to restart if that changes in the first iteration, that might be good
enough.
We don't use an extensive amount of third party code, and I was thinking
that we should move some of that out of the bazel binary itself. For
example, the maven integration could call a locally installed maven instead
of doing it in-process (and if maven isn't installed, it can still shell
out to a simple maven-compat binary that only supports simple download kind
of things). Being able to use maven code in-process is a surprising
by-product of maven being written in Java - if it was written in Python,
we'd have to call something external anyway, just as for - say - PyPI.

—
Reply to this email directly or view it on GitHub
https://github.com/google/bazel/pull/149#issuecomment-96410273.

Closing this one. Follow-up discussion should go to bazel-dev@ I think


Raids: recommended items display

Adds option to show common recommended items for raid rooms in the scouting overlay

Can you provide a screenshot of what this looks like?

apologies, i thought i had one in there and never looked back to check

this was added in runelite/plugin-hub#417


add mixer_fill command and send responseCode and message to callback

Implementation to send MIXER FILL command to casparCG server

sorry I missed this pull request.
I added the MIXER FILL command a while back. If I missed something let me know.


feat(#141): Created better tests structure

GitHub Issue: https://github.com/mjovanc/njord/issues/141
Changes

Created the sqlite folder & split tests out into separate test files
Addressed all warnings within test files
Updated the README with a Testing section

Let me know what you think!
Testing
All tests:

All tests within sqlite folder:

A specific test within the sqlite folder:

Also add this to the contributors section in README.md:
[<img src="https://github.com/Hiccup-za.png?size=72" alt="Hiccup-za" width="72">](https://github.com/Hiccup-za)

Also, add "Testing" to the TOC.

LGTM!


Make sure no data is collected when nobody is logged in.

Currently data is collected never mind if someone is logged in or not. That is inefficient and need to be fixed.

Solved impilictly in #36


Quast HTML output

Quast is actually the one with the bad output.
Alex O found the issue a few weeks ago but we never got back to it.
I published a history. If you click on the "visualize" (eye) button on the HTML output of Quast, it's just printed not rendered:
https://htmltest.cloudve.org/initial/galaxy/u/alexm/h/quast-output
I added Quast to sanitize_whitelist.txt but it didn't fix it

To update this ticket, Quast reports seem to work fine when added to the sanitize whitelist locally.  My entry looks like:
toolshed.g2.bx.psu.edu/repos/iuc/quast/quast/5.0.2+galaxy1
Might be an issue in how this is propagated through the helm wizardry?

Figured out the issue as to why it wasn't propagating. Problem with changing configmaps that are mutable via Rancher/Cloudman is that it will change the readonly one but because the mutable is already there, it won't update it (-n flag in: https://github.com/galaxyproject/galaxy-helm/blob/master/galaxy/templates/_helpers.tpl#L80). We however can't have upgrades re-copy blindly, because if you had a change made from admin UI, it will be overwritten by the configmap upgrade.
I'd therefore suggest we remove mutable configs from Cloudman upgrade list, so users are not surprised by the change not taking effect, because the upgrade will not actually change the live version, just the configmap and not propagate to Galaxy. I can also add a note in the README of the chart then that mutable configs can only be changed via configmap the first time, after which they need to either be modified manually (i.e. exec into one of the handlers and manually change file on the NFS), or do it through UI.
Alternatively, we could add some merge logic to combine the files, although this might open a can of worms that we don't want to deal with rn imo.

Closing this, since it's not a Quast-specific issue, in favor of https://github.com/galaxyproject/galaxy-helm/issues/123


[7.17.3] xpack how to start eslactic with it

Hello,
Using default value my elastic cluster (3 nodes) start (and ready) without problem.
When I try to change and include xpack, It does'nt ready
my only change is :
esConfig:
elasticsearch.yml: |
xpack.security.enabled: true
with kubectl logs on a pod cluster seems not ok
{"type": "server", "timestamp": "2022-07-07T13:38:10,322Z", "level": "INFO", "component": "o.e.c.c.Coordinator", "cluster.name": "elasticsearch", "node.name": "elasticsearch-master-0", "message": "setting initial configuration to VotingConfiguration{DFjcKhFERjGgJvut3ZaKiA,{bootstrap-placeholder}-elasticsearch-master-2,Epb7iLHiSLukhHukKjc3Mg}" }
{"type": "server", "timestamp": "2022-07-07T13:38:11,020Z", "level": "INFO", "component": "o.e.c.s.MasterService", "cluster.name": "elasticsearch", "node.name": "elasticsearch-master-0", "message": "elected-as-master ([2] nodes joined)[{elasticsearch-master-1}{DFjcKhFERjGgJvut3ZaKiA}{zQExuF9fSs6W9roctQoLzQ}{<IP_ADDRESS>}{<IP_ADDRESS>:9300}{cdfhilmrstw} elect leader, {elasticsearch-master-0}{Epb7iLHiSLukhHukKjc3Mg}{eZ6dIk7iTzu6KGghYWYFGw}{<IP_ADDRESS>}{<IP_ADDRESS>:9300}{cdfhilmrstw} elect leader, BECOME_MASTER_TASK, FINISH_ELECTION], term: 1, version: 1, delta: master node changed {previous [], current [{elasticsearch-master-0}{Epb7iLHiSLukhHukKjc3Mg}{eZ6dIk7iTzu6KGghYWYFGw}{<IP_ADDRESS>}{<IP_ADDRESS>:9300}{cdfhilmrstw}]}, added {{elasticsearch-master-1}{DFjcKhFERjGgJvut3ZaKiA}{zQExuF9fSs6W9roctQoLzQ}{<IP_ADDRESS>}{<IP_ADDRESS>:9300}{cdfhilmrstw}}" }
{"type": "server", "timestamp": "2022-07-07T13:38:11,022Z", "level": "WARN", "component": "o.e.c.s.MasterService", "cluster.name": "elasticsearch", "node.name": "elasticsearch-master-0", "message": "failing [elected-as-master ([2] nodes joined)[{elasticsearch-master-1}{DFjcKhFERjGgJvut3ZaKiA}{zQExuF9fSs6W9roctQoLzQ}{<IP_ADDRESS>}{<IP_ADDRESS>:9300}{cdfhilmrstw} elect leader, {elasticsearch-master-0}{Epb7iLHiSLukhHukKjc3Mg}{eZ6dIk7iTzu6KGghYWYFGw}{<IP_ADDRESS>}{<IP_ADDRESS>:9300}{cdfhilmrstw} elect leader, BECOME_MASTER_TASK, FINISH_ELECTION]]: failed to commit cluster state version [1]",
"stacktrace": ["org.elasticsearch.cluster.coordination.FailedToCommitClusterStateException: node is no longer master for term 1 while handling publication",
What's happened ? how to enable xpack and only for a kibana gitlan authentication ?
best
bruno

Hi @shakenfr , if you enable xpack.security.enabled: true you also have to enable xpack.security.transport.ssl.enabled: true
By the way, I'm having a similar issue when trying to enable basic security in elasticsearch.


Bug: 2255499 [fusion-hci-4.14] Onboard clients only if they are on same version as provider - 2

manual cherry-pick of #2308

merge after #2349


[3.2] Describe ImageTexture, Image creation and usage

3.2 version of #42412.
ImageTexture.load() is not in 4.0, so marked as deprecated in documentation.
ImageTexture.set_data() is update() in 4.0.

Thanks!


Test Issue

[Enter feedback here]

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 9f071593-9440-5115-242d-488cf25e9256
Version Independent ID: 746b1c6e-1bcc-21fe-e2be-149e7c10f1d5
Content: Understanding the Teams store submission process - Teams
Content Source: msteams-platform/concepts/deploy-and-publish/appsource/publish.md
Product: msteams
GitHub Login: @laujan
Microsoft Alias: lajanuar

closing this test issue.


Need to rework render function to support classic templates too

Is your feature request related to a problem? Please describe.
render template function only supports components with defined custom rendering.
Describe the solution you'd like
It would be great to rework render function to be universal and make it a standard function for rendering components instead of explicit {{ template "ComponentExample" .ComponentExample }}

Function was excluded in latest refactor. Need to re-implement it

Feature described in https://kyoto.codes/features/alternative-rendering/


add additional geneva URLs

@alfantp : Thanks for your contribution! The author(s) have been notified to review your proposed change.

#sign-off


az login fails

This is autogenerated. Please review and update as needed.
Describe the bug
Command Name
az login
Errors:
To Reproduce:
Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.

Put any pre-requisite steps here...
az login

Expected Behavior
Environment Summary
Linux-5.4.0-70-generic-x86_64-with-debian-buster-sid
Python 3.7.3
Installer: PIP

azure-cli 2.3.1
azure-cli-ml 1.10.0

Extensions:
azure-cli-ml 1.10.0

Additional Context

@jiasli for awareness

Please provide the error message you see when az login fails.
Also, I noticed you are using a very old version 2.3.1. Please install the latest Azure CLI and see if the issue is gone.


[ENH] Add testing data

We can use https://openneuro.org/datasets/ds001600/versions/1.0.0 to get a pepolar, phasediff and 3 different versions of phase1/phase2. I'm checking on whether we can add a 4th version of phase1/phase2 from a 7T magnet (which is of course different from the other 3)

[ ] Create stand-alone archives containing one of each BIDS-compatible fieldmap type.
[ ] Upload somewhere public. UPenn Box or the osf server?
[ ] add to circleci download job

I might be misunderstanding the checklist, but if what you need is access to the OpenNeuro data on CircleCI, you can just use datalad:
datalad install ///openneuro/ds001600
datalad get ds001600/sub-1/fmap

As an example, I do that for preprocessed data in FitLins tests: https://github.com/poldracklab/fitlins/blob/5b3b248d7a4ffb1d771d8de44473dd166ccba039/.circleci/config.yml#L61-L92

That's very useful, I wasn't aware of the markiewicz/datalad:0.0.1 image.
I was thinking I would break these up into independent archives so that each is a valid BIDS dataset and has a single fieldmap in the fmaps directory. Otherwise the IntendedFor's will get messy

Oh, I see. We could probably still take advantage of datalad by pulling it down and creating git branches that remove all but one set of fieldmaps. It would need to be written so that it's idempotent when run on a pre-cached copy and no changes have been made upstream.
For reference, the image is just this. I created it because conda installs were taking more time than the data fetches.

Oh, looks like you can use git worktree to checkout branches into separate directories. That should make things relatively easy to work with.


Fixes #1 unit selection set state no longer using indices

Changed selectedRows state to selectedUnits, uses unique unit ID (${UnitScore.unit_code}${UnitScore.Season}) instead of index, to allow selected rows to persist after changing filters in main dashboard. Added "clear table" button to Comparison table to allow easier clearing of selected units.

Thank you for implementing this @rates37 !


tests: enable all tests for crun

the crun development version implementation got some changes to make it behave like runc and pass all the e2e tests.
Signed-off-by: Giuseppe Scrivano<EMAIL_ADDRESS>

Other than the setenforce, LGTM. Will let @rhatdan weigh in on that.

I'll drop that line once we have a new build

made a new release of crun: https://bodhi.fedoraproject.org/updates/FEDORA-2019-88a7ce090c

/lgtm
Then fix the setenforce 0 in a different PR.  Nice work @giuseppe


Add MR/PR Label Prefix Config Option

Is your feature request related to a problem? Please describe.
I'm using GitLab with scoped labels for my projects. As the suggestion is to name labels like scope::label I would like to be able to use the same style for my netlify-cms editorial workflow labels. At the moment the prefix for the editorial workflow labels is hard-coded to netlify-cms/.
Describe the solution you'd like
I would suggest a configuration option at least for the hosted Git Backends (GitLab, GitHub, Bitbucket) named label_prefix that is netlify-cms/ per default but can be adjusted.
Describe alternatives you've considered
Additional context

Thanks @knoppiks, if anyone would like to work on this see:
https://github.com/netlify/netlify-cms/blob/c33db099ac56edfd993de0212a8085d8d9243ce4/packages/netlify-cms-lib-util/src/APIUtils.ts#L5

Hi, what is the current status of this issue?

Not currently on the radar of anyone if you're looking to contribute 😄

Then I would like to work on this :)

I am thinking 2 possible options for the solution:

First option:
Passing the prefix to APIUtils.ts from top to bottom. More specifically, I can add a config option to config.yml for label prefix and pass this information for all method calls until it hits the APIUtils
Second option:
If I can find a way to access config.yml from APIUtils.ts, I can get this prefix easier.

I prefer the second option, what do you think?
If we will go with the second option, I need guidance for accessing config.yml. I see some packages access to the global config(here), should I do the same for this task?

I am thinking 2 possible options for the solution:

First option:
Passing the prefix to APIUtils.ts from top to bottom. More specifically, I can add a config option to config.yml for label prefix and pass this information for all method calls until it hits the APIUtils
Second option:
If I can find a way to access config.yml from APIUtils.ts, I can get this prefix easier.

I prefer the second option, what do you think?
If we will go with the second option, I need guidance for accessing config.yml. I see some packages access to the global config(here), should I do the same for this task?

Hi @KancerEzeroglu, I think you would need to go with the first option.
APIUtils.ts are just utility functions and should not have access to the entire config or should be able to read config.yml (regardless we also allow initialisation via JavaScript without a config.yml https://www.netlifycms.org/docs/beta-features/#manual-initialization).
The link you provided is for media libraries extensions which work similar to the backends (the config is passed to them).
You can see an example here:
https://github.com/netlify/netlify-cms/blob/277a7117a84f4141fbd4af94774c6806e9ff2648/packages/netlify-cms-backend-github/src/implementation.tsx#L113
Following the same pattern should work. WDTY?

I agree with you and working on this. fyi

Just for the update, I've finished the GitHub part. I need to make an extra one more test. After that, I'll send a PR. And after everything is fine, I'll do the same things for Bitbucket, GitLab, and so on.


High resolution pictures

Hi, could feature/option be added to keep the images in high resolution? KCC downscales them to the resolution of the kindle resolution. Keeping them in the high resolution allows for high detail and sharp image when zooming in on the comic/manga etc
For example, comics from one site are in 1988x3056, and after converting them to Kindle Voyage, the images in the mobi are only 941x1448 (since voyage has 1072x1448 screen)

@T0biasCZe
Isn't the option panel view: 4 panels enough for zoom? You can double tap the image corner and it will zoom.
You could also try the profile Other to disable scaling.

Well HQ setting made the file 1248x1920 instead of 941x1448, but thats only 132.60% increase in resolution. i dont think thats enough to loss lessly have 2x zoom (eg zooming each quarter of the screen)

Could you elaborate?
Why is it needed? What would you expect?
That usage seems quite specific.
Just a thought, another option would be to use a bulk script on your images with some tool like imagemagick before processing with KCC.

1920 is the maximum height supported by kindlegen.

Actually, you are getting quality loss since Kindlegen is downscaling to 1920 instead of KCC.

#500


micromamba not notarized on macOS

Solution to issue cannot be found in the documentation.

[X] I checked the documentation.

Issue
The macOS binary for micromamba is not notarized, so it is blocked from executing until overriding the security warning with admin privileges. I am installing micromamba via this homebrew cask, but I see that that in turn gets it binaries from this repository.
Installed packages
N/A

Environment info
macOS 12.6, Intel x86_64

@yyyasin19 and me are currently working on creating a Formula for homebrew-core. This way, homebrew can manage the signing of the executables.

In the meantime, brew install --cask micromamba --no-quarantine is your best option (or just pressing open on initial startup)

There now is a build of micromamba in homebrew/core. When you do brew uninstall micromamba, then update your brew taps and then brew install micromamba, this issue should be fixed @torfinnberset.

Considering this issue as fixed.
@torfinnberset: please do reopen it if this is not the case.


[WIP] Added more logging to the E2E test apps

This PR modifies all E2E test apps to:

Add more logging to all HTTP requests, by porting the middleware that was created before for one app to all, which injects a request ID if not found too.
HTTP servers log all TCP connections' changing states

Also fixes:

Some apps were still creating new HTTP clients. This makes them re-use the same client
Created a package for utils for E2E test apps so there's less code duplication around.

/ok-to-test

/ok-to-test

/ok-to-test

/ok-to-test

/ok-to-test

/ok-to-test

/ok-to-test

/ok-to-test

/ok-to-test

/ok-to-test

/ok-to-test


OpenClassrooms Bridge - No result

Describe the bug
The bridge OpenClassrooms does not show me any results, whatever the category.
Hello,To Reproduce
Steps to reproduce the behavior:

Go to OpenClassrooms Bridge
Select a category
Generate the rssfeed
See a blank page

Expected behavior
Have results
Desktop (please complete the following information):

OS: Windows 10
Browser : Firefox
Version : 94.0.1

Hi, @sebsauvage! Do you still maintain this bridge? If no, please leave reply not to bother you next time.

Hello.
No sorry, I'm not maintaining this bridge.
9 déc. 2021 02:03:24 Eugene Molotov @.***>:

Hi, @sebsauvage[https://github.com/sebsauvage]! Do you still maintain this bridge? If no, please leave reply not to bother you next time.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub[https://github.com/RSS-Bridge/rss-bridge/issues/2333#issuecomment-989359081], or unsubscribe[https://github.com/notifications/unsubscribe-auth/AAMSJDQGF5NDVEAE564FIADUP755VANCNFSM5IOQHXFA].
Triage notifications on the go with GitHub Mobile for iOS[https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675] or Android[https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub].  [data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAADAAAAAwCAYAAABXAvmHAAAAAXNSR0IArs4c6QAAAARzQklUCAgICHwIZIgAAAAfSURBVGiB7cEBAQAAAIIg/69uSEABAAAAAAAAAAAvBiQwAAGO68BnAAAAAElFTkSuQmCC###24x24:true###][Image de pistage][https://github.com/notifications/beacon/AAMSJDWCS7VESBZ7TUSBONTUP755VA5CNFSM5IOQHXFKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOHL4GX2I.gif]

Thank you for response!
To everyone else - feel free to make PR.

Their new html is annoying with random class identifiers.

Bridge is removed.


Module not found: Can't resolve 'perf_hooks'

I tried to integrate this ttlcache into a Next.js (Typescript) project and receive this error:

Module not found: Can't resolve 'perf_hooks' in '[...]/node_modules/@isaacs/ttlcache'

Steps To Reproduce

Create a fresh Next.js (Typescript) application

yarn create next-app my-app --ts

Install @isaacs/ttlcache package

yarn add @isaacs/ttlcache

Open _app.tsx and create an instance of TTLCache

import "../styles/globals.css";
import type { AppProps } from "next/app";
import TTLCache from "@isaacs/ttlcache";

const cache = new TTLCache({ max: 10, ttl: 1000 });

function MyApp({ Component, pageProps }: AppProps) {
  return <Component {...pageProps} />;
}

export default MyApp;

Run yarn dev, open http://localhost:3000 and observe

warn  -<EMAIL_ADDRESS>Module not found: Can't resolve 'perf_hooks' in '[...]/node_modules/@isaacs/ttlcache'

I closed this because this issue is Next.js specific issue. Node.js apis can be used in getServerSideProps/getStaticProps as discussed here: https://github.com/vercel/next.js/discussions/12607#discussioncomment-12358

Is it ok to make this library works seamlessly on both client-side and server-side, like providing an option for user to choose using only Date.now() 🤔

I am getting this issue in plain old CRA application as well.
Here are my details:

node v16.15.1
"@isaacs/ttlcache": "^1.2.0",
"react": "^18.2.0"

This should work in environments lacking perf_hooks now.
const maybeReqPerfHooks = fallback => {
  try {
    return require('perf_hooks').performance
  } catch (e) {
    return fallback
  }
}
const timeProvider = maybeReqPerfHooks(Date)
const now = () => timeProvider.now()

@MaheshVcsoft (or anyone else) if you are encountering this error, something else is going on, please post a new issue with the details of your setup.


Added new command clusterinfo to kubectl

This is first version of the command. It prints IPs of master and cluster
services. Should be improved once generalized labels are implemented #341.
It requires label kubernet.io/cluster-service=true set for cluster services.
Follow up cl after discussion in #4417.

Satnam, sorry for not continuing in #4417 but I had an issue with git.

To work correctly it requires #4620 to be completed.

I think #4620 is making good progress (for my part, labels for the cluster logging services went in yesterday). No worries about the git confusion -- I have plenty of sympathy.
When displaying information for a service that has an external load balancer why not also display its list of public IPs addresses? After all, in many situations, that's what the user needs to know in order to use the service?

Also, please could you be kind enough to paste some representative output into this conversation thread? Thank you kindly.

Sample output:
$ ./cluster/kubectl.sh clusterinfo
Kubernetes master is running at https://<IP_ADDRESS>
elasticsearch-logging is running at http://<IP_ADDRESS>
kibana-logging is running at http://<IP_ADDRESS>
kube-dns is running at http://<IP_ADDRESS>
heapster is running at http://<IP_ADDRESS>
influxGrafana is running at http://<IP_ADDRESS>

Also rebased with changes from #4620.

@satnam6502
As for now I'm displaying public IPs of nodes but not services just because some cluster services are not exposed publicly. According to what you've written do you mean:

displaying both services and pods
only services with public IP
services with public IP and those pods which services don't have public IP
or maybe sth else?

I mean, in addition to what you want to display, make sure that if a service has an external load balancer set, then print out the list of publicIPs (for GCE this is typically just one IP but our API uses a list of IPs for generality). i say this because -- at least for now -- a common idiom is "define a replication controller over a pod; define a service that operates over the pods controlled by the replication controller; specify an external load balancer for this service" so the user's view of how to use the service is in terms of the external IP address and port  (as set for the load balancer). I see discussion about how to deal with external IPs but for now (and perhaps up to V1.0?) this is likely to be prevalent.

Added displaying of ELBs if exist. Sample output:
$ ./cluster/kubectl.sh clusterinfo
Kubernetes master is running at https://<IP_ADDRESS>
Public IPs of machines on which cluster services run:
elasticsearch-logging is running at http://<IP_ADDRESS>
kibana-logging is running at http://<IP_ADDRESS>
kube-dns is running at http://<IP_ADDRESS>
heapster is running at http://<IP_ADDRESS>
influxGrafana is running at http://<IP_ADDRESS>
Public IPs of cluster services:
elasticsearch-logging is running at http://<IP_ADDRESS>:9200
kibana-logging is running at http://<IP_ADDRESS>:5601

Looks great, thanks. I worry about explaining the distinction between "Public IPs of machines on which cluster services run" vs. "Public IPs of cluster services"? As a user of a cluster, how does someone know to look at http://<IP_ADDRESS>:5601 for the Kibana dashboard from outside the cluster?

I agree that it's confusing. What do you suggest? Merge all entries to one category called "Public cluster IPs"? Or maybe two different categories "Public IPs of cluster services:" and "Public cluster IPs"? Or sth else?

When is it useful for someone to run the kubectl command and get a list of "Public IPs of machines on which cluster services run" which is helpful? I am sure it is useful -- I just don't know from my own use cases. Is there always a public IP address available for "Public IPs of machines on which cluster services run" for all providers?

Got it. Sorry for the confusion. As for now sample output is:
$ ./cluster/kubectl.sh clusterinfo
Kubernetes master is running at https://<IP_ADDRESS>
elasticsearch-logging is running at http://<IP_ADDRESS>:9200
kibana-logging is running at http://<IP_ADDRESS>:5601
kube-dns is running at http://<IP_ADDRESS>
heapster is running at http://<IP_ADDRESS>
influxGrafana is running at http://<IP_ADDRESS>
Ports of services not exported publicly are still missing. Is there any option to workaround it?
PTAL

friendly ping

@piosz: I discussed with @satnam6502 offline and we both felt that exposing the IP is misleading. I would suggest getting rid of IPs and proceeding with this PR.

I've not submitted PR #4959
Do we now need to report the IPs of the the monitoring and cluster level logging services?
Surely the user only needs to know things like:

https://<IP_ADDRESS>/api/v1beta1/proxy/services/monitoring-grafana/
https://<IP_ADDRESS>/api/v1beta1/proxy/services/elasticsearch-logging/
https://<IP_ADDRESS>/api/v1beta1/proxy/services/kibana-logging/
?
What do you think?
FYI @vishh @vmarmol

@satnam6502: +1 to printing the proxy path.

And apologies for the run-around as we changed cluster services to go via the proxy. I'll apologize again in person when I visit Warsaw in the coming months.

No worries. Definitely exposing proxy links is much better for users than exposing node IP. I'll update cl today.

Done. Sample output:
$ ./cluster/kubectl.sh clusterinfo
Kubernetes master is running at https://<IP_ADDRESS>
elasticsearch-logging is running at https://<IP_ADDRESS>/api/v1beta1/services/elasticsearch-logging?namespace=default
kibana-logging is running at https://<IP_ADDRESS>/api/v1beta1/services/kibana-logging?namespace=default
kube-dns is running at https://<IP_ADDRESS>/api/v1beta1/services/kube-dns?namespace=default

That's great, thank you. I did not expect the namespace=default but I guess in the long term it is more precise/safe.

In the same output I was also expecting to see Heapster and influxGrafana?

So we should add "kubernetes.io/cluster-service" label to them. I can do it (in a different CL).

Sorry, I missed the "proxy" in links, so links work now. Fixed that and also omitted namespace=default param (since now it is redundant and could be easily added in the future).

Kicked Travis and waiting for green...

Kicking Travis again. I think Travis might be failing due to gendocs issue?

Travis has failed several times like this:
/home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/docs/ is out of date. Please run hack/run-gendocs.sh
The command "PATH=$HOME/gopath/bin:./third_party/etcd:$PATH ./hack/verify-gendocs.sh" exited with 1.

Let me find out if this is an issue with Travis.

@thockin : suggestion for how to make Travis CI happy? Thanks.

LGTM. Sorry for confusion @piosz. This feature will be super useful!!

Generated docs.
Thank you guys for review and comments! Sorry that it lasted so long.


make/find/use template CONTRACT for external contractors

 Issue by Erkan-Yilmaz
Wednesday Feb 08, 2017 at 14:07 GMT
Originally opened as https://github.com/Erkan-Yilmaz/Gridcoin-tasks/issues/83

mentioned in hangout 8: with Coinomi experiences: delay, ...
the goal is that we really get for what we pay for and in time

if not: the contractor should pay a punishment, pay all back, ...

please share your templates, experiences, ...
thank you!

 Comment by Erkan-Yilmaz
Wednesday Jul 26, 2017 at 16:18 GMT

from IRC:

user1: whoa... going full legal with contract and stuff?
me: user1 at that time it was not clear if Coinomi would return anything. it seemed like lost money that was paid upfront

...

user2: and what entity would be paying them?
me: well, there were thoughts to make perhaps a non-profit, but that was on another matter, some months ago

Create a non-profit organization for Gridcoin?


[Event Request] table 83 Item Journal Line - OnBeforeRevaluationPerEntryAllowed

Dear ALAppExtensions team,
through this issue we would like to request adding event publisher OnBeforeRevaluationPerEntryAllowed to Table 83 Item Journal Line with the following signature:
[IntegrationEvent(false, false)]
local procedure OnBeforeRevaluationPerEntryAllowed(var Rec: Record "Item Journal Line"; var ValueEntry: Record "Value Entry"; var IsHandled: Boolean)
begin
end;

which would be called as follows in RevaluationPerEntryAllowed:
local procedure RevaluationPerEntryAllowed(ItemNo: Code[20]): Boolean
var
	ValueEntry: Record "Value Entry";
	// >>>> New >>>>>>>>>>>>>>>>
	IsHandled: Boolean;
	// <<<< New <<<<<<<<<<<<<<<<
begin
	// >>>> New >>>>>>>>>>>>>>>>
	IsHandled := false;
	OnBeforeRevaluationPerEntryAllowed(Rec, ValueEntry, IsHandled);
	if IsHandled then
		exit(true);
	// <<<< New <<<<<<<<<<<<<<<<
	GetItem;
	if Item."Costing Method" <> Item."Costing Method"::Average then
		exit(true);

	ValueEntry.SetRange("Item No.", ItemNo);
	ValueEntry.SetRange("Entry Type", ValueEntry."Entry Type"::Revaluation);
	ValueEntry.SetRange("Partial Revaluation", true);
	exit(ValueEntry.IsEmpty);
end;

Thanks in advance!

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.
Build ID: 49559.


[New game]: Trio Match

🎮 Game Request
This is a game of matching colors.
Point down the features

This is a game of matching colors. When three match up they will go away and you will get points. Only three will clear if more than three of the same color touch.
Click on a row and the entire row will move to the top of the column.
When the blocks are cleared the blocks on the right will move left in their own row.
The fewer the clicks made the more points you will gain.

Thank you for raising an issue 💗
You can self assign the issue by commenting /assign in comment 😀
Make sure you follow CODE OF CONDUCT and CONTRIBUTING GUIDELINES 🚀
Don’t Forget to ⭐ our GameZone🎮
Make sure you join our Discord🕹

Hey @sabhisharma-ise ! Thank you so much for your raising the issue💗
It’s all yours, you can come anytime again and make some contributions! 🚀
Alone, we can do little, but together we can do so much! 😇


Admin CLI Examples

Ref: #286
NOTE: set-airnode-parameters omitted as the README notes it should not be invoked

Closing because https://yargs.js.org/docs/ doesn't support example per command (only supports global examples).


tautomerRemoveIsotopicHs does not give the expected result

Configuration
RDKit version: 2021.03.5
OS: Windows 10
Python version: 3.9.7
Are you using conda? Yes
If you are using conda, which channel did you install the rdkit from? conda-forge
original_smiles = "CC=CO[2H]"
mol = Chem.MolFromSmiles(original_smiles)

tautomer_params = Chem.MolStandardize.rdMolStandardize.CleanupParameters()
tautomer_params.tautomerRemoveIsotopicHs = True

enumerator = Chem.MolStandardize.rdMolStandardize.TautomerEnumerator(tautomer_params)
mol_can = enumerator.Canonicalize(mol)

print(Chem.MolToSmiles(mol_can))

I would like to preserve deuteriums linked to atoms involved in tautomerism, but I noticed that the canonical tautomer is the same ([2H]O=CCC in this example) whether the tautomerRemoveIsotopicHs parameter is set to True or False.
By setting tautomerRemoveIsotopicHs = False, I would expect the result of tautomer canonicalization to be the same as the input in this case, is my assumption wrong?
Thanks,
Sonia

Hi Sonia, let me explain how this is supposed to work.
The TautomerEnumerator will only work sensibly if the input has no explicit Hs, i.e. no real H atoms in the graph, be it H, D or T.
This is true in many places across RDKit - many algorithm will break or will give unexpected results if real Hs are present in the input molecule.
This is the reason that prompted me to add the RemoveHsParameters.removeAndTrackIsotopes flag, which allows to remove all real Hs from a molecule and convert them to implicit, while keeping track of where H isotopes were originally located, such that AddHs can add them back at the original position.
Example:
from rdkit import Chem
import rdkit.Chem.MolStandardize.rdMolStandardize

original_smiles = "CC=CO[2H]"
mol = Chem.MolFromSmiles(original_smiles)
mol

params = Chem.RemoveHsParameters()
params.removeAndTrackIsotopes = True
mol = Chem.RemoveHs(mol, params)
mol

Now I will convert implicit Hs to real Hs, and then remove all non-isotopic Hs:
Chem.RemoveHs(Chem.AddHs(mol))

As expected, the isotopic Hs has been restored in the original position.
Let's now compute the canonical tautomer of this molecule with only implicit Hs:
tautomer_params = Chem.MolStandardize.rdMolStandardize.CleanupParameters()
tautomer_params.tautomerRemoveIsotopicHs = True
enumerator = Chem.MolStandardize.rdMolStandardize.TautomerEnumerator(tautomer_params)
mol_can = Chem.RemoveHs(Chem.AddHs(enumerator.Canonicalize(mol)))
print(Chem.MolToSmiles(mol_can))
CCC=O

Note that I will get the same result (i.e., no isotopic Hs) even if I set
tautomer_params.tautomerRemoveIsotopicHs = False

The reason is that since the canonical tautomer is different from the input tautomer, the assumption is that deuterium was exchanged with solvent during the tautomerization reaction and hence was removed.
However, if we repeat the experiment starting from the most stable deuterated tautomer, i.e. the canonical one, this time tautomer_params.tautomerRemoveIsotopicHs makes a difference:
d_propanaldehyde = Chem.MolFromSmiles("CC([2H])C=O")
d_propanaldehyde

d_propanaldehyde = Chem.RemoveHs(d_propanaldehyde, params)
d_propanaldehyde

tautomer_params = Chem.MolStandardize.rdMolStandardize.CleanupParameters()
tautomer_params.tautomerRemoveIsotopicHs = False
enumerator = Chem.MolStandardize.rdMolStandardize.TautomerEnumerator(tautomer_params)
d_propanaldehyde_can = Chem.RemoveHs(Chem.AddHs(enumerator.Canonicalize(d_propanaldehyde)))
print(Chem.MolToSmiles(d_propanaldehyde_can))
[2H]C(C)C=O
tautomer_params = Chem.MolStandardize.rdMolStandardize.CleanupParameters()
tautomer_params.tautomerRemoveIsotopicHs = True
enumerator = Chem.MolStandardize.rdMolStandardize.TautomerEnumerator(tautomer_params)
d_propanaldehyde_can = Chem.RemoveHs(Chem.AddHs(enumerator.Canonicalize(d_propanaldehyde)))
print(Chem.MolToSmiles(d_propanaldehyde_can))
CCC=O

The rationale is that in this case by setting tautomer_params.tautomerRemoveIsotopicHs = False we assume that no tautomerization reaction has actually taken place because the canonical tautomer is stable and does not exchange with solvent. This can be useful to avoid that deuterated molecules which are known to be stable to tautomerization are altered by the standardizer.
I hope this makes sense to you and clarifies the behaviour you observed.

Thanks for your quick reply @ptosco, your explanation makes sense from a chemical point of view and clarifies the output I am obtaining, but leaves me with more doubts on what I am trying to do.
I am creating a standardisation pipeline to compare compounds from different sources, and I would like to generate the canonical tautomer and merge compounds whose representation in different databases might refer to different tautomeric forms. By doing this, I don't want to lose the different stereochemistry (both sp2 and sp3) and isotopic composition of the atoms involved in tautomerism, which I thought could be done with the settings
tautomer_params.tautomerRemoveIsotopicHs = False
tautomer_params.tautomerRemoveSp3Stereo = False
tautomer_params.tautomerRemoveBondStereo = False

Based on your explanation, it seems I have two options:

lose D assuming it is exchanged with water during the tautomeric transformation;
retain D, and obtain the keto-tautomer with charged O.

Neither of these options is ideal, as the first one will lead me to wrongly merge compounds with different chemical structures in terms of isotopic composition, and the second will import a wrong structure (positively charged O with 3 bonds) into my library.
The ideal solution for me would be to revert the generation of the canonical tautomer if D (or T) are linked to atoms whose oxidation state changes as a result of tautomerisation, but I cannot think of a way to implement this (and now I am not sure if it makes sense at all!).
If you have any input on this it would be much appreciated


Localized bundle generation failed using localize: true option in angular.json

I got below error using localize: true in angular.json (ng build). When I set localize: false, works fine (surely it doesn't generate files for other languages). This is reproducible on a clean angular 14 project + localize + module federation plugin.
Localized bundle generation failed: Cannot assign to read only property 'message' of object 'SyntaxError: import.meta may appear only with 'sourceType: "module"' (1:4214)
An error occurred inlining file "main.316a7c5df42b3ef1.js"

I believe that this issue is originated to angular-cli. https://github.com/angular/angular-cli/issues/23008

I'm facing similar issue. I can not configure module federation with i18n e.g.: I would like to use en source locale instead of angular default en-US and build localized projects (en, de, pl etc.).
See the example with failed config: https://github.com/dkornel/module-federation-plugin-and-i18n-builds
@manfredsteyer , do you have any knowledge if module federation was designed to work with i18n ?


This isn't sufficiently documented

I just got a bunch of colorful LED strips and wanted to give them the HomeKit treatment with a bit of help from this guide, but halfway through I noticed it was absolute garbage, because it doesn't explain anything. Then I went here and found out that the readme doesn't, either.
My code looks pretty much like the one from the guide. After 8 hours of trying not to hurl my Raspberry Pi into the sun, my energy is pretty much gone, so I haven't been able to refactor that mess into something pretty yet.
Homebridge's output when moving the fancy sliders of Apple's Home app around is basically this:

[5/21/2017, 9:29:46 AM] [Farblicht] setPowerState() successfully set to ON
[5/21/2017, 9:29:47 AM] [Farblicht] setPowerState() successfully set to OFF
[5/21/2017, 9:30:05 AM] [Farblicht] Caching Hue as 0 ...
[5/21/2017, 9:30:05 AM] [Farblicht] _setRGB converting H:0 S:NaN B:100 to RGB:ffNaNNaN ...
[5/21/2017, 9:30:05 AM] [Farblicht] Caching Saturation as 100 ...
[5/21/2017, 9:30:05 AM] [Farblicht] _setRGB converting H:0 S:100 B:100 to RGB:ff0000 ...
[5/21/2017, 9:30:05 AM] [Farblicht] _setRGB converting H:0 S:100 B:100 to RGB:ff0000 ...
[5/21/2017, 9:30:05 AM] [Farblicht] ... _setRGB() successfully set to #ffNaNNaN
[5/21/2017, 9:30:05 AM] [Farblicht] setPowerState() successfully set to ON
[5/21/2017, 9:30:05 AM] [Farblicht] ... _setRGB() successfully set to #ff0000
[5/21/2017, 9:30:05 AM] [Farblicht] ... _setRGB() successfully set to #ff0000
[5/21/2017, 9:30:16 AM] [Farblicht] brightness is currently at NaN %
[5/21/2017, 9:30:16 AM] [Farblicht] ... hue is currently NaN
[5/21/2017, 9:30:16 AM] [Farblicht] power is currently OFF
[5/21/2017, 9:30:16 AM] [Farblicht] ... saturation is currently NaN
[5/21/2017, 9:30:30 AM] [Farblicht] power is currently OFF

I can set colors, but brightness, hue, and all the other fancy options seem to be completely unsupported.
Needless to say, this is unusable. Please provide documentation on what can go into an accessory object in config.json and what parameters and responses Homebridge expects. I suggest doing something like shairport-sync with its config file: Just list everything, put detailed descriptions behind everything and comment out the bits that aren't absolutely critical.
Just please, please don't make me read code to figure out how something works.
Thanks for your consideration.

Just for clarity, I should perhaps add that I'm writing a Flask application that runs on a Raspberry Pi and, when the right URLs are called, interacts with the attached LED strips.
My config.json looks like this:
{
	"bridge": {
		"name": "AmbiencePi",
		"username": "CC:22:3D:E3:CE:32",
		"port": 51826,
		"pin": "031-45-155"
	},
	"description": "Homebridge",
	"accessories": [
		{
			"accessory": "HTTP-RGB",
			"name": "Farblicht",

			"switch": {
				"status": "http://<IP_ADDRESS>:5000/mote/api/v1.0/status",
				"powerOn": "http://<IP_ADDRESS>:5000/mote/api/v1.0/on",
				"powerOff": "http://<IP_ADDRESS>:5000/mote/api/v1.0/off"
			},

			"brightness": {
				"status": "http://<IP_ADDRESS>:5000/mote/api/v1.0/brightness",
				"url": "http://<IP_ADDRESS>:5000/mote/api/v1.0/brightness/%s"
			},

			"color": {
				"status": "http://<IP_ADDRESS>:5000/mote/api/v1.0/set",
				"url": "http://<IP_ADDRESS>:5000/mote/api/v1.0/set/%s",
				"brightness": true
			}
		}
	],
	"platforms": []
}

The brightness part is just a guess, based on scripts by other people, but it's not even working.
I'll need that documentation to get anywhere with this.

You need a plugin to control them. I suggest homebridge-better-http-rgb

Instead of submitting an issue with an entitled "I want this so you need to do it" attitude, perhaps something more constructive? You realize this is an open source hobby project right? Feel free to submit a pull request with some documentation if you feel so inclined.

@GeorgeBgk Got it, thanks!
@rage8885 I won't even dignify that with a full answer.


Storage Volume Copy using in-memory pipe and migration logic

Updates Storage volume copy to use the migration logic with a bi-directional in-memory pipe.
Adds progress tracker indicator to storage migration functions so that when doing a local copy one can disable the progress tracker on one side.

Testsuite passed

Testsuite passed


Support the .spec pattern

At this moment this only works with my-test.test.js but it would be cool to also support .spec

fixed


windowsPb: Add missing when condition to MSVS_2019 install

[x] commit message has one of the standard prefixes
[ ] faq.md updated if appropriate
[ ] other documentation is changed or added (if applicable)
[ ] playbook changes run through VPC or QPC (if you have access)
[ ] VPC/QPC not applicable for this PR
[ ] for inventory.yml changes, bastillion/nagios/jenkins updated accordingly

The playbook will attempt to get a sha checksum of a file that is not present (because msvs2019 is already installed) causing the playbook to fail
"stderr": "Resolve-Path : Cannot find path 'C:\\TEMP\\VS2019_Layout.zip' because it does \r\nnot exist.

Works as expected https://awx2.adoptopenjdk.net/#/jobs/playbook/2087?job_search=page_size:20;order_by:-finished;not__launch_type:sync

ping @steelhead31 for review


[Chromium] Update version 94.0.4606.56 to 96.0.4664.45

Signed-off-by: GitHub<EMAIL_ADDRESS>

Tomorrow I'll push the update transitional to 96, and then to 98


Messages in preview window do not wrap correctly

Messages in preview window do not wrap correctly
Seems to happen universally.

Would it be possible for you to "export" the action and provide the file? It will not contain any data. I just want to take a look at the HTML that is extracted from your editor screen.

Sure, here's an example.
ontask_action_190221_141915.zip

This behavior applies for Canvas actions. Looking into it.

Fixed in release B.4.1


Buildpack works correctly as an online buildpack

As a PCF developer
I want to include the Conjur Buildpack by referencing its Github URL
Example:
https://github.com/cloudfoundry/java-buildpack#usage

See https://github.com/cloudfoundry/nodejs-buildpack/blob/master/bin/supply for an example of building the go binary during supply.


Project page is not updated correctly

Hi,
It seems that the project page is not updated correctly, there is not v10 projects, and some issues that are in v10:
(for example - https://github.com/lightningnetwork/lnd/pull/3601)
are shown in the v9 project page:
https://github.com/lightningnetwork/lnd/projects

There are two different kind of elements we use for development: milestones and projects.
Milestones are for more general, long-term planning/scheduling of issues and PRs. The project board is used for tracking the next upcoming version while it's in development.
Therefore it is correct that the milestone 0.10.0 already exists while the project 0.10.0 does not as long as 0.9.0 is not yet final (or at least in RC).


Hang after successfully saving the cache

After the cache has been successfully saved the action just sits there for a while and does nothing.
I assume this is related to the fallout of moving to Node 20 which causes a 2 minute hang in a lot of GitHub actions. See https://github.com/actions/toolkit/issues/1643 for more details.
I am using jurplel/install-qt-action@v3 and I have experienced this with ArchLinux and Fedora so far.

Looks like there's a workaround
https://github.com/irgaly/xcode-cache/pull/58
https://github.com/masci/datadog/pull/33

Should be resolved with latest changes. I'll make a release now-ish. Let me know if the issue comes back.


Add information from hspace and hbal

The so called Ganeti htools (hbal, hspace and others) can operate locally on the master but also remotely against RAPI endpoints. It would be good to collect and export more cluster data by running these tools periodically and parsing their output to e.g. predict how much instances can still fit into the cluster or to retrieve the cluster score (both current and target score).
These information could be used for graphing or alerting through Prometheus.
We actually already have Python code which runs and parses hspace for statistics purposes and would be happy to donate/incorporate that, if desired.
Two side notes: htools can (at least on Debian) be installed as a separate package. There is no need to install Ganeti itself to have access to hbal et al. Hence it is possible to have a dedicated system with prometheus-ganeti-exporter and htools which connect to a given cluster.
There is also one drawback: RAPI URL and credentials must be passed as command line parameters and will be visible in the process list.
Any ideas or comments on this?

We actually already have Python code which runs and parses hspace for statistics purposes and would be happy to donate/incorporate that, if desired.

That would be great!
Alternatively, I rolled my own script in https://github.com/ganeti/ganeti/issues/47#issuecomment-1730217846 which might be usable here, although it just calls the hbal binary directly (and sometimes fails at that too).

There is also one drawback: RAPI URL and credentials must be passed as command line parameters and will be visible in the process list.

Couldn't this be passed through the environment instead?

There is also one drawback: RAPI URL and credentials must be passed as command line parameters and will be visible in the process list.

Couldn't this be passed through the environment instead?

I think not. But I will double-check the (yay, Haskell :smile:) sourcecode. Environment variables would make things definitely less awkward.
In any case, I will try to open a PR with the code. We can still discuss/improve from there!


Specify Folder to Save PNG

Hi,
I've inserted your JS and the code to my website, but I haven't understand how to save the PNG to a specific folder..
I've inserted this JS, but they open an empty page.. Without signature..
And on the FTP I haven't find the signature file..!

Firma: <canvas id='signature-pad' class='signature-pad' width=400 height=125 style='border:1px solid black'></canvas>

<button id='save-png'>Save as PNG</button>

document.getElementById('save-png').addEventListener('click', function () {
  if (signaturePad.isEmpty()) {
    return alert("Please provide a signature first.");
  }

  var data = signaturePad.toDataURL('image/png');
  console.log(data);
  window.open(data);
});

The code you pasted only opens a new tab that displays the signature image. You can check the updated code of the demo app to see how to actually download the image: https://github.com/szimek/signature_pad/blob/master/example/js/app.js#L35-L48
However, you can't select a folder to save it to, this is handled natively by your browser and most likely all files will end up in somewhere in a Downloads folder, which location depends on your OS.

How to use the download function?
I've try this but not work..! :-(
          document.getElementById('save-png').addEventListener('click', function () {
            download('firme', 'firma1.png');
          });

Scuse me for my noobity

@LuigiMdg Just use the whole code from example folder - it works. Then modify it according to your needs and occasionally check if it still works ;)
If you check the code, you'll notice that the first parameter to the download function is data url that you get from SignaturePad#toDataURL() and the second is the name of the file.
If you have any questions related to JS language itself, please ask them e.g. at https://stackoverflow.com/.
Good luck! :)


Update go client to k8s 1.19

No regressions found when running automation test runs . Found this issue relating to bug in upstream 1.19- https://github.com/rancher/dashboard/issues/1220 .


port tests out of nodeunit

since there is no desire to support nodeunit in the future, see discussion in a PR that I had raised: caolan/nodeunit#356
I just saw tap2junit tool discussed in nodejs/admin#413, wonder if that tool could help us somehow?

Just pointing out that if we replace the existing unit testing framework for this repo, I think we would go with Jasmine.  For consistency purposes, all of our other repos use Jasmine already. It would probably be a major task to migrate to Jasmine.
This is just a comment, not voting in favor of one or the other without a deeper investigation.


Updated the depreacted link in deploy-mobile-apps-using-emm-provider.rst

Summary
Updated the depreacted link in deploy-mobile-apps-using-emm-provider.rst
Ticket Link

NA

Hello @Rutam21,
Thanks for your pull request! A Core Committer will review your pull request soon. For code contributions, you can learn more about the review process here.


[BUG][v2.9] Unable to use Rancher UI proxied Grafana to view most(?) dashboards

Rancher Server Setup

Rancher version: 2.9.0
Installation option (Docker install/Helm Chart): Helm

If Helm Chart, Kubernetes Cluster and version (RKE1, RKE2, k3s, EKS, etc):

104.1.0+up57.0.3
RKE2 v1.28.12+rke2r1

Proxy/Cert Details:

Information about the Cluster

Kubernetes version: v1.28.12+rke2r1 / v1.30.2+rke2r1
Cluster Type (Local/Downstream): both

If downstream, what type of cluster? (Custom/Imported or specify provider for Hosted/Infrastructure Provider): custom, rke2

User Information

What is the role of the user logged in? Admin

Describe the bug

When attempting to view graphs in Grafana that have been proxied through the Rancher UI, some end up erroring out in a funny way. Example shown in the below screenshot.

To Reproduce
Navigate to cluster dashboard. Select Monitoring on the left hand side. Then select Grafana. Open up a dashboard such as "Kubernetes / Compute Resources / Cluster".
Result
The browser window goes white, and the dashboard does not display as expected.
Expected Result
Dashboards are usable.
Screenshots
Attached above.
Additional context
None that I can think of at this moment.

Hiya @blovett - Can you share what version of the monitoring chart you are using here? I mostly want to confirm because there are 2 versions of charts that we shipped for rancher-monitoring in 2.9.0 - so I want to confirm which of the two this issue happens on for you? Or if it happens on both versions?

Hiya @blovett - Can you share what version of the monitoring chart you are using here? I mostly want to confirm because there are 2 versions of charts that we shipped for rancher-monitoring in 2.9.0 - so I want to confirm which of the two this issue happens on for you? Or if it happens on both versions?

Hi. This is with rancher-monitoring-104.1.0+up57.0.3. I have not tried the other version yet. Sorry for leaving this out.

I installed the other version (104.0.0+up45.31.1) on a test cluster and it does not exhibit this behavior. Upgrading it to 104.1.0+up57.0.3 and it does. Downgrading, it doesn't again.

@blovett - Greatly appreciate you clarifying the version and going as far as testing both!
We shipped two versions this release because we were upgrading the underlying components. So the issue seems to only exist in the newer version. From my quick test on my side it seems like the issue seems to be related to some missing or inaccessible web-assets. We'll look into this further and see about getting this fixed.

Digging more into this - just going to dump a stream of w/e logging I can find:
Here's the URL that is failing: /api/v1/namespaces/cattle-monitoring-system/services/http:rancher-monitoring-grafana:80/proxy/chunk.8.8aa451110aebe6a6cc4f.js
From grafana container logs on rancher-monitoring-grafana- pod:
logger=context userId=0 orgId=1 uname= t=2024-08-14T14:21:07.503318251Z level=info msg="Request Completed" method=GET path=/chunk.8.8aa451110aebe6a6cc4f.js status=404 remote_addr=<IP_ADDRESS> time_ms=1102 duration=1.102236101s size=45291 referer="/api/v1/namespaces/cattle-monitoring-system/services/http:rancher-monitoring-grafana:80/proxy/d/efa86fd1d0c121a26444b636a3f509a8/kubernetes-compute-resources-cluster?orgId=1&refresh=10s" handler=notfound

Upon doing a complete wip of site cache (via Application tab of devtools, "Clear Site Data") the UI loading issues no longer break the page visually. However the 404 for the JS file still happens and many graphs are incorrectly rendered. Of note - loading a different dashboard works fine.
I've determined the 404 isn't the root cause of this issue. As the issue happens on dashboards where the 404 does not appear. The issue is actually related to this error in the JS console:
Uncaught (in promise) Error: A listener indicated an asynchronous response by returning true, but the message channel closed before a response was received

This appears to be fixed in the pending rebase for 2.10 bumping to use 61.3.2 upstream.
I suspect that this will be backported to 2.9 and should resolve this issue at that time.

@mallardduck This issue is likely from here https://github.com/grafana/grafana/issues/76180
The fix is included in Grafana v10.3.4, and it worked for me...

@skanakal - I've tested and found we can upgrade to a higher version to fix as well. I have created initial PR to mirror the new versions we will need.

Still need a PR to patch existing affected chart versions to use new images.

Fix merged into dev-v2.9 branch under 104.1.2-rc.1+up57.0.3

Bug verification steps confirmed Grafana dashboards load correctly without issues with below version. The white screen issue appears resolved.
Details :
Rancher : v2.9.3-alpha5
cluster : v1.30.5+rke2r1
monitoring : 104.1.2-rc.1+up57.0.3
logging : 104.1.2-rc.1+up4.8.0
Steps to verify this bug:
Access Grafana via Rancher UI.
Select a dashboard, like "Kubernetes / Compute Resources / Cluster."
Observe for any white screen error on loading the dashboard.
Attached screen shot

Now it's changed to 104.1.2+up57.0.3 version


Errors after zipping file

This may be a configuration issue - but recieving the following errors running from start to GETCODE:
TEST-SPDX/tools-java: zipping into /lfScanningS3/scaffold/2021-10/code/TEST-SPDX/tools-java/spdx-2021-11-03.zip
TEST-SPDX/tools-java: skipping, WhiteSource is disabled
ERROR:root:Failed: could not retrieve folder number for parent folder Software Repository
TEST-SPDX/tools-java: Could not create folder TEST-SPDX
ERROR:root:Failed: could not retrieve folder number for parent folder Software Repository

The error is returned from FOSSDriver / FOSSOlogy.
I'm strongly suspecting a configuration issue with the FOSSOlogy driver.
The error originates from trying to get the "folder number" from Software Repository.
That folder clearly exists on the server.

As suspected, this was a configuration error - changed the username/password and the errors went away


Module build failed

I have installed storybook freshly using  npx -p @storybook/cli@alpha sb init. I expected the storybook should be opened on 'localhost:9009' when I run. But unfortunately it thrown a error message like ERROR in ./src/stories/index.js Module build failed (from ./node_modules/babel-loader/lib/index.js): SyntaxError: D:/repos/kool-kanya/src/stories/index.js: Unexpected token (9:55)
Steps to reproduce
Please provide necessary steps for reproduction of this issue. Describe the exact steps a maintainer has to take to make the problem occur. If the problem is non-trivial to reproduce, please link a repository or provide some code snippets.

Please specify which version of Storybook and optionally any affected addons that you're running

@storybook/react": "^4.0.0-alpha.25

.storybook/config.js
import { configure } from '@storybook/react';

function loadStories() {
  require('../src/stories');
}

configure(loadStories, module);

stories/index.js
import React from 'react';

import { storiesOf } from '@storybook/react';
import { action } from '@storybook/addon-actions';
import { linkTo } from '@storybook/addon-links';

import { Button, Welcome } from '@storybook/react/demo';

storiesOf('Welcome', module).add('to Storybook', () => <Welcome showApp={linkTo('Button')} />);

storiesOf('Button', module)
  .add('with text', () => <Button onClick={action('clicked')}>Hello Button</Button>)
  .add('with some emoji', () => (
    <Button onClick={action('clicked')}>
      <span role="img" aria-label="so cool">
        😀 😎 👍 💯
      </span>
    </Button>
  ));

Do you use babel 6?

@igor-dv yes, I am using Babel 6

Take a look at the MIGRATION guide here
LMK if this didn't work for you.

@igor-dv  Thanks, migrated to babel core 7 and its worked.


Replace KubeProxy Design Draft (Linux Only)

Replace KubeProxy Design Draft (Linux Only)
original google doc
Why We Should Do This
Since we will provide NodePort Service support in AntreaProxy recently, we will then have the ability to remove KubeProxy totally. This will save a lot of cycles and memories, and make it possible to have more control over Service traffic.
Items to be Resolved
We still need the following abilities to do the replacement.
Direct APIServer Access
The KubeProxy may not exist when we’re installing Antrea in the future, this will make Services not work during the installation. Meanwhile, Antrea needs to connect to the Kubernetes Service to watch resources. To overcome this issue, we need both AntreaAgent and AntreaController to be able to connect to the APIServer Pod/Proxy directly.

Node ClusterIP Service Support
The ability to serve ClusterIP Service on the Node. AntreaProxy does not have this ability yet.

Current						Expected
LoadBalancer Service Support
Service Health Check Support
Bootstrap Order Re-arrange
Since the AntreaProxy will take all charge of Service serving, Services will not be available until the first sync of AntreaProxy. Thus, we need to make sure all sub-components rely on Services to be waiting until AntreaProxy is ready.
Detail Design
Direct APIServer Access
Kubernetes Client in Both AntreaAgent and AntreaController
In the current implementation, we use the in-cluster kubeconfig to set up the connection of the Kubernetes client. In the replacement solution, we need to change the server address from the Service address to the IP address of the Kubernetes API Server. There is a PR #1735 for supporting this in Agent. We need a similar PR for the controller also.
Antrea Client for Watching Internal NetworkPolicies
The Antrea client in AntreaAgent uses the Antrea Service to watch internal NetworkPolicies. We make the Antrea client to be initialized after the first sync of AntreaProxy.
APIServer in Both AntreaAgent and AntreaController
The APIServers in Antrea components need to connect to the Kubernetes APIServer to retrieve the Secret. The Secret retrieve will only take one time and it will not happen once the APIServer is up. Since the retrieve implementation is hidden behind the Kubernetes library, we need to rewrite the function in Antrea and override the address of the Kubernetes Service.
Startup Pressure
Once we override the Kubernetes Service, we lose the LB of it. This may cause large pressure on the specified Kubernetes APIServer since the watcher will then keep using it to watch resources.To overcome this issue, we can make the AntreaProxy have the ability to notify if it has been synced. Once the Service is ready, we can then switch to use the Service to take the advantage of the LB. Meanwhile, we should also have a flag to control whether we need to do the switch, since users may use a custom out-of-cluster LB for serving Kubernetes APIServer.Another way to reduce the pressure is to accept multiple override Endpoints, each Antrea component can then randomly pick one Endpoint from the list to connect.
Host ClusterIP Access Support
We use IPTables and IP rules to make it, and one of the design goals is to make the runtime updates on the host as little as possible.Like the NodePort implementation, we use IPSet to match the cluster dress, protocol, and port.
Name: ANTREA-SVC-CLUSTER-SERVICE
Type: hash:ip,port
Revision: 5
Header: family inet hashsize 1024 maxelem 65536
Size in memory: 600
References: 2
Number of entries: 8
Members:
<IP_ADDRESS>,udp:53
<IP_ADDRESS>,tcp:8080

In the mangle table of the iptables, we make a custom chain that marks packets that match the IPSet with mark 0xf2. The chain should be referred to in both OUTPUT and PREROUTING chains.
Chain ANTREA-SVC-CLUSTER (2 references)
target     prot opt source               destination
MARK       all  --  <IP_ADDRESS>/0            <IP_ADDRESS>/0            match-set ANTREA-SVC-CLUSTER-SERVICE dst,dst MARK set 0xf2

In the nat table, we match the packet with 0xf2 mark and masquerade them.
Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination
ANTREA-SVC-MASQ  all  --  <IP_ADDRESS>/0            <IP_ADDRESS>/0

Chain ANTREA-SVC-MASQ (1 references)
target     prot opt source               destination
MASQUERADE  all  --  <IP_ADDRESS>/0            <IP_ADDRESS>/0            mark match 0xf2

To make packets with 0xf2 mark, we add an IP rule to make that packets go to the antrea route table.
from all fwmark 0xf2 lookup antrea

In the antrea route table, we only have one onlink default route which forwards all packets to the antrea-gw0.
default via <IP_ADDRESS> dev antrea-gw0 onlink

We also need to add flow in the pipeline to respond to the virtual IP address' ARP query.
table=20, priority=200,arp,arp_tpa=<IP_ADDRESS>,arp_op=1 \
actions=move:NXM_OF_ETH_SRC[]->NXM_OF_ETH_DST[], \
    set_field:aa:bb:cc:dd:ee:ff->eth_src, \
    load:0x2->NXM_OF_ARP_OP[], \
    move:NXM_NX_ARP_SHA[]->NXM_NX_ARP_THA[], \
    set_field:aa:bb:cc:dd:ee:ff->arp_sha, \
    move:NXM_OF_ARP_SPA[]->NXM_OF_ARP_TPA[], \
    set_field:<IP_ADDRESS>->arp_spa, \
    IN_PORT

Bootstrap Order Re-arrange
The bootstrap order of sub-components in AntreaController only rely on Kubernetes APIServer, thus we don’t need to care since we will use direct connection instead of the Service. We should take care of the bootstrap order of AnteaAgent because subcomponents in AntreaAgent rely on the antrea Service to set up the antrea client.The dependency map of sub-components in AntreaAgent looks like the diagram below.

According to the dependency map, the bootstrap order could be:

Installation
We can still follow the single YAML way to install Antrea, but users need to specify a Kubernetes APIServer Pod address in the config. Since Antrea will use insert operation instead of append option to the IPTables, once Antrea components start, KubeProxy rules will not be hit.
Upgrade
Since AntreaProxy will install IPTables rules before KubeProxy’s, users can just delete the KubeProxy deployment after the installation. There must be short connection breaks since the connection.
Antctl as an installation wizard
We can also install and update commands to Antctl to simplify the installation and upgrade.Instead of modifying a YAML file, only specifying the APIServer Endpoint by using Antctl should be easierAntctl can take responsibility for cleanup works, for e.g., remove KubeProxy rules, remove legacy rules or update API endpoints of Antrea.
KubeProxy Options Compatible
Consider Supporting in future

--azure-container-registry-config string

Path to the file containing Azure container registry configuration information.

--hostname-override string

If non-empty, will use this string as identification instead of the actual hostname.

--cluster-cidr string

The CIDR range of pods in the cluster. When configured, traffic sent to a Service cluster IP from outside this range will be masqueraded and traffic sent from pods to an external LoadBalancer IP will be directed to the respective cluster IP instead

Supported in Alternative Options

--healthz-bind-address ip:port   Default: <IP_ADDRESS>:10256

The IP address with port for the health check server to serve on (set to '<IP_ADDRESS>:10256' for all IPv4 interfaces and '[::]:10256' for all IPv6 interfaces). Set empty to disable.

--masquerade-all

If using the pure iptables proxy, SNAT all traffic sent via Service cluster IPs (this not commonly needed)

Implemented

--master string

The address of the Kubernetes API server (overrides any value in kubeconfig)

--nodeport-addresses stringSlice

A string slice of values which specify the addresses to use for NodePorts. Values may be valid IP blocks (e.g. <IP_ADDRESS>/24, <IP_ADDRESS>/32). The default empty string slice ([]) means to use all local addresses.

--profiling

If true enables profiling via web interface on /debug/pprof handler.

Open Items

How to redirect Service traffic to antrea-gw0 in other ways? Not
What the source IP address will be when accessing the Service IP:Port from localhost? If the source IP address is antrea-gw0's, the MASQUERADE action for Service traffic in POSTROUTING chain is not needed.
Whether Antrea Controller should connect to Kubernetes APIServer directly.

@hongliangl could you clarify what happens for Host ClusterIP access when the endpoints for the Service are not in the Pod Network (they can be host IPs, e.g. for the antrea Service, or even external IPs)? How do we guarantee that the return traffic goes through AntreaProxy in this case (for reverse translation). I think @ruicao93 brought this up at the community meeting yesterday, as he had a similar issue on Windows (https://github.com/vmware-tanzu/antrea/pull/1824).

Some update about open questions from community meetings:

Is the MASQUERADE action necessary? Yes, @hongliangl verified that the source address was selected based on the main route table before packets processed by netfilter where we match ipset and set mark to apply policy routing.
Can antrea-controller rely on AntreaProxy's service proxying instead of adding a configuration to connect kube-apiserver directly? Yes, @weiqiangt verified that antrea-controller can run well after antrea-agent sets up the necessary flows. Currently there's no explicit retry in antrea-controller to wait for services to be available. The ConfigMap.Get call in authentication.ApplyTo in createAPIServerConfig has 30 seconds timeout, I can see tcp retransmission during it:

13:56:22.436193 IP <IP_ADDRESS>.59076 > <IP_ADDRESS>.443: Flags [S], seq<PHONE_NUMBER>, win 64240, options [mss 1460,sackOK,TS val<PHONE_NUMBER> ecr 0,nop,wscale 7], length 0
13:56:23.446227 IP <IP_ADDRESS>.59076 > <IP_ADDRESS>.443: Flags [S], seq<PHONE_NUMBER>, win 64240, options [mss 1460,sackOK,TS val<PHONE_NUMBER> ecr 0,nop,wscale 7], length 0
13:56:25.462214 IP <IP_ADDRESS>.59076 > <IP_ADDRESS>.443: Flags [S], seq<PHONE_NUMBER>, win 64240, options [mss 1460,sackOK,TS val<PHONE_NUMBER> ecr 0,nop,wscale 7], length 0
13:56:29.718224 IP <IP_ADDRESS>.59076 > <IP_ADDRESS>.443: Flags [S], seq<PHONE_NUMBER>, win 64240, options [mss 1460,sackOK,TS val<PHONE_NUMBER> ecr 0,nop,wscale 7], length 0
13:56:37.910231 IP <IP_ADDRESS>.59076 > <IP_ADDRESS>.443: Flags [S], seq<PHONE_NUMBER>, win 64240, options [mss 1460,sackOK,TS val<PHONE_NUMBER> ecr 0,nop,wscale 7], length 0

And if the flows are not ready in 15 seconds (even though the timeout is 30 seconds, the backoff time is beyong the window), the call will return error and the program will exit and restart:
F0305 14:00:12.654096       1 main.go:59] Error running controller: error creating API server config: unable to load configmap based request-header-client-ca-file: Get "https://<IP_ADDRESS>:443/api/v1/namespaces/kube-system/configmaps/extension-apiserver-authentication": dial tcp <IP_ADDRESS>:443: i/o timeout
goroutine 1 [running]:
k8s.io/klog.stacks(0xc0003ca900, 0xc000db4000, 0x131, 0x1e9)
        /go/pkg/mod/k8s.io/klog@v1.0.0/klog.go:875 +0xb9
k8s.io/klog.(*loggingT).output(0x2b923a0, 0xc000000003, 0xc0005c7b20, 0x2ad03af, 0x7, 0x3b, 0x0)
        /go/pkg/mod/k8s.io/klog@v1.0.0/klog.go:826 +0x35f
k8s.io/klog.(*loggingT).printf(0x2b923a0, 0x3, 0x1d24f2b, 0x1c, 0xc000961d58, 0x1, 0x1)
        /go/pkg/mod/k8s.io/klog@v1.0.0/klog.go:707 +0x153
k8s.io/klog.Fatalf(...)
        /go/pkg/mod/k8s.io/klog@v1.0.0/klog.go:1276
main.newControllerCommand.func1(0xc0003ae280, 0xc000153a00, 0x0, 0x8)
        /antrea/cmd/antrea-controller/main.go:59 +0x1e7
github.com/spf13/cobra.(*Command).execute(0xc0003ae280, 0xc00004e0a0, 0x8, 0x8, 0xc0003ae280, 0xc00004e0a0)
        /go/pkg/mod/github.com/spf13/cobra@v0.0.5/command.go:830 +0x2c2
github.com/spf13/cobra.(*Command).ExecuteC(0xc0003ae280, 0x0, 0x0, 0x0)
        /go/pkg/mod/github.com/spf13/cobra@v0.0.5/command.go:914 +0x30b
github.com/spf13/cobra.(*Command).Execute(...)
        /go/pkg/mod/github.com/spf13/cobra@v0.0.5/command.go:864
main.main()
        /antrea/cmd/antrea-controller/main.go:38 +0x52

For the question @ruicao93 @antoninbas asked above, I think not only external endpoints, but also the noencap mode will have the issue that the reply packets will be handled by host directly and reset if we do nothing. I can imagine with current approach we will need to add mark to the connection and use policy routing for the reply packets to be delivered to OVS. Maybe we should take this scenario into consideration when comparing the current approach with TC.

@hongliangl could you clarify what happens for Host ClusterIP access when the endpoints for the Service are not in the Pod Network (they can be host IPs, e.g. for the antrea Service, or even external IPs)? How do we guarantee that the return traffic goes through AntreaProxy in this case (for reverse translation). I think @ruicao93 brought this up at the community meeting yesterday, as he had a similar issue on Windows (#1824).

The implementation for host IPs is a little complicated, and I'll explain it later in detail.

Again I feel we should come up with a single approach to redirect matched host traffic to OVS. Later we will that for Node security policies (but it can be different in that most Node traffic should be directed to OVS) too.

For endpoint in remote host network. E.g,

SRC: <IP_ADDRESS>:12345 DST: <IP_ADDRESS>:8080. In host network, packets will be marked 0x21 in mangle table PREROUTING chain.
SRC: <IP_ADDRESS>:12345 DST: <IP_ADDRESS>:8080. In host network, packets will be masqueraded matching mark 0x21 in POSTROUTING chain on host network after routing to antrea-gw0. <IP_ADDRESS> is the IP address of antrea-gw0.
SRC: <IP_ADDRESS>:12345 DST: <IP_ADDRESS>:8080. In OVS, packets' destination will be modified with DNAT.
SRC: <IP_ADDRESS>:12345 DST: <IP_ADDRESS>:8080. In OVS, packets' source will be modified with setting field method in OVS. If we don't modify source, the return traffic will be routed to antrea-gw0 eventually, not ovs pipeline.
SRC: <IP_ADDRESS>:23456 DST: <IP_ADDRESS>:8080. In host network, packets' will be masqueraded matching mark 0x21 in POSTROUTING chain on host network. If we don't do this, the return traffic cannot return to current host.

The traffic is sent to target endpoint finally. Now we handle the return traffic.

SRC: <IP_ADDRESS>:8080 DST: <IP_ADDRESS>:23456. In host network.
SRC: <IP_ADDRESS>:8080 DST: <IP_ADDRESS>:12345. In host network, we add an onlink route 'ip route add <IP_ADDRESS> via <IP_ADDRESS> dev antrea-gw0 onlink' on host network.
SRC: <IP_ADDRESS>:8080 DST: <IP_ADDRESS>:12345. In OVS, packets' dst will be modified with setting field method in OVS.
SRC: <IP_ADDRESS>:8080 DST: <IP_ADDRESS>:12345. In OVS, packets' source will be recovered because of DNAT.
SRC: <IP_ADDRESS>:8080 DST: <IP_ADDRESS>:12345. In host network, packets' destination will be recovered because of masquerade.

The key to implement NodePort is how to "redirect" the traffic of NodePort into OVS. The NodePort traffice can be from localhost or remote hosts.
Traffic from remote hosts
Environment
This is the test environment. router, endpoint are two network namespaces, and node is a VM. Note that, this is different with real Antrea environment, so there should be some changes when integrating this design with OVS.

Design

The destination of the NodePort traffic may be different IP addresses if the Kubernetes has multiple network adaptors. Here we take eth0 as an example, and assuming that the source IP address is <IP_ADDRESS>, source MAC address is 11:22:33:44:55:66.

Filter the traffic by the destination IP address and the destination port on eth0's ingress.

The destination IP address is the IP address of the network adaptor. Here is <IP_ADDRESS>.
The destination port is the port of NodePort. Here assuming that the port is 30000.
Traffic status:

SRC IP: <IP_ADDRESS>;   SRC MAC: 11:22:33:44:55:66
DST IP: <IP_ADDRESS>;   DST MAC: 00:00:92:68:02:11

"Steal" the traffic filtered by above conditions and put them to gw0's egress. Note that:

The source/destination MAC address is not changed, they should be modified before putting to gw0's egress. Assuming that the source MAC address is 11:22:33:44:55:66, and the destination MAC address is 00:00:92:68:02:11 (eth0's MAC address). In this test environment, the source/destination MAC address should changed to 00:00:10:10:10:01/00:00:10:10:10:02, otherwise , the traffic will be dropped at gw1.
OVS pipeline can also change the source/destination MAC address. So far, I don't integrate TC with OVS.
TC can also do stateless DNAT/SNAT, this is optional, I didn't use this feature for traffic from remotes hosts.
Traffic status:

SRC IP: <IP_ADDRESS>; MAC: 00:00:10:10:10:01; PORT: 56789
DST IP: <IP_ADDRESS>; MAC:  00:00:10:10:10:01; PORT: 30000

The traffic is sent by gw0 and arrives gw1. Here I add DNAT on the current network namespace. All traffic will be redirect to network namespace endpoint via DNAT.

Traffic status:
SRC IP: <IP_ADDRESS>; MAC: 00:00:20:20:20:02; PORT: 56789
DST IP: <IP_ADDRESS>; MAC: 00:00:20:20:20:01; PORT: 30000

The replied traffic arrives gw3 and unDNAT and is sent back via gw1

Traffic status:
SRC IP: <IP_ADDRESS>; MAC: 00:00:10:10:10:02; PORT: 30000
DST IP: <IP_ADDRESS>; MAC: 00:00:10:10:10:01; PORT: 56789

The traffic arrives the ingress of gw0, the traffic is filtered with the source IP address and the source port.

The source IP address is the IP address of the network adaptor where the traffic is from.
The source port is the port of NodePort.
There may be multiple filters on the ingress of gw0. Traffic is redirected to different network adaptors' egress according the source IP address.
The source/destination MAC address  of replied traffic should be modified before the traffic is redirected to target network adaptor's egress.
Traffic status:

SRC IP: <IP_ADDRESS>; MAC: 00:00:92:68:02:11; PORT: 30000
DST IP: <IP_ADDRESS>; MAC: 11:22:33:44:55:66; PORT: 56789

The traffic is sent back then.

Hash
From the process of above, we can see that the key of redirecting NodePort traffic is TC filter. By default, there is an default hash table that has only a hash bucket on each network adaptor's ingress and egress. Every hash bucket can be appended items in list. We can also add another hash table, however, there is an restrict that we can only create a hash with a maximum of 256 buckets, and the hashkey should be set when attaching filter to the hash table.
E.g,

the first command creates a hash table 1 with 256 buckets.
The second command creates a filter matching destination IP address <IP_ADDRESS>

TC u32 filter doesn't have any syntax sugar. hashkey mask 0x000000ff at 20 means that all matching traffic will be distributed to has buckets of hash table 1 according to the result of packet byte offset at 20 & 0x000000ff.
In another word, the offset 20 of packet is TCP source port and destination port.
Why I don't use mask 0x0000ffff, that means matching the full destination as hashkey, however, this will get the same result with using mask 0x000000ff.

tc filter add dev eth0 parent ffff:0 protocol ip prio 99 handle 1: u32 divisor 256
tc filter add dev eth0 parent ffff:0 protocol ip prio 99 u32 link 1: hashkey mask 0x000000ff at 20 match ip dst <IP_ADDRESS>

I have two design of filtering NodePort traffic.

Hello,  @antoninbas @jianjuns @tnqn, I have post the new design and test result.

Hi @hongliangl , I do not understand the following:

The traffic is sent by gw0 and arrives gw1. Here I add DNAT on the current network namespace. All traffic will be redirect to network namespace endpoint via DNAT.
Traffic status:
SRC IP: <IP_ADDRESS>; MAC: 00:00:20:20:20:02; PORT: 56789
DST IP: <IP_ADDRESS>; MAC: 00:00:20:20:20:01; PORT: 30000

The replied traffic arrives gw3 and unDNAT and is sent back via gw1
Traffic status:
SRC IP: <IP_ADDRESS>; MAC: 00:00:10:10:10:02; PORT: 30000
DST IP: <IP_ADDRESS>; MAC: 00:00:10:10:10:01; PORT: 56789

What is gw1 and what is gw3?
"add DNAT on the current network namespace": how you do DNAT? OVS or iptables or TC?

Hi @jianjuns,

What is gw1 and what is gw3?
gw1 and gw3 are two veths. The picture below is the test environment. router and endpoint are network namespaces.  The configuration of them has nothing related with the TC design.

"add DNAT on the current network namespace": how you do DNAT? OVS or iptables or TC?
I do DNAT with iptables here.

@hongliangl should this issue be closed?


different results for some st_equals for arctern and postgis

in arctern the results for the following specific data are all false:
select st_equals_udf(left, right) as geos from test_equals
in postgis, these sqls results are all true
select st_equals('LINESTRING (0 0, 10 10)'::geometry, 'LINESTRING (0 0, 5 5, 10 10)'::geometry);
select st_equals('LINESTRING (10 10, 0 0)'::geometry, 'LINESTRING (0 0, 5 5, 10 10)'::geometry);
select st_equals('LINESTRING(0 0, 1 1)'::geometry, 'LINESTRING(1 1, 0 0)'::geometry);

this is fixed and verified


Suffolk lit lab into screen needs to be reworded

Court leadership would like the intro screen with the id: basic questions intro screen to be reworded as follows:
Step 1. Answer questions that will fill in your form for you.
Step 2. Preview the completed form.

Step 3. Download and print the final form. You will need to deliver it to the court and to the Plaintiff or their lawyer on your own.

Changes made to the Intro Screen and approved by Trial Court.


Fix example config for "max" in readme

This PR fixes the example config in readme which, when copied, does not render properly.

thanks so much for the first PR in this repo! 🥳


DBZ-2522 Fixing index out of bounds for columns excluded from cdc tables

https://issues.redhat.com/browse/DBZ-2522
Updated the data array in the IndicesMapping to match the source column width. This prevents index-out-of-bounds exceptions when you exclude columns in the middle of the table.
Example: A source column looking like [0,1,2,3,4], and the cdc table with [1,4]; the resulting data column for us should be [null, 1, null, null, 4] while the previous implementation would only have an array of size 2 and error out when placing the '4'.

I realize now this commit removed this change to fix tests.
https://github.com/debezium/debezium/pull/1758/commits/dd48172608b604648c586399550245e0cc75bf91
I think the issue is the test themselves, and not the array size. Digging into it

Updated the test to try a different failure condition around data type processing.
There were other failures, but they seemed unrelated to what I changed and might be flakey tests. Will let them rerun with the new commit.

It looks like only unrelated tests are intermittently failing now. If I need to increase the stability of these to advance this PR, just let me know and I can try to find time to dig into them.

LGTM now. The failing test is a known flaky one (tracked as https://issues.redhat.com/browse/DBZ-2344). Thanks, @jgormley61!

@jgormley6 LGTM thanks a lot. I'd like just ask you for two changes

Could you please not use column blacklist name but the new naming?
Could you make a copy of your current test and make it opposite - so instead of using column exclude list use column inlcude list?
The test and behaviour shloud be the same just different config.

I believe I changed all occurrences in my changes to use the new terminology. Should I be updating other existing test to use the new terms as well?
I can add the test Sadly my internet is out this morning (using my phone for this), but I'll try to have that test up asap

Yes please, SqlServerConnectorConfig.COLUMN_BLACKLIST etc. will be removed in future so it is better to keep to the new names in new code.

Oh sorry, I see, I missed the import for the config key. Didn't realize it. I'll fix that.
I quickly wrote the test, but its failing.
It looks to me like COLUMN_INCLUDE_LIST config value is not being currently handled at all.
I can try fixing it as part of this PR or we should create another story to implement it for SQL Server. Currently it looks like only postgres supports it, so would we also need to implement it for mysql (I'm not familiar with any connectors other than the SQL server one)?

It looks like the documentation for SQL server doesn't include column.whitelist or column.include.list either.

I updated to handle the include list for sql server the same way postgres handles it (include list and exclude list can't both be used and include list being the priority). Let me know if there's anything more here I should do for that.

@jgormley6 you are way too fast :-). I intended to tell you it is not necessary to do that but obviously I am late to the game.
WHE the CI passes I'd like to ask you for two three sentences about this feature in SQL Server docs either as part of this PR or another one. Otherwise the code is good to go!

Updated the docs for the include list. I haven't worked with .adoc before so let me know if there's anything fishy about the formatting

Rebased to the latest master and added contibutor to authors - https://github.com/debezium/debezium/pull/1812

Thanks a lot @jgormley6! Re flaky tests, yeah, that's a sore point atm. We're keeping track of them and work to get this sorted out, but it takes some time...


[TT-629] Poll Instant Transactions

Also handles a side case where instant chains refuse to give transaction receipts.

Codecov Report
Attention: 68 lines in your changes are missing coverage. Please review.

Comparison is base (0d70b71) 22.53% compared to head (b9b778b) 20.83%.
Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #735      +/-   ##
==========================================
- Coverage   22.53%   20.83%   -1.70%
==========================================
  Files          28       31       +3
  Lines        3790     4070     +280
==========================================
- Hits          854      848       -6
- Misses       2852     3137     +285
- Partials       84       85       +1

Files
Coverage Δ

blockchain/celo.go
0.00% <ø> (ø)

blockchain/nonce_settings.go
0.00% <0.00%> (ø)

blockchain/ethereum.go
0.00% <0.00%> (ø)

blockchain/transaction_confirmers.go
0.00% <0.00%> (ø)

... and 4 files with indirect coverage changes

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

@smickovskid can you please review and merge if ok ?

Bump on this one, @davidcauchi @smickovskid


Generated changelog and releases don't contain contributor names

The changelog and releases generated by release-plz doesn't include the contributor name or a link to the PRs. This can be added by introducing a release-plz config file.

The #1234 is already included in the commit message when a PR is closed, which creates a link to the PR when viewed on GitHub.


Unknown command app when compiling

I think this is a bug. When I try to compile my project it complains that there isn't a command named "app", but it builds the app anyway. Also, the error doesn't appear when I ask for "app tarball". I am using incremental=false and filter_stdlibs=true with Comonicon v0.12.1,  PackageCompiler v2.0.3 and Julia v1.7

do you have the Comonicon.toml file available somewhere so that I could try the same configuration. it looks like something wrong with the installation procedure.

name = "cadCAD"

[install]
completion = false
quiet = false
optimize = 2

[application]
incremental = false
filter_stdlibs = true

[application.precompile]
execution_file = ["deps/precompile.jl"]

[download]
host = "github.com"
user = "cadCAD-org"
repo = "cadCAD.jl"