Create app archetypes

Apart from the "kitchen sink" app that is the touchstone starter. I think it would be helpful for people if we were to demonstrate an implementation of real-world apps. I suggest:
Social
Posts, user details, starring, search, notifications
Shop
Product tiles, product details, simulated in-app purchase, search, reviews
Photos
Albums, album details, photo details, selection / deletion
Messaging
Message list, message details, searching / removing, sending message
Ping @JedWatson @dcousens thoughts?

I think this would be incredibly useful.

Awesome! Which shall we start with?

IMHO, in order of most familiar to "lots of dev required",  I'd rank them:

Social
Messaging
Shop
Photos

If we wanted to maximise the diversity of how to use Touchstone,  we'd probably want to go bottom up.   If we wanted to get as much done as quickly as possible,  I'd start top down.

Just throwing in a thought on this:
Keystone will need all of those apps as well.
Maybe it would make sense to develop them together from the ground up?
This might later also make it easier to port any keystone app to touchstone. It could even pace the way for a ks/ts app  interoperability standard

What might be more useful initially is an actual bare-bones boilerplate starter app - as @jossmac said, this should really be called "kitchen sink".

@mbrookes so you think we should make more repositories?  That sounds like a nightmare to maintain.   Personally I think it'd be fine to just use touchstonejs-starter for these demonstrations.

@dcousens no, what I'm saying is that there needs to be a starter app with just enough skeleton code to get going, without having to take this kitchen sink demo and strip it down first. Demos are interesting, but a starter app would be more useful initially.
Think rails new <name> or reapp new <name> or even just git clone ...
touchstonejs-starter is a kitchen-sink demo, not a starter. It's great, but it's misnamed.

@mbrookes in short,  you think we should provide 'templates' as such?

I agree the repo is misnamed, this is really more touchstone-demo (never been a big fan of the term "kitchen sink" personally). Also, it's actually the project we host on the touchstone site as the demo.
:+1: for better ks/ts app interoperability, and the concept of aligned demos.
Probably, instead of a touchstone starter (bare-bones) repo, I think we should go down the generator route. You want the minimum amount of functionality in a generator that's still going to be useful, because they are a lot of overhead to maintain. But it would be a much better way to get up and running.
We use Yeoman for Keystone, I don't mind doing the same here because it gives us the infrastructure out of the box, I'd love a standalone CLI tool as well though if anyone wants to step up and put it together (not hard but I don't have time and would prefer to focus on documentation / guides).
Would be great to put that in touchstone itself so you can run touchstonejs new <projectname> and off it goes!
When / if we end up with templates down the track, we could potentially download them from the CLI - imagine something like:
$ touchstonejs new my-project
Welcome to TouchstoneJS! We'll create you a new project.
Which template would you like to use?
[x] Blank
[ ] Demo (see demo.touchstonejs.io)
[ ] Social
[ ] Shop
[ ] Photos
[ ] Messaging
Great, we'll create you a new Blank project.
Would you like to create a new directory? [Y/n] Y
Creating my-project...
(npm install)
(cordova install)
You're ready to start!
Type "cd my-project" then "touchstonejs start" to run the development server.
Type "touchstonejs help" for more information, including how to build and deploy your app.

A generator would certainly be ideal. :ok_hand:   - whatever's easiest for for you guys a quick win but that's maintainable.
Really like the cli / templates approach see ionic getting started for similar, but appreciate that's more work, and there are other things you guys are focused on. Can't wait for docs :+1: - part of the reason I'm struggling dissecting the current starter as a relative beginner.
PS. Agree with you on "kitchen sink", I was just using it to reinforce the distinction between demo and starter.
Thanks for all the hard work guys - Touchstone is going to be amazing!


4.0.0.beta1 unknown attribute transaction_id

After upgrading to 4.0.0.beta1 my tests fail with:
ActiveRecord::UnknownAttributeError: unknown attribute 'transaction_id' for PaperTrail::Version.

I see that new migrations have been added to the generators:
lib/generators/paper_trail/templates/add_transaction_id_column_to_versions.rb
lib/generators/paper_trail/templates/create_version_associations.rb

The only instructions I've found:

PaperTrail can restore three types of associations: Has-One, Has-Many, and Has-Many-Through. In order to do this, you will need to create a version_associations table, either at installation time with the rails generate paper_trail:install --with-associations option or manually.

I don't need this feature.  Is there a way to turn it off so that I can avoid schema changes?  If not, does "manually" mean that I should copy these migrations to my migrations folder?
Given the schema changes and the eager_load! issue with custom initializers, may I suggest adding an "Upgrading to 4.0" section to the readme?

.. pull down the current master branch and give it a whirl .. without those additional migrations migrated

After rollback and switching to master I'm not seeing UnknownAttributeError.  All my tests pass.  Thanks!

The association piece is supposed to be entirely optional

That's wise, you'll probably get more adoption of the new version.

I just pushed up a commit (899c15b)

To play "back seat driver" for a sec: you're duplicating the paper_trail_version_class.column_names.include?('transaction_id') condition in a few places.  I don't know the first thing about the paper_trail codebase, but maybe you could extract a method?

.. pull down the current master branch and give it a whirl .. without those additional migrations migrated

After rollback and switching to master I'm not seeing UnknownAttributeError.  All my tests pass.  Thanks!

The association piece is supposed to be entirely optional

That's wise, you'll probably get more adoption of the new version.

I just pushed up a commit (899c15b)

To play "back seat driver" for a sec: you're duplicating the paper_trail_version_class.column_names.include?('transaction_id') condition in a few places.  I don't know the first thing about the paper_trail codebase, but maybe you could extract a method?

this is happening to us as well.. i hope you are able to releae soon..

this is happening to us as well.. i hope you are able to releae soon..

this is happening to us as well.. i hope you are able to releae soon..

For now, you can use master, or stay on 3.0.6, depending on whether you need to reify associations or not.  Ben has been really helpful and quick with my questions, so I'm sure it won't be long until another beta of 4.0 shows up.

Closed by 899c15b

@batter Is it much trouble to cut another beta release with https://github.com/airblade/paper_trail/commit/899c15b24dd235cd1adfbf521b658babba2e871f in it?  Not a big deal, I've been using your master branch.  Thanks again.

Going to try to cut a new release by end of day today

Just released v4.0.0.beta2

Switching from master to 4.0.0.beta2, not seeing UnknownAttributeError, thanks for cutting another release!


[pulsar-broker] Cleanup already deleted namespace topics.

Cherry pick from #7473.
#7473 has fix the Cleanup already deleted namespace topics issue, but with #8129 involved, changes have been
changed back.
Motivation
We are having frequent issues when user removes cluster from the global namespace where broker from removed-cluster fails to unload topic and namespace bundle still loaded with the broker. It happens when broker from removed-cluster receives below error
17:38:52.199 [pulsar-io-22-28] ERROR org.apache.pulsar.broker.service.persistent.PersistentReplicator - [persistent://prop/global/ns/tp1][east -> west] Failed to close dispatch rate limiter: org.apache.pulsar.client.api.PulsarClientException: Producer was not registered on the connection
:
17:38:52.199 [pulsar-io-22-28] WARN  org.apache.pulsar.broker.service.AbstractReplicator - [persistent://prop/global/ns/tp1][east -> west]] Exception: 'org.apache.pulsar.client.api.PulsarClientException: Producer was not registered on the connection' occured while trying to close the producer. retrying again in 0.1 s
:
17:38:52.351 [pulsar-io-22-37] ERROR org.apache.pulsar.broker.service.persistent.PersistentTopic - [persistent://prop/global/ns/tp1] Error closing topic
java.util.concurrent.CompletionException: org.apache.pulsar.client.api.PulsarClientException: Producer was not registered on the connection
        at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:331) ~[?:?]

Modification
Source Broker should return explicit error-code when producer is already closed and dest-broker from removed-cluster should handle this error and clean up the replicator and topic gracefully.

/pulsarbot run-failure-checks


feat(chord_pattern): add fromQuality factory constructor

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

23 of 23 (100.0%)  changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 100.0%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
678

Relevant Lines:
678

💛  - Coveralls


Not working with database prefix

The joinReactionTotal (scopeJoinReactionTotal) all scoped methods are not working with a database prefix. You get an "column not found" error.

Thank you for the report @tvillager
Could you provide a sample to reproduce it?

@tvillager closing due to inactivity. Feel free to continue conversation here if it is reproducible bug.


full support for internet explorer

Roll-up issue to track differences in IE that will need to be handled to fully support it:

[ ] IE does not fire input events on contenteditable (possible workarounds would be to use MutationObserver (IE11), polling in conjunction with key* events, using hidden input to capture text
[ ] hide the resize handles that IE puts on block content (like images), see this SO question/answer
[ ] IE before "Edge" does not support Selection#extend. This breaks our selection logic- backwards selections without extend are basically impossible without implementing selections non-natively, so perhaps we will just allow them to break on IE pre-Edge.

I have all tests passing and basic support for Edge. Working on IE11.
IE10 is going to be more difficult to get tests happy for, since it does not have the insertText command (and in theory the input event).

IE11 also does not have insertText. Going to try and write a workaround for tests.

https://github.com/bustlelabs/mobiledoc-kit/pull/242 fixes tests for IE Edge

the resize blocks are not in IE11, so I've crossed out that item. We're a go for IE11 support in master!


Support new await syntax

cc https://github.com/rust-lang/rust/pull/60586.

Blocked until the new rustc-ap-syntax gets released.

Blocked on https://github.com/rust-lang/rust/pull/60691.


Small Control values in PPO

I am trying to learn to control a UR5 arm using PPO1(because of my legacy environment issue), rather than PPO2. I have ported the code from run_humanoid.py example, with similar hyper-parameters. The problem is, even if the control range of my model is from -5 to 5, the MlpPolicy that I am using to train generates very small values, (-0.2 to 0.2). Do the trajectory generator or policy clip these values somewhere internally?
Thank you,
Tanvir

Hi @parhartanvir! To answer your question directly, I am pretty sure the actions (or policy outputs) are not being clipped anywhere in the ppo1 (putting @joschu as the source of the ground truth in the loop here). Small actions are probably related to the fact that output layer of MlpPolicy is initialized with small variance, 0.01, here: https://github.com/openai/baselines/blob/1f99a562e3df9eaad96b59d44677006ef54ca1c2/baselines/ppo1/mlp_policy.py#L37
So even though policy can learn to output large actions, if there is not enough reward signal with small actions, the policy will never learn to do so.

apart from that, I'd like to point out that ppo2 should work with legacy environments too, and I'd be much happier debugging issues with ppo2 given that we are deprecating ppo1 :)

@Atcold Not sure I understand the question... you mean given the lower and upper bounds of action space, how to ensure that policy can explore the whole space? The simplest way would be, I guess, to add an action wrapper that scales actions by constant factors (one constant per dimension). More principled approach would be to modify action distribution according to the dimensions of the action space. We did a bit of work on using beta distributions instead of diagonal gaussian in case of continuous action spaces with finite limits. In mujoco, that did not make any difference; but it is possible that in some cases that would be a decent solution.

About the chatroom - don't think such thing exists; I'll bring up a possibility of setting something like that up with the team.

@pzhokhov, my question simply was "Are lower and upper action space bounds used anywhere in the code?".
Yeah, a IRC/Gitter channel would be AMAZING!


Fixed indentation

Fixed indentation for Request Validator example

dup with #349

@idan Could you possibly spend a few minutes going over some of these more smaller issues? The issue backlog is almost 50 strong at the moment, with some having no contributor response for over a month :/

Closing this for #349 since it contains more indentation fixes.


chore: 🤖 clean up build scripts to run in parallel

To be merged after https://github.com/aws-observability/aws-rum-web/pull/148
See diff here: https://github.com/maniator/aws-rum-web/compare/split-out-testcafe...maniator:build-async

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Hi @maniator would you be able to address the merge conflict? Once it's completed, I will go ahead and merge it.

@adebayor123 done! :-)

Hi @maniator would you be able to address the merge conflict? Once it's completed, I will go ahead and merge it.

@adebayor123 done! :-)

Another merge confict :( Could you resolve this one again, then I will merge it in.

Hi @maniator would you be able to address the merge conflict? Once it's completed, I will go ahead and merge it.

@adebayor123 done! :-)

Another merge confict :( Could you resolve this one again, then I will merge it in.

oy... yes I can

@adebayor123 rebased yet again :-)


vscodium for ppc

I saw (and thank you) that you ported vscodium to work on ppc64le, but when I extract it on my ppc64le system I only get vscodium-server; what stops us from having a normal client like other platforms? Is it possible? What would need to be done?

Running the UI is actually possible in PPC64LE, but you need to compile Electron first (since there is no public downloadable binaries for it). IBM has a build script for it here. I had to change very little to get it to work with an older version of GLIBC. Once you have the dist.zip, you can use that (and a few changes to the vscodium build scripts) to get an actual VSCodium UI port. I have to mention some of the issues I've found, tho (at least on my system):

Can't get the terminal window to work
Sometimes fails to launch
Not really stable, specially relating to extensions

careful clicking that electron link ;)

Well... I've actually found a way of getting VSCodium to build and run the UI on PPC64LE without any issues... I'll see if I can publish that latter. (might take time to get it reviewed)

Sweet, would love to connect, I have some cool things in play that Id love to meet new dev friends with

Sweet, would love to connect, I have some cool things in play that Id love to meet new dev friends with
Sorry, can't do 🙁 but good luck!
On another topic, PPC64LE patches are in a PR to upstream VSCodium.


Issues regarding simpleITK operations

Hi,
I have found two issues or rather doubts while using SimpleITK.

When after registration, I am going to write the result image into .nii or .dcm file, all the pixel locations are being changed to integer values

selx = sitk.SimpleElastix()
sitk.WriteImage(selx.GetResultImage(),"testdce.nii")
The Input image have voxels like (103.68,141.32,37.50) <x,y,z>. But there is no float values in testdce.nii. I want the pixels to be as it was in the input file. How to achieve this?

I have encountered a strange issue while using bspline method:
`
`selx = sitk.SimpleElastix()
 selx.SetFixedImage(fixed)
 selx.SetMovingImage(moving)
 parameterMapVector = sitk.VectorOfParameterMap()
 parameterMapVector.append(sitk.GetDefaultParameterMap("rigid"))
 parameterMapVector.append(sitk.GetDefaultParameterMap("affine"))
 parameterMapVector.append(sitk.GetDefaultParameterMap("bspline"))
 selx.SetParameterMap(parameterMapVector)`

Here when I am using bspline, after writing the image into testdce.nii, I have found out that there are some black holes in the image is constructed, there is no sign of these holes in original image.
When I commented out bspline only, it's working appropriately.
Please look into my doubts and waiting for your suggestion.

You appear to be using the SimpleElastix and not just SimpleITK. Can you demonstrate the issue with SimpleITK alone? Otherwise, you should report the problem to SimpleElastix.

duplicate SuperElastix/SimpleElastix#127


Adaptive card doesn't trigger topic in PVA

Describe the bug
Adaptive card doesn't trigger topic in PVA when data value intent is set
Version
Version: 2.1.2
Electron: 8.2.4
Chrome: 80.0.3987.165
NodeJS: 12.13.0
V8: 8.0.426.27-electron.0
Browser

[x] Electron distribution
[x] Chrome
[x] Safari
[ ] Firefox
[x] Edge

OS

[x] macOS
[ ] Windows
[ ] Ubuntu

To Reproduce
Steps to reproduce the behavior:

Create a PVA bot and reuse the Enterprise Assistant Bot 'Bot Tour' dialog
It sends the initial card
Click on 'Calendar' or 'People' button.
The bot is not able to understand the intent in PVA.

Expected behavior
The PVA should be able to handle the Action.Submit with 'intent' name in data and invoke the respective topic.
Screenshots
This is the place where we inform what intent to be invoked. But it doesn't work for PVAs.
Also, where is the documentation for the data attribute the the supported values for Composer or PVAs?

In PVA, even if I have a topic 'CalendarTour' it doesn't get invoked.

Hey @ryub3n,
Could you please kindly share more details for step number 1, regarding how to reuse the 'Bot Tour' dialog?

Create a PVA bot and reuse the Enterprise Assistant Bot 'Bot Tour' dialog

@ryub3n to clarify, are you experiencing this issue specifically when using BotTourDialog.dialog or just in general Adaptive Cards doesn't trigger topics in PVA?

@ramfattah , in general Adaptive Cards don't trigger topics in PVA.

Hi @ryub3n, we are working on this. We will report back once we have updates.
Thanks for your patience.

Hi @ryub3n.
I'm not able to reproduce this issue in PVA after clicking on the Adaptive card 'Calendar' or 'People' buttons.

Sharing steps I took:

Opened PVA bot in Composer
+ Add a dialog, named it BotTourDialog
In the new dialog, I reused Enterprise BotTourDialog.dialog line 1 - 95
In Composer, Bot Responses -> BotTourDialog -> Show code and added the LG code from BotTourDialog.en-us.lg
Then added the LG code from cards.en-us.lg & icons.en-us.lg into BotTourDialog.en-us.lg

Attached is a zip containing final example of BotTourDialog.en-us.zip file

Published the bot in Composer
Tested the Adaptive Card in PVA

Screenshot

Hope this helps, let me know if you have any questions. Thanks.

@ramfattah, you're half way there. I meant that adaptive cards don't trigger Topics.
Continue with the below steps:

Remove the PeopleTour and CalendarTour intents in Composer and add them as topics in PVA.

DELETE:

ADD IN PVA:

The expectation is that a PVA topic can be triggered from an adaptive card as it's basically an intent.

@ryub3n,
Thanks for providing the detailed steps.
I was able to reproduce this issue after adding the PeopleTour and CalendarTour as topics in PVA.

@ryub3n,
Thanks for providing the detailed steps.
I was able to reproduce this issue after adding PeopleTour and CalendarTour as Topics in PVA.

Hi @ryub3n,
Looks like triggering a topic with an adaptive card may not be supported in PVA.
we found a workaround to redirect to a PVA topic without using adaptive cards.
I was able to set up a dialog and redirect to PVA topic using "Begin a Power Virtual Agents Topic" action. Documentation here

closing with a workaround.


closes #52

I think this is a good change because we have the string "check" sometimes where it is reasonable for it to occur, and most of those cases are when there is also "HG".

Thanks!


[App Service] [Discussion] Automatically set default web app setting after creating a new one

#2072 was resolved by the new az configure -d flag via #2414 (yay!), but I wanted to make sure we captured the discussion to automatically set the default web setting when you create a new one. That way, you can call az appservice web create, and then immediately run commands on it (e.g. browse), without needing to specify the name again or explicitly call az configure -d web=foo manually.
Instead of doing (assuming you have a default group already set):
az appservice web create -n foo
az configure -d web=foo
az appservice web browse

You could do this:
az appservice web create -n foo
az appservice web browse

We would like to get more feedback on the current default experience (explicit setting), in the future we will look at implicit setting of defaults on creates (where it makes sense)


ALiBi Support on flash-attetion

ALiBi can support long inference context length. Implement ALiBi with flash attention will be very helpful.

I agree. We welcome community's contribution.

I am wondering if there are any plans of supporting ALiBi with flash attention in a near future?

Bringing this up, I saw that the roadmap has changed. Have you tested the Triton implementation with ALIBI?

As mentioned in the README, we also have an experimental implementation in Triton that support attention bias (e.g. ALiBi): https://github.com/HazyResearch/flash-attention/blob/main/flash_attn/flash_attn_triton.py
Feel free to play with it.

Hey @tridao  Thanks you for all the work ! I am able to do a fwd pass with alibi. Is it normal than I can't do a backward pass ?
Thanks,

Backward pass with the Triton implementation should work (for 1D bias). What error do you see?

 I have disabled all biais in layer as I didn't understand well what bias gradient mean. The error is still there.
The test I am doing :
def test_fwd_and_bwd():
    model = get_transformer_flash_attention(config)
    model.to(0)
    BATCH=1
    SEQ=500
    VOCAB=model.config.vocab_size
    x = (torch.rand((BATCH, SEQ)) * VOCAB).abs().to(torch.int).to('cuda:0')
    y=model(x)
    loss=y.mean()
    loss.backward()
    assert loss is not None

If you have any clue I can investigate more during the day.
Final idea is just to reproduce T5.
Thanks Tridao !

Yeah gradient of the bias isn't supported, so T5 won't work. For 1D bias such as key padding mask and Alibi you shouldn't need gradient.
Feel free to play with the implementation.

All good, understood 👍

Alibi is supported in FlashAttention as of v2.4.0


SPFx Field Customizer - onInit executes multiple times when navigating between folders in same document library

Target SharePoint environment
SharePoint Online
What SharePoint development model, framework, SDK or API is this about?
💥 SharePoint Framework
Developer environment
No response
What browser(s) / client(s) have you tested

[ ] 💥 Internet Explorer
[X] 💥 Microsoft Edge
[X] 💥 Google Chrome
[ ] 💥 FireFox
[ ] 💥 Safari
[ ] mobile (iOS/iPadOS)
[ ] mobile (Android)
[ ] not applicable
[ ] other (enter in the "Additional environment details" area below)

Additional environment details
Versions:

Microsoft Edge version: Version 107.0.1418.26 (Official build) (64-bit)
Chrome version: Version 106.0.5249.119 (Official Build) (64-bit)
SPFx version: 1.15.2
Node.js version: 16.13.0 (64-bit)

Describe the bug / error
I am developing an SPFx Extension Field Customizer that is supposed to run in an document library with many levels of folders and documents.
The problem i am having is that the "onInit" function is executed multiple times when navigating between the folders in the document library. I am executing an request to SharePoint through the PnPJs library from inside the onInit function.
On initial visit to the document library, this is the order of how the default function execute:

If i decide to navigate in to one of these folders from this view, then this is the execution order of the default functions:

If i navigate back up to the root folder and then to a different folder (still one level down). Then this is the execution order of the default functions:

If i right click on a document or a folder in the same view as above to open the object menu, it triggers the default functions again:

This behavior i explained above continues as i navigate to a new folder in the same document library. For every new folder i navigate to, the execution count for the onInit executions increase by one.
Steps to reproduce

Create a new SPFx Extension - Field Customizer with React as a template using yo.

Add these for the console log outputs as show in my pictures above.

Add the column associated with your Field Customizer to a document library.

run gulp serve and test it against the document library.

Expected behavior
I expect the oninit function to be executed only once for each navigation event made between the folders.

@andrewconnell & @VesaJuvonen, do you guys have any insight into this?

@SaadKnowIT - sorry for the silence.
I'm able to repro it and will be working on the fix.

@AJIXuMuK - Glad to here that you are working on this. Waiting for your findings/results.

@SaadKnowIT - the issue has been fixed and will be included in the next release.
To set the expectations: onInit will be called once per customizer/list.
It means that it may not be called if you navigate between 2 lists because the state of the list is cached and the customizer has been already initialized. Don't use this handler to handle navigation of any type.

@AJIXuMuK, thanks for the reply and the fix. Will test it when it is released.
What am i supposed to use to handle navigation events (Navigating between folders) inside the same library/list or when navigating between different libraries/lists.

this should be rolled out worldwide by now


What the reason of symlinked current directory?

Please, help me understand: Why do we have a source directory that is always the same and then link it to current? It seems useless to me:
https://github.com/Unitech/pm2-deploy/blob/master/deploy#L256
First, we run cd for directory that was already linked, do the stuff there, then symlink it once again.
It seems like we should have different release directories or something.
What is the problem behind it? I would make a build somewhere else and then symlink it to current and then restart pm2 to have a zero-downtime restart. But I constrained to make a build on a post-deploy hook which temporarily broke my server.

Probably has to do with pm2 deploy revert rollback feature.

I spent some time puzzling over this and this is the best answer I’ve found. The current symlink seems to be there for compatibility with other tools. Plus, it’s how it’s done by this other tool on which pm2-deploy seems to be based.
It doesn’t have to do with revert, which is implemented by checking out a previous commit.


Issue/187 check projfile or not + add added_letters/added_sentences

In views.py function, apply check project file or not, apply check added_letters and added_sentences

검토 했습니다


Fix: Start all messages with a capital letter

This PR

[x] starts all messages with a capital letter

:information_desk_person: Some messages already do, others don't - it's just an opinion, but I feel it should be consistent. What do you think?

:+1:

+1
My 2 cents: a good fast-follow would be to convert the string literals to single-quotes or add the various vars into the double-quoted string to take better advantage of variable interpolation, instead of concatenation operations. Probably micro-optimization when we have more important concerns, but hey...

@bighappyface
:+1:
Actually, I was also thinking about consistently using sprintf, what do you think?


incompatible_depset_is_not_iterable

According to Bazel CI, this command fails: bazel build... --incompatible_depset_is_not_iterable
File "/home/bazel/.cache/bazel/_bazel_bazel/664e1f005eb545012940353990310184/external/npm_bazel_typescript/internal/common/tsconfig.bzl", line 278, in create_tsconfig
--
  | [((workspace_path + "/") + f.path) for f in files]
  | type 'depset' is not iterable. Use the `to_list()` method to get a list. Use --incompatible_depset_is_not_iterable=false to temporarily disable this check.

Can you take a look at this?

I think you need to update the dependency on rules_nodejs.

need to cut a rules_nodejs release to get that fix...

Okay should be fixed by 09f858aec16969660092cd83d62807247ab11f6e

No, there's still an error:
$ bazel build --incompatible_depset_is_not_iterable ... --nobuild
	File "/.../96315885d9b5519569de96c9f8ab613a/external/npm_bazel_typescript/internal/common/tsconfig.bzl", line 278, in create_tsconfig
		[((workspace_path + "/") + f.path) for f in files]
type 'depset' is not iterable. Use the `to_list()` method to get a list. Use --incompatible_depset_is_not_iterable=false to temporarily disable this check.

(cc @vladmos)

Since tsconfig.bzl was changed, is it just a matter of updating inter-dependencies between rules_nodejs and rules_typescript, @alexeagle?


Cleanup

This is the first draft of PR, I mainly did some cleanup

[x] Create a demo package either to checkout another branch to actually test the package
[x] Formatted the code to be readable
[x] Add Lombok to avoid Java POJO boilerplate code
[ ] Avoid using If everywhere, prefer Optional instead
[ ] Provided a more convenient way to show results (Input & Output).
[ ] Create a Github Maven Repository
[ ] Add more tests for none common cases

Codecov Report

Merging #72 (c089972) into main (d3c604e) will not change coverage.
The diff coverage is 62.50%.

@@            Coverage Diff            @@
##               main      #72   +/-   ##
=========================================
  Coverage     61.11%   61.11%
  Complexity       24       24
=========================================
  Files             3        3
  Lines           126      126
  Branches         17       17
=========================================
  Hits             77       77
  Misses           46       46
  Partials          3        3

Impacted Files
Coverage Δ

...va/com/osscameroon/jsgenerator/util/Constants.java
0.00% <0.00%> (ø)

...meroon/jsgenerator/service/ConvertServiceImpl.java
63.10% <61.11%> (ø)

...ava/com/osscameroon/jsgenerator/util/FileUtil.java
70.58% <75.00%> (ø)

:mega: Codecov can now indicate which changes are the most critical in Pull Requests. Learn more


Missing events in API documentation for View

Describe the bug
change:center change:resolution and change:rotation are no longer listed in the master documentation https://openlayers.org/en/master/apidoc/module-ol_View-View.html As they are still working and it's inconsistent with similar change events in https://openlayers.org/en/master/apidoc/module-ol_Map-Map.html it's presumably unintended and maybe related to https://github.com/openlayers/openlayers/issues/9471.  Also "An View" is a grammar error.

same here https://openlayers.org/en/latest/apidoc/module-ol_interaction_Draw-Draw.html drawstart drawend are not at the doc

same here https://openlayers.org/en/latest/apidoc/module-ol_interaction_Draw-Draw.html drawstart drawend are not at the doc

That was covered by #9471 and has been fixed in OpenLayers 6.  The View documentation is a different issue, it was complete in OpenLayers 5 but some events are missing OpenLayers 6.

See #9773 for my findings on this issue. Unfortunately I was not able to fix it.


Added dark mode to navbar

Issue that this pull request solves
Part of: #490
Proposed changes

Applied dark mode to navbar in website view and media queries

Applied dark mode to navbar in mobile view and pop-up

Brief description of what is fixed or changed
Added dark mode to navigation bar on the website.
Types of changes
Put an x in the boxes that apply

[ ] Bugfix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Documentation update (Documentation content changed)
[ ] Other (please describe):

Checklist
Put an x in the boxes that apply

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[x] My changes does not break the current system and it passes all the current test cases.

Please review @Kajol-Kumari :)

Yeah I will take care of that next time , thank you :)


Bad error message for incorrect dependency

The error message, if someone has the near-sdk dependency as anything like near-sdk = "4.0.0" or near-sdk = "4.1.0-pre.1, the error message is:
ERROR: near-sdk dependency should be a table

Which is very unhelpful and does not indicate why this is failing. Ideally, for previous versions it should indicate what the minimum SDK version supported is, and for ones that don't apply the abi feature, it should either indicate that it is required or be able to add that feature automatically.
cc @shelegdmitriy

Adding this as another error message that is bad:
the package non-fungible-token depends on near-sdk, with features: __abi-generate but near-sdk does not have these features.
This depends on https://github.com/near/near-sdk-rs/pull/893 (which hasn't currently been merged or released) which will be unusable unless you know that.
cc @miraclx


Periodic Net::ReadTimeout errors causing restart for 24/7 Bot

This issue is very sporadic, but today it happened twice within only 7 hours.  Probably the only reason I notice is because I run my telegram bot using PM2, and I get alerts anytime it restarts without manual interaction.
These restarts are happening at times when the bot is not receiving any inputs from either chat users or the machine it lives on.  Here's the full error log:
/home/ifiht/.rvm/rubies/ruby-3.1.3/lib/ruby/3.1.0/net/protocol.rb:219:in `rbuf_fill': Net::ReadTimeout with #<TCPSocket:(closed)> (Net::ReadTimeout)
        from /home/ifiht/.rvm/rubies/ruby-3.1.3/lib/ruby/3.1.0/net/protocol.rb:193:in `readuntil'
        from /home/ifiht/.rvm/rubies/ruby-3.1.3/lib/ruby/3.1.0/net/protocol.rb:203:in `readline'
        from /home/ifiht/.rvm/rubies/ruby-3.1.3/lib/ruby/3.1.0/net/http/response.rb:42:in `read_status_line'
        from /home/ifiht/.rvm/rubies/ruby-3.1.3/lib/ruby/3.1.0/net/http/response.rb:31:in `read_new'
        from /home/ifiht/.rvm/rubies/ruby-3.1.3/lib/ruby/3.1.0/net/http.rb:1609:in `block in transport_request'
        from /home/ifiht/.rvm/rubies/ruby-3.1.3/lib/ruby/3.1.0/net/http.rb:1600:in `catch'
        from /home/ifiht/.rvm/rubies/ruby-3.1.3/lib/ruby/3.1.0/net/http.rb:1600:in `transport_request'
        from /home/ifiht/.rvm/rubies/ruby-3.1.3/lib/ruby/3.1.0/net/http.rb:1573:in `request'
        from /home/ifiht/.rvm/gems/ruby-3.1.3/gems/faraday-net_http-3.0.2/lib/faraday/adapter/net_http.rb:113:in `block in request_with_wrapped_block'
        from /home/ifiht/.rvm/rubies/ruby-3.1.3/lib/ruby/3.1.0/net/http.rb:985:in `start'
        from /home/ifiht/.rvm/gems/ruby-3.1.3/gems/faraday-net_http-3.0.2/lib/faraday/adapter/net_http.rb:112:in `request_with_wrapped_block'
        from /home/ifiht/.rvm/gems/ruby-3.1.3/gems/faraday-net_http-3.0.2/lib/faraday/adapter/net_http.rb:102:in `perform_request'
        from /home/ifiht/.rvm/gems/ruby-3.1.3/gems/faraday-net_http-3.0.2/lib/faraday/adapter/net_http.rb:66:in `block in call'
        from /home/ifiht/.rvm/gems/ruby-3.1.3/gems/faraday-2.7.4/lib/faraday/adapter.rb:45:in `connection'
        from /home/ifiht/.rvm/gems/ruby-3.1.3/gems/faraday-net_http-3.0.2/lib/faraday/adapter/net_http.rb:65:in `call'
        from /home/ifiht/.rvm/gems/ruby-3.1.3/gems/faraday-2.7.4/lib/faraday/request/url_encoded.rb:25:in `call'
        from /home/ifiht/.rvm/gems/ruby-3.1.3/gems/faraday-multipart-1.0.4/lib/faraday/multipart/middleware.rb:28:in `call'
        from /home/ifiht/.rvm/gems/ruby-3.1.3/gems/faraday-2.7.4/lib/faraday/rack_builder.rb:153:in `build_response'
        from /home/ifiht/.rvm/gems/ruby-3.1.3/gems/faraday-2.7.4/lib/faraday/connection.rb:444:in `run_request'
        from /home/ifiht/.rvm/gems/ruby-3.1.3/gems/faraday-2.7.4/lib/faraday/connection.rb:280:in `post'
        from /home/ifiht/.rvm/gems/ruby-3.1.3/gems/telegram-bot-ruby-1.0.0/lib/telegram/bot/api.rb:59:in `call'
        from /home/ifiht/.rvm/gems/ruby-3.1.3/gems/telegram-bot-ruby-1.0.0/lib/telegram/bot/api.rb:46:in `method_missing'
        from standalone.rb:15:in `block in <main>'
        from /home/ifiht/.rvm/gems/ruby-3.1.3/gems/telegram-bot-ruby-1.0.0/lib/telegram/bot/client.rb:20:in `run'
        from /home/ifiht/.rvm/gems/ruby-3.1.3/gems/telegram-bot-ruby-1.0.0/lib/telegram/bot/client.rb:10:in `run'
        from standalone.rb:14:in `<main>'

Seems like there may be an issue between the ruby libraries and Telegram when run for an extended period of time?  Is this something anyone else is seeing, could it be handled in the bot code?

Hi @Ifiht,
looks like it's just a network issue and not related to this gem. I suggest you to rescue Faraday::TimeoutError and try to make request again. Something like this: https://github.com/atipugin/telegram-bot-ruby/blob/master/lib/telegram/bot/client.rb#L31-L40

@atipugin thanks for the suggestion, I was hoping to fix the problem and update here but haven't been able to find time to test - closing for now but if I find anything that could help the gem I'll post back.


Create presets of selected elements

When working with monorepo I often find myself switching between sets of selected folders. It would be nice to have these quickly selected based on previous saved selections.

This is an interesting feature. I also have had cases were I am switching between different sets of folders and basically have to clear my selection each time.
I'm trying to envision what this feature would look like. Perhaps you can help me brainstorm. Maybe I could add some save button to save the current selection of files. Clicking save would cause VS code to request a name for the collection. Then you could select from your collections with a dropdown at the top of the extension UI.
The challenge with this is storing the data persistently. I'm fairly new to VS Code extensions so I'm not sure how feasible this is.

I like that approach. Usage of the copy button/function could trigger snapshots in file selection that can be navigated with ↶ and ↷. Let me ponder on this one a bit more.

@robertpiosik I have just pushed v1.3.0 which adds the arrow buttons. Let me know what you think.


Large errors hll++ at high cardinality

Once I set my cardinality to 100k using hll++, the error rate becomes massive:
approx:  10506
actual:  100000
delta:   89494

My sample code is pretty much cribbed from your test/bench code.  Set the truth variable to 1000 and you get exact result.  Things look good in the 10k range too.  But at 100k... boom.
Complete repro code:
package main

import (
    "github.com/clarkduvall/hyperloglog"
    "fmt"
    "hash/fnv"
    "hash"
)

func hash64(s string) hash.Hash64 {
    h := fnv.New64()
    h.Write([]byte(s))
    return h
}

func main() {
    truth := 100000

    h, err := hyperloglog.NewPlus(16);
    if err != nil {
        fmt.Println(err)
        return
    }

    for i := 0; i < truth; i++ {
            istr := fmt.Sprintf("%s", i)
            h.Add(hash64(istr))
    }
    epp := h.Count()

    fmt.Println("approx: ", epp)
    fmt.Println("actual: ", truth)
    fmt.Println("delta:  ", int64(truth)-int64(h.Count()))
}

Nice find, I'll take a look. Thanks for the repro code!

When debugging this, I found a strangely large number of similar hashes. If you change the hashing line from:
istr := fmt.Sprintf("%s", i)

to
istr := fmt.Sprintf("%s", i*384174)

the results are much better (within a couple hundred). I'm wondering if the fnv hash doesn't handle similar values very well for HLL purposes.
On another note, the reason it explodes after a certain number is because it switches from the sparse representation to the normal representation. The sparse representation doesn't require the hash values to be as well distributed, so the problem doesn't show up until the normal representation is used. Looks like the crossover point is around 59040.

I think you're right.  It's the hashing function.
I switched to fnv64a and results look fine:
approx:  103611
actual:  100000
delta:   -3611

Apparently that variant has better avalanche properties: http://en.wikipedia.org/wiki/Fowler–Noll–Vo_hash_function#FNV-1a_hash

I think this can be closed.  It may be worth-while to update the examples/benchmarks to use the a variant of fnv instead.

I popped a PR updating benchmark so that nobody else makes the same mistake I did:
https://github.com/clarkduvall/hyperloglog/pull/11


Revert "Add ui tests for CLI (#95)"

This reverts commit 669304aa6392806d74f3794597f9ccda500cde67.

Can I ask does this change broken anything? Why do we need to revert it?

If you want to update the snapshot, you can run the tests with TRYCMD=overwrite.
You can find the tips in the CI log.
Testing tests/cli-ui/pyroscope_cli_help_cmd_stdout.toml ... ok
Testing tests/cli-ui/pyroscope_cli_help_flag_stdout.toml ... ok
Update snapshots with `TRYCMD=overwrite`
Debug output with `TRYCMD=dump`

The CI failed because the UI changed, the change introduced by https://github.com/grafana/pyroscope-rs/commit/64a23e0d4c2179608c53832975421d9a4f97fdb8.


[SuperIlc] Pass through unmanaged assets during compilation

Improve quality of life by passing through to the output all inputs that aren't managed assemblies. This also includes the input assembly if we're unable to successfully compile the assembly, which is useful during CPAOT workload bring-up. If the input folder is a an published with dotnet publish --self-contained and contains the runtime, this results in a nice runnable output folder.

@dotnet-bot test this

@dotnet-bot test OSX10.12 Debug and CoreFX tests

@dotnet-bot test OSX10.12 Debug and CoreFX tests

@dotnet-bot test OSX10.12 Debug and CoreFX tests

@dotnet-bot test OSX10.12 Debug and CoreFX tests

The Mac Debug / CoreFX CI is being obtuse. This PR touches SuperIlc only which is not even hooked up into the build so should have no effect on the CI at all.


Add Illinois State University scraper

There were slight differences in the requests for syllabi/Database/ and syllabi/Archive/, but they were annoying enough that I decided to handle the differences explicitly with two different methods.

@HarrisonGregg


Improve stats on CLOmonitor

Summary
This PR contains a few commits contributing to the CLOmonitor score.

This PR is now ready for review. let's improve our Score =)

@tobru can you do a signoff for your commits? git rebase master --signoff so that the DCO check is happy.


Can Rabbitmq scalar be used with regexes ?

Currently the example shown in the documentation is :
triggers:
- type: rabbitmq
  metadata:
    host: RabbitMqHost
    queueLength: '20' # Optional. Queue length target for HPA. Default: 20 messages
    queueName: testqueue

while we want to group some of the queues with the parameter to trigger these deployment.
for eg:
triggers:
- type: rabbitmq
  metadata:
    host: RabbitMqHost
    queueLength: '20' # Optional. Queue length target for HPA. Default: 20 messages
    queueName: platform.*rabbitmq

Is this possible ?

I'm not sure but I don't think so, can you confirm @ahmelsayed?

This is not possible. Possible workaround would be to define multiple triggers and cover all queues by specifying them explicitly. But to have it working correctly, something like this https://github.com/kedacore/keda/issues/733 must be implemented first.

Hi,
Is there an update by chance on if this is going to be possible (using regex for queues)?

I don't think we've made any progress on this.

Hi,
Is there any update? :)


Why not add data corruption fault report to fault queue if ats translation is send ?

https://github.com/riscv-non-isa/riscv-iommu/blob/main/iommu_hw_guidelines.adoc#reliability-availability-and-serviceability-ras

In the cases where the error affects the transaction being processed but otherwise allows the IOMMU to continue providing service, the IOMMU may abort (see Aborting transactions) the transaction and report the the fault by queuing a fault record in the FQ. For PCIe, for example, a "Completer Abort (CA)" response is appropriate to abort the transaction. The following cause codes are used to report such faulting transactions:
DDT data corruption (cause = 268)
PDT data corruption (cause = 269)
MSI PT data corruption (cause = 270)
MSI MRIF data corruption (cause = 271)
Internal data-path error (cause = 272)
First/second-stage PT data corruption (cause = 274)

Why not add data corruption fault at here ?
https://github.com/riscv-non-isa/riscv-iommu/blob/main/iommu_ref_model/libiommu/src/iommu_translate.c#L501
    if ( (cause == 1) || (cause == 5) || (cause == 7) || (cause == 261) ||
         (cause == 263) || (cause == 265) || (cause == 267) ) {
        report_fault(cause, iotval, iotval2, TTYP, DTF,
                     req->device_id, req->pid_valid, req->process_id, req->priv_req);
        goto return_completer_abort;
    }

Thanks. That was missed and updated in PR #339


Need 2 more features to make it complete: automatic calendar updater and respond to inquiries (not confirmed booking)

Need 2 more features to make it complete: automatic calendar updater and respond to inquiries (not confirmed booking)

Thanks for the ideas!
The pricing tool will automatically update your calendar on a regular basis.  Is that what you mean by 'automatic calendar updater'?
The inquiry responder is a feature that has been requested and I hope to add soon.

I mean what they called on smartbnb a pulse
it will automatically show that your calendar was updated 1 hour ago
if you want to do this manually you normally go to the calendar and block/unblock one of the nights
listings with resent updated calendars get higher rankings in search
so basically feature might take action every hour
i am not really planing on to use a pricing tool

I see.  Good idea.  I changed the title so that it's clearer for others and added this to the list of features to add.  Feel free to make changes if you can describe it better.  :)


OpenSSL generated SM2 key throws error on parsing

Trying to parse an SM2 private key created using OpenSSL throws an error - x509: PKCS#8 wrapping contained private key with unknown algorithm: <IP_ADDRESS>97.1.301
Versions:
OpenSSL - v3.1.3
emmansun/gmsm - v0.24.1
Steps to reproduce the error:

Generate SM2 Private key using OpenSSL:
openssl ecparam -name SM2 -genkey -noout -out private-key.pem

Use the returned key string in the below code:

package main

import (
	"encoding/pem"
	"fmt"

	"github.com/emmansun/gmsm/pkcs8"
)

func main() {
	privateKey := "-----BEGIN PRIVATE KEY-----\nMIGIAgEAMBQGCCqBHM9VAYItBggqgRzPVQGCLQRtMGsCAQEEIIfYbABfRJN5ZBkW\nteXxzV0hzNrWBhN0Fmn0cJRqy50XoUQDQgAEbyM/EfFVSXAdxeZ3ovXSAtG3GD1v\nav+xanZVivqzzKU35ILFbXef9YkxHQOpQRRifIj99nJS7SH+cFH5S0jKLw==\n-----END PRIVATE KEY-----\n"
	keyData := []byte(privateKey)
	block, _ := pem.Decode(keyData)
	_, err := pkcs8.ParsePKCS8PrivateKeySM2(block.Bytes)
	fmt.Println(err)
}

Error returned is x509: PKCS#8 wrapping contained private key with unknown algorithm: <IP_ADDRESS>97.1.301

Digging deeper into the code using a debugger, the ObjectIdentifier of this key's algorithm is oidNamedCurveP256SM2 but in this block of the code in line 52 of smx509/pkcs8.go,
https://github.com/emmansun/gmsm/blob/ceff9bfd19fb6de02b948b2b96565469411a41a5/smx509/pkcs8.go#L52-L54
there is a not operator provided in the if condition, resulting in return x509.ParsePKCS8PrivateKey(der) being executed which throws the error.
Am I missing something here or is this a bug? Thanks in advance.

Are you sure this is the pkcs8 private key? Looks more like SEC1 ECC private key.

Per double checking, it seems the sm2 pkcs8 private key format in OpenSSL v3.1.3 is different with my understanding:

Let me double check the change and if there are standards for it.
Thx!

It seems that many people disapprove of this implementation of openssl. https://github.com/openssl/openssl/pull/22529, let me make this library compatible with this implementation first.

Thanks for the quick fix. Does this mean that this library is not compatible with OpenSSL SM2 keys and other functions like Sign/Verify may also not work with OpenSSL generated SM2 keys?

Thanks for the quick fix. Does this mean that this library is not compatible with OpenSSL SM2 keys and other functions like Sign/Verify may also not work with OpenSSL generated SM2 keys?

There are standards for SM2 Sign/Verify, Encrypt/Decrypt and so on, for the SM2 private key pkcs8 , there are no related national standards yet, we just treat SM2 private key similar like other NIST elliptic curves: NIST P224 NIST P256 NIST P382 NIST P521.
// pkcs8 reflects an ASN.1, PKCS #8 PrivateKey. See
// ftp://ftp.rsasecurity.com/pub/pkcs/pkcs-8/pkcs-8v1_2.asn
// and RFC 5208.
type pkcs8 struct {
	Version    int
	Algo       pkix.AlgorithmIdentifier
	PrivateKey []byte
	// optional attributes omitted.
}

// AlgorithmIdentifier represents the ASN.1 structure of the same name. See RFC
// 5280, section <IP_ADDRESS>.
type AlgorithmIdentifier struct {
	Algorithm  asn1.ObjectIdentifier
	Parameters asn1.RawValue `asn1:"optional"`
}

That means Algo.Algorithm is
	// RFC 5480, 2.1.1 Unrestricted Algorithm Identifier and Parameters
	//
	//	id-ecPublicKey OBJECT IDENTIFIER ::= {
	//		iso(1) member-body(2) us(840) ansi-X9-62(10045) keyType(2) 1 }
	oidPublicKeyECDSA = asn1.ObjectIdentifier{1, 2, 840, 10045, 2, 1}

and Alg.Parameters  will be the real Curve OID. I think there are more discussions in https://github.com/openssl/openssl/pull/22529 .

Let’s see if openssl3 will stick to its own opinion or change it to backward compatibility.

Thanks a lot for the detailed explanation. You can close the issue if you'd like.

openssl 3.2.1 will fix it.

openssl 3.2.1 will fix it.

https://github.com/openssl/openssl/pull/22529


sap.m.Text word wrap issue

OpenUI5 version: 1.28 - 1.40.6
Browser/version (+device/version): Windows7, x64. Google Chrome 53
Any other tested browsers/devices(OK/FAIL): IE 11, OK
URL (minimal example if possible): https://gist.github.com/Morion-Self/c14a0f7dff91e4ef49851c7336a3a2b7
Steps to reproduce the problem:

Create sap.m.table with sap.m.Text as colums
Populate table by data, like this

What is the expected result?
IE11

What happens instead?
too early line break in Google Chrome

Any other information? Most words displaying correctly, but some (for expample, Договор) wrap in another line.
Also, result like this will if use sap.m.ObjectListItem.

I think, this is because ObjectListItem also contains sap.m.Text

Hi,
Thanks for reporting the issue.
i was able to reproduce the issue here.
Probably comes from the webkit specific word-break: break-word property.
Tracked internally
Best regards,
Tobias

Hi,
It's fixed in the master branch and downported to 1.38, 1.40, 1.42.
Best,
Teo

I'm re-opening this issue, as the fix should be reverted.

Hi,
This is a Google Chrome issue. It can be reproduced with:
http://jsbin.com/yuvihucuwu/1/edit?html,js,output
We'll report it to Chrome developers.
Best,
Teo

TeodorTaushanov, can you give a link to Chrome issue tracker for this problem?

Here is:
https://bugs.chromium.org/p/chromium/issues/detail?id=655590


restore updating state properties for isGoneFlow and isInvisibleFlow tests

also, don't require the flow to complete because that is an implementation detail and not part of the expected method contract

@wrandall22 this addresses your comment from the other PR. I forgot I had auto-merge enabled otherwise I would have made the change there before merging.
I'm going to stick with the naming of the watch and watchIn methods, If we ever expose them more publicly they would have the following semantics:
val flow = flowOf(1, 2, 3)

flow.watch { doSomething(it)}

flow.watchIn(viewModelScope) { doSomething(it) }


Ramps are too small, should probably be 2x4 rather than 1x2

I updated the app! Let me know if you still feel like they are incorrectly sized!


Removing unit test that was failing to cover the resolution of the nano time

The difference between 2 consecutive calls to System.nanoTime() returns a resolution that depends on the OS implementation. Due to that, the test was failing in an intermittent manner.
Today we decided to remove the test since it was not testing a direct feature of ACS

I did not create an issue on Jira because it's only about removing an unit test. There is no bug fix involved or feature developed.
@miguelaferreira @remibergsma @bhaisaab @DaanHoogland
Could you have a look at this change, please?
Cheers,
Wilder

Thanks!

LGTM

LGTM


should scope matching do substring comparison within a path element?

Currently step 3 of [scope matching][https://slightlyoff.github.io/ServiceWorker/spec/service_worker/index.html#scope-match-algorithm] is:

Let matchingScope be the longest [[key]] in scope to registration map starting with the value of clientURLString.

Currently this wording means that '/some/scope' would match '/some/scopeislong'.  These two scopes represent peers in the sites tree structure.
It seems very surprising that these two match because it prevents you from having separate service workers for separate parts of your tree simply due to one being a substring of the other.

I don't understand the problem? If there are service workers for both those scopes, the scopeislong one will match every path that starts with /some/scopeislong while the other service worker will match every other path that matches /some/scope (including /some/scopeisalsolong). That seems perfectly reasonable to me.
Although I do agree that that step is confusingly written. It almost reads like it is asking for the longest scope where key.startsWith(clientURLString) is true, while it should be the longest scope where clientURLString.startsWith(key) is true.

I guess the longest key does prevent what I am seeing.  I guess this might just be a gecko bug.

This was a bug in the way the service_worker_unregister_and_register() function is used in some SW tests. and can be closed.


Logical not before function name starting with "in"

Example code:
fn inEpsilon(a, b) { 	return math.abs(a - b) < 10**-10 }
echo !inEpsilon(0, 0.1**5.0)
fails to parse. It appears to be the !in having priority over the function name that is causing the issue.

Merged a fix for this into the v0.3.5 branch a while ago


"constraint not satisfiable" error message is not deterministic

In operator-framework/operator-controller#352, I fixed a bug to make sure that operator-controller checks for unsat errors. To complete the fix, I also updated an e2e test to change the assertion on the message that is reported in the Resolved condition.
It turns out the assertion is flaky because the content of the message changes from run to run, only in terms of the order in which the individual constraint messages appear. This is problematic when this error message is written to a kubernetes status object because it results in a reconcile -> update status -> reconcile -> update status loop until we get "lucky" and deppy returns the same message two reconciles in a row.
Is is possible to update deppy to return a deterministic value for the unsat error message?

In addition to making the order deterministic, I'd propose that we use ; instead of , as a delimiter because commas are often present an individual constraint string. Semi-colons make it a bit easier to grok the message, IMO.

For now, I resorted to the following:
// prettyUnsatMessage ensures that the unsat message is deterministic and
// human-readable. It sorts the individual constraint strings lexicographically
// and joins them with a semicolon (rather than a comma, which the unsat.Error()
// function does). This function also has the side effect of sorting the items
// in the unsat slice.
func prettyUnsatMessage(unsat deppy.NotSatisfiable) string {
	sort.Slice(unsat, func(i, j int) bool {
		return unsat[i].String() < unsat[j].String()
	})
	msgs := make([]string, 0, len(unsat))
	for _, c := range unsat {
		msgs = append(msgs, c.String())
	}
	return fmt.Sprintf("constraints not satisfiable: %s", strings.Join(msgs, "; "))
}

Is it reasonable for deppy to handle this itself, regardless of the way the problem was built?

My first instinct is to keep Deppy as thin as possible and not involve it in sorting. It feels more like user domain.
It still might be a good idea to make sure that Deppy outputs errors in a certain order. For example, if we can say "Deppy guarantees to output errors in the same order as it receives input variables" we can let Deppy users decide and control ordering. But I haven't looked into what is going on on the Deppy side yet: maybe this is what already happens.

I think that's reasonable. I think as long as deppy's output is deterministic from its input, it will have met its obligations.

Let's summarise this issue as:

We need to ensure deppy's output is deterministic from its input
If it already is, make sure that there are tests for this or add tests

In addition to making the order deterministic, I'd propose that we use ; instead of , as a delimiter because commas are often present an individual constraint string. Semi-colons make it a bit easier to grok the message, IMO.

I created a separate issue #150 for this.

I did some digging and I can confirm that the order of AppliedConstraints in NotSatisfiable error is non deterministic on Deppy level. So while there was the issue on operator-controller side - we also have on in Deppy.
This can be seen if we remove this piece of code from the Deppy test:
https://github.com/operator-framework/deppy/blob/0a012d2bd7ebc51a134c54d93f7fa8220873e3ee/internal/solver/solve_test.go#L310-L337
Order of AppliedConstraint will constantly be changing.

I think the source of nondeterminism is in this function:
https://github.com/operator-framework/deppy/blob/ddaf504f3bafa717f26d4dc0cd4d16c1631acdfd/internal/solver/lit_mapping.go#L144-L148
We iterate over d.constraints which is a map and order is random in Go.


Release v0.6.6

Before the release, the followings should be fixed:

[ ] ...

and the issue for the key feature should be closed:

[ ] ...

Then,

[ ] Update the version in setup.py
[ ] Update the version in package.xml

After resolve #217

This issue is merged to #217


Find data for leads which we currently do not have data about

Ideas

Website parsing
Better processing of Google results
whatever else

Acceptance Criteria

Research results on how we can find data for additional leads
Possibly some example code (optional)
List of additional resources that we might need (e.g. API keys)

GooglePlaces can be improved by:

Adding full name search
Checking all results (not just the first one) from all search methods and trying to match them by place_id
If no matching place_id was found, try to figure out the best result some other way

Andi's Idea: Using other Sources like Handelsregister to check lead name / address


⚠ Pihole-KA has degraded performance

In 27551bc, Pihole-KA ($URL_PIHOLE_MERZ_NIMBUS) experienced degraded performance:

HTTP code: 200
Response time: 10358 ms

Resolved: Pihole-KA performance has improved in e17639f.


Legend and tooltip items are reversed in Recharts

In our charts, the bars dont match the order of the stacked bar.

Looks like this is a known issue with recharts, so we'll need to work around it: https://github.com/recharts/recharts/issues/209

closed via #15 by implementing a custom tooltip in Recharts


Feat: user agrees terms & conditions

Feature

[x] added a checkbox labeled "I agree to the Terms and Conditions"
[x]  checkbox is unchecked by default
[x] the submit button is disabled untill the checkbox is checked.

This PR closes
Closes #564
Image Preview

@olamide203 Apologies; We already have a PR for fixing this issue https://github.com/boxyhq/saas-starter-kit/pull/574 Would you like to work on something else?


problem with aggregation?

Hi
Please validate that the aggregate syntax is correct. Trying to run your command from powershell, it seems to fail
PS: az policy state list --apply "groupby((policyAssignmentId, policySetDefinitionId, policyDefinitionReferenceId, policyDefinitionId), aggregate($count as numStates))"
az : ERROR: (InvalidApplyInQueryString) 'with' expected at position 116 in 'groupby((policyAssignmentId, policySetDefinitionId, policyDefinitionReferenceId, policyDefinitionId), aggregate( as numStates))'.
At line:1 char:1
+ az policy state list --apply "groupby((policyAssignmentId, policySetD ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (ERROR: (Invalid...s numStates))'.:String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError

This one using AzureRM works fine... I think I'll revert back, as even the outcome is displayed much better than having to process json file.
 Get-AzureRmPolicyState -Filter "IsCompliant eq false" -Apply "groupby((PolicyAssignmentId, PolicySetDefinitionId, PolicyDefinitionReferenceId, PolicyDefinitionId), aggregate($count as NumStates))" -OrderBy "NumStates desc"
`
Thanks

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: ca46c96d-8c88-7c4a-d75f-1c6c45d0f75c
Version Independent ID: 9b8bdef9-8f67-7e82-1ae8-9356b62e84b0
Content: az policy state
Content Source: src/command_modules/azure-cli-resource/azure/cli/command_modules/resource/_help.py
Service: azure-resource-manager
GitHub Login: @rloutlaw
Microsoft Alias: routlaw

Since you are in powershell, the $count will be interpreted as a variable and becomes an empty string, so avoid such behavior, you need to use single quotes.
My PR #8668 will clarify this


Multiple frequency for TimeSeriesPredictor

Hello all,
Is it possible to train the TimeSeriesPredictor on a data set with multiple freq?
if so how? If not, How can we approach such matter

Hi @Mhdaw, this is currently not supported because some models (e.g., DeepAR, ETS) have different # of parameters depending on the frequency of the data.
There are two options:

Replace the timestamps in all of the time series with a dummy frequency (e.g., daily or hourly) and train a single model on all of them.
Train a separate model for each frequency.

It's hard to tell in advance which strategy will result in best forecast accuracy - I recommend that you try out both.

@shchur Thank you for your fast response,
since Chronos models don't use the time feature as an input to the model and treat the data as a simple sequence, Can I replace all timestamps with the lowest frequency on the dataset?
And I will also try a separate model for each frequency

Yes, if you only use the Chronos model, replacing all timestamps with arbitrary values will work.

Thanks

Thanks


Added the device software version with Update 2.

Will roll this change with another change that just came in.


Mutable Layers aren't auto-contrasted

Here's an example, running the image.invert Op.

The output looks blank until you press the contrast button.
It's super easy to fix this by resetting the contrast of any mutable layers. I've actually already coded up the fix on my machine...
The question is: should we reset the contrast?
Factors:

Does ImageJ2/Fiji reset the contrast for mutable outputs? I can't even get this Op to work in Fiji...
Would anybody actually not want it to be autocontrasted? In this case, users might think that the Op doesn't work if we don't autocontrast...

@elevans @ctrueden do you have opinions?

As a consumer I find image outputs that haven't had their contrast auto adjusted to be really annoying. Without fail my first action is to call up the auto contrast function on the image. If I remember correctly there are a few places in ImageJ (maybe plugins?) that apply the auto contrast adjustments to the image before returning them to the user. Thus in my mind there is not particular reason to not return an adjusted image.
I also feel like this could mitigate users being confused with outputs. They may think nothing happened...when indeed it did!

I cannot think of a better solution that auto-adjusting the contrast after running a computer op.
For inplace ops though, I think it could be annoying in some cases. So my vote is yes for computers, no for inplaces.

For inplace ops though, I think it could be annoying in some cases. So my vote is yes for computers, no for inplaces.

Do Modules retain that information? If not, then there's no way for us to know...

For plain modules, there is no such things as a computer. Any "both" input is assumed to be inplace mutation.

Woah, that's news to me. Is that documented, or just what happens in practice?
If that's the case, then the only way I can think of to auto-contrast computer outputs only is to hardcode checks on the name of the output (because Computer outputs are always named out and Inplace outputs are always named arg. Can you think of a better way @ctrueden?

Here is an example: suppose you run a module that adds 10 to every pixel. If you reautoscale, it will look like nothing changed. Whereas the user will probably be expecting everything to get brighter.

Thanks, that helps.

We might be talking past each other here. There is a such thing as a computer op in ImageJ Ops v1. But in the SciJava module system generally, there is no such thing as function/computer/inplace/etc.  Above, I am speaking about the SciJava module system in general; the only things it has for parameter types are INPUT, OUTPUT, and BOTH.
That said, it gets confusing because ImageJ Ops v1 ops are (inheritance) SciJava commands. You can of course tell the difference between a computer op vs an inplace op based on which interfaces/classes it implements/extends. But if you treat them as plain SciJava commands, without special-case checking them for "op-ness", you won't know if they are computers vs inplaces.
I hope by now it is clear that I am not in favor of running ImageJ Ops in such as a way, as normal SciJava modules, via the run methods of ModuleService or CommandService. Ops should only be run via the OpService; I consider the fact that Op extends Command to be a mistake.
With respect to napari-imagej, I don't know how you are running ops right now, but I thought you said you were using the OpService. If that's the case, we can indeed autoscale computer ops, and leave the scaling of existing layers fed into inplaces alone. All I am saying above is that when running something that is not an op, but rather some other kind of SciJava command or script module, executed via ModuleService#run, any BOTH parameter should not autoscaled.


error 0x1a when trying to run HackBGRT on startup

I get the Verification Error 0x1A whenever I try to boot with HackBGRT (works fine with normal Windows Boot Loader).
Computer is an Acer Predator Orion 3000.

And did you read and follow README.md and shim.md which exactly describe the required Secure Boot configuration?


fixed rise intervals count constant used for recording activation

Good find, thank you!

Please leave a note in ChangeLog with credit to yourself for this fix


Make sure lifecycle-Manager image version consistent with kustomize deploy

Issue:
The current deployment of Lifecycle-Manager in KCP using Kustomize does not ensure versioning consistency for Lifecycle Manager deployment.
Background:

The current deployment of Lifecycle-Manager in KCP uses a fixed ref in management plane config .
The image tag used for Lifecycle Manager instance is always latest, which can cause problems if the latest image is not compatible with the current configuration or environment. See image tag.

AC:

[ ] Updating the image tag with the current tag after each PR gets merged, to ensure that the deployed Lifecycle Manager instance matches the source code and configuration of that PR.

The PR is on the management-plane-config repo


⚡ Calculate positive and negative FMA sums in parallel

The FMA currently calculates the sum of the product and the addend first and, in case certain conditions are met, it then negates it.
https://github.com/openhwgroup/cvfpu/blob/eba1bbebd84121a529a4696c5ce792da3013a8bd/src/fpnew_fma_multi.sv#L474-L475
However, two's complement negation requires another full carry chain over the width of the sum, which means that the FMA currently has two sequential carry chains over the entire 3*PRECISION_BITS+4 bits of the sum (one for the addition and another one for the negation). This leads to long paths and timing issues.
A common way to reduce the length of signal paths in a FP adder is to calculate a positive and a negative sum in parallel and then select the correct result with a mux. The advantage is that the carry chains are no longer sequential but parallel instead. The Wally core uses this technique in its FMA: https://github.com/openhwgroup/cvw/blob/main/src/fpu/fma/fmaadd.sv
This PR modifies the FMA of CVFPU accordingly. We have verified that it is equivalent to the current implementation with Synopsys VC formal's sequential equivalence check (the script to reproduce the check is posted in a comment below).
Initial results from preliminary PD runs in FC suggest that TNS is reduced and timing ameliorated.

The following script can be used to verify that the proposed changes are sequentially equivalent to the current implementation with Synopsys VC formal 's sequential equivalence check (start VC formal via vcf -fmode SEQ, then source this script):
set_fml_appmode SEQ

set flist_golden [list \
 "common_cells/lzc.sv" \
 "cvfpu/src/fpnew_pkg.sv" \
 "cvfpu/src/fpnew_classifier.sv" \
 "cvfpu/src/fpnew_rounding.sv" \
 "cvfpu/src/fpnew_fma_multi.sv" \
]
set flist_impl [list \
 "common_cells/lzc.sv" \
 "cvfpu/src/fpnew_pkg.sv" \
 "cvfpu/src/fpnew_classifier.sv" \
 "cvfpu/src/fpnew_rounding.sv" \
 "cvfpu/src/fpnew_fma_multi_new.sv" \
]

analyze -format sverilog -library spec -vcs $flist_golden +incdir+.
analyze -format sverilog -library impl -vcs $flist_impl +incdir+.

elaborate_seq -spectop fpnew_fma_multi -impltop fpnew_fma_multi

map_by_name -clock spec.clk_i

create_clock -period 100 spec.clk_i
create_reset spec.rst_ni -sense low

fvassume -expr {spec.src_fmt_i == 0}
fvassume -expr {spec.src2_fmt_i == 0}
fvassume -expr {spec.dst_fmt_i == 0}

sim_run -stable
sim_set_state -uninitialized -apply 0

check_fv -block

report_proofs

Note that the +incdir+ needs to point to a path where the common_cells/registers.svh header can be found. cvfpu/src/fpnew_fma_multi_new.sv contains the changes of this patch, while cvfpu/src/fpnew_fma_multi.sv and all other source files are the current version from develop. Different source and destination formats can be tried manually (unfortunately, runtime explodes when attempting to constrain these more loosely via, e.g., spec.src_fmt_i inside {0,1,2,3,4}).
We have run the script for various source and destination formats and proved that the changes are equivalent:
  Summary Proofs:
   ----------------------------------------------------------------------------------------------------------------------
    VpId |           Name |      Type |         Parent |     #A |     #C |     #S |     #F |     #I |    Status |     %
   ----------------------------------------------------------------------------------------------------------------------
       0 |         seqdef |      root |            nil |     13 |      5 |     13 |      0 |      0 |   success |   100
       0 |      seqdef-rw |        or |         seqdef |      - |      - |      - |      - |      - |         - |     -
       0 |          rw1_1 |       int |      seqdef-rw |      5 |      5 |      5 |      0 |      0 |   success |   100
       0 |       rw1_1-ur |        or |          rw1_1 |      - |      - |      - |      - |      - |         - |     -
       0 |           ur_1 |      leaf |       rw1_1-ur |      4 |      5 |      4 |      0 |      0 |   success |   100
       0 |      rw1_1-dcp | decompose |          rw1_1 |      - |      - |      - |      - |      - |         - |     -
       0 |         idcp_1 |      leaf |      rw1_1-dcp |      4 |      5 |      4 |      0 |      0 |   success |   100
   ----------------------------------------------------------------------------------------------------------------------

Thanks a lot for the review!

A thing to consider would be whether this architectural change should be behind a parameter as it trades area for propagation delay, but I'll leave that up to the maintainers!

Well, actually, in our trials the area was slightly reduced by this change but that is probably a side effect of the improved timing.

@lucabertaccini do you want to have a parameter or are we good to merge "as is"?

Thanks a lot @michael-platzer! This is great :) Considering that the area gets slightly better as a side effect of the reduced timing pressure, I think we can merge it "as is" and avoid adding an extra parameter.


Longhorn v0.7.0, On kubernetes 1.16.4, volumes never attach

This seems to be the same as the closed issue #771, which didn't have a resolution.  I'm upgrading my test cluster from kubernetes 1.15.3 to 1.16.4, and under 1.16.4 the volumes got stuck in a detach/attach loop.
I'm attaching a support bundle with the logs
longhorn-support-bundle_135c5a72-0ac6-4769-9137-01b9236d3b31_2020-02-05T07-47-56Z.zip

I think the relevant bit of the log from the attacher:
I0205 08:09:34.737652       1 leaderelection.go:258] successfully renewed lease longhorn-system/external-attacher-leader-driver-longhorn-io
I0205 08:09:39.773362       1 leaderelection.go:258] successfully renewed lease longhorn-system/external-attacher-leader-driver-longhorn-io
I0205 08:09:44.808410       1 leaderelection.go:258] successfully renewed lease longhorn-system/external-attacher-leader-driver-longhorn-io
I0205 08:09:45.885272       1 controller.go:175] Started VA processing "csi-2c77e0e26cb5feca82ac601c7406a364e9ed0c975453e24fbd12e165411a0674"
I0205 08:09:45.885374       1 csi_handler.go:89] CSIHandler: processing VA "csi-2c77e0e26cb5feca82ac601c7406a364e9ed0c975453e24fbd12e165411a0674"
I0205 08:09:45.885403       1 csi_handler.go:116] Attaching "csi-2c77e0e26cb5feca82ac601c7406a364e9ed0c975453e24fbd12e165411a0674"
I0205 08:09:45.885456       1 csi_handler.go:249] Starting attach operation for "csi-2c77e0e26cb5feca82ac601c7406a364e9ed0c975453e24fbd12e165411a0674"
I0205 08:09:45.885787       1 csi_handler.go:209] PV finalizer is already set on "container-registry"
I0205 08:09:45.885843       1 csi_handler.go:542] Found NodeID dev04 in CSINode dev04
I0205 08:09:45.885917       1 csi_handler.go:169] VA finalizer is already set on "csi-2c77e0e26cb5feca82ac601c7406a364e9ed0c975453e24fbd12e165411a0674"
I0205 08:09:45.885969       1 csi_handler.go:183] NodeID annotation is already set on "csi-2c77e0e26cb5feca82ac601c7406a364e9ed0c975453e24fbd12e165411a0674"
I0205 08:09:45.886013       1 connection.go:180] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I0205 08:09:45.886057       1 connection.go:181] GRPC request: {"node_id":"dev04","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":1}},"volume_context":{"diskSelector":"","nodeSelector":"","numberOfReplicas":"\u0003","staleReplicaTimeout":"\u0000"},"volume_id":"container-registry"}
I0205 08:09:45.924901       1 connection.go:183] GRPC response: {}
I0205 08:09:45.926313       1 connection.go:184] GRPC error: rpc error: code = Aborted desc = The volume container-registry is attaching
I0205 08:09:45.926358       1 csi_handler.go:412] Saving attach error to "csi-2c77e0e26cb5feca82ac601c7406a364e9ed0c975453e24fbd12e165411a0674"
I0205 08:09:45.951984       1 csi_handler.go:423] Saved attach error to "csi-2c77e0e26cb5feca82ac601c7406a364e9ed0c975453e24fbd12e165411a0674"
I0205 08:09:45.952114       1 csi_handler.go:99] Error processing "csi-2c77e0e26cb5feca82ac601c7406a364e9ed0c975453e24fbd12e165411a0674": failed to attach: rpc error: code = Aborted desc = The volume container-registry is attaching
I0205 08:09:45.953170       1 controller.go:141] Ignoring VolumeAttachment "csi-2c77e0e26cb5feca82ac601c7406a364e9ed0c975453e24fbd12e165411a0674" change
I0205 08:09:49.848076       1 leaderelection.go:258] successfully renewed lease longhorn-system/external-attacher-leader-driver-longhorn-io
I0205 08:09:54.881673       1 leaderelection.go:258] successfully renewed lease longhorn-system/external-attacher-leader-driver-longhorn-io

Looking at another project using CSI, it looks like longhorn is mishandling the error.  This coment:
https://github.com/intel/pmem-csi/issues/103#issuecomment-446289994
Led me to the CSI spec:
https://github.com/container-storage-interface/spec/blob/master/spec.md#controllerpublishvolume
Which says that if the volume has already been published than it should just continue successfully, ignoring the error.

To get clean logs, I scaled down the deployment trying to attach, so the volume is in a detached state before this next log run:
I0205 08:22:47.149587       1 leaderelection.go:258] successfully renewed lease longhorn-system/external-attacher-leader-driver-longhorn-io
I0205 08:22:52.191089       1 leaderelection.go:258] successfully renewed lease longhorn-system/external-attacher-leader-driver-longhorn-io
I0205 08:22:57.225708       1 leaderelection.go:258] successfully renewed lease longhorn-system/external-attacher-leader-driver-longhorn-io

Now, I scale up and get the following logs:
leader-driver-longhorn-io
I0205 08:23:37.556272       1 leaderelection.go:258] successfully renewed lease longhorn-system/external-attacher-leader-driver-longhorn-io
I0205 08:23:42.593116       1 leaderelection.go:258] successfully renewed lease longhorn-system/external-attacher-leader-driver-longhorn-io
I0205 08:23:43.587922       1 controller.go:175] Started VA processing "csi-2c77e0e26cb5feca82ac601c7406a364e9ed0c975453e24fbd12e165411a0674"
I0205 08:23:43.589903       1 csi_handler.go:89] CSIHandler: processing VA "csi-2c77e0e26cb5feca82ac601c7406a364e9ed0c975453e24fbd12e165411a0674"
I0205 08:23:43.591066       1 csi_handler.go:116] Attaching "csi-2c77e0e26cb5feca82ac601c7406a364e9ed0c975453e24fbd12e165411a0674"
I0205 08:23:43.591786       1 csi_handler.go:249] Starting attach operation for "csi-2c77e0e26cb5feca82ac601c7406a364e9ed0c975453e24fbd12e165411a0674"
I0205 08:23:43.592879       1 csi_handler.go:209] PV finalizer is already set on "container-registry"
I0205 08:23:43.593735       1 csi_handler.go:542] Found NodeID dev04 in CSINode dev04
I0205 08:23:43.594562       1 csi_handler.go:177] VA finalizer added to "csi-2c77e0e26cb5feca82ac601c7406a364e9ed0c975453e24fbd12e165411a0674"
I0205 08:23:43.595242       1 csi_handler.go:191] NodeID annotation added to "csi-2c77e0e26cb5feca82ac601c7406a364e9ed0c975453e24fbd12e165411a0674"
I0205 08:23:43.653369       1 connection.go:180] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I0205 08:23:43.654396       1 connection.go:181] GRPC request: {"node_id":"dev04","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":1}},"volume_context":{"diskSelector":"","nodeSelector":"","numberOfReplicas":"\u0003","staleReplicaTimeout":"\u0000"},"volume_id":"container-registry"}
I0205 08:23:47.629111       1 leaderelection.go:258] successfully renewed lease longhorn-system/external-attacher-leader-driver-longhorn-io
I0205 08:23:52.673066       1 leaderelection.go:258] successfully renewed lease longhorn-system/external-attacher-leader-driver-longhorn-io
I0205 08:23:57.721620       1 leaderelection.go:258] successfully renewed lease longhorn-system/external-attacher-leader-driver-longhorn-io
I0205 08:23:58.653746       1 connection.go:183] GRPC response: {}
I0205 08:23:58.657399       1 connection.go:184] GRPC error: rpc error: code = DeadlineExceeded desc = context deadline exceeded
I0205 08:23:58.658201       1 csi_handler.go:412] Saving attach error to "csi-2c77e0e26cb5feca82ac601c7406a364e9ed0c975453e24fbd12e165411a0674"
I0205 08:23:58.683277       1 csi_handler.go:423] Saved attach error to "csi-2c77e0e26cb5feca82ac601c7406a364e9ed0c975453e24fbd12e165411a0674"
I0205 08:23:58.684089       1 csi_handler.go:99] Error processing "csi-2c77e0e26cb5feca82ac601c7406a364e9ed0c975453e24fbd12e165411a0674": failed to attach: rpc error: code = DeadlineExceeded desc = context deadline exceeded
I0205 08:23:58.684684       1 controller.go:175] Started VA processing "csi-2c77e0e26cb5feca82ac601c7406a364e9ed0c975453e24fbd12e165411a0674"
I0205 08:23:58.685090       1 csi_handler.go:89] CSIHandler: processing VA "csi-2c77e0e26cb5feca82ac601c7406a364e9ed0c975453e24fbd12e165411a0674"
I0205 08:23:58.685532       1 csi_handler.go:116] Attaching "csi-2c77e0e26cb5feca82ac601c7406a364e9ed0c975453e24fbd12e165411a0674"
I0205 08:23:58.685946       1 csi_handler.go:249] Starting attach operation for "csi-2c77e0e26cb5feca82ac601c7406a364e9ed0c975453e24fbd12e165411a0674"
I0205 08:23:58.686601       1 csi_handler.go:209] PV finalizer is already set on "container-registry"
I0205 08:23:58.687106       1 csi_handler.go:542] Found NodeID dev04 in CSINode dev04
I0205 08:23:58.687555       1 csi_handler.go:169] VA finalizer is already set on "csi-2c77e0e26cb5feca82ac601c7406a364e9ed0c975453e24fbd12e165411a0674"
I0205 08:23:58.687998       1 csi_handler.go:183] NodeID annotation is already set on "csi-2c77e0e26cb5feca82ac601c7406a364e9ed0c975453e24fbd12e165411a0674"
I0205 08:23:58.689544       1 connection.go:180] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I0205 08:23:58.689263       1 controller.go:141] Ignoring VolumeAttachment "csi-2c77e0e26cb5feca82ac601c7406a364e9ed0c975453e24fbd12e165411a0674" change
I0205 08:23:58.690268       1 connection.go:181] GRPC request: {"node_id":"dev04","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":1}},"volume_context":{"diskSelector":"","nodeSelector":"","numberOfReplicas":"\u0003","staleReplicaTimeout":"\u0000"},"volume_id":"container-registry"}
I0205 08:23:58.711942       1 connection.go:183] GRPC response: {}
I0205 08:23:58.714483       1 connection.go:184] GRPC error: rpc error: code = Aborted desc = The volume container-registry is attaching
I0205 08:23:58.714970       1 csi_handler.go:412] Saving attach error to "csi-2c77e0e26cb5feca82ac601c7406a364e9ed0c975453e24fbd12e165411a0674"
I0205 08:23:58.732964       1 controller.go:141] Ignoring VolumeAttachment "csi-2c77e0e26cb5feca82ac601c7406a364e9ed0c975453e24fbd12e165411a0674" change
I0205 08:23:58.735538       1 csi_handler.go:423] Saved attach error to "csi-2c77e0e26cb5feca82ac601c7406a364e9ed0c975453e24fbd12e165411a0674"
I0205 08:23:58.736496       1 csi_handler.go:99] Error processing "csi-2c77e0e26cb5feca82ac601c7406a364e9ed0c975453e24fbd12e165411a0674": failed to attach: rpc error: code = Aborted desc = The volume container-registry is attaching
I0205 08:23:59.684878       1 controller.go:175] Started VA processing "csi-2c77e0e26cb5feca82ac601c7406a364e9ed0c975453e24fbd12e165411a0674"
I0205 08:23:59.684981       1 csi_handler.go:89] CSIHandler: processing VA "csi-2c77e0e26cb5feca82ac601c7406a364e9ed0c975453e24fbd12e165411a0674"
I0205 08:23:59.685010       1 csi_handler.go:116] Attaching "csi-2c77e0e26cb5feca82ac601c7406a364e9ed0c975453e24fbd12e165411a0674"
I0205 08:23:59.685061       1 csi_handler.go:249] Starting attach operation for "csi-2c77e0e26cb5feca82ac601c7406a364e9ed0c975453e24fbd12e165411a0674"
I0205 08:23:59.685368       1 csi_handler.go:209] PV finalizer is already set on "container-registry"
I0205 08:23:59.685445       1 csi_handler.go:542] Found NodeID dev04 in CSINode dev04
I0205 08:23:59.685518       1 csi_handler.go:169] VA finalizer is already set on "csi-2c77e0e26cb5feca82ac601c7406a364e9ed0c975453e24fbd12e165411a0674"
I0205 08:23:59.685567       1 csi_handler.go:183] NodeID annotation is already set on "csi-2c77e0e26cb5feca82ac601c7406a364e9ed0c975453e24fbd12e165411a0674"
I0205 08:23:59.685611       1 connection.go:180] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I0205 08:23:59.685630       1 connection.go:181] GRPC request: {"node_id":"dev04","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":1}},"volume_context":{"diskSelector":"","nodeSelector":"","numberOfReplicas":"\u0003","staleReplicaTimeout":"\u0000"},"volume_id":"container-registry"}
I0205 08:23:59.797008       1 connection.go:183] GRPC response: {}
I0205 08:23:59.798577       1 connection.go:184] GRPC error: rpc error: code = Aborted desc = The volume container-registry is attaching
I0205 08:23:59.798624       1 csi_handler.go:412] Saving attach error to "csi-2c77e0e26cb5feca82ac601c7406a364e9ed0c975453e24fbd12e165411a0674"
I0205 08:23:59.819903       1 controller.go:141] Ignoring VolumeAttachment "csi-2c77e0e26cb5feca82ac601c7406a364e9ed0c975453e24fbd12e165411a0674" change
I0205 08:23:59.825779       1 csi_handler.go:423] Saved attach error to "csi-2c77e0e26cb5feca82ac601c7406a364e9ed0c975453e24fbd12e165411a0674"
I0205 08:23:59.826188       1 csi_handler.go:99] Error processing "csi-2c77e0e26cb5feca82ac601c7406a364e9ed0c975453e24fbd12e165411a0674": failed to attach: rpc error: code = Aborted desc = The volume container-registry is attaching
I0205 08:24:02.779071       1 leaderelection.go:258] successfully renewed lease longhorn-system/external-attacher-leader-driver-longhorn-io

Yep, that's the exact same issue.  ControllerPublishVolume times out, and then longhorn is no longer able to detect that a previous call succeeded even if it times out.  I'm going to try to look at the following things:

(Fix) increase timeout
(Fix) gracefully handle the timed out exception
(Workaround) figure out why ControllerPublishVolume takes so long

I built a 1.16.4 cluster from scratch and was able to get longhorn attaching volumes, so it looks like this bug only happens with ControllerPublishVolume times out.
What I think the cause of my timeouts is that during deployment of the cluster, I installed minio without generating a required certificate so that I had four instances attaching and detaching (since their mounted secret wasn't available) continuously for two weeks or so (I built the cluster than got the flu and came back to it after I was better).
My workaround for the moment is to not let it get in the state where there are huge timeouts, but the underlying cause is still there, and it is a bug.  longhorn needs to be able to detect when resuming from a timed out ControllerPublishVolume if the call actually succeeded.

@dmayle If the volume is already attaching, Longhorn currently reports error since it's not ready yet; but if it's already attached, Longhorn will not try to attach it again.
Can you try to create a volume and attach it to the UI? The attach should take about 5 seconds maximum, otherwise something might be wrong.

@yasker I've tried creating a new volume and attaching, and I've got the same problem.  While performing a cluster upgrade, longhorn gets into a state where ControllerPublishVolume times out, and then this attach problem comes up.
It's the exact same problem as linked above in the pmem-csi github, which mentions the exact same problem occurring in ceph caused by a failure to implement this part of the CSI spec.

I'm still looking for workarounds to prevent the ControllerPublishVolume from timing out becuase I have a production cluster that I can't upgrade because of this bug.

@dmayle As you can see in https://github.com/longhorn/longhorn-manager/blob/96edc510d24a65a6ebb3dbe39480913bf3c61cd4/csi/controller_server.go#L209 , if the volume is already attached, we will not return error for CSI call. So I think the problem here might be different.
The operation through UI (e.g. create volume, attach volume) is not going through the CSI interface. Can you show us the longhorn-manager log? There should be some clues on why the attaching get stuck there.

I've traced through my logs, and I finally figured out that iscsid was no longer running on the node in question.  I found this interesting, because I'd never activated the iscsid service, not realizing that it was necessary  (I added the iscsi tools, just didn't know I needed to activate the service).  I think iscsid will start up based on some iscsiadm commands.
After activating the service on all of my worker nodes, I rebuilt the cluster from scratch and put my workload onto it.  While upgrading this time, I made one change to my operating procedure: before upgrading a node, I disabled scheduling and deleted all replicas on that node.
Having done both of these things, I am able to successfully upgrade my cluster. (Well, the three leaders are upgraded, and I'm currently on my fourth of five worker nodes in this cluster).  After it completes, I'm going to run one last cluster rebuild and upgrade.  This one will be with the iscsid change, but I won't manually drain the node replicas.  Assuming everything works correctly, I'll report back here and suggest that the requirements in README.md updated so that #3 says something like,
3.  `open-iscsi` has been installed on all the nodes of the Kubernetes cluster, and `iscsid` enabled.

Hi @dmayle , we've updated the README.
I wasn't sure it's about iscsid because if open-iscsi is not installed, the manager will error out immediately due to unable to satisfy the requirement. But it seems we failed to take open-iscsi installed but iscsid not running into consideration.

@yasker  in my case, I found logs saying that the mount command was failing because device /dev/longhorn/my-volume didn't exist.
I saw the mention of iscsiadm in the logs, and then looked up how to use the tool online.  When I ran sample commands to check the state of iscsi, that's when I saw it reported that iscsid wasn't running.  Starting the service immediately fixed the attach problem.
I checked my other hosts (both this cluster and the production cluster), and in all other nodes, iscsid was running, even though the service was not activated, which is what led me to this.
At this point, I'm in the same place on my final upgrade test (on worker node four out of five total) without the manual drain.  The upgrade is succeeding.
As to open-iscsi starting iscsid automatically, that is not the case on my distro, but I've added it to my list of notes for bare metal machines.


Decode kasplex deploy/mint, works, but room for improvement

First attempt at decoding Kasplex deploy/mint. It works, but there are certainly room for improvement

Improved parsing. Should be good enough for release now

Reopening