Test-Connection cmdlet displaying unwanted data.

Steps to reproduce
The Test-Connection cmdlet is displaying unwanted data as part of the result.

Code:

Test-Connection www.microsoft.com -Count 1 -Quiet

Expected behavior
It should display just the word: True

Actual behavior
Pinging www.microsoft.com [<IP_ADDRESS>] with 32 bytes of data:
Reply from <IP_ADDRESS>: bytes=32 time=17ms TTL=58
Ping complete.
True

Environment data

> $PSVersionTable

Name                           Value
----                           -----
PSVersion                      6.1.0-preview.2
PSEdition                      Core
GitCommitId                    v6.1.0-preview.2
OS                             Microsoft Windows 10.0.16299
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
PSRemotingProtocolVersion      2.3
SerializationVersion           <IP_ADDRESS>
WSManStackVersion              3.0

This works properly in 5.1 but not in 6 so I've reclassified the issue as a bug.

The reason is that current code calls WriteInformation (blindly?).
See line 751 of TestConnectionCommand.cs, also line 775 and line 783.
Temporary workaround to stop the information from being displayed to the host is to use InformationAction common parameter. Example:
Test-Connection www.microsoft.com -Count 1 -Quiet -InformationAction Continue

From scripting perspective, this wouldn't be an issue, as the textual information is never written to the pipeline, and no, the textual output is not part of result data, which are defined as things sent to the pipeline. Also, Quiet switch is defined to return simpler results (int or bool, instead of record objects). I have to admit that one might not expect InformationRecord with Quiet. However, knowing the reason, I say we'd better keep InformationAction and Quiet decoupled.
In PowerShell 5.1, Test-Connection does not seem to call WriteInformation at all. By the way, the default value for $InformationPreference in PowerShell 5.1 and PowerShell Core 6.0.2 is SilentlyContinue. The author of the issue might have a different effective value when he reproduced the issue. (Perhaps PS 6.2 Core changed the default value for $InformationPreference? I'm not sure.)
If PS 6.2 Core had $InformationPreference defaulted to SilentlyContinue, the textual information wouldn't be there unless the user explicitly asks for it.

I need more feedback.
The problem is that in the script mode and in interactive mode it should work in different ways. In interactive mode the user will probably prefer to see the progress (bar) as it happens with the ping.exe command. This applies to other parameters too.
@mklement0 If you have time then your help would be useful.

@GeeLaw:
Good find, but it is -InformationAction Ignore that is needed in this case to work around the bug.
$InformationActionPreference still defaults to SilentlyContinue, and that shouldn't change.
The bug is that the WriteInformation() calls you link to mistakenly use the PSHOST tag, which effectively bypasses the $InformationActionPreference value and also -InformationAction SilentlyContinue (but, as stated, -InformationAction Ignore is effective in suppressing the output).
What said WriteInformation() calls do is effectively what Write-Host does in order to unconditionally display its output (by design).
@iSazonov: I haven't really looked at the behavior of other cmdlets with progress bars, but with -Quiet I would not expect a progress bar, even when calling interactively.

@mklement0 Thanks for the reply and the correction on PSHostTag. It seems to me that InformationPreference (InformationAction) won't accept Ignore? This is true in 5.1 and 6.0.2. Perhaps in 6.1 it's changed. (Is InformationActionPreference a new alias for InformationPreference?)
Also, my first comment was mistaken, for that it sets InformationAction to Continue. The correct workaround is to discard stream 6 (information stream) by redirecting it to $null, i.e.,
Test-Connection www.microsoft.com -Count 1 -Quiet 6> $null

Check the correctness by the following:
Write-Host 'Hello, world' 6> $null

It should write nothing to the host.

@GeeLaw:

It seems to me that InformationPreference (InformationAction) won't accept Ignore?

Yes, the preference variable doesn't accept Ignore, but the common parameter does.
That is, you can suppress the information stream (number 6) for a given invocation, but not categorically for the whole scope - by design.
Therefore, the following two statements are equivalent:
Test-Connection www.microsoft.com -Count 1 -Quiet 6> $null
Test-Connection www.microsoft.com -Count 1 -Quiet -InformationAction Ignore

In other words: both 6> $null and -InformationAction Ignore are effective workarounds.

Is InformationActionPreference a new alias for InformationPreference?

No, the name has always been $InformationPreference, following the pattern of $VerbosePreference, $WarningPrefrerence, and $DebugPrefrerence.
From what I can tell it's only the -ErrorAction / $ErrorActionPreference pair where the preference variable retains the Action part.

As an aside regarding the prohibition of Ignore as the value of action preference variables:

There's a fundamental design problem with this restriction, because automatically defined preference variables are used to propagate common-parameter values inside advanced functions - see #1759

The restriction is not enforced at assignment time, which means you won't see the problem until the preference is (possibly implicitly) applied the next time - see #4348

More generally, in any scope but the global one no validation is performed at all; compare $ErrorActionPreference = 'bogus' to & { $ErrorActionPreference = 'bogus' } - see
#3483.

@mklement0 I was asking about the alias because you mentioned it as InformationActionPreference.
To my best knowledge, -XxxAction (and -Verbose and -Debug) simply sets the corresponding preference variable inside the invoked cmdlet. As documented in about_ help topics, specifically we have the following:
The value of the -InformationAction parameter, if used, overrides the current value of the $InformationPreference variable.

Within the command or script in which it is used, the InformationAction common parameter overrides the value of the $InformationPreference preference variable, which by default is set to SilentlyContinue.

I interpret this as setting a local preference variable, as can be validated by the following demonstration:
function test-func { [cmdletbinding()]param() process { $InformationPreference } }
test-func # gives SilentlyContinue
test-func -InformationAction Continue # gives Continue
test-func -InformationAction Ignore # gives Ignore

In 5.1 and 6.0.2, InformationAction does not accept Ignore, as can be demonstrated by the following snippet:
function test-func { [cmdletbinding()]param() process { write-information 'writing' } }
test-func -InformationAction Ignore

produces
Write-Information : The value Ignore is not supported for an ActionPreference variable. The provided value should be used only as a value for a preference
parameter, and has been replaced by the default value. For more information, see the Help topic, "about_Preference_Variables."
At line:1 char:57
+ ...  { [cmdletbinding()]param() process { Write-Information 'writing' } }
+                                           ~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Write-Information], NotSupportedException
    + FullyQualifiedErrorId : System.NotSupportedException,Microsoft.PowerShell.Commands.WriteInformationCommand

I wonder if that changed in 6.1.

There's another interesting thing: $VerbosePreference and $DebugPreference actually accepts a wider range than that can be set by the corresponding common parameter. E.g., it is possible to set $VerbosePreference to Inquire by explicit assignment, but not possible by using -Verbose switch because, after all, it is a switch and maps $True/$False to Continue/SilentlyContinue.
As far as I know, except for those two switches, the preference-related common parameters map exactly to the corresponding preference variables in the local scope.

Another round of playing around shows the following code works:
Write-Information 'writing' -InformationAction Ignore

But that would be cumbersome to developers, as we have to guard Write-Information with the check $InformationAction -eq 'Ignore', and avoid calling it in the first place, e.g.,
Function Test-Inf1
{
    [CmdletBinding()] Param ( )
    Process { Write-Information 'Hello, world!' }
}
Function Test-Inf2
{
    [CmdletBinding()] Param ( )
    Process { If ($InformationPreference -ne 'Ignore') { Write-Information 'Hello, world!' } }
}
Test-Inf1 -InformationAction Ignore # writes an error
Test-Inf2 -InformationAction Ignore # okay

It seems that the same applies to authors writing cmdlets with C#.

@GeeLaw

I was asking about the alias because you mentioned it as InformationActionPreference.

Oops! Sorry - hadn't noticed my own typo.

simply sets the corresponding preference variable inside the invoked cmdlet.

Yes, and the rest of what you demonstrate is the subject of the aforementioned #1759.
In short: The design flaw is that disallowing Ignore as the preference-variable value clashes with using automatically set local preference-variable instances  in order to propagate common-parameter values.
More specifically:

In 5.1 and 6.0.2, InformationAction does not accept Ignore, as can be demonstrated by the following snippet:

Since it matters with respect to resolving the problem, let me point out that -InformationAction does accept Ignore, and it's only said design flaw that causes the problem if and when the preference is applied in the context of advanced functions .
This problem persists as of PowerShell Core v6.1.0-preview.2.
Compiled cmdlets, by contrast, do not have this problem, which is why Write-Host foo -InformationAction Ignore works as intended, for instance.

To summarize, we've been discussing two unrelated problems here:

The design flaw with respect to Ignore as an action-preference variable value, which is already being tracked in #1759.

This issue's original topic, the misbehaving Test-Connection cmdlet, which mistakenly uses the PSHOST tag in its calls to .WriteInformation.

The solution to the latter is to simply omit the tag, as the following example demonstrates:
# With $InformationAction at its default, 'SilentlyContinue', this invocation is silent.
# If you set it to 'Continue', 'foo' prints.
& { [CmdletBinding()]param() $PSCmdlet.WriteInformation('foo', [string[]] @()) }

So I came across this because I am experiencing the same issue where -Quiet is not being respected.  I looked at the code and it's a little difficult to follow and very over-complicated for what it’s doing, but I didn't see anywhere that is actually looking for this as a matter of suppressing output, nor am I seeing where it's switching from outputting a string to a boolean, which is what should be outputted when the -Quiet parameter is passed.
I did submit a PR last year (#2537) to add Test-Connection to the code and that was rejected at the time because “@PowerShell/powershell-committee would not accept this PR as it would introduce compatibility problems in the future”, so I was surprised to see the inclusion of this cmdlet now but not with the code of the original PR but rather brand new code that doesn’t provide all the expected functionality.
The biggest issue I come across is in my scripts I perform checks like this, “if(Test-Connection <IP_ADDRESS> -Quiet)” to branch into my logic, but with the -Quiet parameter not being respected, the branch always returns True because there’s not a False or Null.  So this makes the command still completely unusable to me, and also since it’s been included in new releases of PowerShell, that makes upgrading very touchy for me.  Please, fix this as it looks like it’s been several months since the issue was first reported.  Either that, or re-instate the original PR, doesn’t matter as long as the functionality is returned.
I did submit a PR last year to add Test-Connection to the code and that was rejected at the time because they didn't want to add legacy cmdlets, so I was surprised to see the inclusion of this cmdlet now but not with the code of the original PR that was complete and functioned exactly as the "legacy" version did.

If we have Test-Connection in 5.1 and 6.x, I would expect them to have the same output. I know that implementation is different, but users should not care about that. The current Test-Connection in 6.1 behaves differently giving us a different output then in 5.1. Besides that, -Quiet parameter is practically useless.

It seems this problem still stands, as of right now (PSVersion = 6.2.0) the cmdlet continues to display the "ping" information even if QUIET is present (adding -InformationAction Ignore does the trick but it means modules/scripts using the Test-Connection cmdlet will have to be updated, not cool)

Would be really nice if we could get this fixed for 7.0. I'm more than happy to contribute the code, but we need an implementation spec to follow as there was very little agreement on earlier attempts to improve this.
cc @SteveL-MSFT @iSazonov

I come back to this from time to time. Now I believe that we could solve this by using an explicit separation interactive and non-interactive scenarios with -Intercative switch. With the parameter we could implement rich user-friendly console output. It seems it is not too inconvenient for the user to type this parameter (as "-i"). Without the switch we'd do strong typed output without verbous console output that is good for script scenarios.

Given that no other cmdlet uses such a parameter, I don't think it makes sense to have such duality. A cmdlet should do it's task and behave consistently; Interactive behaviour should not be separated from how it otherwise behaves.
For example, look at Get-ChildItem. Interactively, it in very useful thanks to the default formatter display. No changes are needed to also make it useful for automation purposes; the same command that works interactively also works in a script.
That's unnecessary complexity, I feel.

We could do the interactive output by default and enhance Quiet parameter to suppress the output in scripts.

Again... I don't see the need to have a different output interactively vs in scripts.
If the cmdlet simply behaves like other cmdlets and outputs unusable data instead of being completely unique and outputting all its data in a difficult to capture or parse text stream (this is PowerShell, not Bash), there is absolutely no need for a change in behaviour.
-Quiet is a switch used by the original cmdlet in Windows PowerShell to give a pure True/False response, instead of outputting objects. Breaking that convention is a bad idea, in my opinion.
This cmdlet should behave in a manner consistent with existing cmdlets. There is no reason to have it alone use the current behaviour. In its current iteration it behaves more as one would expect a Unix utility to operate, very different to how other PowerShell cmdlets operate.

Again... I don't see the need to have a different output interactively vs in scripts.

Can you demo desired output in all supported scenarios? Note that meta information that the cmdlet outputs is very important.

outputting all its data in a difficult to capture or parse text stream

The cmdlet does strong typed object output.  (The problem is that it is impossible to build these objects and display simultaneously because there is "meta" information.)

-Quiet is a switch used by the original cmdlet in Windows PowerShell to give a pure True/False response, instead of outputting objects. Breaking that convention is a bad idea, in my opinion.

That Windows PowerShell cmdlet only supports ping for which the concept of connection does not exist. This was initially controversial design. Right name for them would be Test-Ping ot Test-ICMP.
Current cmdlet supports ip "connections". Although I would prefer something like "Test-Connectivity".

In its current iteration it behaves more as one would expect a Unix utility to operate, very different to how other PowerShell cmdlets operate.

No, the cmdlet does strong typed object output. And the console output was made to look like utilities. But at the same time, you may see that in fact it is richer and more useful.
The problem is that this output cannot be obtained using the capabilities of the formatting subsystem and it is necessary to make a direct output to the console. (Note that this does not get mixed up with output objects in the output stream)

The output can be obtained with the formatting system, if we structure the data objects in a more robust manner. I submitted a PR with a prototype already. A cmdlet that displays data in a form that doesn't properly reflect the underlying object data (e.g., by displaying data from subproperties instead of properly structuring the object class) is generally misleading at in my mind to be avoided wherever possible.
I will examine this further and put together a more complete example of what I perceive to be desired output. Test-Connection in Windows PowerShell may have been a controversial design, but I think it was a step in the right direction, if a very incomplete one.

@iSazonov I wasn't agree with you after the first read BUT after testing, I understand your point of View
$a=Test-Connection www.microsoft.com
$b=Test-Connection www.microsoft.com -Quiet

The $a and $b values are what I expect BUT I don't want the extra output.
Select-String has also a Quiet Parameter and it has nothing to do with "interactive" behavior
I'm agree that this command need an additional parameter to change the behaviour (Quiet or not).
I prefer the parameter 'Interactive' not by default but maybe a switch parameter "NoInteractive" is a better adjustment to let priority on interactive usage.

Alright, here's what I would consider to be a somewhat more useful implementation.
General Points

All current host/information output is relegated to the -Verbose stream. Currently that isn't used at all, and this is a perfect use case for it.
No progress bar unless specified with a -ShowProgress switch.
Remove the -Ping switch (it is the default behaviour).

Primary Output
Test-Connection www.google.com

Information from the Replies property of the output object should be included in the main output object, and primary output mode should be multiple objects, each representing a single ping attempt/reply object.
Buffer data is generally irrelevant, as it cannot be specified, and only a BufferSize property should be exposed. Replies.Buffer property should remain private.
Replies.Options property should be hidden from default formatting.
Results output as a table, grouped by Destination address (in the case that multiple destinations are specified).

Output visual mockup
Command used:
$Result = Test-Connection www.google.com
$Data = foreach ($Reply in $Result.Replies) {
    [PSCustomObject]@{
        Source = $Result.Source
        Destination = $Result.Destination
        Address = $Reply.Address
        RoundtripTime = $Reply.RoundtripTime
        BufferSize = $Reply.Buffer.Length
        Options = $Reply.Options
    }
}
$Data | Format-Table -GroupBy Destination -Property Source, Address, RoundtripTime, BufferSize

Resulting output:
   Destination: www.google.com
Source  Address       RoundtripTime BufferSize
------  -------       ------------- ----------
WS-JOEL <IP_ADDRESS>            36         32
WS-JOEL <IP_ADDRESS>            21         32
WS-JOEL <IP_ADDRESS>            25         32
WS-JOEL <IP_ADDRESS>            25         32

Test-Connection www.google.com -TraceRoute

Each hop should be output as a separate object, each containing the PingReply objects as a property accessible by hidden from formatting.
Main TraceRouteResult object should contain either ETS or class properties that calculate summary data from their four PingReplies.
Note: This object type we're using is currently bugged, and all PingReply objects report TtlExpired as their status. Recommend investigating progress of fix for .NET Core 3, or designing custom solution for TraceRoute backing to resolve the issue.
Output as a table, grouped by DestinationHost (Why is this property name different to that of the other object type used for standard pings?)

Output visual mockup
Command used:
$Result = Test-Connection www.google.com -TraceRoute
$Data = foreach ($Reply in $a.Replies) {
    [PSCustomObject]@{
        Hop = $Reply.Hop
        Source = $a.Source
        Destination = $a.DestinationHost
        DestinationAddress = $a.DestinationAddress
        Replies = $Reply.PingReplies
        RoundtripTimes = $Reply.PingReplies.RoundtripTime
        HopAddress = $Reply.PingReplies[0].Address
        BufferSize = $Reply.PingReplies.ForEach{$_.Buffer.Length}
        Options = $Reply.PingReplies[0].Options
    }
}

$Data | Format-Table -GroupBy Destination -Property Hop, RoundtripTimes, DestinationAddress, HopAddress, BufferSize

Resulting output:
   Destination: www.google.com
Hop RoundtripTimes DestinationAddress HopAddress     BufferSize
--- -------------- ------------------ ----------     ----------
  1 {0, 0, 0}      <IP_ADDRESS>      <IP_ADDRESS>
  2 {0, 0, 0}      <IP_ADDRESS>      <IP_ADDRESS>
  3 {0, 0, 0}      <IP_ADDRESS>      <IP_ADDRESS>
  4 {0, 0, 0}      <IP_ADDRESS>      <IP_ADDRESS>
  5 {0, 0, 0}      <IP_ADDRESS>      <IP_ADDRESS>
  6 {0, 0, 0}      <IP_ADDRESS>      <IP_ADDRESS>
  7 {0, 0, 0}      <IP_ADDRESS>      <IP_ADDRESS>
  8 {0, 0, 0}      <IP_ADDRESS>      <IP_ADDRESS>
  9 {0, 0, 0}      <IP_ADDRESS>      <IP_ADDRESS>
 10 {0, 0, 0}      <IP_ADDRESS>      <IP_ADDRESS>
 11 {0, 0, 0}      <IP_ADDRESS>      <IP_ADDRESS>
 12 {0, 0, 0}      <IP_ADDRESS>      <IP_ADDRESS>
 13 {0, 0, 0}      <IP_ADDRESS>      <IP_ADDRESS>
 14 {32, 28, 20}   <IP_ADDRESS>      <IP_ADDRESS>

I am firmly of the belief that if we're presenting data to the user, it should be readily accessible in a programmatic fashion, with similar surface structure to what is being displayed onscreen. Renaming of properties or burying data one or two levels deep in a property of the output object only invites confusion, bug reports, frustration, and a significant decrease in overall usability.

Oh, @vexx32 I see  you never diagnosed a network. Your proposal was implemented by me in the first step and then rejected as not suitable for use in an interactive session. For example, we can look at a blank screen for a very long time after running a command Test-Connection www.google.com -TraceRoute. So implementation was changed to show an output (string or progress bar) for every response.

All current host/information output is relegated to the -Verbose stream. Currently that isn't used at all, and this is a perfect use case for it.

My suggetion above was to introduce Interactive switch to split interactive and script scenarious. You suggest do the same with Verbose switch which is even more unnatural practice.

No progress bar unless specified with a -ShowProgress switch.

String output and progress bar is in current implementation as two alternatives. We need only one. Progress bar is used in Windows PowerShell cmdlet. My preference is string output in interactive session. It is much more convenient.
And we never suppress a progress bar with a switch. We have $ProgressPreference for script scenarios. Some cmdlets shows a progress bar only for long operations by timer.

Remove the -Ping switch (it is the default behaviour).

Best practice is to use explicit parameters in scripts. It makes code more readable. It was not necessary in Windows PowerShell cmdlet where only ping was implemented. New cmdlet implements more functionalities and we need new explicit parameter for every ones.

Your proposal was implemented by me in the first step and then rejected as not suitable for use in an interactive session. For example, we can look at a blank screen for a very long time after running a command Test-Connection www.google.com -TraceRoute. So implementation was changed to show an output (string or progress bar) for every response.

The progress display isn't necessary with the split object format, as we can quite easily see progress as each object is submitted to output. The only reason it's needed here is because we don't output data to the pipeline as it is retrieved like every other PowerShell cmdlet does.

My suggestion above was to introduce Interactive switch to split interactive and script scenarious. You suggest do the same with Verbose switch which is even more unnatural practice.

-Verbose is a common parameter and thus a far more natural choice for a cmdlet than a completely new switch. We needn't reinvent the wheel here.

Best practice is to use explicit parameters in scripts. It makes code more readable. It was not necessary in Windows PowerShell cmdlet where only ping was implemented. New cmdlet implements more functionalities and we need new explicit parameter for every ones.

I'm neither here nor there on this, but typically a cmdlet's default behaviour does not have a switch. For example, we don't have a -Loud switch as the reverse of -Quiet.

The progress display isn't necessary with the split object format, as we can quite easily see progress as each object is submitted to output.

With your proposal above we collect "ping" objects in "meta" object and it is impossible output the "ping" objects in real time - we can output in pipeline only the "meta" object - user will see blank display in all time.

-Verbose is a common parameter and thus a far more natural choice for a cmdlet than a completely new switch.

I am not aware of any cmdlet that was outputting significant information to the verbose stream. We always use this stream to display additional diagnostic information so that we understand the process of running a cmdlet.

typically a cmdlet's default behaviour does not have a switch.

It is right for single parameter set. Now we have many parameter set and we need to explicitly designate each.

With your proposal above we collect "ping" objects in "meta" object and it is impossible output the "ping" objects in real time - we can output in pipeline only the "meta" object - user will see blank display in all time.

The meta object is unnecessary. As mentioned in the above proposal, objects would be created and output for each PingReply or trace hop. The mock up is not the final code, merely an easy to duplicate format to illustrate the idea. Each entry in the table would be output one by one. Please read the full proposal.

I am not aware of any cmdlet that was outputting significant information to the verbose stream. We always use this stream to display additional diagnostic information so that we understand the process of running a cmdlet.

I'm also not aware of any cmdlet that is routinely outputting "significant" information to the information/host and not to output except buried several levels deep in another object.

It is right for single parameter set. Now we have many parameter set and we need to explicitly designate each.

I don't think there is much usefulness in doing so, but it doesn't create a significant amount of harm I suppose. I simply think it a waste of time; I'm not sure that many people will see much use in specifying a switch for the default behaviour.

The meta object is unnecessary.

It is critical necessary.
This approach makes the cmdlet a useless thing. If you have a network where there are problems then try to run a diagnostic using that cmdlet. You cannot do this. You will throw it and take ping and traceroute utilities. But now you can do the same diagnostics with the new cmdlet both in the console and, for example, in the monitoring system with a script. I understand that it is difficult to understand if you do not do regular network diagnostics. See how many parameters these native utilities have, especially in Unix versions. All of them are important for diagnosis. The art of diagnosis consists in using their combinations and magical meanings. I tried to add all this to the new cmdlet.

I simply think it a waste of time

Using positional parameters help you :-)

The short summary for PowerShell Committee.
Current cmdlet was designed

to get portable cmdlet for all supported platforms. Really we have still some issues in .Net Core so not all features works on Unix planforms
to get features of popular tools like ping, traceroute, pathping, Portqry.exe and etc
to get useful output objects in scripts suitable for both simple and in-depth network accessibility analysis
to get useful console output with header and footer. Notice that the cmdlet displays even more useful information in some scenarios than native prototope utility.
to allow future inhancements like remote testing "Test-Connection -Source ... -Destination ..."

The main problem is how to combine interactive console output (interactive scenario) and output of objects in pipeline (script scenario). My current suggestion is to make a splitting either with a parameter (-Interactive) or with new cmdlet (Show-Connectivity for interactive scenario and Test-Connectivity for script scenario).
Also I'd suggest change name of the cmdlet to Test-Connectivity which is more accurate. It will also allow free use of the old Windows cmdlet through proxying (WCM) without name conflict.

@iSazonov can you provide an example of such a diagnostic that requires there to be a meta object that hides all the data away? My proposal is to move the information from the meta object into each PingReply object; I do not see how that would decrease the utility of the cmdlet.

How do you put statistics from footer to first "ping" object if you output the object immediately after creation?

What footer information? The only footer information from a ping is Ping complete.
There are no statistics in the current meta objects that I can see anywhere; they just contain all of the objects with the same information that are being rendered as string data on the information stream, just in a less usable format.

The point here is to keep the cmdlets (in this case Test-Connection) the same way on both versions (WinPS and Pwsh) adding a switch to the cmdlet to in this case to 'disable' that output would be wrong since like I stated before it will mean every script/module using this cmdlet will have to be updated, the solution is making it the same way on both versions

@NJ-Dude Windows cmdlet is based on WMI and it is impossible to port it with backward compatibility. Also 5.1 was frozen - no additions will be in future.

@NJ-Dude Windows cmdlet is based on WMI and it is impossible to port it with backward compatibility. Also 5.1 was frozen - no additions will be in future.

I understand and know that, I'm just saying the SYNTAX and functionality should be the same, meaning, if using QUIET doesn't display any output then it shouldn't display any output regardless of the flavor of PS being used.

I'm just saying the SYNTAX and functionality should be the same

It is impossible. Windows Compatibility module is single way to get the old functionality.

A cmdlet's output should be able to be stored and used without having to also manually suppress unwanted display to the console that is somehow separate to the main output.
There are no other cmdlets that behave this way, where the most useful data is in string form on the information stream. There is no reason for it to behave this way. All data should be on the output stream, period. Additional data is on the verbose or debug streams as needed. Use of the information stream in this manner is quite literally unprecedented for a cmdlet that ships with PowerShell itself.
And as mentioned, there is no data in the footer you mention that needs to be specifically in a footer; it's all available from the beginning or as each response is processed.

Queuing this up for a Committee discussion
Soooo I've lost track of the thread a little here, having trouble down in the weeds, but my basic take without regards to implementation is this:

Data should be output line-by-line. I think that that's the current case with both Test-Connections and ping.exe et al
Data should be returned as a structured object. The formatting should be agnostic of this fact. (As horrible as this is, I've seen a formatter that emits a JSON string to the console despite the fact that it's a PSObject under the hood. My point is simply that it can be done.) Formatting is also a place where we're allowed to change whatever we want without breaking changes. (Also strongly agree with @vexx32 that we should be careful about column headers that don't match the property names. It's occasionally necessary for readability, but it also drives me crazy.)
-Quiet should emit nothing except True/False as a Boolean, just like Windows PowerShell.
If there's more information we need to emit than the default case (which should be more than the minimal boolean -Quiet case), -Verbose sounds reasonable, but I haven't thought about it enough. (This is also where I lose the thread, hard to tell what more folks want up above).
Imitating the exact same object (cimv2\Win32_PingStatus) with all the same properties as Windows PowerShell is impossible (because .NET Core, WMI, etc.), but we should try to get it as close as possible.
I don't know about progress. My high-level take is that progress still drives everyone crazy because it's slow (despite our optimizations), but also that it doesn't matter so much in non-interactive because everyone is setting$ProgressPreference anyway.

Sounds good to me!
Progress mainly annoys the heck out of me because it can't be handled in the command call; you have to set $ProgressPreference. I really wish that were a common parameter.... But I have another issue griping about that, so let's not get into that here! :smile:

@PowerShell/powershell-committee reviewed this.  We agreed that Test-Connection should emulate the behavior as was in Windows PowerShell 5.1 as closely as possible (including -Quiet).  This means that a PingStatus output object (dropping the win32_) should be emitted for each reply having the properties in the default format and any additional ones that are available.  Progress should not be used.
Would someone be willing to author a short RFC that shows the cmdlet syntax along with proposed output format for review?

@stevel-msft I'd be happy to. :)

I love the sound of that.
Thanks

Kinda interesting that PR 3125 covered all of this but the use of Test-Connection was rejected, but now we’ve come full circle. What about looking back at 3125?

Looking at it briefly, it looks like it essentially replaced Test-Connection with a differently-implemented command on Unix platforms to try to emulate the Windows command? Am I reading that right?
I don't think that's the best option available to us; consistency across both platforms in the implementation of the command as a whole is more valuable. It may have some interesting ideas we can utilise though, I'm sure!
I'll drop a link to the draft RFC when I'm done writing it & feel free to comment, adjust, etc... I'd love to hear some more viewpoints on this. 🙂

My specific use case for prompting this was based on the desire to use PowerShell Core on Linux, but the implementation was fully tested across both Windows and Linux.  It was meant to replace the missing command in it's entirety.

When will we see this?  In 6.2.2? When will that likely land?
(was funny to see this thread raged from April 2018 to now over -quiet being noisy. Seems such a no-brainer breaking change)

I can get this code written pretty easily I think, I'm just waiting on the RFC to be accepted. Soon as that happens I'll get it done and submit a PR for this. :smile:

Oh I thought the status was that it was approved (not knowing exactly what states there are or what the full process is). But thanks for the update. Still a pity that it's taken over 12 months to make it quiet :)

over 12 months to make it quiet

I expected more feedback

@vexx32 you can start coding this and put it behind an Experimental flag in case any feedback comes in that changes the current proposal

@SteveL-MSFT I've already got a mostly-working implementation. I'll look to submit a PR w/ some Experimental flags soon so we can talk about the code more concretely. 💖

After thinking in last days about the desired behavior, I would prefer to have interactive behavior with a minimum of parameters by default (fast typing and user friendly display), which would be convenient in an interactive session. And converting to script style with additional parameters (this implies different types in output).

Could you elaborate a bit more on how you think that could work @isazonov?

@vexx32 Do you ask about implementation or UX design?

Mainly what you think the UX would be like for that, I think. :)

In interactive session best UX is

minimal typing
It means:

ping is default
switching to another mode (traceroute and etc.) with one parameter

user friendly output
It means:

emulate ping.exe (tracert.exe and others) output on console host like I tried in the demo code - with headers, footers and informative lines well formatted. No need to think about output types - they is not used, only displayed.
add parameters to switch to script mode - that is to suppress output of user-friendly text on console host and emit strong typed objects. No need to format this output. We discussed Quiet which returns True/False but we need a parameter(s) to emit other raw strong typed objects (like -RawOutput). It is acceptable UX to use additional parameters in scripts.

Thanks, I think I understand what you're getting at a bit better now.
I don't really see the need for a dual mode like this, though? No other cmdlets in PowerShell have this split between interactive and "script mode" parameters.
If you wanted the exact output of ping / tracert, why wouldn't you just use those utilities directly?
PowerShell has never made a significant effort to completely mimic an existing command; I think Get-ChildItem is probably the closest, but it's almost the only one to do so.
If we wanted to thoroughly emulate ping / tracert display like you say, I would suggest we instead have that as a separate cmdlet or function, e.g., Show-Connection, rather than clutter Show-Command with extra parameters that have no existing precedent or need within PowerShell.

I don't really see the need for a dual mode like this, though? No other cmdlets in PowerShell have this split between interactive and "script mode" parameters.

We have gaps in out formatting system. For example, we have an issue with request to have header/footer dir. There are other scenarios where it would be convenient.

If you wanted the exact output of ping / tracert, why wouldn't you just use those utilities directly?

I do :-). These native utilities are very powerful because they are low-level but they are frozen. We can do smarter things than they do - this is the essence of the appearance of this cmdlet (and not just to make a ping) - if you take a deep look at how the addresses and names in the cmdlet are now formed and how they are output, you will see that there are much more useful than in the native ping. It looks like tricks but it is really very useful.

Of course we have gaps in the formatting system. But I don't think the answer is to implement a custom solution every single time. That just creates more problems and makes it even more difficult to improve or rewrite the formatting system to be more effective across the board.
Yeah, you've done some great work with it! I very much appreciate that. The current mode of output is not useful for anything but interactive use. We could simply remove the output step and call it Show-Connection and then utilise what you've written in a more PowerShell-focused solution for Test-Connection itself as I outline in the RFC.

#10419

@vexx32 I just tried your PR and it improves it quite a bit, but I noticed that in Windows PowerShell, it emits PingReply individually to the pipeline so you get something that looks like:
PS C:\Users\slee> Test-Connection localhost

Source        Destination     IPV4Address      IPV6Address                              Bytes    Time(ms)
------        -----------     -----------      -----------                              -----    --------
SLEE-DESKTOP  localhost       <IP_ADDRESS>        ::1                                      32       0
SLEE-DESKTOP  localhost       <IP_ADDRESS>        ::1                                      32       0
SLEE-DESKTOP  localhost       <IP_ADDRESS>        ::1                                      32       0
SLEE-DESKTOP  localhost       <IP_ADDRESS>        ::1                                      32       0

However, with your change, all the ping replies are members of a single object:
PS> Test-Connection localhost

Source       Destination Replies
------       ----------- -------
slee-desktop localhost   {System.Net.NetworkInformation.PingReply, System.Net.NetworkInformation.PingReply, System.Net.NetworkInformation.PingReply, System.Net.N…

Are we not able to get back the Windows PowerShell behavior?

@SteveL-MSFT That object output has not changed from the initial implementation in PS Core; the success output has always had the single object. 🙂
As mentioned in the closing comments of the PR, that is only a partway implementation of the RFC to help simplify reviews. I will be submitting a follow-up shortly. Just have to rebase that branch to remove the now-duplicate commits and submit the rest of it to get much closer to true parity with Windows PowerShell's implementation. It will still differ somewhat (as can be seen from the RFC we reviewed a handful of weeks back) but will be hopefully much more versatile. 😊

In release 7.0.0 Test-Connection -Quiet still gives
Test-Connection: Testing connection to computer 'scrubbed' failed: Cannot resolve the target name.
instead of
False

@chvol could you please open a new issue for that with as much detail as possible?
Thanks! 🙂


Broken Link

hi,
Link : “https://github.com/orgs/Your-First-Open-Source-Project/people”
this link shared on the registration screen has exploded. would like to report.

Use the new link instead - https://github.com/orgs/yfosp/people


INFO Trying again after blcok polling failed:Block data unavailable,block was likely unceld(blcok hash = )

When I sync the aurora node of test network or main netwoerk on graph-node, it always come to the problem like the screen pic. It looks like it is polling too fast.

@Jackie-Zhong  This should be resolved by now, please reopen if this issue will reoccur again.


Fix to Issue #57 has unintended consequences

Checkout this revision: 85b6087b2917014530096c4f65c78c15189f966a
then run:

ds = prtDataGenUnimodal;
class = prtClassFld;
ds = class.kfolds(ds,10);
disp(ds)

You will see ds has userData is a [10 x 1] struct.      Because in the first instance we make a ds with a 1x100 userData, and then we make a 1 x (100100) userData, then (1 x (100100*100)) etc.  That's... no good.
Note that, as far as I can tell, this does NOT happen in the parent commit: f441f1ce2b9ad1e763a1efebffb9686c158fc074
The problem is that userData was never intended to be used to hold information from different cross-validation folds - userData was always intended to be a single, global set of information for the dataSet, and mucking with it inside kfolds breaks that assumption.
The bug above only really matters if you run cross-validaton on the same data set a bunch of times - it is admittedly not the worst problem in the world, but it can be weird.   If userData was actually BIG this problem can be significant - I don't necessarily want 1000 copies of userData for my 1000 folds.
I don't know what the right answer is.  Usually I'd suggest putting whatever this information that you want to store in userData on every fold in the ACTION (and use the second output of kfolds or crossValidate) but that's clearly sub-optimal and confusing.
We could ADD a new field to dataSets - kfoldsUserData, which is an nFolds x 1 array of userData?  Then in cross-Validate, we can change  prtDataSetBase.crossValidateCombineFoldResults to do this:

    for i = 1:length(dsTestCell)
         dsOut.kfoldsUserData(i) = dsTestCell(i).userData;
    end

Thoughts?

Actually, Miles just convinced me a better solution is to just provide the individual dataSets out as an optional 3 (crossValidate) or 4th (kfolds) output - then you can do:

   [dsAgg,ascts,dsFolds] = class.kfolds(ds,10);

where dsFolds(1).userData, dsFolds(2).userData, etc. have the right things.

Then what should the userData for dsAgg look like?

I think its the userData from the original ds?  (the input)

So my workaround has been to index the very big userData @peterTorrione referenced originally. It gets really big because I am doing 10-fold xval as well as one-vs-all classification. So I have a double for look that goes something like this:
for iClass = 1:nClasses
     for iFold = 1:nFold
        userDataIdx = iClass*(iFold-1)+iClass;
        data_i_need = dsOut.userData(userDataIdx).fieldName
      end
end

I like Miles' suggestion to output the individual data sets from kfolds, and then I can use each data set's observationInfo() instead. We would still need to figure out how to fill observationInfo() in a one-vs-all scheme.
Ultimately I think it would be easiest if my snippet would look more like:
for iClass = 1:nClasses
     for iFold = 1:nFold
        data_i_need = dsOutPerFold.observationInfo(iClass).fieldName
      end
end

Oh holy cow.  prtClassBinaryToMary does the same thing!  Wow.  That's... interesting.
I think I understand the use case.  I don't think observationInfo works here, b/c it has to be nObservations x 1.  And, I think we always thought a "pure" action shouldn't muck with userData b/c you (the user) might put stuff in userData, and you want it to come out the same way you put it in.  I propose one of the following -

The idea above, which basically results in:

[dsAgg,acts,dsOutPerFold] = ...
for iClass = 1:nClasses
     for iFold = 1:nFold
        data_i_need = dsOutPerFold(iFold).userData(iClass).fieldName
      end
end

Per here - https://github.com/covartech/PRT/issues/57   I am pretty sure we want to NOT use observationInfo here b/c observationInfo is "special" and gets handled properly, e.g., in ds.retainObservations(...), and has to be nObservations x 1.
To avoid smooshing other people's userData, I would actually do something like

   function self = runAction(self,ds)
        ....
        ds.userData.prtClassFld = struct(all my info I want to store)
   end

If I was doing that in prtClassFld.  In general this is probably safe

   function self = runAction(self,ds)
        ....
        ds.userData.(class(self)) = struct(all my info I want to store)
   end

We can also make a new field of prtDataSet* - call it "actionData".  actionData is intended to be mucked with by actions and do all of the above with actionData rather than userData.  We have to be careful that whenever we make multiple dataSets (e.g., in cross-validate, in prtClassBinaryToMary) we always concatenate actionData arrays if they are not empty I guess?

Thoughts?
THen you would do basically

for iClass = 1:nClasses
     for iFold = 1:nFold
         data_i_need = dsOutPerFold(iFold).actionInfo.prtClassBinaryToMary(iClass).classifierName.fieldName
      end
end

But I'm not 100% sure that's right either, and I need to go pick up my family...

I'm on board with either of these. Option 1 satisfies my needs (as does my hack mentioned above), but if Option 2 allows for userData to be used as originally intended, that's fine too.


connect over and over without running the tests in some browser types

When run using grunt-karma and karma-webdriver-launcher on TestingBot, this happens (last lines of debug output) when selecting IE/Edge or Safari as the browser:
05 10 2015 13:07:08.941:DEBUG [karma]: A browser has connected on socket 8umWWi9aYhE8BOTWABuW
05 10 2015 13:07:08.966:DEBUG [karma]: A browser has connected on socket WsmXr8C1QwhwNQSsABuX
05 10 2015 13:07:08.974:DEBUG [karma]: A browser has connected on socket 9QRh8j9cCGA0U3ohABuY
05 10 2015 13:07:09.000:DEBUG [karma]: A browser has connected on socket LNELo4n0K8mDbJmeABuZ
05 10 2015 13:07:09.036:DEBUG [karma]: A browser has connected on socket PLLCxhu5g9f4HDRgABua
05 10 2015 13:07:09.049:DEBUG [karma]: A browser has connected on socket 8tdQPyioFU6ROMf0ABub
05 10 2015 13:07:09.051:DEBUG [karma]: A browser has connected on socket _cakZ8yxwEJuWji0ABuc
05 10 2015 13:07:09.095:DEBUG [karma]: A browser has connected on socket yNc45itdtUuqLmr8ABud
05 10 2015 13:07:09.102:DEBUG [karma]: A browser has connected on socket q0nEceAxvGJv_rs2ABue
05 10 2015 13:07:09.110:DEBUG [karma]: A browser has connected on socket BuwsT40IzH5fPw9cABuf
05 10 2015 13:07:09.125:WARN [launcher]: microsoftedge via Remote WebDriver have not captured in 60000 ms, killing.
05 10 2015 13:07:09.214:DEBUG [karma]: A browser has connected on socket u6gK14JRLzfDzZTWABug
05 10 2015 13:07:09.216:DEBUG [karma]: A browser has connected on socket bJORwwlfJxKc2x8bABuh
05 10 2015 13:07:09.217:DEBUG [karma]: A browser has connected on socket lV-zcdQMbTvM5Bm0ABui
05 10 2015 13:07:09.218:DEBUG [karma]: A browser has connected on socket VzHQwByzj5uGQ68UABuj
05 10 2015 13:07:09.219:DEBUG [karma]: A browser has connected on socket rsig26WXriTz-MzBABuk
05 10 2015 13:07:09.251:DEBUG [karma]: A browser has connected on socket r3-Fy7p7WRXS_NguABul
05 10 2015 13:07:09.290:DEBUG [karma]: A browser has connected on socket XgQ2-ircgnCA4rd4ABum
05 10 2015 13:07:09.330:DEBUG [karma]: A browser has connected on socket Ja3K7VgGAXKBDXPrABun
05 10 2015 13:07:09.332:DEBUG [karma]: A browser has connected on socket O76Vct3TLawydmxMABuo
05 10 2015 13:07:09.334:DEBUG [karma]: A browser has connected on socket rhIuBXDyU7enRpSOABup
05 10 2015 13:07:09.366:DEBUG [karma]: A browser has connected on socket tacI3YkzDkrftGqUABuq
05 10 2015 13:07:09.370:DEBUG [karma]: A browser has connected on socket _CqMiEg5_pbmzwvTABur
05 10 2015 13:07:09.402:DEBUG [karma]: A browser has connected on socket w9iVCXMzxxQUGBLRABus
05 10 2015 13:07:09.442:DEBUG [karma]: A browser has connected on socket 2zPDydlLeUZtydeKABut
05 10 2015 13:07:09.451:DEBUG [karma]: A browser has connected on socket e7s8rl1Xh8RWoq0DABuu
05 10 2015 13:07:09.454:DEBUG [karma]: A browser has connected on socket rNBvpFszGXiNCvILABuv
05 10 2015 13:07:09.476:DEBUG [karma]: A browser has connected on socket CBuxKv1YAFN13uiKABuw
05 10 2015 13:07:09.479:DEBUG [karma]: A browser has connected on socket UVoGqWF2iCdO1sU3ABux
05 10 2015 13:07:09.514:DEBUG [karma]: A browser has connected on socket 8Hjy6gToJQgyN5KlABuy
05 10 2015 13:07:09.551:DEBUG [karma]: A browser has connected on socket Hv14eS9q-G8CO31jABuz
05 10 2015 13:07:11.373:INFO [WebDriver]: Killed Karma test.
05 10 2015 13:07:11.374:DEBUG [launcher]: Process microsoftedge via Remote WebDriver exited with code -1
05 10 2015 13:07:11.375:DEBUG [temp-dir]: Cleaning temp dir C:\Users\Herby\AppData\Local\Temp\karma-87557015
05 10 2015 13:07:11.384:ERROR [launcher]: microsoftedge via Remote WebDriver failed 2 times (timeout). Giving up.
05 10 2015 13:07:11.386:DEBUG [karma]: Run complete, exiting.
05 10 2015 13:07:11.406:DEBUG [launcher]: Disconnecting all browsers
Warning: Task "karma:testingbot-sequence-0" failed. Use --force to continue.

Aborted due to warnings.

D:\Herby\Work\foo\bar>

Other browsers tend to work (FF and Chrome do).
Output is the same kind for safari 8 and ie 11. You can see the videos of how it performed on these browsers here:
https://testingbot.com/tests/6136705
https://testingbot.com/tests/6136713
https://testingbot.com/tests/6136718

Same for iphone 8.1: https://testingbot.com/tests/6136730

Karma, this is still an issue. IE/Edge on testingbot just keep showing "reconnecting in  ms...". Are you going to fix it? It may be the problem with @testingbot tunnel, but it may be problem in your code.

Can you please check if this is happening with any other launcher?

??? Other launchers do not use socket.io. This is karma problem.
Friedel Ziegelmayer wrote:

Can you please check if this is happening with any other launcher?
—
Reply to this email directly or view it on GitHub
https://github.com/karma-runner/karma/issues/1606#issuecomment-154368654.

Ah, you mean "launcher" in strictly karma definition of the word. I may try the recently created karma-testingbot-launcher...

Terminate batch job (Y/N)? n

This issue is marked as needing author feedback, and the author has deleted their GitHub account. Closing.


Unwanted model.TGT files in the CT2 release

There are  a few files in CT2 release that is not required as far as I can see.

vocab/model.TGT
vocab/vocab.SRC
vocab/vocab.TGT

vocab/model.SRC works as sentencepiece model - for en->indic and indic->en. For detokenizing, just joining the items in output of translator is enough. We got the translations working well with this setup for both en->indic and indic->en directions.
(Please let me know if I am mistaken in the usage of CT2 model. )

In the CT2 release, the files vocab/vocab.SRC and vocab/vocab.TGT are included but these are not necessary for the inference. These files are only provided for the purpose of understanding and inspecting the tokens present in the vocabulary.
The file vocab/model.SRC is essential for applying the SPM (SentencePiece) tokenization.
For detokenization, a simple approach is to concatenate the items in the output and replace the '▁' character with a space (' '). This method should work without the need for the vocab/model.TGT file, we just went with SPM decode for modularization purposes.

Thanks. I was using SPM docode and noticed that it adds a significant time to the overall translation time. Removing it saved at least 25% of total translation in my tests.


Tooltip display wrong color when fill: {type:"gradient"}

Description
When I am usins a stacked bar chart and want to use a gradient fill property, the colour of the tooltip is wrong when I hover it. It always display the same colour (the one of the first element) instead of the colour of the element on which I am hovering (which is what it usually does)
Steps to Reproduce

create a stacked bar chart
add in the options : {fill:{type:"gradient"}}
Hover your graph

Expected Behavior
The colour of the tooltip should be the same as the one of the elements on which I am hovering
Actual Behavior
Screenshots

Reproduction Link
https://codepen.io/pgredigui/pen/JjLzNBv

duplicate of #4755


[gradle plugin] test code refactoring

I know this is not a matter of high priority but the gradle plugin tests got a bit messy over time. This PR attempts to clean that up and provide a more expressive way to setup and execute gradle plugin tests.

Can you also please rebase this branch so that we see the build really go green.

rebased onto master


Implementation local symbol loading

Hi everyone,
This enables loading symbols from the local workstation on Linux and Windows. Having the symbols included in the binary on the instance also still works (as before).

Hi everyone,
the linux build breaks because of a bug in clang7 which is related to my suggestion of std::from_chars.
I have to update the toolchain to a more recent version of clang7, but this will take some time.
@antonrohr I think the best solution for now is to revert the std::from_chars change. :grimacing:
I'm sorry for that hustle.
We could ignore the CI results but then we will have a broken continuous build until I managed to
updated the toolchain.

@antonrohr I think the best solution for now is to revert the std::from_chars change. 😬

Would using absl::SimpleAtoi be an option? I believe this was using std::stoul before, which can throw an exception when the string can't be parsed into a number. absl::SimpleAtoi simply returns a boolean whether parsing was succesful.

@antonrohr I think the best solution for now is to revert the std::from_chars change. grimacing

Would using absl::SimpleAtoi be an option? I believe this was using std::stoul before, which can throw an exception when the string can't be parsed into a number. absl::SimpleAtoi simply returns a boolean whether parsing was succesful.

If I remember correctly the number-string is base 16. SimpleAtoi supports unfortunately only base-10.

@antonrohr I think the best solution for now is to revert the std::from_chars change. grimacing

Would using absl::SimpleAtoi be an option? I believe this was using std::stoul before, which can throw an exception when the string can't be parsed into a number. absl::SimpleAtoi simply returns a boolean whether parsing was succesful.

If I remember correctly the number-string is base 16. SimpleAtoi supports unfortunately only base-10.

Ah, too bad. Thanks for clarifying.


How to run gpt2 model after building the engine successfully?

After running this:
!python3 TensorRT-LLM/examples/gpt/build.py --model_dir=./c-model/gpt2m/1-gpu --use_gpt_attention_plugin --remove_input_padding
The engine gets successuly built. In the directory called "engine_outputs" I have now 2 files config.json and gpt_float16_tp1_rank0.engine. How do i now use this engine robustly in my pipeline? How do i measure the inference speed?
i tried run.py's command: !python3 TensorRT-LLM/examples/run.py --max_output_len=8 but it doesn't seem intuitive, also if I add arguments like --no_add_special_tokens it says unrecognized. I changed files like run.py to build the engines for 0.7.1 version #871

Please follow the bug_report.yml template to share the infomation of your issue, thanks.

Hi @SehajDxstiny would u please try our latest code base to see if the issue still exists?
And do u still have further issue or question now? If not, we'll close it soon.


Debugging Information

It would be lovely if you could build with a CLAP_WRAPER_VERBOSE=TRUE cmake flag or some such which made the wrapper verbose in how it finds, loads, and so on the subordinate clap wrapper. In live right now I get to "loading clap-saw-demo.clap" and then a failure. Some chattiness along the way on what went wrong might be great!

I found the bug in Ableton :) Live does not recognize the VST3 calling sequence. I filed that as a bug at Ableton.
The wrapper is updated accordingly.
Nevertheless, some debugging or information would be neat.


[Question] Auth config when using SAP API Business Hub

Hi there 👋,
many thanks for the great work!
When using S/4HANA Cloud sandbox with APIKey, API keys are required instead of basic auth. How can I best integrate this in the current workload?
Fyi I tried to patch CostumHeaders containing the APIkey in the script src/api/src/business-partner/business-partner.service.ts before executing the request, for example for getBusinessPartnerbyID:
  async getBusinessPartnerById (id: string): Promise<BusinessPartner> {
    return await businessPartnerApi
      .requestBuilder()
      .getByKey(id)
      .select(businessPartnerApi.schema.BUSINESS_PARTNER)
      .addCustomHeaders({ APIKey: '<MY-API-KEY>' })
      .execute({
        url: (process.env.ODATA_URL),
        username: (process.env.ODATA_USERNAME),
        password: (process.env.ODATA_USERPWD)
      })
  }

The header is attached in the request, but I still got 500 when testing locally. Do you have any ideas on this?
Thank you!

dedicated guidance for APIKey handling added


Delete docs for ThirdPartyResources

ThirdPartyResources (TPRs) were removed in v1.8. It is misleading to still show the docs for TPRs for >=v1.8 versions of the docs website. If one needs to read the docs for TPRs, they can switch to <=v1.7 versions of the docs website.
However, docs about migration of TPRs to CRDs are still not removed because one can also migrate from v1.7 (when CRDs were introduced) to v1.7+.
/assign sttts

/lgtm
/approve

/assign @bradamant3
for approval

@ajaeger regards to Nürnberg. Great to see Suse in the project 👍

Hope, the /hold was appropriate here - if not, please tell me...

Definitely relevant! :)
Thanks for pointing it out, @ajaeger! Updated, ptal.

@ajaeger I think yes. But I am not so much into the website processes. In the main repo one only puts a hold if there is something seriously wrong, or to hold it when something is approved and lgmt'ed already, but should not merge yet. Usually one trusts people that comments are addressed, even without a hold. And you see @nikhita is more than fast with that :-)

/lgtm (not sure whether it counts since I'm not in the OWNERS file)
/nohold
@nikhita, you're fast :+1:  Thanks!

/no-hold

/hold cancel

lgtm maybe without the text behind?

(not sure whether it counts since I'm not in the OWNERS file)

It should count if you are a member of the Kubernetes github org. Maybe it doesn't apply the lgtm because of the statement after /lgtm? Can you try with just /lgtm and check?

/lgtm

thanks,  @nikhita and @sttts

In case the folks in this PR are still looking at comments:
Unless completely out of band because the approver is an admin in the K8s org, the /approve command has effect only for sig-docs-maintainers. This is to make sure that docs contributions are reviewed for minimal docs standards.
Also in case it helps: with the Hugo migration /data/tasks.yml has no meaning. It should be removed from master and any upcoming release branches as soon as we can get to it (many more urgent Hugo migration issues ahead of it, however).
Will approve in a follow-on comment, but waiting a small bit to hope that this comment helps explain about docs process.

Thanks for the explanation, @Bradamant3!

Also in case it helps: with the Hugo migration /data/tasks.yml has no meaning. It should be removed from master and any upcoming release branches as soon as we can get to it (many more urgent Hugo migration issues ahead of it, however).

Created https://github.com/kubernetes/website/pull/8559


Trident & Red Hat CodeReady possible ?

Describe the bug
I try to deploy  trident in a codeready openshift cluster. I got the failure:
./tridentctl install -n trident -d DEBU Initialized logging.                          logLevel=debug DEBU Running outside a pod, creating CLI-based client.  FATA Install pre-checks failed; could not initialize Kubernetes client; could not get OpenShift server version
Environment

Trident version: 19.10 and v20.01.0-beta.1
Trident installation flags used: ./tridentctl install -n trident -d
Kubernetes version: [v1.14.6+17b1cc6
Kubernetes orchestrator: Server Version: 4.2.10
CodeReady Version: crc version crc version: 1.3.0+918756b OpenShift version: 4.2.10 (embedded in binary)

To Reproduce
Deploy Red Hat CodeReady
Deploy Trident
Expected behavior
Trident container deployed
Additional context
Is it possible to use trident with code ready?

Found the problem.  CodeReady has a dedicated oc command and Client Version is empty/unknown
output from oc version code ready:
.crc/bin/oc_old version
Client Version: unknown
Server Version: 4.2.10
Kubernetes Version: v1.14.6+17b1cc6

output from oc OKD:
oc version
oc v3.11.0+0cbc58b
kubernetes v1.11.0+d4cacc0
features: Basic-Auth GSSAPI Kerberos SPNEGO

Server https://api.crc.testing:6443
kubernetes v1.14.6+17b1cc6

With the oc from OKD I can deploy trident:
./tridentctl  logs
trident-controller log:
time="2020-01-10T11:42:08Z" level=info msg="Running Trident storage orchestrator." binary=/usr/local/bin/trident_orchestrator build_time="Wed Oct 30 13:17:04 EDT 2019" version=19.10.0
time="2020-01-10T11:42:08Z" level=info msg="Initializing HTTP REST frontend." address="<IP_ADDRESS>:8000"
time="2020-01-10T11:42:08Z" level=info msg="Added frontend." name="HTTP REST"
time="2020-01-10T11:42:08Z" level=info msg="Initializing HTTPS REST frontend." address=":8443"
time="2020-01-10T11:42:08Z" level=info msg="Added frontend." name="HTTPS REST"
time="2020-01-10T11:42:08Z" level=info msg="Initializing K8S helper frontend." namespace=trident
time="2020-01-10T11:42:08Z" level=info msg="K8S helper determined the container orchestrator version." gitVersion=v1.14.6+17b1cc6 version=1.14+
time="2020-01-10T11:42:08Z" level=info msg="Added frontend." name=k8s_csi_helper
time="2020-01-10T11:42:08Z" level=info msg="Initializing CSI frontend." name=crc-rk2fc-master-0 version=19.10.0
time="2020-01-10T11:42:08Z" level=info msg="Enabling controller service capability." capability=CREATE_DELETE_VOLUME
time="2020-01-10T11:42:08Z" level=info msg="Enabling controller service capability." capability=PUBLISH_UNPUBLISH_VOLUME
time="2020-01-10T11:42:08Z" level=info msg="Enabling controller service capability." capability=LIST_VOLUMES
time="2020-01-10T11:42:08Z" level=info msg="Enabling controller service capability." capability=CREATE_DELETE_SNAPSHOT
time="2020-01-10T11:42:08Z" level=info msg="Enabling controller service capability." capability=EXPAND_VOLUME
time="2020-01-10T11:42:08Z" level=info msg="Enabling controller service capability." capability=CLONE_VOLUME
time="2020-01-10T11:42:08Z" level=info msg="Enabling volume access mode." mode=SINGLE_NODE_WRITER
time="2020-01-10T11:42:08Z" level=info msg="Enabling volume access mode." mode=SINGLE_NODE_READER_ONLY
time="2020-01-10T11:42:08Z" level=info msg="Enabling volume access mode." mode=MULTI_NODE_READER_ONLY
time="2020-01-10T11:42:08Z" level=info msg="Enabling volume access mode." mode=MULTI_NODE_SINGLE_WRITER
time="2020-01-10T11:42:08Z" level=info msg="Enabling volume access mode." mode=MULTI_NODE_MULTI_WRITER
time="2020-01-10T11:42:08Z" level=info msg="Added frontend." name=csi
time="2020-01-10T11:42:08Z" level=info msg="Initializing Trident CRD controller frontend." namespace=trident
time="2020-01-10T11:42:08Z" level=info msg="Creating event broadcaster."
time="2020-01-10T11:42:08Z" level=info msg="Setting up CRD event handlers."
time="2020-01-10T11:42:08Z" level=info msg="Added frontend." name=crd
time="2020-01-10T11:42:08Z" level=info msg="Activating HTTP REST frontend." address="<IP_ADDRESS>:8000"
time="2020-01-10T11:42:08Z" level=info msg="Activating HTTPS REST frontend." address=":8443"
time="2020-01-10T11:42:08Z" level=warning msg="Persistent state version not found, creating." OrchestratorAPIVersion=1 PersistentStoreVersion=crdv1
time="2020-01-10T11:42:08Z" level=info msg="Added 0 existing volume(s)"
time="2020-01-10T11:42:08Z" level=info msg="Trident bootstrapped successfully."
time="2020-01-10T11:42:08Z" level=info msg="Activating K8S helper frontend."
time="2020-01-10T11:42:08Z" level=info msg="Activating CRD frontend."
time="2020-01-10T11:42:08Z" level=info msg="Activating CSI frontend."
time="2020-01-10T11:42:08Z" level=info msg="Starting Trident CRD controller."
time="2020-01-10T11:42:08Z" level=info msg="Waiting for informer caches to sync."
time="2020-01-10T11:42:08Z" level=info msg="Listening for GRPC connections." name=/plugin/csi.sock net=unix
time="2020-01-10T11:42:08Z" level=info msg="Starting workers."
time="2020-01-10T11:42:08Z" level=info msg="Started workers."
time="2020-01-10T11:42:35Z" level=info msg="Added a new node." handler=AddOrUpdateNode node=crc-rk2fc-master-0

Hi @cnu80, the problem arises because of the oc binary not reporting the version in the desired format. As long as you ensure the oc binary is present in the path (of version 4.2) the installer should work.

thanks for effort, I will open a ticket on the code ready project.


⚠ Bofrost.es has degraded performance

In 9255989, Bofrost.es (https://www.bofrost.es) experienced degraded performance:

HTTP code: 200
Response time: 1475 ms

Resolved: Bofrost.es performance has improved in f95714c after 10 minutes.


Remove katpoint

switch from katpoint library to astroplan and astropy

switched to astropy in https://github.com/rishadeb/whatsup/pull/7


New configuration parameter added : allowSerializedHeaders

Only used for TCP when transferExchange is true. When set to true, serializable objects in headers and properties will be added to the exchange.
see issue https://issues.apache.org/jira/browse/CAMEL-10343

@oscerd looks good to merge.

@davsclaus sure :-)
I'm taking care of https://github.com/apache/camel/pull/1191 and then I'll merge this one :-)

@davsclaus, since it's a new parameter do we need to merge only on master branch?

Yeah lets do that

Thanks @nd4av. The PR has been merged! :-) Can you close the PR?
Can you please update the documentation too? Here is the doc page: http://camel.apache.org/netty4.html
You'll need a signed Apache ICLA: http://apache.org/foundation/how-it-works/legal.html
When you will be registered we will grant you Karma to edit documentation.
Thanks!

@oscerd , my ICLA has been approved and I'm registred as<EMAIL_ADDRESS>Thanks for granting.

Dear Nicolas Duffillot,
This message acknowledges receipt of your ICLA, which has been filed in the Apache Software Foundation records.
If you have been invited as a committer, please advise the project PMC that your ICLA has been filed.
Warm Regards,
-- Craig L Russell
Secretary, Apache Software Foundation

Hello @nd4av ,
I'll grant you Karma and let you know! Many thanks for your contributions! :-)

@nd4av Are you nicolas.duffillot on confluence?

Yes, @nd4av = nicolas.duffillot !

I've granted Karma to your account.

OK, thanks.
Is it available as of 2.18 or is it back fixed in earlier versions ?

@nd4av , since it's a new option it is available only from 2.18

Done. Thanks again.

Can you close the PR please ? :-) Thanks!


Bug: annotations are not applied

Looks like there's a regression in 3.0.0 and none of the below annotations apply.
    ingress.appscode.com/annotations-service: |
        {
            "service.beta.kubernetes.io/aws-load-balancer-backend-protocol": "http",
            "service.beta.kubernetes.io/aws-load-balancer-proxy-protocol": "*",
            "service.beta.kubernetes.io/aws-load-balancer-ssl-cert": "arn:aws:acm:..."
        }

    ingress.appscode.com/annotations-service: '{"service.beta.kubernetes.io/aws-load-balancer-ssl-cert": "arn:aws:acm:..."`}'

    ingress.voyager.appscode.com/annotations-service: |
        {
            "service.beta.kubernetes.io/aws-load-balancer-backend-protocol": "http",
            "service.beta.kubernetes.io/aws-load-balancer-proxy-protocol": "*",
            "service.beta.kubernetes.io/aws-load-balancer-ssl-cert": "arn:aws:acm:..."
        }

    ingress.voyager.appscode.com/annotations-service: '{"service.beta.kubernetes.io/aws-load-balancer-ssl-cert": "arn:aws:acm:..."`}'

Oops sorry, I'm getting interesting behavior... Confirming first.

Looks like only the multi-line annotation doesn't get picked up.
    ingress.appscode.com/annotations-service: |
        {
            "service.beta.kubernetes.io/aws-load-balancer-backend-protocol": "http",
            "service.beta.kubernetes.io/aws-load-balancer-proxy-protocol": "*",
            "service.beta.kubernetes.io/aws-load-balancer-ssl-cert": "arn:aws:acm:..."
        }

Also -- the behavior is flaky when updating this annotation multiple times.

@sadlil , can you add  a test for the multi-line annotations?

@tamalsaha i have manually tested this issue against current master branch. This issue is not occurring. I have tried updating the annotations multiple times, targeted updates happens as expected. But we need to stay sharp to maintain both yaml multi-line format and json format. Any invalid json can update the ingress but it will not update the target service or pod.
And single line json map must be within '{}' (single quotes)
@julianvmodesto I am closing the issue. Feel free to reopen if our upcoming release doesn't work for you.


CombatEdit raises the attack damage of SwordItem if lowered with a custom class

This probably affects all tools, but testing with Antrhopophagy, the knives (whose damage is set here) gets raised by 4 attack damage.
With CombatEdit:

Without CombatEdit:

Should be fixed with this commit in release 1.3.2.

Should be fixed with this commit in release 1.3.2.


Auto save is not working for .md files

translationCore Create - v1.4.0 build 113-e01d8f1
Auto save does not function as it does for the TSVs. here is the cosole:

log into the app.
open any .md file.
make edits and then turn off the wifi.
try to save. you see a warning message. close the file.
turn on the wifi and open the same file. Note that the changes are not saved.

@elsylambert waiting for deploy build

@elsylambert @klappy
Confirming: looks like autosave IS working for MD files now on the refactor branch

This is not resolved in translationCore Create - v1.4.2 build 132-6493c98. Tried to save without the internet and the save was not working.


On Mac, it should use Xamarin.Mac instead of MonoMac

MonoMac was an early version of the Mac bindings, and it predates our understanding of how to properly manage memory in the combined .NET and Objective-C worlds.
Xamarin.Mac is an evolution of MonoMac which benefits from all the investments that we did in memory management and startup for iOS and is actively maintained.
Xamarin.Mac is also fully open source: http://github.com/xamarin/xamarin-macios

@kekekeks

@migueldeicaza Is Xamarin.Mac compatible with .NET Core already? Last time I've asked on gitter, I was told, that Xamarin.Mac requires a set of runtime patches for Mono to work.

I think another issue was building on Linux and on Windows (without a Mac host). Xamarin.Mac tooling also seems to not be compatible with .NET Core SDK, i. e. one can't just set TargetFramework property to xamarinmac10 and get it working.

It does not.
MonoMac is broken by design, you are free to keep using it, but it will just produce broken applications.

We could potentially make a separate Xamarin.Mac backend, but the main problem with that is being unable to even build libraries targeting Xamarin.Mac from Windows or Linux.
So having it in the main repo will pretty much force us to use OS X for Avalonia development and greatly limit the amount of people who can contribute.

We can use https://github.com/onovotny/MSBuildSdkExtras to target Xamarin.Mac from the .NET Core SDK projects.

Targets have to already be available on the system, so probably not. We could have it conditionally compile only on the windows solution build.

The problem with Xamarin.Mac is that unlike Xamarin.iOS targets and reference assemblies aren't available even on Windows.

That's what I thought, but I was told otherwise (or that it's a bug if they aren't available) during my internship.

The targets are available on my Windows installation at %VSINSTALLROOT%\MSBuild\Xamarin\Mac.

Our MonoMac bindings are interfering somehow with VSCode/Omnisharp's Intellisense. Removing it makes the whole repo work in vscode under linux (i tested it with or without mono in the system).

It works perfectly fine with Rider and Avalon Studio, so it's probably a bug in OmniSharp.

Visual Studio on Windows can open the Xamarin.Mac projects on Windows
It wont run them, but it will build them.


update missing deps (standalone examples)

currently missing dependencies taken from the main repository.
this PR fix this by explicitly specify deps in examples/package.json

@AsaAyers it works if you wrap  with a function
the original code used 0.14, I just followed it
about 'redux' yes, it depend on it right here:
https://github.com/rackt/redux-router/blob/master/examples/basic/index.js#L3
about peerDependency, I'm unfamiliar with it.
I googled it and I understand it for module which used as a plugin?
like.. jQuery plugin or something? it it the case here?

redux is also required in src/client.js.  If redux-router were going to state that it depends on redux it could be a dependency or peerDependency. In this scenario lets say my project depends on redux@2 and redux-router depends on redux@3
If it is a dependency npm will happily install<EMAIL_ADDRESS>for my code to use and<EMAIL_ADDRESS>would be provided for redux-router to use. I don't know what problems this would cause, but I suspect it would produce errors and I'm sure it's not what is intended. Any errors here would happen at runtime.
If instead, it were a peerDependency then npm can complain that it can't find a single version of redux that satisfies both dependencies. This avoids the runtime error and instead causes the failure to show up during npm install
The 3rd option is to just not specify that dependency. It currently works because my project's dependencies will install a version of redux and redux-router will just use whatever it gets from require('redux'). The biggest problem here is that it would be possible to install a redux that isn't compatible with the router and you won't get an error about it until runtime.

I see.
so, if I understand it right,
redux-router should have a peerDependency of redux
while the examples should have it has dependency

I'd leave that up to the owners here to decide, but that's the option I think I'd chose. I haven't upgraded to npm@3 yet, so this is based on my experience with npm@2 and what I have read about the upcoming changes.

We experimented with peerDependency, this is still causing trouble with some npm versions. I think the current state of dependencies is looks fine and it doesn´t seem to cause any major issues.


[C++] Memory Usage of Parquet Stream Writer

Currently I am working on a tool that needs to write a previously unknown amount of data to a parquet file.
The parquet::StreamWriter seems to be the correct solution for this.
I noticed that the memory usage of process increases over time.
This would not be the case if I were to write to a csv file, for example.
The used memory seems to roughly double every time it grows. This indicates that the memory is allocated by a std::vector or a container type which has the same exponential growth behavior.
Is there a reason that the stream writer requires this increasing amount of memory as the output file grows?
arrow version: release 5.0.0
minimal example:
#include <fstream>
#include <iostream>
#include <string>

#include "arrow/io/file.h"
#include "parquet/exception.h"
#include "parquet/stream_writer.h"

constexpr size_t COLUMNS = 100;
constexpr size_t ROWS_IN_ROW_GROUP = 4000;

constexpr size_t ROW_GROUPS = 5000;

std::shared_ptr<parquet::schema::GroupNode> get_schema() {
    parquet::schema::NodeVector fields;
    for (int i = 0; i < COLUMNS; ++i) {
        fields.push_back(parquet::schema::PrimitiveNode::Make(std::string("column_field_uint32_") + std::to_string(i),
                                                              parquet::Repetition::OPTIONAL, parquet::Type::INT32,
                                                              parquet::ConvertedType::UINT_32));

        fields.push_back(parquet::schema::PrimitiveNode::Make(std::string("column_field_float_") + std::to_string(i),
                                                              parquet::Repetition::OPTIONAL, parquet::Type::FLOAT,
                                                              parquet::ConvertedType::NONE));

        fields.push_back(parquet::schema::PrimitiveNode::Make(std::string("column_field_uint16_") + std::to_string(i),
                                                              parquet::Repetition::OPTIONAL, parquet::Type::INT32,
                                                              parquet::ConvertedType::UINT_16));

        fields.push_back(parquet::schema::PrimitiveNode::Make(std::string("column_field_uint8_") + std::to_string(i),
                                                              parquet::Repetition::OPTIONAL, parquet::Type::INT32,
                                                              parquet::ConvertedType::UINT_8));
    }

    return std::static_pointer_cast<parquet::schema::GroupNode>(
        parquet::schema::GroupNode::Make("schema", parquet::Repetition::REQUIRED, fields));
}

// print the memory usage by printing the line starting with VmSize: of /proc/self/status
void print_memory_usage() {
    std::ifstream file;
    file.open("/proc/self/status");
    std::string line;
    const std::string vmsize = "VmSize:";
    while (file) {
        std::getline(file, line);
        auto idx = line.find(vmsize);
        if (idx != std::string::npos) {
            std::cout << line.substr(idx + vmsize.length());
            break;
        }
    }
    file.close();
}

int main() {
    std::shared_ptr<arrow::io::FileOutputStream> outfile;

    PARQUET_ASSIGN_OR_THROW(outfile, arrow::io::FileOutputStream::Open("very_big.parquet"));

    parquet::WriterProperties::Builder builder;
    builder.compression(parquet::Compression::SNAPPY);

    parquet::StreamWriter os{parquet::ParquetFileWriter::Open(outfile, get_schema(), builder.build())};
    os.SetMaxRowGroupSize(0);  // row groups will be ended manually with os << parquet::EndRowGroup.

    for (int i = 0; i < ROW_GROUPS; ++i) {
        if (i % 100 == 0) {  // print memory usage every 100 row groups
            std::cout << "group " << i << " from " << ROW_GROUPS << ", memory usage: ";
            print_memory_usage();
            std::cout << std::endl;
        }

        for (int j = 0; j < ROWS_IN_ROW_GROUP; ++j) {
            for (int k = 0; k < COLUMNS; ++k) {
                os << static_cast<uint32_t>(i * k * j);
                os << static_cast<float>(i * k * j * 0.3333);
                os << static_cast<uint16_t>(i * k * j * 2);
                os << static_cast<uint8_t>((i * k * j) % 255);
            }
            os << parquet::EndRow;
        }
        os << parquet::EndRowGroup;
    }
}

The produced output is:
group 0 from 5000, memory usage:           51492 kB
group 100 from 5000, memory usage:        249672 kB
group 200 from 5000, memory usage:        286528 kB
group 300 from 5000, memory usage:        360200 kB
group 400 from 5000, memory usage:        360200 kB
group 500 from 5000, memory usage:        360200 kB
group 600 from 5000, memory usage:        507556 kB
group 700 from 5000, memory usage:        507556 kB
group 800 from 5000, memory usage:        507556 kB
group 900 from 5000, memory usage:        507556 kB
group 1000 from 5000, memory usage:       507556 kB
group 1100 from 5000, memory usage:       802260 kB
group 1200 from 5000, memory usage:       802260 kB
group 1300 from 5000, memory usage:       802260 kB
group 1400 from 5000, memory usage:       802260 kB
group 1500 from 5000, memory usage:       802260 kB
group 1600 from 5000, memory usage:       802260 kB
group 1700 from 5000, memory usage:       802260 kB
group 1800 from 5000, memory usage:       802260 kB
group 1900 from 5000, memory usage:       802260 kB
group 2000 from 5000, memory usage:       802260 kB
group 2100 from 5000, memory usage:      1391668 kB
group 2200 from 5000, memory usage:      1391668 kB
group 2300 from 5000, memory usage:      1391668 kB
group 2400 from 5000, memory usage:      1391668 kB
group 2500 from 5000, memory usage:      1391668 kB
group 2600 from 5000, memory usage:      1391668 kB
group 2700 from 5000, memory usage:      1391668 kB
group 2800 from 5000, memory usage:      1391668 kB
group 2900 from 5000, memory usage:      1391668 kB
group 3000 from 5000, memory usage:      1391668 kB
group 3100 from 5000, memory usage:      1391668 kB
group 3200 from 5000, memory usage:      1391668 kB
group 3300 from 5000, memory usage:      1391668 kB
group 3400 from 5000, memory usage:      1391668 kB
group 3500 from 5000, memory usage:      1391668 kB
group 3600 from 5000, memory usage:      1391668 kB
group 3700 from 5000, memory usage:      1391668 kB
group 3800 from 5000, memory usage:      1391668 kB
group 3900 from 5000, memory usage:      1391668 kB
group 4000 from 5000, memory usage:      1391668 kB
group 4100 from 5000, memory usage:      2570104 kB
group 4200 from 5000, memory usage:      2570104 kB
group 4300 from 5000, memory usage:      2570104 kB
group 4400 from 5000, memory usage:      2570104 kB
group 4500 from 5000, memory usage:      2570104 kB
group 4600 from 5000, memory usage:      2570104 kB
group 4700 from 5000, memory usage:      2570104 kB
group 4800 from 5000, memory usage:      2570104 kB
group 4900 from 5000, memory usage:      2570104 kB

The short, though maybe unsatisfying, answer is that it is row group information and it is expected.  The parquet file writer is going to keep some data in memory for each row group that is created.  This is used to populate the footer of the parquet file.  To verify this simply rerun your experiment with 1 or 10 rows per row group and you will see the exact same memory usage patterns.
One thing to note is that you are creating 400 columns even if you are only using 100 of them.  This makes the row group metadata about 4x larger than it needs to be.
There is probably room for improvement as to what information we really need to keep.  I don't know that any serious optimization passes have been done on this front.  I believe the answer is usually to use large row groups and multiple files if needed to prevent ever creating a single file with too much row group metadata information.

The used memory seems to roughly double every time it grows. This indicates that the memory is allocated by a std::vector or a container type which has the same exponential growth behavior.

This is probably just an artifact of the way you are measuring memory usage.  The VSZ/RSS numbers tend to grow in chunks because malloc will try and reduce the # of allocations it needs to make.

Thanks for the answer!

One thing to note is that you are creating 400 columns even if you are only using 100 of them. This makes the row group metadata about 4x larger than it needs to be.

I am actually  also using all 400 columns but i see that my example is a little confusing.

This is probably just an artifact of the way you are measuring memory usage. The VSZ/RSS numbers tend to grow in chunks because malloc will try and reduce the # of allocations it needs to make.

Ah yes i forgot about that.


[CI]: Run Integration Tests in GH Actions

We might be able to do some basic integration tests, or rather just run stuff and expect it to work (no idea how to perform an actual integration test for this)
Given:
https://stackoverflow.com/a/57549440/12970202
it seems we can use a docker container in GH Actions so we might be able to setup a particular database to run some basic migrations against the current database

already part of vnext


UnionToIntersection

I use this library all the time for reference but I couldn't find this type in here:
https://stackoverflow.com/questions/56213182/how-to-merge-the-properties-of-all-objects-in-a-union-type/56213252
type UnionToIntersection<U> =
    (U extends any ? (k: U) => void : never) extends
    ((k: infer I) => void) ? I : never

this? :sweat_smile:
would you expect it to be in intersection rather than union?

Lol. I couldn't find it! I think intersection was the word that I was unaware of. I was thinking in my head something called MergeUnion. Is intersection the name for the & operator?

hahaha. I think so yeah! :)


Unable to deploy notebook or tensorboard servers

Describe the bug
When trying to setup a notebook server or a tensorboard server when deploying kubeflow on eks I get the following error:
[403] Could not find CSRF cookie XSRF-TOKEN in the request.
Steps To Reproduce
Follow the Vanilla guide for v1.3 aws deployment of kubeflow
Expected behavior
Expected behaviour was to deploy a notebook or tensorboard server
Environment

Kubernetes version 1.19
Using EKS: Latest
None, was deployed using an EC2 instance however

Screenshots
If applicable, add screenshots to help explain your problem.
Additional context
I assume this isn't working because the load balancer provides http not https and that is why it is failing. I see posts about similar where they use APP_SECURE_COOKIES=false or something similar. However it doesn't say where to put that. I assume in the env file?

This is actually urgent for me. I would love a response if possible

Hi @DanielLund, looking into this. I was able to reproduce the issue

Since the load balancer endpoint is not using HTTPS, the solution as the documentation suggests is to set APP_SECURE_COOKIES: false in all the respective webapp deployments. for e.g. in this case you would change in the jupyter-web-app deployment
kubectl edit deployments -n kubeflow jupyter-web-app-deployment
under spec.template.spec.containers.env add the environment variable
- env:
  - name: APP_SECURE_COOKIES
    value: "false"

and restart the deployment for the changes to take effect
kubectl -n kubeflow rollout restart deployment/jupyter-web-app-deployment
However, this approach is insecure and I would not recommend this to use in production. Next step is to document how to expose the deployment over HTTPS. I will update #67

Hi @DanielLund, Please find the updated documentation here - https://github.com/awslabs/kubeflow-manifests/issues/67#issuecomment-1059566247

@surajkota I cannot thank you enough for this. This is fantastic


community web page  for fission yeast international meetings

It would be great to have a list of these, with a link to the conference website and , if avaiable to the electronic abstract book.
I'm trying to find something of Takashi's, that I think was in Japan  (Tokyo meeting).
Lost...

I'm stuck because I can't remember the gene name :(

Not urgent (I found the gene).
Do we already link to some abstract books anyway?
I thought we did but O could n't find them

The last four meetings are listed on the Documents download page, https://www.pombase.org/documents. The list could move to its own page if you prefer, but the individual meeting pages already link to almost everything I've been able to find to date.
The 2013 London meeting is the only one with its only abstract book PDF linked right now. I also have a password-protected PDF for the 2015 Kobe meeting. The password may be the reason we haven't linked it up, but I think I could make a passwordless copy.
I don't have any of the information on the meetings before I joined PomBase, i.e. before 2011. I can see how many conference website URLs I can dig out of the email archive, and which ones still work.

OK, once we have a bosic list we can ask if any ex conferrence organiszers can supply an electronic abstract book, or link.
I will also see what I have Re past meetings.
@bahler probably you have a list maybe?

Hi Jurg,
If you click on the link in the bottom of this e-mail it will take you to the full thread ;)
We want to create a page with links to past conferences, and if possible abstract books.
I wondered if you had any info....or is there a scientific committee who can provide the info?
Val

I don’t have a list of pombe meetings, but this could be put together based on distributed information. I should have a PDF of Pombe 2013.
Cheers,
-Jürg
From: Val Wood<EMAIL_ADDRESS>Sent: 11 October 2018 15:43
To: pombase/website<EMAIL_ADDRESS>Cc: Bahler, Jurg<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [pombase/website] community web page for fission yeast international meetings (#962)
Hi Jurg,
If you click on the link in the bottom of this e-mail it will take you to the full thread ;)
We want to create a page with links to past conferences, and if possible abstract books.
I wondered if you had any info....or is there a scientific committee who can provide the info?
Val
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/pombase/website/issues/962#issuecomment-428977438, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ANk7VKWBlUgChC7DSnCSbdo0jPpu7iZYks5uj1kRgaJpZM4XBDmW.

I think we can close this ticket.
But how do I get to this page (which menu is it under?)
https://www.pombase.org/documents

which menu is it under?

Downloads

makes sense!

I looked under 3 others, I don't know why I didn't look here!


[Bugfix] Workflow Fix

fixes https://github.com/code100x/cms/issues/19 https://github.com/code100x/cms/issues/23
This PR addresses three issues:
The requirement is to run the workflow on each pull request made to this repository. To achieve this, we need to use the push trigger instead of pull_request.
It resolves several conflicts between eslint and prettier.
a. It changes single quotes to double quotes.
b. The eslint indentation check is removed as prettier and eslint indent do not work together due to differing indentation styles. Reference: https://stackoverflow.com/questions/56337176/prettier-and-eslint-indents-not-working-together
c. Removed unnecessary semicolon.

@hkirat, you can now merge this PR. After it's merged, I'll create another PR to resolve all the formatting issues.

This will require a wholesale lint
Will do after PRs reach ~0

Okay, @hkirat.


fix(msgPublish): payload changed after send msg

Issue Number
Example: #705
What is the new behavior?
Please describe the new behavior or provide screenshots.
Does this PR introduce a breaking change?

[ ] No

Specific Instructions
Are there any specific instructions or things that should be known prior to review?
Other information

When sending a message, isLoadData is undefined, which will not affect the sending of the message, nor will it affect the isNewPayload parameter


Metrics collection does not work correctly with the default AWS EC2 metadata options for AWS Node Templates

Is an existing page relevant?
No response
What karpenter features are relevant?
AWS Node Templates
How should the docs be improved?
Unsure if the docs need improved as this is probably related to certain types of metrics collection: specifically adding tags to any metrics generated in a cluster, on a node created by an AWS Node Template, using Datadog, but it may affect other users.
Opening this issue for future searchers more than anything, it can probably just be closed.
When using the default metadata options suggested by the AWS Node Template documentation, access to the IMDS from the host is restricted. In our use case this stopped all metrics generated in the cluster from being tagged, making them unavailable for graphing / monitors.
Setting the following resolved the issue - this way the other suggested security defaults should be used.
spec:
  metadataOptions:
    httpTokens: optional

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Thanks for opening the ticket, if this becomes a repeated issue we may consider adding a FAQ entry for it.


Error on pod config

Hi, I'm getting an error while trying to run the last command in the tutorial. This command is supposed to create the sample pod that uses the SecretProviderClass and the user-assigned identity.
The error I'm getting is:
Error from server (BadRequest): error when creating "pod.yaml": Pod in version "v1" cannot be handled as a Pod: json: cannot unmarshal object into Go struct field PodSpec.spec.containers of type []v1.Container

Thanks for any pointers on how to solve this issue!

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: ec9ff712-aa4c-4e95-ba96-2405010d9f83
Version Independent ID: f553a5a4-a20f-8efd-3712-e64112c35676
Content: Provide an access identity to the Azure Key Vault Provider for Secrets Store CSI Driver for Azure Kubernetes Service (AKS) secrets - Azure Kubernetes Service
Content Source: articles/aks/csi-secrets-store-identity-access.md
Service: azure-kubernetes-service
GitHub Login: @Nickomang
Microsoft Alias: nickoman

So the error on the config is that the "containers" field should be a list. So adding a '-' to the left of "name" should fix the issue. The yaml file would end up like this:
# This is a sample pod definition for using SecretProviderClass and the user-assigned identity to access your key vault
kind: Pod
apiVersion: v1
metadata:
  name: busybox-secrets-store-inline-user-msi
spec:
  containers:
      - name: busybox
        image: registry.k8s.io/e2e-test-images/busybox:1.29-4
        command:
          - "/bin/sleep"
          - "10000"
        volumeMounts:
        - name: secrets-store01-inline
          mountPath: "/mnt/secrets-store"
          readOnly: true
  volumes:
    - name: secrets-store01-inline
      csi:
        driver: secrets-store.csi.k8s.io
        readOnly: true
        volumeAttributes:
          secretProviderClass: "azure-kvname-user-msi"

Cheers!

@userr2232
Thanks for your feedback! We will investigate and update as appropriate.

@userr2232
Thanks for reporting this! We have created a PR for this issue and the changes should go live soon.


ARTEMIS-1874 fix NPE setting object property

Upstream Issue: https://issues.apache.org/jira/browse/ARTEMIS-1874
Issue: https://issues.jboss.org/browse/JBEAP-15516
Upstream PR: https://github.com/apache/activemq-artemis/pull/2336
This is just a backport of the fix into the 1.x branch. I've setup the PR just to make the backport easy for devs, if you prefer to cherry-pick this fix later on yourself, feel free to simply close this PR! (or ask me to do it).
Note that the commit is already in the 2.6.3.jbossorg-x branch : https://github.com/rh-messaging/jboss-activemq-artemis/commit/4d492bea0ee5fb00dcceca9366375cb87d32fe94

That last note should be your upstream reference. :)

That last note should be your upstream reference. :)

I thought the link to the commit in the branch will be more definitive :)

Coupled with https://issues.jboss.org/browse/ENTMQBR-1970 or not?

I've added it, next to the JBEAP one, it can't hurt to have it mention for sure.

Just to check on things: this PR has been merged yet, is there anything I should be actively doing to make it happen? Or should I just be patient :)


Width have a BUG

What is this ?

what was happend?

I uninstall VS-code
I delete .code file form user file
I can't do this no more,I need some help

##FYI

This line here can also change it's width.

AND THIS IS RIDICULOUS

What problem am I supposed to be focusing on? The viewport of the editor is too narrow?
雖然我看到中文，你是指編輯器的視窗縮小了範圍嗎? 還是說英文好了。
(Translation: Even though I saw Chinese (in your image), are you referring to your editor's window being narrow? I guess it's best to reply in English.)

好好,我说中文吧,我的意思是这个编辑器的宽度应该是不可以调整的,在其他设备上使用的时候没有遇到过这种情况,编辑窗口本身竟然还能调节宽度,而且在某且情况下,比如说我把Terminal 放在右侧,然后这个宽度就会缩小并且需要你手动改变宽度,我不知道这是什么原因造成的,我在设置里没有找到EditorWidth的任何信息,究竟是什么原因.

@isidorn  You really is my savior!!!Thank you so much.