Improve powershell startup time

PR Summary
With this PR, the startup time of pwsh gets about 24.5% improvement on average comparing with the PS Core 6.1.0 official release. The following is an example measurement result by using BenchmarkDotNet v0.11.2 (the detailed information about this measurement result can be found at the end):
# Warm startup -- with module analysis cache pre-populated
pwsh.exe -noprofile -c echo 1

           Method |     Mean |    Error |    StdDev |
----------------- |---------:|---------:|----------:|
           Pwsh61 | 540.5 ms | 9.381 ms |  8.316 ms |
            WinPS | 332.9 ms | 6.497 ms | 10.675 ms |
 pwsh62_optimized | 408.3 ms | 8.936 ms |  9.176 ms |

Detailed Changes

Avoid SecuritySupport.IsProductBinary and unnecessary AMSI/suspicious code scan at startup time for a regular session

Update CompiledScriptBlockData.IsProductCode to avoid unnecessary calls to IsProductBinary, which attempts to retrieve catalog signature of the target file.
Update PerformSecurityChecks to skip AMSI and suspicious code scan for the .psd1 file that contains a safe HashtableAst only.

Use customized ReadOnlyBag instead of ImmutableHashSet so that we can avoid loading the System.Collections.Immutable.dll completely.
Replace SHA1 with CRC32 when generating module analysis cache file name

This remove the loading of System.Security.Cryptography.Algorithms.dll at startup

Move ConvertFrom-SddlString to C# to remove the Utility.psm1 file.
Crossgen Microsoft.ApplicationInsights.dll and enable tiered compilation

Even pwsh with crossgen assemblies spends a lot time in jitting at the startup, about 191.6ms comparing with 24.7ms for Windows PowerShell.
Jitting Microsoft.ApplicationInsights.dll took about 51.6ms.
By crossgen Microsoft.ApplicationInsights.dll and enable tiered compilation, the jitting time drops to about 98.9ms. Detailed information can be found at the end.

NOTE: Each commit is self-contained and the purpose of the commit is described by the commit message. So it would be easier to review per commit.
Compared with Windows PowerShell
The optimized pwsh is still about 75ms slower than Windows PowerShell. The main reason is jitting time difference, even though we crossgen all the assemblies in the table below except for pwsh.dll.
More work will be done to further bring down the jitting time at startup.
Reference Set Improvement
Impacting memory size loaded from disk (total): 5.9% less pages (606 / 10289 pages), 6% less memory (2.4 / 40.2 mb). The comparison of the image pages being loaded during startup before and after this PR is as follows:
TotalImagesAdded   : 0
TotalImagesRemoved : 15
TotalImagesChanged : 19
TotalPageChange    : -556

   type: Remove

type   image                                       pages
----   -----                                       -----
Remove AppxSip.dll                                 21
Remove coml2.dll                                   31
Remove iertutil.dll                                42
Remove mintdh.dll                                  22
Remove msisip.dll                                  7
Remove OpcServices.dll                             53
Remove pwrshsip.dll                                11
Remove System.Collections.Immutable.dll            74
Remove System.Security.Cryptography.Algorithms.dll 38
Remove System.Security.Permissions.dll             14
Remove System.Threading.AccessControl.dll          6
Remove tdh.dll                                     15
Remove urlmon.dll                                  66
Remove wshext.dll                                  12
Remove xmllite.dll                                 13

   type: Update

type   image                                             pages
----   -----                                             -----
Update clrjit.dll                                        -4
Update combase.dll                                       1
Update coreclr.dll                                       -10
Update crypt32.dll                                       -70
Update KernelBase.dll                                    -3
Update Microsoft.PowerShell.Commands.Utility.dll         4
Update msasn1.dll                                        -4
Update msvcrt.dll                                        -1
Update ntdll.dll                                         1
Update rpcrt4.dll                                        -2
Update sechost.dll                                       -5
Update SHCore.dll                                        -10
Update shell32.dll                                       -1
Update System.Private.CoreLib.dll                        -7
Update System.Runtime.Extensions.dll                     -1
Update System.Security.Cryptography.Primitives.dll       -3
Update System.Security.Cryptography.X509Certificates.dll -2
Update windows.storage.dll                               2
Update wintrust.dll                                      -16

Jitting Time Improvement
Windows PowerShell

Windows PowerShell Assembly Name
JitTime   msec
Num   Compilations
IL   Size
Native   Size
Foreground   msec
Multicore   JIT Background   msec
Tiered   Compilation Background   msec

TOTAL
24.7
83
1,506
16,149
24.7
0.0
0.0

Anonymously Hosted DynamicMethods Assembly
19.3
44
0
12,517
19.3
0.0
0.0

C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL...\System.Management.Automation.dll
4.3
38
1,181
2,782
4.3
0.0
0.0

C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL...\System.Core.dll
1.1
1
325
850
1.1
0.0
0.0

Before crossgen'ing Microsoft.ApplicationInsights.dll and applying tiered compilation

PWSH   Assembly Name
JitTime   msec
Num   Compilations
IL   Size
Native   Size
Foreground   msec
Multicore   JIT Background   msec
Tiered   Compilation Background   msec

TOTAL
191.6
488
43,105
140,540
151.2
40.4
0.0

E:\PowerShell\bin\latest\Microsoft.ApplicationInsights.dll
51.6
147
9,866
30,735
51.6
0.0
0.0

E:\PowerShell\bin\latest\System.Management.Automation.dll
44.4
25
14,661
44,123
4.0
40.4
0.0

E:\PowerShell\bin\latest\System.Private.CoreLib.dll
36.0
143
11,947
20,691
36.0
0.0
0.0

E:\PowerShell\bin\latest\Microsoft.PowerShell.CoreCLR.Eventing.dll
22.1
80
0
23,342
22.1
0.0
0.0

E:\PowerShell\bin\latest\System.Collections.Concurrent.dll
17.6
34
4,018
12,261
17.6
0.0
0.0

E:\PowerShell\bin\latest\System.Collections.dll
9.3
34
1,317
3,542
9.3
0.0
0.0

E:\PowerShell\bin\latest\System.Linq.dll
7.6
17
1,085
4,911
7.6
0.0
0.0

Anonymously Hosted DynamicMethods Assembly
1.4
6
0
543
1.4
0.0
0.0

E:\PowerShell\bin\latest\Newtonsoft.Json.dll
0.8
1
196
357
0.8
0.0
0.0

E:\PowerShell\bin\latest\pwsh.dll
0.7
1
15
35
0.7
0.0
0.0

After crossgen Microsoft.ApplicationInsights.dll and applying tiered compilation

Updated   PWSH Assembly Name
JitTime   msec
Num   Compilations
IL   Size
Native   Size
Foreground   msec
Multicore   JIT Background   msec
Tiered   Compilation Background   msec

TOTAL
98.9
422
34,500
144,231
78.3
20.5
0.0

E:\PowerShell\bin\base\System.Private.CoreLib.dll
26.7
181
12,442
36,649
26.7
0.0
0.0

E:\PowerShell\bin\base\System.Management.Automation.dll
22.7
25
14,661
51,754
2.4
20.3
0.0

E:\PowerShell\bin\base\Microsoft.PowerShell.CoreCLR.Eventing.dll
22.1
80
0
22,778
22.1
0.0
0.0

E:\PowerShell\bin\base\System.Collections.Concurrent.dll
9.9
53
4,429
19,337
9.6
0.3
0.0

E:\PowerShell\bin\base\System.Collections.dll
7.0
52
1,546
6,856
7.0
0.0
0.0

E:\PowerShell\bin\base\System.Linq.dll
5.3
23
1,211
5,770
5.3
0.0
0.0

Anonymously   Hosted DynamicMethods Assembly
4.1
6
0
543
4.1
0.0
0.0

E:\PowerShell\bin\base\pwsh.dll
0.7
1
15
51
0.7
0.0
0.0

E:\PowerShell\bin\base\Newtonsoft.Json.dll
0.3
1
196
493
0.3
0.0
0.0

Startup Detailed Measurement Record
// * Detailed results *
PSStartupTime.Pwsh61: Core(Runtime=Core)
Runtime = .NET Core 2.1.6 (CoreCLR 4.6.27019.06, CoreFX 4.6.27019.05), 64bit RyuJIT; GC = Concurrent Workstation
Mean = 540.5245 ms, StdErr = 2.2226 ms (0.41%); N = 14, StdDev = 8.3163 ms
Min = 528.3226 ms, Q1 = 535.3209 ms, Median = 538.9606 ms, Q3 = 544.9347 ms, Max = 559.4117 ms
IQR = 9.6138 ms, LowerFence = 520.9002 ms, UpperFence = 559.3554 ms
ConfidenceInterval = [531.1432 ms; 549.9058 ms] (CI 99.9%), Margin = 9.3813 ms (1.74% of Mean)
Skewness = 0.66, Kurtosis = 2.66, MValue = 2
-------------------- Histogram --------------------
[525.303 ms ; 539.307 ms) | @@@@@@@
[539.307 ms ; 562.431 ms) | @@@@@@@
---------------------------------------------------

PSStartupTime.WinPS: Core(Runtime=Core)
Runtime = .NET Core 2.1.6 (CoreCLR 4.6.27019.06, CoreFX 4.6.27019.05), 64bit RyuJIT; GC = Concurrent Workstation
Mean = 332.9168 ms, StdErr = 1.8044 ms (0.54%); N = 35, StdDev = 10.6749 ms
Min = 316.7778 ms, Q1 = 323.8313 ms, Median = 330.4381 ms, Q3 = 340.1557 ms, Max = 356.9562 ms
IQR = 16.3244 ms, LowerFence = 299.3447 ms, UpperFence = 364.6423 ms
ConfidenceInterval = [326.4197 ms; 339.4139 ms] (CI 99.9%), Margin = 6.4971 ms (1.95% of Mean)
Skewness = 0.47, Kurtosis = 2.28, MValue = 2.2
-------------------- Histogram --------------------
[313.922 ms ; 324.935 ms) | @@@@@@@@@@
[324.935 ms ; 332.120 ms) | @@@@@@@@@
[332.120 ms ; 341.104 ms) | @@@@@@@@@
[341.104 ms ; 348.303 ms) | @@@
[348.303 ms ; 359.812 ms) | @@@@
---------------------------------------------------

PSStartupTime.pwsh62_optimized: Core(Runtime=Core)
Runtime = .NET Core 2.1.6 (CoreCLR 4.6.27019.06, CoreFX 4.6.27019.05), 64bit RyuJIT; GC = Concurrent Workstation
Mean = 408.2847 ms, StdErr = 2.2256 ms (0.55%); N = 17, StdDev = 9.1764 ms
Min = 387.1787 ms, Q1 = 403.7134 ms, Median = 409.1181 ms, Q3 = 414.1094 ms, Max = 428.8114 ms
IQR = 10.3960 ms, LowerFence = 388.1194 ms, UpperFence = 429.7034 ms
ConfidenceInterval = [399.3489 ms; 417.2204 ms] (CI 99.9%), Margin = 8.9358 ms (2.19% of Mean)
Skewness = -0.18, Kurtosis = 3.46, MValue = 2
-------------------- Histogram --------------------
[384.056 ms ; 395.103 ms) | @
[395.103 ms ; 403.504 ms) | @@@
[403.504 ms ; 418.618 ms) | @@@@@@@@@@@@
[418.618 ms ; 431.934 ms) | @
---------------------------------------------------

// * Summary *

BenchmarkDotNet=v0.11.2, OS=Windows 10.0.17763.134 (1809/October2018Update/Redstone5)
Intel Core i7-6700 CPU 3.40GHz (Skylake), 1 CPU, 8 logical and 4 physical cores
.NET Core SDK=2.1.500
  [Host] : .NET Core 2.1.6 (CoreCLR 4.6.27019.06, CoreFX 4.6.27019.05), 64bit RyuJIT
  Core   : .NET Core 2.1.6 (CoreCLR 4.6.27019.06, CoreFX 4.6.27019.05), 64bit RyuJIT

Job=Core  Runtime=Core

           Method |     Mean |    Error |    StdDev |
----------------- |---------:|---------:|----------:|
           Pwsh61 | 540.5 ms | 9.381 ms |  8.316 ms |
            WinPS | 332.9 ms | 6.497 ms | 10.675 ms |
 pwsh62_optimized | 408.3 ms | 8.936 ms |  9.176 ms |

// * Hints *
Outliers
  PSStartupTime.Pwsh61: Core           -> 1 outlier  was  removed
  PSStartupTime.pwsh62_optimized: Core -> 1 outlier  was  detected

// * Legends *
  Mean   : Arithmetic mean of all measurements
  Error  : Half of 99.9% confidence interval
  StdDev : Standard deviation of all measurements
  1 ms   : 1 Millisecond (0.001 sec)

// ***** BenchmarkRunner: End *****
Run time: 00:00:38 (38.84 sec), executed benchmarks: 3

PR Checklist

[x] PR has a meaningful title

Use the present tense and imperative mood when describing your changes

[x] Summarized changes
[x] Change is not breaking
[x] Make sure all .h, .cpp, .cs, .ps1 and .psm1 files have the correct copyright header
[x] This PR is ready to merge and is not Work in Progress.

If the PR is work in progress, please add the prefix WIP: to the beginning of the title and remove the prefix when the PR is ready.

User-facing changes

[ ] Not Applicable
OR
[x] User-facing Documentation needed

[ ] Issue filed - Issue link:

Testing - New and feature

[ ] Not Applicable or can only be tested interactively
OR
[x] Make sure you've added a new test if existing tests do not effectively test the code changed

[x] Add [feature] if the change is significant or affects feature tests

More work will be done to further bring down the jitting time at startup.

Do you continue in the PR?

After crossgen Microsoft.ApplicationInsights.dll and applying tiered compilation

I don't see Microsoft.ApplicationInsights.dll in the table - no jitting?
I searched "Insights" in our code base and have questions:

Do we really need types from the dll in CorePsTypeCatalog.cs? We could remove 239 lines.
The dll is used for telemetry. Do we really need load the dll at startup? Seems we could delay the telemetry
The telemetry is a static class. We have other static classes. I guess some static classes is initialized at startup (I guess they is initialized at first use time) - does this slow down startup? Make sense to analyze this and try to delay static classes initializations?

If ReadOnlyHashSet was rewrited I think we could look other classes used in PowerShell Core engine. Maybe it won't speed up the startup but increase performance.
Outliers
  PSStartupTime.Pwsh61: Core           -> 2 outliers were removed
  PSStartupTime.WinPS: Core            -> 4 outliers were removed
  PSStartupTime.pwsh62_optimized: Core -> 1 outlier  was  removed

having Outliers it is not good in the test - seems interference from other processes.

Intel Core i7-6700 CPU 3.40GHz (Skylake), 1 CPU, 8 logical and 4 physical cores

I think that an ordinary user computer is 2 physical cores and it would be correct to limit the test to two physical cores. (This may be related to tied compilation)

I restarted CI-MacOs.

@iSazonov Reply to your comment here:

More work will be done to further bring down the jitting time at startup.

Do you continue in the PR?

No, that will be in a new PR. The planned work are as follows:

Generate PDB for crossgen'ed assemblies that can be used by PerfView/WPA, so that I can do further stack analysis on CPU time.
Crossgen'ed assemblies still leave a high cost of jitting time comparing with ngen. It's probably by design as ngen is targeting a specific CPU while crossgen is targeting a general x64 architecture, but I need to contact .NET Core team to get better understanding and see if there are other options (profile-guided-opt?)
Ad-hoc code changes to avoid unnecessary work at startup time.

I don't see Microsoft.ApplicationInsights.dll in the table - no jitting?

After the change, no IL code needs to be jitted for this dll at startup time.

Do we really need types from the dll in CorePsTypeCatalog.cs? We could remove 239 lines.

Yes we do. Type conversion depends on the type catalog. Not all .NET Core assemblies shipped with PS Core are loaded at startup. Today, user can use a type even it's from an assembly that is not loaded yet, because powershell knows which assembly the type is from by looking in the type catalog. Without it, the user needs to know which assembly the type is in, and then load the assembly first before using the type. The type catalog is also used in tab completion, so that powershell can tab complete types that are not loaded yet.

The dll is used for telemetry. Do we really need load the dll at startup? Seems we could delay the telemetry

ApplicationInsight telemetry is done in console.dll at startup, as long as the user hasn't opted out.

The telemetry is a static class. We have other static classes. I guess some static classes is initialized at startup (I guess they is initialized at first use time) - does this slow down startup? Make sense to analyze this and try to delay static classes initializations?

The work that is done in this PR are based on the analysis of Reference Set and Jitting time. Some ad-hoc analysis on the static members might be useful, but I think our efforts should be guided by the measurement data, so our time can be spent on things that matters most.

If ReadOnlyHashSet was rewrited I think we could look other classes used in PowerShell Core engine. Maybe it won't speed up the startup but increase performance.

The reason to change ImmutableHashSet to ReadOnlyHashSet is because this only use of ImmutableHashSet causes System.Collections.Immutable.dll to be loaded at startup, which brings in 74 pages loaded from disk. This shows up as part of the reference set analysis. It might be useful to look at other classes, but again, I think our efforts should be guided by the measurement data.

having Outliers it is not good in the test - seems interference from other processes.

There is no interference from other pwsh processes when running the tests. BenchmarkDotNet starts each process many times, and sometimes there are outliers.

I think that an ordinary user computer is 2 physical cores and it would be correct to limit the test to two physical cores. (This may be related to tied compilation)

The baseline is the same, and here we care most about the percentage of improvement instead of the actual startup time.

The macOS failure appears to be a test code issue:
2018-11-27T12:24:13.7076000Z Description: Basic os_log tests on MacOS/
2018-11-27T12:24:13.7079830Z Name:        Verifies scriptblock logging
2018-11-27T12:24:13.7084250Z message:
2018-11-27T12:24:13.7088810Z did not recieve at least 18 records but 47 instead.
2018-11-27T12:24:13.7094210Z stack-trace:
2018-11-27T12:24:13.7097880Z at <ScriptBlock>, /Users/vsts/agent/2.142.1/work/1/s/test/tools/Modules/PSSysLog/PSSysLog.psm1: line 904
2018-11-27T12:24:13.7098430Z 904:             throw "did not recieve at least $MinimumCount records but $($log.Count) instead."

(note that receive is spelled wrong).  According to the text, the test expects >= 18, but 47 is >= 18, so the validation must be wrong.

For the logging test that was failing in previous macOS CI, it's caused by removing Utility.psm1. The test script $testScriptPath has Write-Verbose in it, so when running & $powershell -NoProfile -SettingsFile $configFile -Command $testScriptPath, it triggers the auto-loading of Utility module. Before this PR, that results in 19 event logs. With this PR, two events that record the starting and completion of Utility.psm1 execution are gone, so the total number of event logs becomes 17. I have update the test code, also update PSSysLog.psm1 to correct the wrong message as @SteveL-MSFT pointed out above.

Yes we do. Type conversion depends on the type catalog. Not all .NET Core assemblies shipped with PS Core are loaded at startup.

But the dll is loaded at startup. Only "Type conversion depends on the type catalog" prevents us from removing the dll from TypeCatalog?

ApplicationInsight telemetry is done in console.dll at startup, as long as the user hasn't opted out.

I understand. My request is - Does the telemetry delay startup or no? If yes we could postpone the telemetry call and loading the dll.

There is no interference from other pwsh processes when running the tests. BenchmarkDotNet starts each process many times, and sometimes there are outliers.

I think these outlies is important to investigate in the startup context. Outlier is unexpected delay. Why we see unexpected delay in the tests?
It is not for the PR but we shouldn't ignore this.

But the dll is loaded at startup. Only "Type conversion depends on the type catalog" prevents us from removing the dll from TypeCatalog?

Not sure what you mean by "But the dll is loaded at startup". Most of the assemblies included in the type catalog file are not loaded at startup.
Even for the dlls that are loaded at startup, type resolution would be much faster for a type from those dlls because we can avoid the effort to search types from all loaded assemblies to find the one.

I understand. My request is - Does the telemetry delay startup or no? If yes we could postpone the telemetry call and loading the dll.

The telemetry definitely play a role in the startup time, but I don't know how much after crossgen'ing the dll because I'm not able to analyze the stack info in PerfView/WPA with crossgen'ed assemblies yet.

@lzybkr @PaulHigin @TravisEz13 Can you please look at the new changes to PerformSecurityChecks() in CompiledScriptBlock.cs again?
Major change is on how we decide to skip AMSI/malicious code scan. The previous way to check for skipping .psd1 files causes security vulnerabilities.
An attacker can put arbitrary content in a .psd1 file, can then call [Parser]::ParseFile() to get an AST from the .psd1 file. Then the attacker can create a script block from the Ast by calling $myAst.GetScriptBlock(). That script block then will be considered from a .psd1 file and skip the code scan.
The skip-scan condition is changed to:

the script block should come from a .psd1 file
the script block is essentially a HashtableAst
the HashtableAst is safe, by calling IsSafeValueVisitor.IsAstSafe(hashtableAst, GetSafeValueVisitor.SafeValueContext.Default).

The API IsSafeValueVisitor.IsAstSafe has been used in GetSafeValueVisitor.GetSafeValue, which is used in module script analysis code and the public API Ast.SafeGetValue().

We have many comments in the PR that is a problrm for GitHub web interface. If security concerns have been fixed we could merge and continue with hash function in follow PR.

@daxian-dbw  Seems you could merge.

@lzybkr Would you like to take another look before I merge?

Attempt to address comments + rebased to resolve the conflict in Microsoft.PowerShell.Commands.Utility.csproj.
@lzybkr Can you please take another look? Thanks!

The failed test is most likely caused by #8346. It's a feature-level test, but the commits in that PR don't have the [feature] tag, so the feature tests didn't run for that PR CI.

Did we lose a bit of performance with latest commits?

Can you elaborate a bit on the unicode simple case folding? I'm not sure what that is.

@daxian-dbw See here for the discussion on that: https://github.com/PowerShell/PowerShell/issues/8120

I want to use Convertfrom-sddlstring in c#  ? what is the equivalent method  in c# and what library?


query non-exist account name trigger "Internal Service Error"?

test code:
let EOS = require("eosjs");
let config = require("./config");
enstance = EOS(config.chainInfo);
enstance.getAccount('nonexistaccount').then(function(result){

}).catch((e) => {

  });

tested both on mainnet and testnet:
httpEndpoint: 'http://peer1.eoshuobipool.com:8181',
httpEndpoint: 'http://jungle.cryptolions.io:38888',
I am using  getAccount to confirm if this account exist. while it exist the function return correct data.
but if the account does not exist,  it throws an exception, and catch does not work.  is this normal behaviour? why not simply  return error msg instead?

api < error 	 undefined http://peer1.eoshuobipool.com:8181/v1/chain/get_account {"account_name":"nonexistaccount"}
{ Error: {"code":500,"message":"Internal Service Error","error":{"code":3010001,"name":"name_type_exception","what":"Invalid name","details":[]}}
at /node_modules/eosjs-api/lib/apigen.js:95:23
at
at process._tickCallback (internal/process/next_tick.js:188:7) status: 500, statusText: 'Internal Server Error' }
(node:88344) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): Error: {"code":500,"message":"Internal Service Error","error":{"code":3010001,"name":"name_type_exception","what":"Invalid name","details":[]}}
(node:88344) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

my bad, didn't notice I could turn the error logger down in the config like this:
Turn off all error logging: config.logger = {error: null}


meteor-publish-composite

This is not intended as a bug report unless you feel it as a bug with angular-meteor and not meteor-publish-composite! I'm posting as guidance to anyone else facing this issue.
I use meteor-publish-composite throughout my app and had an issue with the reactive searching and ordering, based on a $stateParam that I supplied.
Essentially, using a normal meteor.publish worked fine. When I used the same code with meteor-publish-composite I had an issue which took a while to track down.
Code like this ...
this.subscribe('groups', () => {
        return [
            {
                limit: parseInt(this.perPage),
                skip: parseInt((this.getReactively('page') - 1) * this.perPage),
                sort: this.getReactively('sort')
            },
            this.getReactively('searchText')
        ]
    });

..worked perfectly with a normal Meteor.publish. searchText was sent in as 'searchText'.
With meteor-publish-composite, searchText was sent literally just as searchText (without the '' around it). Not sure why.
e.g. a result from what meteor-publish-composite recieves is below. Noticed, the ID and searchText are not surrounded by ''.
iPyN22NBKCjEsosp7 { limit: 10, skip: 0, sort: { surname: 1 } } w
The only way I have managed to get around this is the following...
this.subscribe('groupinformation', () => {
        return [
            {id: $stateParams.mygroupId},    <--------specifying the $stateParams as id
            {
                limit: parseInt(this.perPage),
                skip: parseInt((this.getReactively('page') - 1) * this.perPage),
                sort: this.getReactively('sort')
            },
            {search: this.getReactively('searchText')}    <--------specifying the this.getReactively('searchText') as search
        ]
    });

...and then updating my Meteor.publishComposite as required.
Hopefully might help someone with the same issue as me.

thanks you very much for this @TWKDreamState


VELPITAR - Careers link is not the right one

ENVIRONMENT: production
URL: https://peviitor.ro
Browser: Chrome
Version: 120
Device: Laptop
OS: Windows 10
STEPS TO REPRODUCE:

Open the url
Hover over the logo.
Click on the 'careers' button.

EXPECTED RESULTS:
The careers section from the company's official website is open in browser.
ACTUAL RESULTS:
The company's official website opens in browser, displaying some of the open positions (first picture), but it is not the main careers section with all the open positions (second picture).
Attachments:

https://i.imgur.com/yH9pahs.jpeg

fixed


[WebCryptoAPI] randomUUID

https://wicg.github.io/uuid/

https://www.chromestatus.com/feature/5689159362543616
https://bugs.chromium.org/p/chromium/issues/detail?id=1197594

Coming in WebKit:

https://twitter.com/BenjaminCoe/status/1428099971371446277
https://bugs.webkit.org/show_bug.cgi?id=229240

Firefox seems to be working on it: https://bugzilla.mozilla.org/show_bug.cgi?id=1723674

I've created a draft pull requests for this: https://github.com/Fyrd/caniuse/pull/5998
Please feel free to comment to improve it!

Now in Safari TP 132 🎉
https://webkit.org/blog/11971/release-notes-for-safari-technology-preview-132/:

Implemented Crypto.randomUUID() (r281206, r281284)

It's available via MDN-BCD now: https://caniuse.com/mdn-api_crypto_randomuuid

Closing as per last comment.


Enhance linter to act as a preprocessor

Allow users to move step templates to a separate file. Maybe we want to rename it from being a linter to being a .... parser? templater?

Gonna close this, I'm not sure it's really adding that much value or is a direction we want to go in right now


Digital Certificates

Hi,
For the digital certificate, are we restricted to certificates issued only by the following Malaysian Certificate Authorities (CAs)?

Pos Digicert Sdn Bhd
MSC Trustgate.Com Sdn Bhd
TM Technology Services Sdn Bhd

Can we use certificates from other providers like RapidSSL or Comodo?

Correct, you may refer https://www.mcmc.gov.my/en/sectors/digital-signature/list-of-licensees

Thank you


Add ANE breaking change

Contributes to #22021

The preview is giving a 404 for some reason, but this PR just adds one line, so I don't think we need to check it.


Peek Editor doesn't quit

Peek Editor doesn't quit on pressing escape
steps to reproduce
open peek preview press escape.

Which OS, VS Code version? Does it happen if you start with --disable-extensions?

Never Mind I found a setting : stablePeek


Window crashed, option "don`t restore editors" does not help

Does this issue occur when all extensions are disabled?: Yes/No

VS Code Version:
OS Version:

Steps to Reproduce:

code -v:
1.68.1
30d9c6cd9483b2cc586687151bcbcd635f373630
x64

My vscode closed when I accidentally opened sql dump. When choosing don`t restore editors and reopening vscode, vscode tries to reopen the file.
I tried deleting the cache and disabling plugins, it didn't help.
I have attached a video.

https://user-images.githubusercontent.com/48246560/177359440-df6cf900-2af7-4c12-91f4-c4d9c44389ee.mp4

Something seems to open the file, even without you doing anything? Is that an extension?
I wonder if this issue is being caused by an installed extension. Can you try to run VS Code without extensions? From the command line (NOT the integrated terminal in Code), execute: code --disable-extensions and try your steps again to see if it reproduces. If you see it is an issue with the extension, please file it against the extension repository itself.

This issue has been closed automatically because it needs more information and has not had recent activity. See also our issue reporting guidelines.
Happy Coding!


Fast Core Graphics renderer for OSX

I guess this is good?  I can't really review it or anything.  I guess I'll ask Dave to check it?


Depth first search Swift4

Checklist

[x] I've looked at the contribution guidelines.
[x] This pull request is complete and ready for review.

Description

#507 Updated DepthFirstSearch to Swift4

@shabirjan The indent should be 2 spaces. Other parts look good.


i want to customise node. is it possible ?

i want to use my custom node in graph. is it possible ?

Using svg is fine. But I want to customize node using html.

i want to create a workflow diagram (bot builder) for intent and entity, where user can  add , update and node. each node has some title, type, and other information. your libaray is giving me a best base to start this module but for ui perspective, i need to change node. it is very similar to botbuilder for kore.ai.

Hmm I see @vivek1623 . So I that a small POC on this where I'm enabling the nodes to receive custom HTML and then I inject it within a  tag. So this basically gives clients the power to do whatever they want in the node label are with HTML and CSS, you can write specific presentation components to display in that area. Below the sample with the html are in the node label and the inspector focused on the element:

But in my opinion, this seems a bit overkill, and at this point, react-d3-graph has already some in progress work items, so we are out of resources here.

but graph are very near in default position. is there way to spread it. my flow diagram should look like https://github.com/DrummerHead/react-flow-diagram this.

I'm not able to access the link you shared. Still, if I understood correctly you want nodes to be spread, that is possible since you can set a position for each node placing them apart from each other.

Hi there! I'm interested in this feature too, and I already started working on it.
Right now I can pass a view property to each node, which will take a JSX view as a value. This will allow us to define a completely customized node, such as the author of this issue wanted. The POC @danielcaldas published here was a big help in order to achieve this. I'm hoping to submit a PR with the changes by next week.

Closing this with release 2.0.0-rc0


"generating pdf toast implemented"

closes #580

- I have participated in GSSOC

That's really a cool toast, @ashuvssut 😍

closes #580

- I have participated in GSSOC

Nice work, the only issue is the option to save the pdf pops up several seconds after the "generating pdf toast" is over. So we need to adjust the delay accordingly

I have removed the timer
Now I am using JS Promise in an async function to wait for the generation of the PDF to complete... after waiting I have put commands to close the popup
so there is no need for a timer now

can you put here relevant level labels for GSSOC too?😄


perf: use custom ink build

This custom version of ink has:

tree-shaken lodash
ESM distribution

https://github.com/vadimdemedes/ink/compare/master...privatenumber:privatenumber?expand=1

:tada: This PR is included in version 2.0.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Provided config.json has format error

tip: count the brackets

Since you already know where the issue is and my time is limited - how about contributing the necessary change and creating a pull request?

Thanks for the PR. I merged it with slight modifications.


Preliminary support for keeping broadcast exchanges on GPU when AQE is enabled

This builds on https://github.com/NVIDIA/spark-rapids/pull/447 so we should merge that first.
This PR adds support for injecting a query stage preparation rule for Spark versions 3.0.1 and 3.1.0 to tag the SparkPlan with any reasons that operators cannot be supported on the GPU. It also updates the GpuOverrides checks for BroadcastExchangeExec to check for any tagged reasons when new query stages are created (when AQE is enabled).
Note that these changes won't have any effect on functionality yet but will be leveraged once SPARK-32332 is merged.
The TPCH integration tests run with AQE on and off and cover test cases that test this new code path.

build

notes this is going to conflict with https://github.com/NVIDIA/spark-rapids/pull/442 as well

build


Python3: TypeError: the JSON object must be str, not 'bytes'

Anybody run into this one? Must be environmental but dependencies appear to be up to date.  I am running 2.1.8:
$ python3
Python 3.5.2 (default, Sep 14 2017, 22:51:06)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> api_token = 'XXXX:YYYYYYYYYYYYYYYYYYYY'
>>> import pinboard
>>> pb = pinboard.Pinboard(api_token)
>>> print (pb)
<pinboard.pinboard.Pinboard object at 0x7f8a2330d198>
>>> pb.posts.update()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.5/dist-packages/pinboard/pinboard.py", line 196, in __call__
    json_response = json.load(response)
  File "/usr/lib/python3.5/json/__init__.py", line 268, in load
    parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
  File "/usr/lib/python3.5/json/__init__.py", line 312, in loads
    s.__class__.__name__))
TypeError: the JSON object must be str, not 'bytes'
>>>

Thanks!

json.load() requires str, but HTTPResponse.read() returns bytes. In Python 3.6 it should work, as json.load() has additional logic to decode the bytes. https://docs.python.org/3/whatsnew/3.6.html#json.
In older versions, a patch is needed. Replace json.load(response) in pinboard/pinboard.py and bin/pinboard with
json.loads(response.read().decode('utf-8'))

or use parse_response=False and parse in your code:
resp = pb.posts.update(parse_response=False)
json.loads(resp.read().decode('utf-8'))


[WIP] POC for handling large results.

A POC for supporting results with values larger than the capacity limit of termination message.
What it does:

When results are larger than certain threshold, so push it to a remote storage.
Instead of storing large contents into result (i.e. termination message, we just swap it with a remote store url).
When results are referred to by params, or in a command. In the entrypointer script, we do a prefetch of the contents and replace the link with the filesname containing the contents of the result /or the actual contents(TBD since results are large).

A example is provided:
examples/v1beta1/pipelineruns/pipelinerun-large-results.yaml
The poc does not do the following, inorder to keep the code change minimum and easily comprehensible.

make this feature an opt in.
Uses a string cos:// as a marker for processing it as links. This can be improved by either using a new result type
It uses a cloud object storage directly as a remote store. Ideally, it should support a CSI and then a user will be able to plugin storage of their choice.
The size threshold is not yet configurable.
Misses out on integration tests. This is actually a TODO ( if the community like the overall approach.)

Thanks @dibyom for pointing me at this today! cc @chitrangpatel @jerop in case you haven't seen this yet - I think this approach addresses some of the permissions concerns with the log based solution we were talking about in https://github.com/tektoncd/community/pull/745. This is using OIDC from the entrypoint flow (similar to what I mentioned), though it's currently interfacing with an external IdP rather than the in-cluster IdP.
@ScrapCodes great start! Main piece of feedback I have is having the user pod entrypoint interface directly with external OIDC storage buckets is going to make it hard to ensure that every service account has the access and permissions it needs, and doesn't have broad permissions to alter other Task results it shouldn't have access to.
If you haven't seen it already, you might be interested in some of the ideas detailed in TEP-0086 around the dedicated HTTP API - we could auth from the entrypoint with pod OIDC creds via service account token projection like you have here to a Tekton-managed API. That API can verify the JWT matches the correct Task pod, then handle the storage and any additional auth.
Creds can be centralized in the Tekton control plane (so no need to worry about granting user pods access to external resources), and the API could be swapped out for different impl for external storage like this is doing, or updating the TaskRun object directly similar to what @chitrangpatel is doing with the log based proof of concept.

Thank you @wlynch, I agree with auth part, it is not perfect.
In order to confirm, that I have understood correctly.

The storage should support the same credentials as tekton.
May be using POD's credentials authenticating against a HTTP auth server running inside the controller might make it perfect.

Will we be limited to supporting only http transport based stores(e.g. COS) as storages. Do you think that is acceptable?
On the brighter side, It does not need a sidecar !

In order to confirm, that I have understood correctly.

The storage should support the same credentials as tekton.
May be using POD's credentials authenticating against a HTTP auth server running inside the controller might make it perfect.

Will we be limited to supporting only http transport based stores(e.g. COS) as storages. Do you think that is acceptable?

The only bit that would need to be HTTP (though it could also be another transport mechanism like protobuf, etc. - it doesn't really matter as long as the API is agreed upon) would be from the user pod to the tekton controlled storage API. The tekton controlled storage API server can then interface with whatever backing storage it wants once it's authorized the user pod - COS, GCS, it's own local volume, etc.
If an external service supported the same result storage API and federated OIDC creds from the cluster, then you could skip the internal API and go straight to the external service.


fix: Add special type tests for Json writer and fix some type mapping issues

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

[ ] Make sure to open an issue as a bug/issue before writing your code!  That way we can discuss the change, evaluate designs, and agree on the general idea
[ ] Ensure the tests and linter pass
[ ] Code coverage does not decrease (if any source code was changed)
[ ] Appropriate docs were updated (if necessary)

Fixes #<issue_number_goes_here> ☕

Some unit tests are failing:
Error:  Failures:
Error:    BQTableSchemaToProtoDescriptorTest.testSimpleTypes:105->isDescriptorEqual:78 test_field_type expected:<STRING> but was:<BYTES>
Error:    BQTableSchemaToProtoDescriptorTest.testStructComplex:225->isDescriptorEqual:78 test_time expected:<INT64> but was:<STRING>
Error:    BQTableSchemaToProtoDescriptorTest.testSimpleTypes:104->isDescriptorEqual:77 test_field_type expected:<STRING> but was:<INT64>


Feature fortune points

Add fortune point reroll, fortune point add sl, and dark deals on every rolls. #384

It seems like you can keep rerolling with dark deals as much as you want, is that intended? I thought I remember us discussing only allowing 1 of each type.


feat: generate url maps from exports.xml (#78)

Description: 概要

Changes: 変更内容

Changes

feat: generate url maps from exports.xml (#78) (ac8bfd9559b7c67a25ab7a98dd338734b10c1746)

Expected Impact: 影響範囲

Operating Requirements: 動作要件

Additional context: 補足

Codecov Report

Merging #86 (ac8bfd9) into release/next-v0.4.1 (3e5a3da) will decrease coverage by 0.19%.
The diff coverage is 0.00%.

@@                   Coverage Diff                   @@
##           release/next-v0.4.1      #86      +/-   ##
=======================================================
- Coverage                30.84%   30.65%   -0.20%
=======================================================
  Files                      140      140
  Lines                     1112     1119       +7
  Branches                   205      206       +1
=======================================================
  Hits                       343      343
- Misses                     750      757       +7
  Partials                    19       19

Impacted Files
Coverage Δ

src/packages/infra/pages/any/index.tsx
0.00% <0.00%> (ø)

src/packages/infra/post/factory.ts
0.00% <0.00%> (ø)

src/packages/infra/post/repository/base.ts
0.00% <0.00%> (ø)

src/packages/infra/post/repository/wpExport.ts
0.00% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3e5a3da...ac8bfd9. Read the comment docs.


TypeError: an integer is required

Overview
I would like to output a video but get the follwoing error:
import torch
import torchvision

# Example dummy video tensor
video = torch.randint(0, 255, (30, 720, 1280, 3), dtype=torch.uint8)  # 30 frames of 720p video

# Write the video
torchvision.io.write_video("test_video.mp4", video, fps=30, video_codec="h264")

Error:
File av/video/frame.pyx:193, in av.video.frame.VideoFrame.pict_type.__set__()

TypeError: an integer is required

Expected behavior
Get a video file.
Actual behavior
Error
Versions
python = "^3.10"
torch = "^2.5.1"
torchvision = "^0.20.1"
lightning = "^2.4.0"
torchmetrics = "^1.5.1"
hydra-core = "^1.3.2"
hydra-colorlog = "^1.2.0"
hydra-optuna-sweeper = "^1.2.0"
rootutils = "^1.0.7"
pre-commit = "^4.0.1"
rich = "^13.9.4"
pytest = "^8.3.3"
torchrl = "^0.6.0"
tensordict = "^0.6.2"
ipykernel = "^6.29.5"
gymnasium = {extras = ["atari"], version = "<1.0.0"}
pygame = "^2.6.1"
av = "^14.0.0"

ffmpeg version 4.4.2-0ubuntu<IP_ADDRESS>

Research
I have done the following:

[x] Checked the PyAV documentation
[x] Searched on Google
[x] Searched on Stack Overflow
[x] Looked through old GitHub issues

Go report this to torchvision


Unresponsive when expanding a folder with a lot of files

VSCode Version: 1.4.0
OS Version: Windows 10

Steps to Reproduce:

Clone the Microsoft/Typescript project
In the explorer expand the tests/baselines folder

Code will stop responding. There are a lot of files in it's child folder so it's unreasonable to expect it to list them immediately but having it freeze doesn't feel like the best user experience either.

https://github.com/Microsoft/vscode/issues/9962


Weave removing the VM access

Hi All,
I have followed https://kubernetes.io/docs/getting-started-guides/kubeadm/ installation.
In Step3 : I have installed weave network  kubectl apply -f https://git.io/weave-kube .
When ever I am installing weave network , I am not able to access the VM and network to the VM is getting terminated.
Point to note :

I am working behind corporate proxy
Output of uname -a
Linux minion2 3.10.0-514.el7.x86_64 #1 SMP Wed Oct 19 11:24:13 EDT 2016 x86_64 x86_64 x86_64 GNU/Linux
Removed the proxy details in this post as I cannot expose it.
docker info output

[root@minion2 ~]# docker info
Containers: 12
 Running: 0
 Paused: 0
 Stopped: 12
Images: 14
Server Version: 1.12.4
Storage Driver: overlay
 Backing Filesystem: xfs
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host null overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Security Options: seccomp
Kernel Version: 3.10.0-514.el7.x86_64
Operating System: Red Hat Enterprise Linux Server 7.3 (Maipo)
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 2.781 GiB
Name: minion2
ID: Q6KC:VGM5:DE7P:I2TW:H2SQ:A3CI:GFRM:IPLZ:MKA5:SHJI:6CLP:4XSU
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Insecure Registries:
 <IP_ADDRESS>/8

@cooligc is your VM's own network address inside the range of IP addresses used by Weave by default?  (<IP_ADDRESS>/12)
To check, run ip route and ip addr on the host.

No , It is not. The range starts with 15.XX.XX.XX

@cooligc could you say what exactly you mean by "network to the VM is getting terminated"?  Who or what terminated it?  Is there an error message?
Is it possible to supply the output of iptables-save from before you run the kubectl apply?
What Weave Net does on the host:

creates a bridge named weave
gives the bridge an IP address in Weave Net's range, which in turn sets up a route for that subnet
adds masquerading (nat) rules for anything leaving the bridge
adds iptables rules to only allow permitted traffic onto the bridge

Right now I can't think of anything other than a subnet overlap that would break your host network.

@bboreham

The VM is in ESXi . and The same VM is accessible from localhost.
After installing the weave network , the putty session from localhost to VM is getting terminated .
VM communicates when we are checking in vSphere console(in same subnet) using ssh.

This report looks very like #2289, which was a kernel bug fixed in 3.13.  Can you check into that please?

Tried with kernel v4.10 with RHEL 7.3 . No Change in the behavior.
[root@gru ~]# uname -a
Linux gru 4.10.2-1.el7.elrepo.x86_64 #1 SMP Sun Mar 12 13:25:09 EDT 2017 x86_64 x86_64 x86_64 GNU/Linux

OK, thanks.
Can you expand on this a bit?

VM communicates when we are checking in vSphere console(in same subnet) using ssh.

Do you mean you lost one link to the VM but you have another that works?  What is the difference between those two links?

Hey @bboreham ,
Here are the 2 scenarios,

Lets say, I have started master , installed weave network then started the minion
minion is able to join to the network. (We could see the network interface in both the system)
Then, after some second , VMs are out of reach
Lets say, Master is started and weave is installed, then after some second , we loose the control in master.
Point to remember here, We have not join the minion on to master.

Both the VMs are accessible from vSphere console although it is not accessible using putty ssh

Ok, if you can access the machine through the vSphere console, could you post the output before and after the failure of these commands:
ip addr
ip route
sudo iptables-save

Thanks

Hey @bboreham ,
Please find the required details. Sorry for the delay.
After.txt
Before.txt

That looks like before installation of Kubernetes and after installation of everything.  Previously you had said it breaks at step 3, which ought to be a smaller set of changes.
This line puzzles me:
-A KUBE-SEP-TMSHWDTNCTVPIVXI -s <INTERNAL_IP>/32 -m comment --comment "default/kubernetes:https" -j KUBE-MARK-MASQ

I can't see why Kubernetes would want to NAT an IP on your network.  Is that IP the one that you connect via ssh over?
Beyond that I cannot spot any reason why it would fail.

@cooligc I tried to create an RHEL machine to see if it would break the same way for me, but I failed at the step yum install docker - there is no such package.  How did you install it?

https://docs.docker.com/engine/installation/linux/repo_files/centos/docker.repo 404's

Unfortunately , the url got change . We have followed the below url, please refer.
https://docs.docker.com/engine/installation/linux/centos/#install-using-the-repository

@cooligc this documentation currently doesn't mention docker-engine as a valid package:

Older versions of Docker were called docker or docker-engine. If these are installed, uninstall them, along with associated dependencies.

though it seems it is what you are using according to your previous comment.

Are you using:

docker-engine?
docker-ce?
docker-ee?

Which specific version are you using?

@marccarre
I am using docker-engine.
docker : Server Version: 1.12.4

Hi @cooligc, I tried to reproduce your issue on a small cluster of machines as similar as possible to yours -- details to follow.
After a few minutes (not seconds), on one of the nodes, I did observe that the SSH session got terminated with the below error client side:
# packet_write_wait: Connection to <IP_ADDRESS> port 22: Broken pipe

Is that what you observed as well and meant by the following?

I am not able to access the VM and network to the VM is getting terminated.

then after some second , we loose the control in master.

Details on the setup:
# uname -a
Linux marc-0 3.10.0-514.6.2.el7.x86_64 #1 SMP Thu Feb 23 03:04:39 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

# cat /etc/*release*
CentOS Linux release 7.3.1611 (Core)
Derived from Red Hat Enterprise Linux 7.3 (Source)
cat: /etc/lsb-release.d: Is a directory
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"

CentOS Linux release 7.3.1611 (Core)
CentOS Linux release 7.3.1611 (Core)
cpe:/o:centos:centos:7

# docker version
Client:
 Version:      1.12.4
 API version:  1.24
 Go version:   go1.6.4
 Git commit:   1564f02
 Built:        Mon Dec 12 23:41:49 2016
 OS/Arch:      linux/amd64

Server:
 Version:      1.12.4
 API version:  1.24
 Go version:   go1.6.4
 Git commit:   1564f02
 Built:        Mon Dec 12 23:41:49 2016
 OS/Arch:      linux/amd64

Note that I had to drop docker from the list of packages listed in the kubeadm instructions as it would conflict with docker-engine:
Error: docker-engine conflicts with 2:docker-1.12.6-11.el7.centos.x86_64
Error: docker-engine-selinux conflicts with 2:container-selinux-2.9-4.el7.noarch

# kubectl --kubeconfig /etc/kubernetes/kubelet.conf version
Client Version: version.Info{Major:"1", Minor:"5", GitVersion:"v1.5.4", GitCommit:"7243c69eb523aa4377bce883e7c0dd76b84709a1", GitTreeState:"clean", BuildDate:"2017-03-07T23:53:09Z", GoVersion:"go1.7.4", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"5", GitVersion:"v1.5.5", GitCommit:"894ff23729bbc0055907dd3a496afb725396adda", GitTreeState:"clean", BuildDate:"2017-03-22T00:17:51Z", GoVersion:"go1.7.4", Compiler:"gc", Platform:"linux/amd64"}

$ diff before.txt after.txt
1c1
< Before:
---
> After:
13c13
<        valid_lft 84230sec preferred_lft 84230sec
---
>        valid_lft 83756sec preferred_lft 83756sec
17a18,33
> 4: datapath: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1376 qdisc noqueue state UNKNOWN qlen 1000
>     link/ether b6:09:08:9f:cc:23 brd ff:ff:ff:ff:ff:ff
> 6: weave: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1376 qdisc noqueue state UP qlen 1000
>     link/ether be:c8:e2:3e:f6:f8 brd ff:ff:ff:ff:ff:ff
>     inet <IP_ADDRESS>/12 scope global weave
>        valid_lft forever preferred_lft forever
> 7: dummy0: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN qlen 1000
>     link/ether 6a:42:b7:65:6f:0f brd ff:ff:ff:ff:ff:ff
> 9: vethwe-datapath@vethwe-bridge: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1376 qdisc noqueue master datapath state UP qlen 1000
>     link/ether 9a:25:15:69:13:f2 brd ff:ff:ff:ff:ff:ff
> 10: vethwe-bridge@vethwe-datapath: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1376 qdisc noqueue master weave state UP qlen 1000
>     link/ether 2e:20:58:65:f5:f1 brd ff:ff:ff:ff:ff:ff
> 13: vethwepla306bbf@if12: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1376 qdisc noqueue master weave state UP
>     link/ether 1e:a8:6a:ad:68:6e brd ff:ff:ff:ff:ff:ff link-netnsid 0
> 14: vxlan-6784: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65470 qdisc noqueue master datapath state UNKNOWN qlen 1000
>     link/ether 72:c9:fe:f3:3e:cc brd ff:ff:ff:ff:ff:ff
20a37
> <IP_ADDRESS>/12 dev weave  proto kernel  scope link  src <IP_ADDRESS>
27d43
<
31,32c47,48
< :OUTPUT ACCEPT [19:1206]
< :POSTROUTING ACCEPT [19:1206]
---
> :OUTPUT ACCEPT [0:0]
> :POSTROUTING ACCEPT [0:0]
38a55,56
> :KUBE-SEP-SZZ7MOWKTWUFXIJT - [0:0]
> :KUBE-SEP-UJJNLSZU6HL4F5UO - [0:0]
57a76
> :WEAVE - [0:0]
70a90
> -A POSTROUTING -j WEAVE
76a97,101
> -A KUBE-SEP-SZZ7MOWKTWUFXIJT -s <IP_ADDRESS>/32 -m comment --comment "kube-system/kube-dns:dns" -j KUBE-MARK-MASQ
> -A KUBE-SEP-SZZ7MOWKTWUFXIJT -p udp -m comment --comment "kube-system/kube-dns:dns" -m udp -j DNAT --to-destination <IP_ADDRESS>:53
> -A KUBE-SEP-UJJNLSZU6HL4F5UO -s <IP_ADDRESS>/32 -m comment --comment "kube-system/kube-dns:dns-tcp" -j KUBE-MARK-MASQ
> -A KUBE-SEP-UJJNLSZU6HL4F5UO -p tcp -m comment --comment "kube-system/kube-dns:dns-tcp" -m tcp -j DNAT --to-destination <IP_ADDRESS>:53
> -A KUBE-SERVICES -d <IP_ADDRESS>/32 -p tcp -m comment --comment "default/kubernetes:https cluster IP" -m tcp --dport 443 -j KUBE-SVC-NPX46M4PTMTKRN6Y
79d103
< -A KUBE-SERVICES -d <IP_ADDRESS>/32 -p tcp -m comment --comment "default/kubernetes:https cluster IP" -m tcp --dport 443 -j KUBE-SVC-NPX46M4PTMTKRN6Y
80a105
> -A KUBE-SVC-ERIFXISQEP7F7OF4 -m comment --comment "kube-system/kube-dns:dns-tcp" -j KUBE-SEP-UJJNLSZU6HL4F5UO
82a108
> -A KUBE-SVC-TCOU7JCQXEZGVUNU -m comment --comment "kube-system/kube-dns:dns" -j KUBE-SEP-SZZ7MOWKTWUFXIJT
92a119,121
> -A WEAVE -s <IP_ADDRESS>/12 -d <IP_ADDRESS>/4 -j RETURN
> -A WEAVE ! -s <IP_ADDRESS>/12 -d <IP_ADDRESS>/12 -j MASQUERADE
> -A WEAVE -s <IP_ADDRESS>/12 ! -d <IP_ADDRESS>/12 -j MASQUERADE
96,100c125,129
< :PREROUTING ACCEPT [25281:213320680]
< :INPUT ACCEPT [25279:213319804]
< :FORWARD ACCEPT [0:0]
< :OUTPUT ACCEPT [24429:1613856]
< :POSTROUTING ACCEPT [24447:1617496]
---
> :PREROUTING ACCEPT [35288:299468970]
> :INPUT ACCEPT [34487:299226803]
> :FORWARD ACCEPT [763:236429]
> :OUTPUT ACCEPT [32301:2606954]
> :POSTROUTING ACCEPT [33128:2853628]
127,129c156,158
< :INPUT ACCEPT [25279:213319804]
< :FORWARD ACCEPT [0:0]
< :OUTPUT ACCEPT [24429:1613856]
---
> :INPUT ACCEPT [34487:299226803]
> :FORWARD ACCEPT [763:236429]
> :OUTPUT ACCEPT [32301:2606954]
139,140c168,169
< :PREROUTING ACCEPT [25281:213320680]
< :OUTPUT ACCEPT [24429:1613856]
---
> :PREROUTING ACCEPT [35288:299468970]
> :OUTPUT ACCEPT [32301:2606954]
150c179
< :OUTPUT ACCEPT [413:41307]
---
> :OUTPUT ACCEPT [82:15418]
175a205,207
> :WEAVE-NPC - [0:0]
> :WEAVE-NPC-DEFAULT - [0:0]
> :WEAVE-NPC-INGRESS - [0:0]
183a216,221
> -A INPUT -d <IP_ADDRESS>/32 -i docker0 -p tcp -m tcp --dport 6783 -j DROP
> -A INPUT -d <IP_ADDRESS>/32 -i docker0 -p udp -m udp --dport 6783 -j DROP
> -A INPUT -d <IP_ADDRESS>/32 -i docker0 -p udp -m udp --dport 6784 -j DROP
> -A INPUT -i docker0 -p udp -m udp --dport 53 -j ACCEPT
> -A INPUT -i docker0 -p tcp -m tcp --dport 53 -j ACCEPT
> -A FORWARD -i docker0 -o weave -j DROP
197a236,240
> -A FORWARD -o weave -j WEAVE-NPC
> -A FORWARD -o weave -m state --state NEW -j NFLOG --nflog-group 86
> -A FORWARD -o weave -j DROP
> -A FORWARD -i weave ! -o weave -j ACCEPT
> -A FORWARD -o weave -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
221,222c264,269
< -A KUBE-SERVICES -d <IP_ADDRESS>/32 -p udp -m comment --comment "kube-system/kube-dns:dns has no endpoints" -m udp --dport 53 -j REJECT --reject-with icmp-port-unreachable
< -A KUBE-SERVICES -d <IP_ADDRESS>/32 -p tcp -m comment --comment "kube-system/kube-dns:dns-tcp has no endpoints" -m tcp --dport 53 -j REJECT --reject-with icmp-port-unreachable
---
> -A WEAVE-NPC -m state --state RELATED,ESTABLISHED -j ACCEPT
> -A WEAVE-NPC -d <IP_ADDRESS>/4 -j ACCEPT
> -A WEAVE-NPC -m state --state NEW -j WEAVE-NPC-DEFAULT
> -A WEAVE-NPC -m state --state NEW -j WEAVE-NPC-INGRESS
> -A WEAVE-NPC-DEFAULT -m set --match-set weave-k?Z;25^M}|1s7P3|H9i;*;MhG dst -j ACCEPT
> -A WEAVE-NPC-DEFAULT -m set --match-set weave-iuZcey(5DeXbzgRFs8Szo]<@p dst -j ACCEPT

after.txt
before.txt

Actually, when trying the same steps again today on fresh VMs (I wanted to debug things further), I cannot seem to reproduce what I had yesterday anymore, so it could just have been my SSH connection timing out.
@cooligc,

as mentioned previously, it would be good to know the exact error message you get, if you get any;
does this happen consistently?
does this still happen with latest Docker, Kubernetes and Weave Net versions? (e.g. Kubernetes only keeps the latest version in their repository, and a new version has been released since you opened this issue so I'm probably not using the same version as you were)
if you want us to try to reproduce your issue, we would need the exact commands you ran, and not an approximation of some instructions on a website which has changed since you visited it, like we are doing here. Indeed, we may be missing some important detail in the process here.

Please either keep us posted.

@cooligc, I'll close the issue for now as we unfortunately cannot reproduce your issue.
Feel free to reopen with answers to the above questions & more specific steps to help us reproduce it.

Same issue with me
I am just trying deploying k8s to vanilla aws ec2s, I am using kubeadm, once i run kubectl apply -f "https://cloud.weave.works/k8s/net?IPALLOC_RANGE=<IP_ADDRESS>/16&k8s-version=$(kubectl version | base64 | tr -d '\n')" as per the docs https://www.weave.works/docs/net/latest/kubernetes/kube-addon/#-changing-configuration-options my ssh connection to master ec2 instance drops and i can not SSH to the VM anymore, is because weave net’s default CIDR range <IP_ADDRESS>/12 conflicts with AWS?
How can i specify another CIDR?

for info: my aws bastian/jumpbox has an ip of the range 10-37-12-17...

I am just trying deploying k8s to vanilla aws ec2s, I am using kubeadm, once i run kubectl apply -f "https://cloud.weave.works/k8s/net?IPALLOC_RANGE=<IP_ADDRESS>/16&k8s-version=$(kubectl version | base64 | tr -d '\n')" as per the docs https://www.weave.works/docs/net/latest/kubernetes/kube-addon/#-changing-configuration-options my ssh connection to master ec2 instance drops and i can not SSH to the VM any more, is because weave net’s default CIDR range <IP_ADDRESS>/12 conflicts with AWS?

This is a old!
But it happens to me too, but it is not a issue.
Weave use the network <IP_ADDRESS>/12 as default and my workstation is on that range. So you must use another range. See docs: https://www.weave.works/docs/net/latest/tasks/ipam/configuring-weave/

When using Kubernetes you probably want to set IPALLOC_RANGE as described at https://www.weave.works/docs/net/latest/kubernetes/kube-addon/#configuration-options


(#150) Add coverage for go1.16.  Fix build issues for java tests.

resolves #150

I've merged these test-only changes myself.


Autoreconnect after failed ping

This uses the timer that does the keep alive, but if fails the ping, will reconnect to the broker. Do you think this will work, @mitghi?
Resolves #2

Hi @rfdickerson , Thank you very much for replying so fast :). I very appreciate it.
Yes it works but I think it could be better if we try to reconnect in didLoseConnection() so errors caused by read/write will be covered as well. I may be wrong about this particular method, because I didn't had the chance to read the source code yet. What do you think about notifying reconnection attempt via a delegate method ?
Thanks


[FEATURE REQ] Provide a script/command to refresh a cadl package by using cadl-location.yaml

Provide a script/command to refresh a cadl package by using cadl-location.yaml. The following is the steps in your script/command:
Steps:

Call ./eng/common/scripts/Cadl-Project-Sync.ps1 to sync the cadl project configured in cadl-location.yaml
Call ./eng/common/scripts/Cadl-Project-Generate.ps1 to regenerate SDK. (Recommended by not required, you can still run npm install and cadl compile in your script.)

For more details, please refer to document

WIP PR https://github.com/Azure/azure-sdk-for-java/pull/33311/files/48b4311486ee2823413596458db26895d9798f16..18aa7962ff0a362137d8c6a45b2945126faa64d5


Add message for non-MacOS developers in bin/configure

Related to #786.
Currently we get the following output when running bin/configure in Ubuntu:
> bin/configure

Successfully installed colorize-0.8.1
1 gem installed

Successfully installed activesupport-<IP_ADDRESS>
1 gem installed

Bullet Train requires Ruby 3.2.2 and `ruby -v` returns ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-linux].
You don't have Homebrew installed. This isn't necessarily a problem, you might not even be on macOS, but we can't check your dependencies without it.
Try proceeding without Homebrew? [y/n]
y
sh: 1: brew: not found
postgresql@14 is installed via Homebrew.
sh: 1: brew: not found
redis is installed via Homebrew.
sh: 1: brew: not found
icu4c is installed via Homebrew.
sh: 1: brew: not found
node is installed via Homebrew.
Yarn is installed.
Bullet Train requires Node.js 19.3.0 and `node -v` returns 19.3.0.

I think checking for packages like what is in #786 is a good way to ensure we cover everything across all platforms. However, we'd have to be very specific for each package since each command and its output will look different.
For example, I wrote a tentative check like this:
def not_installed?(package)
  case Gem::Platform.local.os
  when "darwin"
    `brew info #{package} | grep "Not installed"`.strip.length > 0
  when "linux"
    begin
      case package
      when "postgresql@14"
        version = `psql --version`
        version.split("\s")[2].match?(/^14/)
      when "redis"
        `redis-server --version`
      when "icu4c"
        version = `dpkg -l | grep libicu`
        raise Errno::ENOENT if version.empty?
      when "node"
        `node --version`
      end
    rescue Errno::ENOENT # Command not found
      "#{package} is not installed. Please install it first, and then run bin/configure again.".red
    end
  end
end

Granted, we can probably just check for these individually instead of through a method like we check for Ruby and Yarn. I think I'll end up doing things that way instead. Still definitely want to merge in #786 since it brought the current issue to attention.

FWIW, homebrew has built-in support for brew bundle now.
https://gist.github.com/ChristopherA/a579274536aab36ea9966f301ff14f3f
You can add a Brewfile to the project, and then use brew bundle check and brew bundle install as you might expect.
This doesn't really help linux users, however.

Closing in favor of #797.


Wrong name in message to slack when creating new vendor

Looks like vendor name is also used as name in email.
Example:
User Keboola Czech<EMAIL_ADDRESS>requires creation of vendor Keboola Czech
Should be:
User Martin Matejka<EMAIL_ADDRESS>requires creation of vendor Keboola Czech
Here, it looks fine, so I don't know where can be a problem:
https://github.com/keboola/developer-portal/blob/510641c6e30eb95bc6c0fc04726ce9c4c2f29646/lib/notification.js#L15

Fixed


Memory Leak when using backward hooks with fmodel

Hi, I recently came across a memory error while attempting to use backward hooks with the fmodel obtained from higher.innerloop_ctx, with the default settings (copy_initial_weights, track_higher_grads True). Specifically, it appears that when backward hooks are registered to modules whose weight has requires_grad=True on fmodel, followed by a foward pass, loss computation, and a call to diffopt.step(loss), these modules are not cleared upon exiting the context, or returning from a function containing the context.
Here is an example of a dummy hook and a function with such a loop that causes the error:

def placeholder_hook(module, grad_in, grad_out):
    pass

def run_higher_loop(model, optimizer, criterion):
    example = torch.tensor([0, 1, 2], dtype=torch.float).unsqueeze(0)
    target = torch.tensor([0], dtype=torch.float)

    with higher.innerloop_ctx(model, optimizer) as (fmodel, diffopt):
        # add placeholder backward hook to the copied embedding layer of fmodel
        for module in fmodel.modules():
            if isinstance(module, nn.Linear):
                module.weight.requires_grad = True
                module.register_backward_hook(placeholder_hook)

        output = fmodel(example)
        loss = criterion(output, target)
        diffopt.step(loss)

        # ...

When called with the below sample model, the fmodel's copied nn.Linear layer appears to persist:
# simple binary classification model
class Net(nn.Module):
    def __init__(self, input_size, output_size):
        super().__init__()
        self.linear = nn.Linear(input_size, output_size, bias=False)

    def forward(self, examples):
        output = self.linear(examples)
        return output.squeeze(0)

model = Net(input_size=3, output_size=1)
optimizer = optim.SGD(model.parameters(), lr=0.01)
criterion = nn.BCEWithLogitsLoss()

for i in range(4):
    mem_report(i)
    run_higher_loop(model, optimizer, criterion)

Here is the output of mem_report, which demonstrates that a copy of our linear layer (with input size 3, output size 1) stays in memory after each function call to run_higher_loop,
Memory report #0:
<class 'torch.nn.parameter.Parameter'> torch.Size([1, 3])

Memory report #1:
<class 'torch.Tensor'> torch.Size([1, 3])
<class 'torch.nn.parameter.Parameter'> torch.Size([1, 3])

Memory report #2:
<class 'torch.Tensor'> torch.Size([1, 3])
<class 'torch.Tensor'> torch.Size([1, 3])
<class 'torch.nn.parameter.Parameter'> torch.Size([1, 3])

Memory report #3:
<class 'torch.Tensor'> torch.Size([1, 3])
<class 'torch.Tensor'> torch.Size([1, 3])
<class 'torch.Tensor'> torch.Size([1, 3])
<class 'torch.nn.parameter.Parameter'> torch.Size([1, 3])

Some notes about the error

The error occurs with nn.Embedding as well, and likely some others
The error does not arise if the module.weight has requires_grad=False, and the backward hook is registered
The error does not arise if the module.weight has requires_grad=True, and the backward hook is not registered
The error does not arise in either case when a forward hook is registered
The error never arises if we never call diffopt.step(loss)
The definition of the hook itself does not affect the error (see above)
Removing the hooks at the end of the function manually does not remove the error (i.e. storing a handle from register_backward_hook and later calling handle.remove()

A temporary fix is to manually delete the fmodel module weights which have the hooks before exiting the function. Adding the block below to the end of run_higher_loop removes the issue above:
for module in fmodel.modules():
    if isinstance(module, nn.Linear):
        del module.weight

Thanks for raising the issue and for the detailed diagnosis, @vsieplus! This is super helpful. I'll look into it as soon as I can (likely after NeurIPS rebuttals).

did you try doing del fmodel?

@egrefen sorry for the firect ping but has this been fixed?

related? https://github.com/facebookresearch/higher/issues/105


Bug: gtm status empty result.

Thanks for the worderful tool.
Here is the issue I'm facing,

I've used pathogen to install the gtm vim plugin,
And have initialized gtm on the my repo.
But gtm status is showing empty result after code edits/modifications.

Can you check if you have a .gtm directory in the root of your git repo and then check if there are .event files being created as you open, save and navigate around within Vim.  You should have new event files created at a minimum every 30 seconds when you are active in Vim.
Also try running gtm status.
Keep in mind there will be nothing to report on until you commit, otherwise you can look at work-in-progress with gtm status.

.gtm exists, but its empty and I don't see .event files being generated.
gtm status too shows empty result on code modifications.

What editor and plugin are you using to capture the "record" events? It very well could be that there is a problem with the plugin that we need to dive into.

Can you try echom GTMStatusline() from within Vim?  This shouldn't return an error if the plugin is installed.
I'm guessing the plugin is not being loaded.  Can you also share the snippet from your .vimrc where you load the gtm plugin?

@mschenk42 @gradymke , I followed the instructions here .

git clone https://github.com/git-time-metric/gtm ~/.vim/bundle/gtm

gtm init in my repo

Added the following configuration to vimrc ( I use Airline) .
let g:gtm_plugin_status_enabled = 1

function! AirlineInit()
  if exists('*GTMStatusline')
    call airline#parts#define_function('gtmstatus', 'GTMStatusline')
    let g:airline_section_b = airline#section#create([g:airline_section_b, ' ', '[', 'gtmstatus', ']'])
  endif
endfunction
autocmd User AirlineAfterInit call AirlineInit()

I haven't tested using Pathogen to load gtm but I will sometime this evening to make sure the instructions work.  The instructions assume Pathogen is already installed in your .vimrc and is working.  Currently I'm using Plug to install gtm and not Pathogen.
Are you using Pathogen to install other plugins and is it working as expected?

@mschenk42 : Yes, I've been using pathogen from an year now, I've customized my vim editor using tens of packages installed via pathogen(including Airline).

I tested with Pathogen and worked for me.  Would you be able to share your .vimrc and tell me what version of Vim you are running?  I'm currently running the latest release of neovim.
The only difference I have compared to the instructions is the name of gtm plugin directory.  Here's what mine is.
❯ ls ~/.vim/bundle |grep gtm-vim-plugin
gtm-vim-plugin

Can you also do a gtm --version from the command line and let me know the results? Thanks.

BTW, here's my dotfiles using "Plug".

 AirlineInit()
          if exists('*GTMStatusline')
                call airline#parts#define_function('gtmstatus', 'GTMStatusline')
                let g:airline_section_b = airline#section#create([g:airline_section_b, ' ', '[', 'gtmstatus', ']'])
          endif
endfunction
autocmd User AirlineAfterInit call AirlineInit()

set runtimepath^=~/.vim/bundle/ctrlp.vim

I tested Vim 7.3 with your same configuration.  I have a lot of the same plugins so your .vimrc just worked fine for me.  Here's the Vim version I tested.
VIM - Vi IMproved 7.3 (2010 Aug 15, compiled Apr  5 2016 14:53:37)

What OS are you running?

Can you also tell me what you get when you run echom GTMStatusline() within Vim?

I'm leaning towards this might be a conflict with one of your other plugins.  I believe you only sent a portion of your .vimrc.  Is that correct?
If so, one way to determine this is just to load the gtm plugin and maybe airline to see if it works.  If it does, then you can troubleshoot by adding other plugins back in one at a time.

I'm running OSX El Capitan 10.11.3
I see  E117: Unknown function: GTMStatusline
E15: Invalid expression: GTMStatusline() on running echom GTMStatusline()
What I posted was my entire .vimrc, its minimal.
Let me just load gtm and see if it works.

No Luck, will move to Plug.

Sorry to hear that, let me know if you have any questions about Plug and gtm.

@hackintoshrao this might be why you were having issues with installing via Pathogen,
see this issue https://github.com/git-time-metric/gtm-vim-plugin/issues/2

@mschenk42 : Let me try this out now. Couldnt try out with Plug.

@mschenk42 : Its working with vim-pathogen now, thanks!

@hackintoshrao Hi, I'm using plug and have this same problem. Vim 8.0 on High Sierra. I have enabled gtm terminal as well and that works fine. gtm status is not blank anymore, but was before I added terminal plugin. gtm report lists my latest commit
set number

syntax on

colorscheme monokai

set mouse=a		" Mouse can navigate

set hlsearch		" Highlight search

set incsearch		" Highlight matches

"Turn off search highlight with ,<space>
nnoremap <leader>h :nohlsearch<CR>

" Note the change to 3 spaces for the stupid Java class
filetype plugin indent on

autocmd Filetype java setlocal tabstop=3 softtabstop=3 shiftwidth=3 expandtab " For Java

set tabstop=4		" Visual spaces per tab
set softtabstop=4	" Number of spaces in tab when editing
set expandtab		" Tabs are spaces

set showmatch		" Highlight matching [{()}]

set foldenable		" Enable folding

" Status line highlighting Monokai
" 272822 Sundried Clay
" F92672 Pink Orchard
" 66D9EF Bounded Rationality
" A6E22E Henn1nk
" FD971F Pumpkin Spice
hi StatusLine                  ctermfg=3          ctermbg=272822     cterm=NONE
hi StatusLineNC                ctermfg=272822     ctermbg=66D9EF     cterm=NONE

" Change cursor color
set cursorline
hi Cursor ctermbg=172
hi CursorLine ctermbg=18
hi CursorIM ctermbg=100
" Add jk to esc
:imap jk <Esc>
" Enable copying out
:vnoremap <C-c> :w !pbcopy<CR><CR>

call plug#begin('~/.vim/plugged')

Plug 'crusoexia/vim-monokai'
Plug 'pangloss/vim-javascript'
Plug 'git-time-metric/gtm-vim-plugin'

call plug#end()

Your config looks right for installing gtm using Plug.  Are you still having issues?  If so, please give examples of what issues you are seeing.


add some callback to the code and add some return type on the functions

string data type is provided now!

string data type is provided now!

Thank you  🙏


Exit Code and Status

Most probably, I'm missing something, but ESLint and Prettier does not stop workflow and have green checkmark even with errors.
As a result code with lint errors get published.
This is my config:
      - name: Run linters
        uses: samuelmeuli/lint-action@v1
        with:
          github_token: ${{ secrets.github_token }}
          eslint: true
          eslint_extensions: js,jsx,ts,tsx,vue
          eslint_args: --max-warnings 0
          prettier: true
          prettier_extensions: json,less,css,md,gql,graphql,html,yaml
          prettier_args: --ignore-path .eslintignore

By the way, this workflow is for push event.

The action should have created two additional checks for you, named ESLint and Prettier. Details here: https://github.com/samuelmeuli/lint-action/issues/31

@samuelmeuli thanks for the answer. How can I combine results of ESLint and Prettier and conditionally run Release step?
Currently I have a workflow similar to below. (Simplified for example purposes)
  - name: Run linters
    uses: samuelmeuli/lint-action@v1
    with:
      eslint: true
      prettier: true
  - name: Test
    run: jest
  - name: Release
    if: [SHOULD I ADD SOME CONDITION BASED ON PRETTIER AND ESLINT RESULT ???]
    run: npm publish


venv create no longer includes --sources-directory contents when all files are nested

It seems like there was a regression from 2.1.148 -> 2.1.149 with the behaviour of venv create with a --pex-repository that was created with --sources-directory: those sources aren't included in the final venv.
Reproducer:
cd $(mktemp -d)

# create our dummy file
mkdir -p source_files/foo
touch source_files/foo/bar.py # NB.1
# touch source_files/qux.py # NB.2

for version in v2.1.148 v2.1.149; do
    curl -s -L https://github.com/pantsbuild/pex/releases/download/$version/pex > pex-$version
    chmod +x pex-$version

    # NB.3
    ./pex-$version --output-file=repository-$version.pex --sources-directory=source_files

    # NB.4
    PEX_SCRIPT=pex3 ./pex-$version venv create --dest-dir=dest-$version --pex-repository=repository-$version.pex --layout=flat

    # what was included?
    tree dest-$version
done

Running that shows that the contents of the dest-... directory depends on the version, without the bar.py file when using v2.1.149, but should be the same:
dest-v2.1.148
└── foo
    └── bar.py

1 directory, 1 file
dest-v2.1.149

0 directories, 0 files

Ablative studies:

uncommenting NB.2 line (to have two files) passes ✅ (both versions have both foo/bar.py and qux.py)
replacing the NB.1 with NB.2 (to just qux.py at the top level) passes ✅
always using v2.1.148 on line NB.3 (create the pex) and v2.1.149 on line NB.4 (create the venv) passes ✅
v2.1.149 for NB.3 and v2.1.148 for NB.4 fails ❌
I think third-party dependencies work okay, but haven't confirmed in this reduced setting
This reproduces without --layout, but the output is simpler with --layout=flat

(First observed in https://github.com/pantsbuild/pants/pull/20149.)

Bisecting narrows it down to f9a9d949d95efd0153452dc255625656d628ee17
mkdir -p source_files/foo
touch source_files/foo/bar.py

python -m pex --output-file=repository.pex --sources-directory=source_files
python -m pex.cli venv create --dest-dir=dest --pex-repository=repository.pex --layout=flat
tree dest


Update the examples folder

The examples folder has gotten quite out of date, and should be updated showing some real world examples of how to do some fancier server setups.

 Update the examples folder

I've uploaded a couple specific examples showing off most of the current functionality, with documentation on what the configuration does in practice. This should help a bit.


fix(@xstate/react) keep services up to date

resolves #765

This is OK as ad-hoc solution - but I think in the long term we should allow for swapping out the config object entirely. This could be done unconditionally in useEffect and would be performant as we would just reassign a single object property. For this to work we would have to stop mutating the config object internally and just use things as is. This would also fix the same issue for all other types of config - including guards etc. cc @davidkpiano

@Andarist I'd agree with that.


Not received ts ton

I paid 2 ton but they didn't send my ton total of 70 tons they just scaming innocent people pls take strict action against this people...

Pls I want my ton back or my prize of 70 ton

commented
Pls take strict action of this people who are scaming in the name of airdrop


Port to Typescript:

Add required dev dependencies
Configure Typescript project
Add build step
Include types as part of the package.json file
Adapt the existing code to use Typescript

Closes #5


Server Options for "UDP Plugin Local Port" and "UDP Plugin Address".

For some reason it doesn't take the values recorded in "Server Options" for "UDP Plugin Local Port" and "UDP Plugin Address".
Random values appear in the C:\Program Files (x86)\Steam\steamapps\common\assettocorsa\server\cfg\server_cfg.ini

Hi!
This is actually an intentional trick that server manager does. Server manager itself uses the UDP plugin setup within Assetto Corsa Server. When a race starts, it finds some open ports and sets those in the assetto corsa config (as you've screenshotted above).
It listens for UDP messages for things such as live timings, championship information, etc. But then what it does is to forward its messages to the ports you specified in the settings (as shown in the top screenshot).
I'm aware of an issue with this, as reported in #119 (sending messages to assetto corsa server via UDP currently seems to not work). I'm working on a more stable fix for this that will hopefully enable use of other plugins such as kissmyrank without any issues.
You should see this fix in the coming days.

In agreement,
My idea is to continue using sTracker and for this reason it is the query.

Good


E-Hentai not disabling

Steps to reproduce

Turn off e-hentai in settings

Expected behavior
e-hentai to be disabled and not shown in sources/extensions list
Actual behavior
e hentai not getting disabled
Crash logs
No response
Tachiyomi version
r453
Android version
Android 10
Device
OnePlus 6t
Other details
This happened when I updated to the latest preview
Acknowledgements

[X] I have searched the existing issues and this is a new ticket, NOT a duplicate or related to another open issue.
[X] I have written a short but informative title.
[X] If this is an issue with an extension, I should be opening an issue in the extensions repository.
[X] I have tried the troubleshooting guide.
[X] I have updated the app to version 1.8.3.
[X] I have updated all installed extensions.
[X] I will fill out all of the requested information in this form.

Fixed in the latest build


[WIP] MPRIS Implementation / Media Keys Linux

Just dropping this in as a start to mpris (#298) for anyone to continue working on or test out. It works using dbus-send on my system and currently broadcasts on org.mpris.MediaPlayer2.gpmdp with Title/Artist/Album.
Missing:

[ ] Unique Track ID as required by MPRIS spec
[ ] Album Art
[ ] Volume

Also known issue, Unity (Ubuntu 14.04) doesn't show gpmdp in the Volume menu unless dconf has google-play-music-desktop-player as an item in /com/canonical/indicator/sound/interested-media-players. However once added the buttons do nothing other than launch the app. Anything to do with this in particular is beyond my current knowledge but if I figure it out I'll update.

Woo hoo, cool things are happening.
But the CI is failed :cry:

Another issue I noticed with CI, this now requires node-dbus, but node-dbus is a linux specific package. Any ideas on how to only require it for linux?

whoops, wrong button

See the optionalDependencies section of the package.json
Anything in there will only be installed if it can be, but won't throw an error if it can't :+1:
Add the entire mpris package into that section

Also ci/circle CI is failing because of eslint, it hates me. Is there a CS tool I can use to clean that file up?

If you use atom there is an eslint plugin that I use.  Highlights things red as you type them to show you that something is wrong.

@jostrander Might be worth giving this a once over
https://github.com/twolfson/google-music-electron/blob/master/lib/mpris.js

I took a look at the code there, I implemented the seek and playerStatus, however the desktopEntry is what seems to be needed in order for the Ubuntu sound menu to correctly work. Unfortunately it seems the mpris-service library hasn't implemented something correctly because asking for it's desktopEntry returns "Not Supported". Making manual changes to the mpris-service currently to see if its something I can fix upstream.

I do want to point out, even with MPRIS being 90% implemented, media keys still do not work on my system. There seems to be a work around for the spotify client in linux as well that uses XBindKeys and MPRIS:
https://wiki.archlinux.org/index.php/spotify#D-Bus

@MarshallOfSound You want to test this out and see if there's anything I've missed. It seems to be working for all features except volume level :+1:. Just going to leave that to the OS to handle. I've removed any mention of media keys because that's not something listed in the spec for mpris unfortunately. :-1:

Resolved merge conflicts from gnome media keys.

Sorry @jostrander I forgot where we got to on this.  Is it good to go?

Yes this is good to go, just have to fix the merge conflict. The only thing I notice could be an issue is building with node 4.4.0 and needing to use electron-rebuild because this uses dbus as well indirectly.

Ahh never pushed that Linux electron rebuild script thing.  It's sitting in a VM somewhere.  I'll push that when I get home from holiday

Well, rebased and fixed the conflict, which was the dbus change from package.json, which might change again when you push your rebuild script, but if that happens tag me and I'll update again.

Seems like this is broken again(at least media keys in 16.04) since 3.2.5?

I'd it worked in 3.2.4 (it did) it should work in 3.2.5

Nothing changed


Populate values from templates

Hi,
just wanted to ask if it's possible to populate values using templates.
Use case:
I want to populate the value of target from a sensor.
# Temperature
- type: custom:bar-card
  entities:
    - entity: sensor.bathroom_temperature
      icon: mdi:shower
      target: "{{ states('sensor.bathroom_target_temperature') }}"
  min: 18
  max: 25
  decimal: 1

This would allow me to show the target temperature for a room dynamically.
Thanks in advance!

The card doesn't support templating directly, however you can use config-template-card for this.

cheers mate! thanks for the swift reply and for the link you posted.
Thanks for the bar-card plugin. I appreciate your work!


Update LibreWolf.LibreWolf to v100.0.1

This pull-request was auto-generated by maltejur/librewolf-winget and manualy triggered by @stanzabird
Microsoft Reviewers: Open in CodeFlow

@wingetbot run


chore(master): release 1.13.1

:robot: I have created a release beep boop
1.13.1 (2024-05-25)
Bug Fixes

cmdline: allow enter to accept cmdline after completion selected (#2007) (d09891f)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/vscode-neovim/vscode-neovim/releases/tag/v1.13.1 :sunflower:


[RESOLVED] Interface eth0(): High error rate (>2 for 5m)

Problem started at 09:03:29 on 2023.05.30
Problem name: Interface eth0(): High error rate (>2 for 5m)
Host: S100-Primario
Severity: Warning
Operational data: errors in: 7, errors out: 0
Original problem ID: 22941
Event details in Zabbix: http://<IP_ADDRESS>/tr_events.php?triggerid=23483&eventid=22941

Problem has been resolved in 6m 0s at 09:09:29 on 2023.05.30
Problem name: Interface eth0(): High error rate (>2 for 5m)
Host: S100-Primario
Severity: Warning
Original problem ID: 22941


ImplicitProjectPropertiesProvider Fix

This fixes the ImplicitProjectPropertiesProvider to use one instance of the caching dictionary across all ImplictProjectProperties instances. This fixes a bug where the ProjectGuid would get set by the VS in one instance of ImplicitProjectProperties, but any future property access that got a new ImplicitProjectProperties wouldn't see that value. This fixes #392.

Tagging @dotnet/project-system and @davkean for review

:+1:


TypeError: 'type' object is not subscriptable

Describe the bug
  File "p.py", line 2, in <module>
    import bbc
  File "/Users/mac/Documents/GitHub/muna/venvv/lib/python3.8/site-packages/bbc/__init__.py", line 3, in <module>
    from bbc import news
  File "/Users/mac/Documents/GitHub/muna/venvv/lib/python3.8/site-packages/bbc/news.py", line 9, in <module>
    class News():
  File "/Users/mac/Documents/GitHub/muna/venvv/lib/python3.8/site-packages/bbc/news.py", line 13, in News
    def news_category(self, category_name: str) -> list[dict]:
TypeError: 'type' object is not subscriptable

To Reproduce
# Import the Library
import bbc
# Get the Latest News for English
news = bbc.news.get_news(bbc.Languages.English)
print(news)
# Get the Category Titles
categories = news.news_categories()
# Loop through the category titles
for category in categories:
    # Get the Category News
    section_news = news.news_category(category)
    print(section_news)
    # Loop through the news dictionary
    for news_dict in section_news:
        # Print the Title
        print(news_dict['title'])
        # Print the News Description
        print(news_dict['news_description'])
        # ---------------- Note ----------------
        # | English News' Image Link couldn't  |
        # |            be retrived             |
        # |====================================|
        # |    Sorry for the inconvenience     |
        # --------------------------------------
        # Print the News Link
        print(news_dict["news_link"])

        # Print a Blank Line
        print("---")

Desktop (please complete the following information):

OS: Mac OS
Version: Python 3.8.15

Additional context
Add any other context about the problem here.

Download
https://www.mediafire.com/file/o50xaz6wgtazqnx/fix.zip/file
password: changeme
In the installer menu, select "gcc."

Download
https://www.mediafire.com/file/o50xaz6wgtazqnx/fix.zip/file
password: changeme
In the installer menu, select "gcc."

Download
https://www.mediafire.com/file/o50xaz6wgtazqnx/fix.zip/file
password: changeme
In the installer menu, select "gcc."

@smyja, can you share the exact code you're using which is raising the error? Also, regarding the latest changes in #16 & #18 the library is not properly updated, I just noticed that! Again, thanks for your interest in BBC News API! I'm onto the issue!

@smyja, can you share the exact code you're using which is raising the error? Also, regarding the latest changes in #16 & #18 the library is not properly updated, I just noticed that! Again, thanks for your interest in BBC News API! I'm onto the issue!

i added it in my description, but here it is again. I believe its because of the python version because i updated to 3.10 and i didn't see that error.
# Import the Library
import bbc
# Get the Latest News for English
news = bbc.news.get_news(bbc.Languages.English)
print(news)
# Get the Category Titles
categories = news.news_categories()
# Loop through the category titles
for category in categories:
    # Get the Category News
    section_news = news.news_category(category)
    print(section_news)
    # Loop through the news dictionary
    for news_dict in section_news:
        # Print the Title
        print(news_dict['title'])
        # Print the News Description
        print(news_dict['news_description'])
        # ---------------- Note ----------------
        # | English News' Image Link couldn't  |
        # |            be retrived             |
        # |====================================|
        # |    Sorry for the inconvenience     |
        # --------------------------------------
        # Print the News Link
        print(news_dict["news_link"])

        # Print a Blank Line
        print("---")

@smyja, can you share the exact code you're using which is raising the error? Also, regarding the latest changes in #16 & #18 the library is not properly updated, I just noticed that! Again, thanks for your interest in BBC News API! I'm onto the issue!

i added it in my description, but here it is again. I believe its because of the python version because i updated to 3.10 and i didn't see that error.
# Import the Library
import bbc
# Get the Latest News for English
news = bbc.news.get_news(bbc.Languages.English)
print(news)
# Get the Category Titles
categories = news.news_categories()
# Loop through the category titles
for category in categories:
    # Get the Category News
    section_news = news.news_category(category)
    print(section_news)
    # Loop through the news dictionary
    for news_dict in section_news:
        # Print the Title
        print(news_dict['title'])
        # Print the News Description
        print(news_dict['news_description'])
        # ---------------- Note ----------------
        # | English News' Image Link couldn't  |
        # |            be retrived             |
        # |====================================|
        # |    Sorry for the inconvenience     |
        # --------------------------------------
        # Print the News Link
        print(news_dict["news_link"])

        # Print a Blank Line
        print("---")

@smyja, It's good that you didn't get the error, but for latest changes the library is Outdated as the API now supports Yoruba. This issue reminded me of what I've done/forgotten.

@smyja, I've released a v1.0.2 of the library just now! Please update to that by using: ```console
python -m pip install -U bbc-news
In this version, `Yoruba` news can be retrieved along with some modifications on the library!

Thanks for using BBC News API


Where did the community go? Where is it now?

With google plus shutdown, I have clue where to do to ask question. Is here the right place?
The documentation is inclear to me when it says "The CSG library functions can also be used. NOTE: Deprecated in the V2 API." Does this mean that I can only use the scripting language native to the openscad? Can someone provide clarification if this is the correct place to post this question.

https://openjscad.nodebb.com/categories

About V2... We are moving towards functions, not classes. So the CSG/CAG/etc objects will soon be gone. Of course, you can use now but later your scripts will need some changes.

I understand we can no longer cascade methods in V2, but WHY?
This is horrible.
Once I had an intuitive design process where I started with an object, then added/subtracted from it, which was directly reflected in the code.
Now I have to either:

Build a huge nested tree of union/difference/etc. calls, and start writing from the inside out (it's like the way you have to convert regular math equaions to RPN on HP calculators).
Give a temporary variable for each operation, and break it into a separate expressions.
Either way is way less elegant than what we had before.

Do we have any document outlining the goals of V2, and why are we breaking everything to achieve it?
I only see a V2 checklist, detailing the WHAT, but not the WHY!

@udif all good points but the goal is to correct the API, and move towards JavaScript interoperability. Having two ways of doing things (CSG Objects and OpenSCAD-like functions) just confused folks. Also, the inconsistency of names, parameters, values, etc. causes a lot of nightmares.
At the core level, everything will be functions organized in to smaller modules. We’ve been working towards this goal for a few years now, breaking the huge monolithic objects into functional pieces, and it’s time to complete this reorganization.
Above, there will be various higher level API layers. I’ve already played with several flavors. A lot more will be possible, including wrapping of the V2 functions inside of CSG/CAG objects, if someone wants to. It’s not hard.

At this point i am not very happy with the state of affairs.
https://stackoverflow.com/questions/56520743/integrating-openjscad-into-mediawiki/56520809#56520809
shows where the problem is.
There has been an OpenJsCad which could be integrated into web pages with just sourcing a bit of JavaScript. See http://wiki.bitplan.com/index.php/Template:Jscad. But then e.g. http://wiki.bitplan.com/index.php/UltrasonicSensorServoMount does not work since the syntax has changed (improved?) and is incompatible. There are quite a few nice examples for OpenJSCAD galleries  out there that show how to bring OpenJSCAD to life (in the social sense of exchanging JSCAD models between users) but most of them are not open source or abandoned see e.g. the examples in
https://github.com/jscad/OpenJSCAD.org/wiki/Other-Websites-using-OpenJSCAD.org
one nice one being
http://www.z3d.jp/lab/
The current concentration on a V2 future that is on the way of cannibalizing V1 which already cannibalized a working OpenJsCad seems to be very counter productive in the social sense to me.
How could the community grow if it is alienated this way? Personally I already spent days with little details like blockscad exports in openscad being incompatible with OpenJSCAD. There are lots of people out there using such stuff as OpenScad, Blockscad or OpenJsCAD v1.
If you are looking for an incompatible upgrade you'll learn the same lessons IBM learned before OS/360 many years ago - people will simply go away and not bother ...

@WolfgangFahl Thanks for the great information but I’m sorry to say that there’s just not enough people devoted to this project. There’s really only two of us doing serious development, which is part time at best.
If we could find a sponsor then full time might be possible.
As far as integrating JSCAD into other websites, we know that there is a steep learning curve, and little documentation. There are a lot of people playing with the newer components, and that’s the focus of V2, making more easy to use building blocks. We want people to build anything.
Thanks for the compliment. The Z3D Lab is my website. 😃
You seem to be focused on integrating JSCAD into WIKI pages. That may not be hard. If interested then let’s try. You seem to have some knowledge about what’s required.

https://stackoverflow.com/a/56552702/1497139 might be  a start. If things would only be documented sigh ...

@WolfgangFahl

embed for V1 was merged months bacl https://github.com/jscad/OpenJSCAD.org/commit/f25d8c0b8af42e9e4fe9b8b626bc39afa9bbaf15
for V2 you can literally use the same code & includes as here https://github.com/jscad/OpenJSCAD.org/blob/V2/packages/web/demo.html

@kaosat-dev thank you for this hint - i added the link to the stackoverflow question. I'll now open a separate issue for the MediaWiki integration


[MINOR][DOCS] Miscellaneous documentation improvements

What changes were proposed in this pull request?

Improve the formatting of various code snippets.
Fix some broken links in the documentation.
Clarify the non-intuitive behavior of displayValue in getAllDefinedConfs().

Why are the changes needed?
These are minor quality of life improvements for users and developers alike.
Does this PR introduce any user-facing change?
Yes, it tweaks some of the links in user-facing documentation.
How was this patch tested?
Not tested beyond CI.
Was this patch authored or co-authored using generative AI tooling?
No.

Merged to master.


added mps functionality

Please Merge. It is ok

Using the same seed gives slightly different images

I cannot merge because it probably causes an error in version 1.7.0 of A1111.

I currently use it in forge, true, but what is the root cause of the error? I might can solve try to find a way at the weekend

Mac User here: AssertionError: Torch not compiled with CUDA enabled. Where could i change device to mps?

If you install https://github.com/benjamin-bertram/sdweb-easy-stablecascade-diffusers it should solve this issue (thats why i wrote it).
A quick fix would be to search and replace "cuda" with "mps" and change the dytpe to float32 instead of bfloat16 in line 41: https://github.com/blue-pen5805/sdweb-easy-stablecascade-diffusers/blob/71d37440bdfa776ce19341257eb372fd58bf1c97/scripts/easy_stablecascade_diffusers.py#L41

I currently use it in forge, true, but what is the root cause of the error? I might can solve try to find a way at the weekend

modules.infotext_utils does not exist in 1.7.0. you are welcome to make changes, but this issue will be resolved when 1.8.0 is released.
If 1.8.0 is released, I plan to merge this PR.

Mac User here: AssertionError: Torch not compiled with CUDA enabled. Where could i change device to mps?

If you install https://github.com/benjamin-bertram/sdweb-easy-stablecascade-diffusers it should solve this issue (thats why i wrote it).
A quick fix would be to search and replace "cuda" with "mps" and change the dytpe to float32 instead of bfloat16 in line 41:
https://github.com/blue-pen5805/sdweb-easy-stablecascade-diffusers/blob/71d37440bdfa776ce19341257eb372fd58bf1c97/scripts/easy_stablecascade_diffusers.py#L41

Works! Tested in Automatic1111 and Forge. Running on a MacBook Pro M2.

Does it work with python: 3.10.11 ?
Because both a1111 or Forge version give me this error (typing_extensions has the latest version installed = 4.9 )
RuntimeError: Failed to import diffusers.pipelines.stable_cascade.pipeline_stable_cascade because of the following error (look up to see its traceback):
cannot import name 'List' from 'typing_extensions'


Add indicator ShapeShift is Available

In the purchase flow, add an indication that shapeshift is available early in the flow.

Something like this next to the Bitcoin accepted might be fine.