Connection reset generated by Envoy

I am using EnvoyProxy version : 1.15.2. My usecase is a reverse proxy solution meant for multi-tenant system.
As part of my connection manager configuration :
- name: envoy.http_connection_manager
   config:
     stat_prefix: ingress_http
     codec_type: AUTO
     common_http_protocol_options:
       max_connection_duration: 120s

Intermittently, my customers are seeing "Connection reset by Peer" exception in their code. I took TCPDump for one such customer, and saw following behavior :
19:15:19.286535 IP (tos 0x0, ttl 255, id 57091, offset 0, flags [DF], proto TCP (6), length 60)
    <IP_ADDRESS>.44332 > <IP_ADDRESS>.8080: Flags [S], cksum 0x1566 (incorrect -> 0xfb6a), seq 849832810, win 62727, options [mss 8961,sackOK,TS val<PHONE_NUMBER> ecr 0,nop,wscale 7], length 0

19:15:19.288045 IP (tos 0x0, ttl 252, id 0, offset 0, flags [DF], proto TCP (6), length 60)
    <IP_ADDRESS>.8080 > <IP_ADDRESS>.44332: Flags [S.], cksum 0x8538 (correct), seq<PHONE_NUMBER>, ack 849832811, win 26847, options [mss 1460,sackOK,TS val 297489952 ecr<PHONE_NUMBER>,nop,wscale 8], length 0

19:15:19.288053 IP (tos 0x0, ttl 255, id 57092, offset 0, flags [DF], proto TCP (6), length 52)
    <IP_ADDRESS>.44332 > <IP_ADDRESS>.8080: Flags [.], cksum 0x155e (incorrect -> 0x1af8), ack 1, win 491, options [nop,nop,TS val<PHONE_NUMBER> ecr 297489952], length 0

19:15:19.288211 IP (tos 0x0, ttl 255, id 57093, offset 0, flags [DF], proto TCP (6), length 672)
    <IP_ADDRESS>.44332 > <IP_ADDRESS>.8080: Flags [P.], cksum 0x17ca (incorrect -> 0x93a8), seq 1:621, ack 1, win 491, options [nop,nop,TS val<PHONE_NUMBER> ecr 297489952], length 620: HTTP, length: 620

19:15:19.289166 IP (tos 0x0, ttl 252, id 16385, offset 0, flags [DF], proto TCP (6), length 52)
    <IP_ADDRESS>.8080 > <IP_ADDRESS>.44332: Flags [.], cksum 0x1a09 (correct), ack 621, win 110, options [nop,nop,TS val 297489952 ecr<PHONE_NUMBER>], length 0

19:15:19.296689 IP (tos 0x0, ttl 252, id 16386, offset 0, flags [DF], proto TCP (6), length 505)
    <IP_ADDRESS>.8080 > <IP_ADDRESS>.44332: Flags [P.], cksum 0xdb48 (correct), seq 1:454, ack 621, win 110, options [nop,nop,TS val 297489953 ecr<PHONE_NUMBER>], length 453: HTTP, length: 453

19:15:19.296699 IP (tos 0x0, ttl 255, id 57094, offset 0, flags [DF], proto TCP (6), length 52)
    <IP_ADDRESS>.44332 > <IP_ADDRESS>.8080: Flags [.], cksum 0x155e (incorrect -> 0x16c1), ack 454, win 488, options [nop,nop,TS val<PHONE_NUMBER> ecr 297489953], length 0

19:15:32.166437 IP (tos 0x0, ttl 255, id 57095, offset 0, flags [DF], proto TCP (6), length 723)
    <IP_ADDRESS>.44332 > <IP_ADDRESS>.8080: Flags [P.], cksum 0x17fd (incorrect -> 0x2472), seq 621:1292, ack 454, win 488, options [nop,nop,TS val<PHONE_NUMBER> ecr 297489953], length 671: HTTP, length: 671

19:15:32.191067 IP (tos 0x0, ttl 252, id 16387, offset 0, flags [DF], proto TCP (6), length 779)
    <IP_ADDRESS>.8080 > <IP_ADDRESS>.44332: Flags [P.], cksum 0xe552 (correct), seq 454:1181, ack 1292, win 115, options [nop,nop,TS val 297491242 ecr<PHONE_NUMBER>], length 727: HTTP, length: 727

19:15:32.191079 IP (tos 0x0, ttl 255, id 57096, offset 0, flags [DF], proto TCP (6), length 52)
    <IP_ADDRESS>.44332 > <IP_ADDRESS>.8080: Flags [.], cksum 0x155e (incorrect -> 0xd9e7), ack 1181, win 483, options [nop,nop,TS val<PHONE_NUMBER> ecr 297491242], length 0

19:16:17.470597 IP (tos 0x0, ttl 255, id 57097, offset 0, flags [DF], proto TCP (6), length 723)
    <IP_ADDRESS>.44332 > <IP_ADDRESS>.8080: Flags [P.], cksum 0x17fd (incorrect -> 0x3f3d), seq 1292:1963, ack 1181, win 483, options [nop,nop,TS val<PHONE_NUMBER> ecr 297491242], length 671: HTTP, length: 671

19:16:17.491355 IP (tos 0x0, ttl 252, id 16388, offset 0, flags [DF], proto TCP (6), length 785)
    <IP_ADDRESS>.8080 > <IP_ADDRESS>.44332: Flags [P.], cksum 0xcb4d (correct), seq 1181:1914, ack 1963, win 121, options [nop,nop,TS val 297495772 ecr<PHONE_NUMBER>], length 733: HTTP, length: 733

19:16:17.491367 IP (tos 0x0, ttl 255, id 57098, offset 0, flags [DF], proto TCP (6), length 52)
    <IP_ADDRESS>.44332 > <IP_ADDRESS>.8080: Flags [.], cksum 0x155e (incorrect -> 0x11ca), ack 1914, win 478, options [nop,nop,TS val<PHONE_NUMBER> ecr 297495772], length 0

19:16:28.241150 IP (tos 0x0, ttl 255, id 57099, offset 0, flags [DF], proto TCP (6), length 723)
    <IP_ADDRESS>.44332 > <IP_ADDRESS>.8080: Flags [P.], cksum 0x17fd (incorrect -> 0x828f), seq 1963:2634, ack 1914, win 478, options [nop,nop,TS val<PHONE_NUMBER> ecr 297495772], length 671: HTTP, length: 671

19:16:28.259600 IP (tos 0x0, ttl 252, id 16389, offset 0, flags [DF], proto TCP (6), length 785)
    <IP_ADDRESS>.8080 > <IP_ADDRESS>.44332: Flags [P.], cksum 0x917e (correct), seq 1914:2647, ack 2634, win 126, options [nop,nop,TS val 297496849 ecr<PHONE_NUMBER>], length 733: HTTP, length: 733

19:16:28.259617 IP (tos 0x0, ttl 255, id 57100, offset 0, flags [DF], proto TCP (6), length 52)
    <IP_ADDRESS>.44332 > <IP_ADDRESS>.8080: Flags [.], cksum 0x155e (incorrect -> 0xde0d), ack 2647, win 473, options [nop,nop,TS val<PHONE_NUMBER> ecr 297496849], length 0

19:17:25.287512 IP (tos 0x0, ttl 255, id 57101, offset 0, flags [DF], proto TCP (6), length 723)
    <IP_ADDRESS>.44332 > <IP_ADDRESS>.8080: Flags [P.], cksum 0x17fd (incorrect -> 0x83c7), seq 2634:3305, ack 2647, win 473, options [nop,nop,TS val<PHONE_NUMBER> ecr 297496849], length 671: HTTP, length: 671

19:17:25.289906 IP (tos 0x0, ttl 252, id 16390, offset 0, flags [DF], proto TCP (6), length 52)
    <IP_ADDRESS>.8080 > <IP_ADDRESS>.44332: Flags [R.], cksum 0xe7b4 (correct), seq 2647, ack 3305, win 131, options [nop,nop,TS val 297502552 ecr<PHONE_NUMBER>], length 0

In this, because Envoy didnt generate [F.] flag, client never knew connection was triggered for close and received reset exception. Any ideas, why it can happen this way. Is there any property I am not making use of which can help in this scenario.
This is not happening in all the cases, but only for few of them.  Any leads in this would be very helpful.
Note : configurations like delayed_close_timeout, drain_timeout, max_stream_duration are not set in configuration and defaults are used for them.

Hi guys,
Can someone help me in this?
Thanks.

@prateek0394 we have similar problems in our environment. To avoid such errors you should set the max connection time of the HTTP client (your customer) lower then the max connection time on Envoy. This way the client will terminate the connection and you don't have the case where Envoy is closing the connection and at the same time the client tries to send data.
In many cases it probably also makes more sense to use idle_timeout, because then you close the connection only if it was not used for that period.
From what I understand in your tcpdump the following thing happens:
19:15:19 connection start
19:17:19 (+120s max_connection_duration)
  -> connection termination starts
  -> header `Connection: close` is set
19:17:24 (+5s drain_timeout)
  -> no new data accepted, no FIN sent yet
  -> requests are answered with connection reset
19:17:25 (+1s delayed_close_timeout) envoy sends FIN

If a request is sent after the drain_timeout and before the delayed_close_timeout elapses Envoy answers your request with a connection reset.


Add tail -f functionality (flag -f)

Use this module: https://github.com/nxadm/tail

No need for the module nxadm/tail. Just used fsnotify from the standard library.

Closed with PR #9


Sonic builder [unstable]

Error in sonic builder on unstable in fresh installation [in system]

This is because of the fixes in this issue where we decided not to build rust unless reset is True. Maybe it can be modified to make this check only if rust is installed. https://github.com/threefoldtech/jumpscaleX_builders/issues/37

Fixed by this https://github.com/threefoldtech/jumpscaleX_builders/commit/3e1c06d291a8f096dd9577c687ba5f235eceb27b

verified on branch unstable :
root@jsx-test:~# curl https://raw.githubusercontent.com/threefoldtech/jumpscaleX_core/unstable/install/jsx.py?$RANDOM > /tmp/jsx;
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 34333  100 34333    0     0   279k      0 --:--:-- --:--:-- --:--:--  279k
root@jsx-test:~# chmod +x /tmp/jsx;
root@jsx-test:~# /tmp/jsx install -b unstable


[Feature] Sync for manifests and helm chart

Why are these changes needed?
The CRD inconsistency between ray-operator/config/crd/bases/ and helm-chart/kuberay-operator/crds/ is possible to cause bugs like #538. Hence, we need to check the consistency in GitHub Actions.

Related issue number
Closes #446
#538
Checks
#565 is the control group.

[ ] I've made sure the tests are passing.
Testing Strategy

[ ] Unit tests
[x] Manual tests
[ ] This PR is not tested :(

Nice! Do we still need a PR with some logic to keep RBAC in sync?

Nice! Do we still need a PR with some logic to keep RBAC in sync?

Oh, I think we need RBAC synchronization. I will open an issue.


Issue with running HISAT 2.2 example from vignette

Hello,
I attempted to run the example described in the vignette. However, it appears to run into an issue when mapping reads to the yeast rRNA sequences using HISAT2. I'm attaching one of the log files. From what I can tell, it is breaking because HISAT2 is expecting a minimum read length of 20. I'm using HISAT2 v.2.2.0.
02_hisat2_rrna.log

Hi Alex,
Thanks for filing the issue. That's a strange observation, because although it's logical, this is not a problem we've had before. The vignette works for several of us on several computers and we haven't had this hisat2 fail.
@acope3 what version of cutadapt are you using?
@FlicAnderson what version of hisat2 works for you?
Edward

My version of hisat2 is hisat2-2.1.0.
@acope3, please add the number of lines and size of the file  vignette/tmp/WTnone/trim.fq:
$ wc -l  vignette/tmp/WTnone/trim.fq
$ du -sb vignette/tmp/WTnone/trim.fq

Hi @acope3  - Same as @mikej888 , I'm using hisat2-2.1.0, but I'm happy to investigate further if you're able to provide more info as Mike recommended - I'll be able to compare your logs with mine and see if we can figure out where the error is coming from.  It's not something I've seen before either, but thanks for bringing it up!

Request that @acope3 updates us if this still needs help, otherwise we'll close the issue.

Apologies, I've been in the middle of finishing up my dissertation, so this got lost in the shuffle. Based on the attached file, I'm using cutadapt 2.10 with Python 3.6.8.
01_cutadapt.log

My version of hisat2 is hisat2-2.1.0.
@acope3, please add the number of lines and size of the file  vignette/tmp/WTnone/trim.fq:
$ wc -l  vignette/tmp/WTnone/trim.fq
$ du -sb vignette/tmp/WTnone/trim.fq

Also, please attach the content of vignette/logs/<DATE>/WTnone/WTnone_01_cutadapt.log

These two commands 3809372 and 106710847, respectively.

A diff between 01_cutadapt.log (renamed to cutadapt_orig.log) from a release 2.0-beta tag regression test run against @acope3's log:
$ diff 01_cutadapt.orig.log 01_cutadapt.log
1c1
< This is cutadapt 1.18 with Python 3.7.3
---
> This is cutadapt 2.10 with Python 3.6.8
3,4c3,4
< Processing reads on 4 cores in single-end mode ...
< Finished in 3.67 s (4 us/read; 15.76 M reads/minute).
---
> Processing reads on 8 cores in single-end mode ...
> Finished in 3.44 s (4 us/read; 16.82 M reads/minute).
10d9
< Reads that were too short:              11,228 (1.2%)
18c17
< Sequence: CTGTAGGCACC; Type: regular 3'; Length: 11; Trimmed: 958926 times.
---
> Sequence: CTGTAGGCACC; Type: regular 3'; Length: 11; Trimmed: 958926 times
24,26c23,25
<   A: 29.0%
<   C: 17.8%
<   G: 29.8%
---
>   A: 29.1%
>   C: 17.9%
>   G: 29.9%
28c27
<   none/other: 1.1%
---
>   none/other: 0.9%
83d81
<

File sizes from the regression test run are identical to those provided by Alex:
$ wc -l vignette-20200304-2.0.beta/vignette-20200304-2.0.beta/tmp/WTnone/trim.fq
3809372 vignette-20200304-2.0.beta/vignette-20200304-2.0.beta/tmp/WTnone/trim.fq
$ du -sb vignette-20200304-2.0.beta/vignette-20200304-2.0.beta/tmp/WTnone/trim.fq
106710847       vignette-20200304-2.0.beta/vignette-20200304-2.0.beta/tmp/WTnone/trim.fq

On EDDIE cutadapt 2.10 was used (this run used the Nextflow implementation of the RiboViz workflow which is why the log file is named differently and is within work but it uses the same cutadapt command-line invocation):
$ diff work/01/08e517f2c93d669077d4c493eb73f5/.command.log 01_cutadapt.log
1,4c1,4
< This is cutadapt 2.10 with Python 3.7.6
< Command line parameters: --trim-n -O 1 -m 5 -a CTGTAGGCACC -o trim.fq SRR1042855_s1mi.fastq.gz -j 0
< Processing reads on 1 core in single-end mode ...
< Finished in 13.98 s (15 us/read; 4.14 M reads/minute).
---
> This is cutadapt 2.10 with Python 3.6.8
> Command line parameters: --trim-n -O 1 -m 5 -a CTGTAGGCACC -o vignette/tmp/WTnone/trim.fq vignette/input/SRR1042855_s1mi.fastq.gz -j 0
> Processing reads on 8 cores in single-end mode ...
> Finished in 3.44 s (4 us/read; 16.82 M reads/minute).
23,27c23,27
<   A: 29.2%
<   C: 18.0%
<   G: 30.1%
<   T: 22.3%
<   none/other: 0.4%
---
>   A: 29.1%
>   C: 17.9%
>   G: 29.9%
>   T: 22.2%
>   none/other: 0.9%

$ wc -l work/01/08e517f2c93d669077d4c493eb73f5/trim.fq
3809372 work/01/08e517f2c93d669077d4c493eb73f5/trim.fq
$ du -sb work/01/08e517f2c93d669077d4c493eb73f5/trim.fq
106710847       work/01/08e517f2c93d669077d4c493eb73f5/trim.fq

Again, identical file sizes.

My 02_hisat2_rrna.log is:
/home/ubuntu/hisat2-2.1.0/hisat2-align-s version 2.1.0
64-bit
Built on login-node03
Wed Jun  7 15:53:42 EDT 2017
Compiler: gcc version 4.8.2 (GCC)
Options: -O3 -m64 -msse2 -funroll-loops -g3 -DPOPCNT_CAPABILITY
Sizeof {int, long, long long, void*, size_t, off_t}: {4, 8, 8, 8, 8, 8}
952343 reads; of these:
  952343 (100.00%) were unpaired; of these:
    467194 (49.06%) aligned 0 times
    485149 (50.94%) aligned exactly 1 time
    0 (0.00%) aligned >1 times
50.94% overall alignment rate

Alex is using HISAT 2.2.0. HISAT 2.2.0 is not listed at ftp://ftp.ccb.jhu.edu/pub/infphilo/hisat2/downloads. Googling revealed http://daehwankimlab.github.io/hisat2/ which announced the HISAT 2.2.0 release 2/6/2020 available via http://daehwankimlab.github.io/hisat2/download/. So I used the new release:
$ wget -O hisat2-2.2.0-Linux_x86_64.zip https://cloud.biohpc.swmed.edu/index.php/s/hisat2-220-Linux_x86_64/download
$ unzip hisat2-2.2.0-Linux_x86_64.zip
$ export PATH=~/hisat2-2.2.0:$PATH
$ hisat2 --version
/home/ubuntu/hisat2-2.2.0/hisat2-align-s version 2.2.0
64-bit
Built on parkch-desktop
Thu Feb  6 15:10:16 CST 2020
Compiler: gcc version 4.8.5 20150623 (Red Hat 4.8.5-39) (GCC)
Options: -O3 -m64 -msse2 -funroll-loops -g3 -DPOPCNT_CAPABILITY -std=c++11
Sizeof {int, long, long long, void*, size_t, off_t}: {4, 8, 8, 8, 8, 8}

After running (again via the Nextflow version of RiboViz):
$ nextflow run /home/ubuntu/riboviz/prep_riboviz.nf -params-file vignette/vignette_config.yaml -ansi-log false

Unlike Alex's attempt, the run completes and hisat2 runs fine. But the log data is different to that from HISAT2 2.1.0:
$ cat work/2c/aa5ec51591efa4535f01624ad658f7/.command.log
/home/ubuntu/hisat2-2.2.0/hisat2-align-s version 2.2.0
64-bit
Built on parkch-desktop
Thu Feb  6 15:10:16 CST 2020
Compiler: gcc version 4.8.5 20150623 (Red Hat 4.8.5-39) (GCC)
Options: -O3 -m64 -msse2 -funroll-loops -g3 -DPOPCNT_CAPABILITY -std=c++11
Sizeof {int, long, long long, void*, size_t, off_t}: {4, 8, 8, 8, 8, 8}
  File "/home/ubuntu/hisat2-2.2.0/hisat2_read_statistics.py", line 182
    length_map = sorted(length_map.iteritems(), key=lambda (k,v):(v,k), reverse=True)
                                                           ^
SyntaxError: invalid syntax
952343 reads; of these:
  952343 (100.00%) were unpaired; of these:
    466472 (48.98%) aligned 0 times                # formerly 467194 (49.06%) aligned 0 times
    485871 (51.02%) aligned exactly 1 time     # formerly 485149 (50.94%) aligned exactly 1 time
    0 (0.00%) aligned >1 times
51.02% overall alignment rate

Numerous regression tests fail starting with the outputs from hisat2:
$ pytest -vs riboviz/test/regression/test_regression.py --expected=vignette-nextflow --nextflow --check-index-tmp --skip-workflow

riboviz/test/regression/test_regression.py::test_hisat_fq[vignette/tmp-WTnone-nonrRNA.fq] FAILED
riboviz/test/regression/test_regression.py::test_hisat_fq[vignette/tmp-WTnone-unaligned.fq] FAILED
riboviz/test/regression/test_regression.py::test_hisat_fq[vignette/tmp-WT3AT-nonrRNA.fq] FAILED
riboviz/test/regression/test_regression.py::test_hisat_fq[vignette/tmp-WT3AT-unaligned.fq] FAILED
riboviz/test/regression/test_regression.py::test_hisat2_sam[vignette/tmp-WTnone-orf_map.sam] FAILED
riboviz/test/regression/test_regression.py::test_hisat2_sam[vignette/tmp-WTnone-rRNA_map.sam] FAILED
riboviz/test/regression/test_regression.py::test_hisat2_sam[vignette/tmp-WT3AT-orf_map.sam] FAILED
riboviz/test/regression/test_regression.py::test_hisat2_sam[vignette/tmp-WT3AT-rRNA_map.sam] FAILED
...

To summarise - 4 issues:

Alex's issue --read-lengths arg must be at least 20 and the failure of hisat2.
Why Alex gets this issue and we don't
SyntaxError: invalid syntax error in the log.
Different outputs from hisat2 2.1.0 vs. 2.2.0.

For 1. Alex's issue --read-lengths arg must be at least 20 and the failure of hisat2:
--read-lengths arg must be at least 20

a user asked about this on https://www.biostars.org/p/432115/ and then followed up with:

I does seem some sequences in my fastq(s) were not minimum length. I will have to filter for length before running. Problem solved. Thanks for your reply.

Another user commented that:

I also got this error with hisat2 v2.2.0 but v2.1.0 works fine. Must be a recent change...?

For 3. SyntaxError: invalid syntax error in the log a user asked about this on https://www.biostars.org/p/435194/. The response was:

Yeah, this seems to be a problem related to python3 whereas the script expects python2.

They specified Python 2 and then got:
Use of uninitialized value $read_len_str in string ne at ./hisat2 line 293.

To me this is not a good solution. Python 2 is end-of-life!

Why Alex gets this issue and we don't

This is unclear.

Different outputs from hisat2 2.1.0 vs. 2.2.0.

This is unclear. https://github.com/DaehwanKimLab/hisat2/releases/tag/v2.2.0 documents the key changes to functionality.
A quick inspection of the TPMS_collated.tsv file shows minor changes in certain values:
If HISAT 2.1.0 is used:
ORF	WTnone	WT3AT
YAL001C	116.7	953
YAL002W	212.8	365.8
YAL003W	199013.8	100458.6
YAL005C	249986.3	239069
YAL007C	6223	3821.5
...

If HISAT 2.1.2 is used:
ORF	WTnone	WT3AT
YAL001C	116.7	952.6
YAL002W	212.8	365.7
YAL003W	199005.8	100409.3
YAL005C	249976.2	238951.6
YAL007C	6222.7	3819.6
...

A decision needs to be made whether the differences in these, and other outputs, are acceptable.

Personally issue 1. and 3. would make me wary of using HISAT 2.2.0 until it is more stable and that we should recommend users continue with HISAT 2.1.0 for now.

Thank you, @mikej888. Perhaps it is worth noting that I run into the same issue with HISAT 2.2.0 when running it independently of RiboViz. I can install HISAT 2.1.0 to see if that fixes things on my end.

I ran riboviz on our lab's server with hisat2 v2.2.0 and did not encounter the error I originally posted. Unlike last time, I also ran riboviz within a conda environment. I reran riboviz outside of a conda environment on the computer I was using when I first generated the error and encountered the same issue with hisat2 v2.2.0.

I propose that we close this issue as it isn't causing any problems.

I propose that we close this issue as it isn't causing any problems.

I agree.


fix docstring

Thanks :) Caught while running tests against the 1M scheme

Thanks :) Caught while running tests against the 1M scheme

Thanks for catching that!


Change MQTT protocol value to 3

Protocol value 3 indicates MQTT version 3.1
Fixes #23

Where did you get that protocol value 3 means mqtt 3.1? The only paho mqtt documentation I can find says that the protocol should be 'MQTTv3.1' or 'MQTTv3.11'. Im glad that this works for you but I'm a little hesitant to merge it until i can better understand why. I do appreciate the PR though

The source code for the client shows that MQTTv31 is 3 and MQTTv311 is 4.
The examples show using just MQTTv31 without quotes, because it is an environment variable in client.py.


minor fixes

There was an error while validating job eafe4545-a455-4113-bea5-c51cba8d494e Message:
Result status: Failed
There was a problem accessing Github repository 'https://github.com/Microsoft/cordova-docs.git'.
Details:Microsoft.LightweightJobEngine.Core.Language.ContextCommandBase+ContextCommandException:
Result status: Failed
There was a problem accessing Github repository 'https://github.com/Microsoft/cordova-docs.git'.
---> System.AggregateException: One or more errors occurred. ---> System.ArgumentException: Error clonning repository ---> LibGit2Sharp.NotFoundException: Branch 366 not found. Available branches: origin/master
at Microsoft.DPS.Services.LibGit2SharpSourceControlProvider.GetBranch(Repository repo, String name)
at Microsoft.DPS.Services.LibGit2SharpSourceControlProvider.<>c__DisplayClass30.<>c__DisplayClass35.b__2a(KeyValuePair2 branch)    at System.Collections.Generic.List1.ForEach(Action1 action)    at Microsoft.DPS.Services.LibGit2SharpSourceControlProvider.<>c__DisplayClass30.<CloneRepository>b__28()    at Microsoft.Practices.EnterpriseLibrary.TransientFaultHandling.RetryPolicy.<>c__DisplayClass1.<ExecuteAction>b__0()    at Microsoft.Practices.EnterpriseLibrary.TransientFaultHandling.RetryPolicy.ExecuteAction[TResult](Func1 func)
at Microsoft.Practices.EnterpriseLibrary.TransientFaultHandling.RetryPolicy.ExecuteAction(Action action)
at Microsoft.DPS.Services.LibGit2SharpSourceControlProvider.CloneRepository(String url, IDictionary2 branches)    --- End of inner exception stack trace ---    at Microsoft.DPS.Services.LibGit2SharpSourceControlProvider.CloneRepository(String url, IDictionary2 branches)
at Microsoft.DPS.Services.LibGit2SharpSourceControlProvider.CloneRepository()
at Microsoft.DPS.Services.LibGit2SharpSourceControlProvider.GetVersionAsync(String path)
at Microsoft.DPS.Commands.IdentifyDocumentsToPublish.CheckExternalServicesCommand.Do(IDictionary2 context)    --- End of inner exception stack trace ---    --- End of inner exception stack trace ---    at Microsoft.LightweightJobEngine.Core.Language.ContextCommandBase.ThrowIfErrorsLogged()    at Microsoft.DPS.Commands.IdentifyDocumentsToPublish.CheckExternalServicesCommand.Do(IDictionary2 context)
at Microsoft.LightweightJobEngine.Core.Language.JobWorkflow1.RunCommands(Guid jobId, IDictionary2 context, CancellationTokenSource cancelSignal)


scala tests significantly slower than before

99bb8f5f:
$ time ./scala_plugin_cross_deps.t
real	0m9.973s
user	0m17.177s
sys	0m1.757s

5a466386 (master at the time of writing):
$ time ./scala_plugin_cross_deps.t
real	1m24.381s
user	1m56.779s
sys	0m7.027s

I'll bisect later today to find the specific commit causing the issue.

Yep, 5a4663866941436fbf6eaba90e2723f4b62b6fa5 is the commit that slows it down (which makes sense, since it touches the relevant codepath).
I don't know much about that specific commit so I'll ping @jackkoenig

Specific lines of code causing the problem:
https://github.com/sifive/wit/blob/5a4663866941436fbf6eaba90e2723f4b62b6fa5/lib/wit/scalaplugin.py#L216-L218


tixati: 2.87 -> 2.88

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://repology.org/metapackage/tixati/versions.
meta.description for tixati is: "Torrent client"
meta.homepage for tixati is: "http://www.tixati.com"
meta.changelog for tixati is: ""
Updates performed

Version update

To inspect upstream changes
Impact

Checks done (click to expand)

built on NixOS
The tests defined in passthru.tests, if any, passed
Warning: no invocation of /nix/store/g2fp8i6ay3gfabxrw88x4fj8gj7lxd4s-tixati-2.88/bin/tixati had a zero exit code or showed the expected version
0 of 1 passed binary check by having a zero exit code.
0 of 1 passed binary check by having the new version present in output.
found 2.88 with grep in /nix/store/g2fp8i6ay3gfabxrw88x4fj8gj7lxd4s-tixati-2.88
found 2.88 in filename of file in /nix/store/g2fp8i6ay3gfabxrw88x4fj8gj7lxd4s-tixati-2.88

Rebuild report (if merged into master) (click to expand)

1 total rebuild path(s)

1 package rebuild(s)

1 x86_64-linux rebuild(s)
0 x86_64-darwin rebuild(s)

First fifty rebuilds by attrpath
tixati

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/g2fp8i6ay3gfabxrw88x4fj8gj7lxd4s-tixati-2.88 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A tixati https://github.com/r-ryantm/nixpkgs/archive/5a30b5898f9cba290e15e1b240f36b042db9cb23.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/g2fp8i6ay3gfabxrw88x4fj8gj7lxd4s-tixati-2.88
ls -la /nix/store/g2fp8i6ay3gfabxrw88x4fj8gj7lxd4s-tixati-2.88/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

tixati

Maintainer pings
cc @volth for testing.

Launches and seems to work.


Allow puppetlabs-concat 2.x

@bmjen does this sound good to you ?

Thanks @brandonweeks. The .fixtures.yml file also needs to be updated.

@bmjen do you want it pinned to 2.1.0? I left it alone since it wasn't pinned to any version at the moment.

@brandonweeks I think unpinning it entirely and having it use master should be fine.

@bmjen concat was already unpinned here.

Cool. Thanks @brandonweeks !


DISABLE_CHUNKING property is not passing content-length header to the back-end  intermittently

Description:
Disable chunking property is not passing content-length header when we send the requests with large payloads (conent length more than 40000).Also this behavior is intermittent.
This can happen due to content length header being set before the PassThroughHttpSSLSender calculates the message size. If it takes longer time to content-length calculation and if the the TargetRequest start is executed before that, the "Transfer-Encoding" header will be sent.
Steps to reproduce:
Create an API with a POST resource.
Add a mediation extension (in sequence) with following property

Invoke the API in high frequence with the large payload.

I’m closing this issue because it has been inactive for more than two years. Please reopen if you still encounter this issue with the latest version.


NTLM auth support

<EMAIL_ADDRESS>commented:

Which version of PhantomJS are you using? Tip: run 'phantomjs --version'.
1.8
Please provide any additional information below.
There is currently no support for NTLM authentication.
(NTLM is common in the Enterprise field and this is a must in my project)
also see https://groups.google.com/forum/#!topic/phantomjs/EoZ10-9puMk
thanks.

Disclaimer:
This issue was migrated on 2013-03-15 from the project's former issue tracker on Google Code, Issue #1037.
:star2:   3 people had starred this issue at the time of migration.

just tried https://github.com/sdesalas/trifleJS and windows auth works since it's using IE.

I'm using Phantomjs v2.0. Is there any example of passing username, password and domain for the NTLM authentication. So far, I'm not able to get a screen capture working for NTLM using phantomjs.

Hi @rajeshpaddy , just use: "http://username:password@localhost/myapp" as the URL for your request. Worked for me.

Hi, I have the same problem that @pkbharigopal :
In my enterprise I am behind a firewall with NTLM/Kerberos authentication configured. None of the options --proxy-auth, --proxy worked for me. I am using PhantomJS 2.0.

Checkout triflejs. It solved my authentication problem.

Sorry @rajeshpaddy,
I do not understand if I have to use the "triflejs" project or watch the piece of code to solve the connection problem

Hi Everyone,
Nobody knows answer to my problem (or question ) ?

You can manipulate request header to send Kerberos token to hit Kerberized web pages from PhantomJS. Using below -
page.customHeaders = {"Authorization": "Negotiate " + [token]};
Also I had to change cli args to --ignore-ssl-error = true to get it working.
Use https://docs.oracle.com/javase/7/docs/api/org/ietf/jgss/GSSManager.html and https://docs.oracle.com/javase/7/docs/api/org/ietf/jgss/GSSContext.html to generate Kerberos token.

I've gotten phantomjs to work with ntlm authentication using the cntlm proxy. The method that I used is detailed below if that would be helpful to anyone else encountering this issue. http://stackoverflow.com/questions/31209449/cntlm-proxy-with-phantomjs

@itslittlejohn :+1: Thanks for sharing!

This is more than possible to implement kerberos authentication in phantomJS using https://github.com/christkv/kerberos.
And I haven't tried it, but @vaishali33's solution looks like the only straightforward one.


Wink Aros Fixes

Description:
Wink Aros only supports 3 Fan Modes:
Low
Medium
High
Fan Mode had a Typo and wasn't represented in the UI
Related issue (if applicable): fixes #
Pull request in home-assistant.github.io with documentation (if applicable): home-assistant/home-assistant.github.io#<home-assistant.github.io PR number goes here>
Example entry for configuration.yaml (if applicable):

Checklist:
If user exposed functionality or configuration variables are added/changed:

[ ] Documentation added/updated in home-assistant.github.io

If the code communicates with devices, web services, or third-party tools:

[ ] Local tests with tox run successfully. Your PR cannot be merged unless tests pass
[ ] New dependencies have been added to the REQUIREMENTS variable (example).
[ ] New dependencies are only imported inside functions that use them (example).
[ ] New dependencies have been added to requirements_all.txt by running script/gen_requirements_all.py.
[ ] New files were added to .coveragerc.

If the code does not interact with devices:

[ ] Local tests with tox run successfully. Your PR cannot be merged unless tests pass
[ ] Tests have been added to verify that the new code works.

Hi @geekofweek,
It seems you haven't yet signed a CLA. Please do so here.
Once you do that we will be able to review and accept this pull request.
Thanks!

Looks good to me, didn't know anyone that had one of these so it wasn't ever really tested. Glad to see someone is getting to use it!

I'm probably one of the few people that have one.  It's not my primary unit so it wasn't much of a big deal but I just started writing some automations around it and testing it before I'll start using it over the summer.


Device login support for Linux.

Seems you need a gofmt, otherwise LGTM

Why is it not supported for Windows target?
Why is it hardcoded?

This project is deprecated, please use the Packer built-in Azure builder.


Program Stuck with a pdf

While extracting text from this pdf:
http://ihassociation.org/wordpress/wp-content/uploads/2015/06/2014-ASHP-Handbook-web-edition.pdf
The program just gets stuck.
Reproduce:
import pdftotext

with open('./2014-ASHP-Handbook-web-edition.pdf', 'rb') as f:
    pdf = pdftotext.PDF(f)
text = "\n\n".join(pdf)

I tried to iterate through the pages and there is a specific page in which the program is stuck on.
The program is stuck and uses 100% cpu which means it keeps processing something.
If its a dependecies problem, adding timeout for processing would be good.

You can use the multiprocessing module to perform long-running tasks with a timeout. Here is a commented example script I called readpdf.py:
import multiprocessing
import sys
import time

import pdftotext

def main():
    path = sys.argv[1]
    text = read_pdf(path)
    if text is None:
        sys.exit("timed out trying to read the PDF")
    print(text)

def read_pdf(path, timeout=5):
    # A queue to pass data between processes
    q = multiprocessing.Queue()

    # Start a new process
    p = multiprocessing.Process(target=read_pdf_process, args=(path, q,))
    p.start()

    # Wait some time for it to finish
    p.join(timeout=timeout)

    if p.is_alive():
        # The process is still running but we are out of time, so terminate it
        p.terminate()

    # Based on the exit code, either get the text that was extracted, or None
    # if we weren't able to finish
    p.join()
    if p.exitcode == 0:
        text = q.get()
    else:
        text = None

    q.close()
    return text

def read_pdf_process(path, q):
    with open(path, "rb") as f:
        pdf = pdftotext.PDF(f)
    text = "\n\n".join(pdf)
    q.put(text)
    q.close()

if __name__ == "__main__":
    main()

On a typical PDF, you get the text:
$ python readpdf.py abcde.pdf
abcde.

On your troublesome PDF it will time out after five seconds:
$ python readpdf.py 2014-ASHP-Handbook-web-edition.pdf
timed out trying to read the PDF

I hope it helps.

Closing, since this is something you should solve in your own code. I shouldn't change this module itself to do this on my side, since spinning up another process would be expensive and surprising to most users.


Prevent users starting assignments that are already past due

Right now they can click "begin assignment" even if the assignment is already closed

Closed by b3d56df9e9f902604ede691fbaaebb3773642a63


Incorrect "ITM" color if market price fully breaches a vertical spread

The corrected code for the PerformITMcalculation function located in ActiveTrades.cpp should be as follows. This code will eventually find its way into a future update/release.
void PerformITMcalculation(AppState& state, std::shared_ptr<Trade>& trade) {
    // Puts have aready been sorted before calls by strike price
    const ImU32 RED = clrRed(state);
    const ImU32 GREEN = clrGreen(state);
    int ITMCOLOR = 0;

    double strike_price = 0;

    // Iterate forward over the open legs to process the Puts but iterate in
    // reverse to process the Calls in order to properly ensure that the ITM
    // color is not reset if strike price is fully through a vertical spread.
    for (const auto& leg : trade->open_legs) {
        if (leg->underlying != Underlying::Options) continue;
        if (leg->put_call != PutCall::Put) continue;
        strike_price = AfxValDouble(leg->strike_price);
        if (trade->ticker_last_price < strike_price) {
            if (leg->open_quantity < 0) ITMCOLOR = RED;    // credit
            if (leg->open_quantity > 0) ITMCOLOR = GREEN;  // debit
        }
    }

    // Process the Calls in reverse
    for (int i = trade->open_legs.size() - 1; i >= 0; --i) {
        auto leg = trade->open_legs[i];
        if (leg->underlying != Underlying::Options) continue;
        if (leg->put_call != PutCall::Call) continue;
        strike_price = AfxValDouble(leg->strike_price);
        if (trade->ticker_last_price > strike_price) {
            if (leg->open_quantity < 0) ITMCOLOR = RED;    // credit
            if (leg->open_quantity > 0) ITMCOLOR = GREEN;  // debit
        }
    }

    trade->itm_text = (ITMCOLOR ? "ITM" : "");
    trade->itm_color = (ITMCOLOR ? ITMCOLOR : clrTextLightWhite(state));
}

Issue resolved and implemented in all cross platform versions of TradeTracker (versions 5.0+)


SPIKE Inline validation

Issue https://github.com/silverstripe/silverstripe-elemental/issues/1068
DO NOT MERGE
This PR is only for investigative purposes

There's so much going on here it's hard to see what's doing what. Can you please add a few notes in the PR description explaining what all this PR is doing?

I don't think this PR is worth peer reviewing, it's not in a good state and it's not complete. It's not meant to be merged, it's kind of just a dumping ground for code I used to do investigation

Is there anything in this PR that's useful for understanding and/or evaluating the high level approach you're recommending in the issue? Currently the issue is more of an info dump than a "this is the recommended approach" so it's a bit hard to follow from that angle.

Closing this for now - we may end up copying some bits from this when we implement this validation in https://github.com/silverstripe/silverstripe-elemental/issues/329 but it likely won't use this exact PR.


Eca/mission critical

@bennage - first draft

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/cloud-adoption/operations/fundamentals.md
:white_check_mark:Succeeded
View

docs/cloud-adoption/operations/overview.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Great Article, and timely too as I need to have this discussion with a customer next week.

:warning: Validation status: warnings

File
Status
Preview URL
Details

docs/cloud-adoption/operations/operational-fitness-review.md
:warning:Warning
View
Details

docs/cloud-adoption/operations/_images/ofr-flow.png
:white_check_mark:Succeeded
View

docs/cloud-adoption/operations/overview.md
:white_check_mark:Succeeded
View

docs/cloud-adoption/operations/operational-fitness-review.md

Line 9: [Warning] Invalid file link:(~/cloud-adoption/guide/pillars).
Line 21: [Warning] Invalid file link:(~/cloud-adoption/cloud-adoption/overview.md).
Line 25: [Warning] Invalid file link:(~/cloud-adoption/checklist/scalability.md).
Line 25: [Warning] Invalid file link:(~/cloud-adoption/checklist/availability.md).
Line 25: [Warning] Invalid file link:(~/cloud-adoption/resiliency/index.md).
Line 29: [Warning] Invalid file link:(~/cloud-adoption/resiliency/#define-your-availability-requirements).
Line 82: [Warning] Invalid file link:(~/cloud-adoption/example-scenario/apps/ecommerce-scenario.md).
Line 120: [Warning] Invalid file link:(~/cloud-adoption/guide/pillars.md).
Line 122: [Warning] Invalid file link:(~/cloud-adoption/guide/design-principles/index.md).
Line 124: [Warning] Invalid file link:(~/cloud-adoption/resiliency/index.md).
Line 126: [Warning] Invalid file link:(~/cloud-adoption/patterns/index.md).

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

:warning: Validation status: warnings

File
Status
Preview URL
Details

docs/cloud-adoption/operations/operational-fitness-review.md
:warning:Warning
View
Details

docs/cloud-adoption/operations/_images/ofr-flow.png
:white_check_mark:Succeeded
View

docs/cloud-adoption/operations/overview.md
:white_check_mark:Succeeded
View

docs/cloud-adoption/operations/operational-fitness-review.md

Line 9: [Warning] Invalid file link:(~/guide/pillars).
Line 29: [Warning] Invalid file link:(~/resiliency/#define-your-availability-requirements).

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/cloud-adoption/operations/_images/ofr-flow.png
:white_check_mark:Succeeded
View

docs/cloud-adoption/operations/operational-fitness-review.md
:white_check_mark:Succeeded
View

docs/cloud-adoption/operations/overview.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/_bread/toc.yml
:white_check_mark:Succeeded

docs/cloud-adoption/operations/_images/ofr-flow.png
:white_check_mark:Succeeded
View

docs/cloud-adoption/operations/index.md
:white_check_mark:Succeeded
View

docs/cloud-adoption/operations/operational-fitness-review.md
:white_check_mark:Succeeded
View

docs/cloud-adoption/operations/overview.md
:white_check_mark:Succeeded
View

docs/toc.yml
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/_bread/toc.yml
:white_check_mark:Succeeded

docs/cloud-adoption/operations/_images/ofr-flow.png
:white_check_mark:Succeeded
View

docs/cloud-adoption/operations/index.md
:white_check_mark:Succeeded
View

docs/cloud-adoption/operations/operational-fitness-review.md
:white_check_mark:Succeeded
View

docs/cloud-adoption/operations/overview.md
:white_check_mark:Succeeded
View

docs/toc.yml
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/_bread/toc.yml
:white_check_mark:Succeeded

docs/cloud-adoption/operations/_images/ofr-flow.png
:white_check_mark:Succeeded
View

docs/cloud-adoption/operations/index.md
:white_check_mark:Succeeded
View

docs/cloud-adoption/operations/operational-fitness-review.md
:white_check_mark:Succeeded
View

docs/cloud-adoption/operations/overview.md
:white_check_mark:Succeeded
View

docs/toc.yml
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/_bread/toc.yml
:white_check_mark:Succeeded

docs/cloud-adoption/operations/_images/ofr-flow.png
:white_check_mark:Succeeded
View

docs/cloud-adoption/operations/index.md
:white_check_mark:Succeeded
View

docs/cloud-adoption/operations/operational-fitness-review.md
:white_check_mark:Succeeded
View

docs/cloud-adoption/operations/overview.md
:white_check_mark:Succeeded
View

docs/toc.yml
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/_bread/toc.yml
:white_check_mark:Succeeded

docs/cloud-adoption/operations/_images/ofr-flow.png
:white_check_mark:Succeeded
View

docs/cloud-adoption/operations/index.md
:white_check_mark:Succeeded
View

docs/cloud-adoption/operations/operational-fitness-review.md
:white_check_mark:Succeeded
View

docs/cloud-adoption/operations/overview.md
:white_check_mark:Succeeded
View

docs/toc.yml
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Imba

I am going to start building the Realworld application using Imba.
Imba treats DOM elements as a first-class citizens. These elements are compiled to an inline dom, which is an order of magnitude faster than todays virtual dom implementations.
The project repository : imba-realworld-example-app
Work in progress

fork the realworld repository
create README
develop the application

Feel free to help me 😃

@anishkny Hi Anish, I just finish implementation.
I don't know how can I host it to have an online demo

Looks great. Couple of notes:

I noticed that the profile pic in top navbar is too big when logged in. Could you fix that?
Also, you can probably exclude dist folder from the repo as a best practice (just add it to .gitignore file).

For a demo, you have a few options:

Netlify
GitHub Pages

We can discuss over at your repo...
Great work! 🎉🎉🎉

Thanks for your feedback, issue is fixed.
I think I can't ignore dist because dist/index.html isn't generated from another folder.
I added demo link on readme


feat: Use typescript-eslint@v6's reworked configs

Is your feature request related to a problem? Please describe.
Following up on #1035 -> #1036: typescript-eslint is soon going to release a new v6 major version with reworked configurations. You can read up on it here: https://typescript-eslint.io/blog/announcing-typescript-eslint-v6-beta
Specifically, the configurations are reworked so that the recommended equivalents to what create-t3-app uses now are:

'plugin:@typescript-eslint/recommended-type-checked'
'plugin:@typescript-eslint/stylistic-type-checked'

Describe the solution you'd like to see
I'd like to send a PR to try the new configs out here. I like this repo and think you'll all do a good job of letting us know if you hate the changes! ❤
https://github.com/typescript-eslint/typescript-eslint/issues/6760
Describe alternate solutions
I suppose create-t3-app could opt to stay on typescript-eslint@v5, or use your own bespoke lint configs... neither option seems very appealing to me. 😉
Additional information
Please let me know if I'm being annoying / overly intrusive with this. The v6 has been a long time coming, and I really want to get the configs right.

Not annoying at all! People use CT3A and other starters as references to copy/paste from, so it's really important that our configs are as good as possible. Happy to get this in as soon as it's ready :)


[fix] Revised CODEOWNERS file

Fixes Issue
Closes #864
Changes proposed

I added @ujjawaltyagii to the CODEOWNERS file
Screenshots

Note to reviewers

@ujjawaltyagii and @rupali-codes, a couple of errors came up, so helping me solve them would be greatly appreciated.

Could you please provide more details about the errors that came up ?

Could you please provide more details about the errors that came up ?

Hi @ujjawaltyagii. I figured out the error. The file is valid now! :)

Could you please provide more details about the errors that came up ?

Hi @ujjawaltyagii. I figured out the error. The file is valid now! :)

is it ready to merge :)

@ujjawaltyagii @CBID2 👀

@CBID2 just to confirm are you done with the changes ? so that we can proceed to merge this PR


Knative tests consistently failing locally

Is there an existing issue for this?

[X] I have searched the existing issues

Current Behavior
Knative tests don't work if run locally: https://github.com/Kong/kubernetes-ingress-controller/pull/2124#issuecomment-1004710075
Expected Behavior
Knative tests should be green whether run locally or by CI
Steps To Reproduce
https://github.com/Kong/kubernetes-ingress-controller/pull/2124#issuecomment-1004710075

Kong Ingress Controller version
HEAD / pre-2.1

Kubernetes version
all

Anything else?
No response

Doesn't seem to happen anymore.


rstudio-server: Initial commit for rstudio-server and associated wrapper

Motivation for this change
Documented in more detail at https://discourse.nixos.org/t/adding-rstudio-server-r/8644/4, but in short, I wanted to be able to run rstudio-server inside a nix-shell so that I could get plots faster than using X-forwarding.
Things done
Modified the rstudio build code to build the server version, with some more tweaks.
Coarse diff between rstudio  and rstudio-server:

Nix file is under the same directory as rstudio. Called server.nix
Added to top-level as rstudio-server
Calls wrap program on each binary
Adds readline6 and pam dependencies for building
Removes reference to icons (build failed if left in)

Coarse diff between rstudioWrapper and rstudioServerWrapper

Uses the same libPath wrapping on the rsession binary
Also adds a fontconfig dependency and wrapping to fix the plot fonts issue noted here
Wraps rserver to point to the wrapped rsession

Potential To-Do's

I think the rstudio-server build could be refactored into the same nix file as the rstudio build, adding an argument to toggle between the two.
I think this could be converted into a NixOS module and made to work with pam. For now it's simple enough to write a a small script to simple authentication (this can be passed to rserver as the --auth-pam-helper argument if memory serves.)
I think that rstudioWrapper and rWrapper should be given the same fontconfig fix as rstudioServerWrapper but perhaps that's best for another PR.

CC @ehmry @changlinli @ciil

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[ ] NixOS
[ ] macOS
[ x ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ x ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[ x ] Fits CONTRIBUTING.md.

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/adding-rstudio-server-r/8644/6

Thank you for adding the server! Are you planning to add a corresponding NixOS module for the service?

Hi @alexvorobiev, I've actually never used NixOS so I wouldn't know where to start with adding a module. My colleague @bcdarwin  has offered to help creating a module if this PR goes through.

Any update on this PR?

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/adding-rstudio-server-r/8644/7

* I think the `rstudio-server` build could be refactored into the same nix file as the `rstudio` build, adding an argument to toggle between the two.

If rstudio-server used mkDerivation, I would say use passthru.unwrapped to allow a reference to the underlying package, but I'm not sure if runCommand filters those attrs

This is still important

What is pending for this update to be released?

@cfhammill, are you available to make the review edits? I am in need of this as well.

sorry for the delay, I'm back looking at this now and will hopefully have a chance to make the requested revisions soon

It's worth revisiting the rstudio derivation as it has changed considerably
with the latest version bump: $NIX_QT5_TMP is no-longer used and qmake
is referred to as an explicit input.

thanks @jbedo I was wondering if redoing this PR off the updated rstudio/default.nix, this was the push I needed.

I have the new version of rstudio-server building based on the updated rstudio derivation. A couple of pain points
The rstudio-server service doesn't work, you have to use the rserver binary directly. This is probably the best we can do for non-nixOS systems.
rserver as far as I can tell requires a few system directories that need to be writeable. I never noticed this before because my systems had those directories around from non-nix installs of rstudio server. the directories in question are

/var/run/rstudio-server (this needs to have global write with the sticky bit set according to rserver, but ostensibly this location can be changed with the --server-data-dir flag)
/var/lib/rstudio-server (needs some write permissions, not sure if it needs to be global, but as far as I can tell this cannot be moved to another directory)

so a user wanting to run rstudio-server would run
sudo mkdir /var/run/rstudio-server && sudo chmod a+wt /var/run/rstudio-server
sudo mkdir /var/lib/rstudio-server && sudo chmod a+wt /var/lib/rstudio-server
once, and then to launch the server the user would run
rserver --auth-none 0
possibly in the background.
If anyone has any clever ideas to avoid the global directories I'd welcome a contribution. We could probably patch the CMake files or the c++ source code to override these default locations and point them elsewhere, not sure where, I think write perms within the nix store is a no-go, if we can't think of a sensible default, an upstream PR to add an argument to move the /var/lib/rstudio-server location might be possible.

Systemd will create them for you, use RunTimeDirectory= and StateDirectory=
options, along with DynamicUser. You can ripgrep nixos/modules/services
for some examples.

@bcdarwin has offered to help turn this into a module, do you mean this would work for a non-nixOS system? If so can you give me more detail on what to do?

@jbedo this should be ready for a final review and (hopefully) merge

Builds and runs a working server instance after manual directory creation as described above.

I'll go through it today/tomorrow, but I wouldn't think QT is needed for a
service. A NixOS module would be helpful and would resolve the directory
creation issue (on NixOS machines). It shouldn't be complicated.

@jbedo, @bcdarwin removed the qt dependency and emailed me a patch, I've applied it. Do you require the module before merging this or can that be for the next PR

I can likely make the module this weekend if still outstanding.

I think we can easily merge the server and desktop versions of the
expression, there's a small number of differences. I'm happy to help
with this.
Re module: I think we should make one, can be a shallow one, it would
make testing easier. Also happy to help with this.

thanks @jbedo, I think consolidating this and the local version is a good idea. I'll give you and @bcdarwin collaborator access to the branch so you can push any changes you would like

I think with a wrapper and after some testing this should be good. The history is quite messy so I guess we should clean it up or do a squash merge.

@bcdarwin do you have time to do the module this week? Otherwise I think we're almost there!

Added a basic nixos module. Basic functionality is there, but for some reason the file browser window is a bit messed up.

@cfhammill I added you as a maintainer in the nixos module, is that ok?

Thanks @jbedo! I know @bcdarwin was working on this as well, curious to see if he has any suggestions. I see the module is calling rserver directly, I think that is probably preferable to using the included rstudio-server shell script. What's wrong with the file browser window?
And yes happy to be a maintainer for this, not sure how useful I will be, but I'll try to help address any issues.

Oh also, I didn't see any reference to /var/run/rstudio-server, was this no longer needed?

rstudio-server just calls their systemd service, so more of less the
same thing.  The file browser just shows a lot of entries for /, none
of which work properly.  There could be some config I've missed, I'm not
an rstudio expert. Feel free to replace with what @bcdarwin has done, I
just had some time this morning to push this along.

Oh also, I didn't see any reference to /var/run/rstudio-server, was this no longer needed?

That's handled by RuntimeDirectory="rstudio-server".

Thanks for clarifying, hopefully Ben will see this thread and weigh in. Weird about the files, hopefully between the three of us we can figure it out. I'm going to spin up a NixOS vm so I can test at some point.

The module starts up and seems to work but systemd times out and restarts it every 90s; my guess is we need to add the PIDFile= to serviceConfig so systemd knows what process to look for.

Also, at the risk of bikeshedding, for consistency with other distributions we should probably rename the service from rstudio to rstudio-server.

edit: this pid file isn't being created (or is very short-lived), so something else is wrong.

Process wasn't forking, fixed the systemd service.

Also, at the risk of bikeshedding, for consistency with other distributions we should probably rename the service from rstudio to rstudio-server.

Done.
I've added a crude test, it would be good to improve it if somebody knows how to do a more meaningful check from curl.

For some reason, the running R process is owned by user rstudio-server instead of the intended user.

We also might want to default www-address=<IP_ADDRESS> in /etc/rstudio/rserver.conf, disallowing remote login by default.  (I prefer to create an SSH tunnel than to run rstudio-server on the network.)

For some reason, the running R process is owned by user rstudio-server instead of the intended user.

Who's the "intended user?" Generally with services they're run under a unique UID and GID to isolate them from other accounts on the system, and DynamicUser does this for us. We could parameterise a run user and group instead which would allow running as any specified user.

We also might want to default www-address=<IP_ADDRESS> in /etc/rstudio/rserver.conf, disallowing remote login by default. (I prefer to create an SSH tunnel rather than to run rstudio-server on the network.)

This would really be my preference too, but it would differ from the default upstream. I'm happy to do this for extra security if the change in behaviour isn't going to unduly confuse rstudio users.

We also might want to default www-address=<IP_ADDRESS> in /etc/rstudio/rserver.conf, disallowing remote login by default. (I prefer to create an SSH tunnel rather than to run rstudio-server on the network.)

This would really be my preference too, but it would differ from the default upstream. I'm happy to do this for extra security if the change in behaviour isn't going to unduly confuse rstudio users.

In my opinion, if the rstudio user is enabling and configuring the NixOS module themselves, they ought to be able to find the relevant setting in the NixOS options, while if a sysadmin or analyst is configuring it for end users, then that administrator should communicate the situation to them.

For some reason, the running R process is owned by user rstudio-server instead of the intended user.

Who's the "intended user?" Generally with services they're run under a unique UID and GID to isolate them from other accounts on the system, and DynamicUser does this for us. We could parameterise a run user and group instead which would allow running as any specified user.

On Debian, if I navigate to localhost:8787, there's a login prompt allowing one to enter shell credentials and then the spawned rsession is run as that user rather than as rstudio-server (allowing home directories, file permissions etc to work as expected, and preventing the user from manipulating rstudio-server state, which is a security issue).  I naively tried adding PAM to this module (with/without rootOK = true; which seems like it might be required here) but it didn't help.

That's a much better config, I'll have a look at the debian package and
see how it's done.

Converted to forking root process and configured pam. @cfhammill @bcdarwin if possible could you test? File browser seems to be fixed too.

@jbedo this is working great for me now.  I guess time to squash and merge?

All sounds good to me, I didn't test myself, booted in to a nixos vm, tried a few things couldn't get basic stuff going and gave up, but if you both are giving thumbs up I'm sure it's fine.
Shall I squash in prep for merge or @jbedo will you squash merge?

It would be better if you could squash this down to a couple of commits, one adding the rstudio-server expression and another adding the module + tests. There's now a conflict with the ids to resolve, just needs to be bumped.

Squashed and uid/gid updated.

Added the new module to the release notes as per updated contributing guidelines.
@jbedo should be ready to merge.

LGTM too, I'll see if I can find an rstudio-server user to test this more but pretty confident in the basic functionality as @bcdarwin has also verified it.

Resolved. Added a crappy test for it too.

Resolved. Added a crappy test for it too.

Works now, thanks.

Rebased and resolved merge conflicts again.

We can merge this once the checks have passed, hopefully before we have to deal with further conflicts. @jonringer I think further issues are likely to be more fundamental problems with the underlying R/RStudio expressions rather than the rstudio-server module, which we can deal with separately.

I'm fine with incremental improvement. And I'm not familiar with r to really guide to the desired end state.
Thanks for reviewing it


Async commit does not ensure linearizability

Bug Report
Read in the same session
Read with max u64 immediately after an async commit transaction. Because the lock in the previous transaction may be not committed yet, and locks can be ignored if we read with max u64. So we may not read the latest changes from the last async commit transaction.
Causal consistency across nodes
Example:
At first the values of k1 and k2 are all 0, they can locate in different regions.
We commit k1=1@100 at first. The max_read_ts of the k2 region leader is 50.
Then, we prewrite and commit k2=1@51 using async commit.
Now, a transaction with start_ts=80, reads k1 and k2 in the same transaction. We get k1=0, k2=1.
We break an external consistency that k2 commits later than k1.

For the first problem, we can change the check_ts_conflict function. If the lock is from an async commit transaction, we cannot ignore it even if the read ts is max u64.

@youjiali1995
A more concrete example:

A external client starts two sessions A and B.
Session A starts T1, start_ts = 1.
Session B starts T2, start_ts = 2.
Session A prewrites k1=1.
Session B prewrites k2=1 using async commit, min_commit_ts = 51.
Client starts session C, it starts a transaction T3, start_ts = 80.
Session A commits T1 with commit_ts = 100.
Session B commits T2 with commit_ts = 51.
Session C reads k1 and k2.

Can Jepsen test find this? Does anyone know if our Jepsen test is still running?

I see. It's another kind of causal reverse. We can't avoid it but I think it's not very severe because It rarely happens. We can implement causality tokens to solve it which is passed between transactions if needed.

It's quite similar to the causal reverse crdb mentioned in their blog. So I think it can be detected by Jepsen too. @jackysp

If a user depends on external consistency. We can disable async commit. Actually, we can't offer the true external consistency even with the timestamp oracle which requires serializability isolation.

This anomaly requires one client maintains two transactions at the same time, are there any existing application do it this way?

This anomaly requires one client maintains two transactions at the same time, are there any existing application do it this way?

Usually operations with causality are done in a single transaction. But maybe one client receives success and then inform another client to commit? I don't know...
But this is indeed a change in consistency guarantee which should be mentioned and make users aware of.

I can think of an example, an application A call service B inside a transaction, and service B do another transaction to get a value, then application A use that value to commit its transaction.
But this case application A and service B are separated components, the inconsistency should not matter much.

Maybe there are some applications that will use this form to simulate some savepoint or nested transaction usages, which may start multiple transactions and decide the next step based on these results.

We can tell the users that the async commit does not fully ensure external consistency.

Async commit ensures that any transaction can observe the effects of all transactions that committed before the start of it.
Async commit doesn't ensure that the concurrent transactions' serial order is consistent with the order in which transactions can be observed to commit.

I think the first one is the most needed while the second one is not.

@youjiali1995
A more concrete example:
1. An external client starts two sessions A and B.

2. Session A starts T1, start_ts = 1.

3. Session B starts T2, start_ts = 2.

4. Client starts session C, it starts a transaction T3, start_ts = 80.

5. Session A prewrites k1=1.

6. Session A commits T1 with commit_ts = 100.

7. Session B prewrites k2=1 using async commit, min_commit_ts = 51.

8. Session B commits T2 with commit_ts = 51.

9. Session C reads k1 and k2.

Session B triggers commit after it receives the success from session A (between 6 and 7)

BTW, maybe we can get a ts from PD before starting prewrite phase, then we can carry it with the prewrite request to primary key. The order of A and B can be ensured in this case because the new ts from PD must be greater than the A's commit ts.
I am not sure if it is enough for any situation.

@gengliqi Previously I talked with @5kbpers about this idea too.
I think maybe it's better to ensure external consistency by default because that's what we guarantee before.

@gengliqi Previously I talked with @5kbpers about this idea too.
I think maybe it's better to ensure external consistency by default because that's what we guarantee before.

I agree.

@gengliqi Previously I talked with @5kbpers about this idea too.
I think maybe it's better to ensure external consistency by default because that's what we guarantee before.

+1

@gengliqi Previously I talked with @5kbpers about this idea too.
I think maybe it's better to ensure external consistency by default because that's what we guarantee before.

So the latency increased by half of the async commit saved.

So the latency increased by half of the async commit saved.

Not that much. A commit operation requires raft replication. So if we also count the RPC from TiDB to TiKV, that's at least two RPCs in total. TSO has only one, and can be easily in batch.

So the latency increased by half of the async commit saved.

Not that much. A commit operation requires raft replication. So if we also count the RPC from TiDB to TiKV, that's at least two RPCs in total. TSO has only one, and can be easily in batch.

How should we control this behavior, do we need to add another system variable for this?

How should we control this behavior, do we need to add another system variable for this?

Yes

It's a bug of async commit, which is a feature still under development. It shouldn't affect any production user.
Lower its severity to major. But still mark it as a blocker for GA of async commit.

It's a bug of async commit, which is a feature still under development. The feature is not enabled by default and it shouldn't affect any production user.
Lower its severity to major. But still mark it as a blocker of GA of async commit.

There is a still a problem if our goal is to enable this new feature by default for GA.

I think it's not a GA blocker, the causality between two on-going transactions is not something a normal application would depend on.

@sticnarf is this addressed by https://github.com/pingcap/tidb/pull/20276 or is this a separate issue?

方便问下这个提交跟之前的方案有什么区别咩？看起来主要想法依然是在commit之前从TSO获得一个时间戳来作为commit_ts，跟支持异步之前主要的区别是什么呢？

方便问下这个提交跟之前的方案有什么区别咩？看起来主要想法依然是在commit之前从TSO获得一个时间戳来作为commit_ts，跟支持异步之前主要的区别是什么呢？

之前获取 commit_ts 是在第一阶段提交（prewrite）之后获取，现在这是在 prewrite 之前获取。
从总延迟的角度看，以前是 prewrite + TSO + commit primary lock，现在缩减为 TSO + prewrite，少了一次提交 primary lock 的延迟。

方便问下这个提交跟之前的方案有什么区别咩？看起来主要想法依然是在commit之前从TSO获得一个时间戳来作为commit_ts，跟支持异步之前主要的区别是什么呢？

之前获取 commit_ts 是在第一阶段提交（prewrite）之后获取，现在这是在 prewrite 之前获取。
从总延迟的角度看，以前是 prewrite + TSO + commit primary lock，现在缩减为 TSO + prewrite，少了一次提交 primary lock 的延迟。

commit primary lock是指把提交信息写入一个中心么？我记得之前看Percolator的时候，这个commit其实只是本地检查lock列，然后写时间戳而已，感觉从论文上没有额外的RPC了，而且现在的方案也应该有这个流程，这个减少commit primary lock主要是指？

方便问下这个提交跟之前的方案有什么区别咩？看起来主要想法依然是在commit之前从TSO获得一个时间戳来作为commit_ts，跟支持异步之前主要的区别是什么呢？

之前获取 commit_ts 是在第一阶段提交（prewrite）之后获取，现在这是在 prewrite 之前获取。
从总延迟的角度看，以前是 prewrite + TSO + commit primary lock，现在缩减为 TSO + prewrite，少了一次提交 primary lock 的延迟。

commit primary lock是指把提交信息写入一个中心么？我记得之前看Percolator的时候，这个commit其实只是本地检查lock列，然后写时间戳而已，感觉从论文上没有额外的RPC了，而且现在的方案也应该有这个流程，这个减少commit primary lock主要是指？

RPC 数量没有减少，但以前需要等到提交 primary lock 后才能向用户返回成功。现在在 prewrite 结束时就能返回成功了。所以从用户的角度，少了一次 RPC 的延时。
在 issue 中提问题可能会对其他关注 issue 的人产生信息干扰，建议来我们 Slack channel 来讨论：https://slack.tidb.io/invite?team=tikv-wg&channel=sig-transaction&ref=community-sig


Icon Request

com.supercell.brawlstars

Well I need this to come when you got clash of clans icon already ¯_(ツ)_/¯
org.freedownloadmanager.fdm

Its good to have a nice icon for my downloader too :3

Hello! Weʼre switching to a new system for icon requests in order to be able to better manage them. From now on, icon requests can be submitted here: https://forms.gle/Fx8vZAiWdW1Tyjo57. If your icon request hasnʼt yet been fulfiled, it has been copied to our new icon request database, meaning no action is required. Consequently, this issue will be closed.


Do not import Shared Module in AppModule

as per style guide, shared module is intended to be imported only in feature modules and not in app or core.

I was reading the style guide right now but I can't find the place where the import of shared module in app module is discouraged? I only found

Do declare components, directives, and pipes in a shared module when those items will be re-used and referenced by the components declared in other feature modules.
Consider using the name SharedModule when the contents of a shared module are referenced across the entire application.
Avoid providing services in shared modules. Services are usually singletons that are provided once for the entire application or in a particular feature module.

From this I understand that it's about components (pipes, directives, ...) which are used across the entire application. Which includes also app component which implements the main layout like toolbar, sidenav and footer so it needs access to these material components.
I fully agree with not including any services (providers) in shared module because they would end up with multiple instances instead of singleton.
So please, can you elaborate bit more on why not to include shared module in app module?

@cynx  In my opinion, this is an incorrect interpretation and I have imported SharedModule into whatever module needs its functionality, including the AppModule, which conforms to @tomastrajan 's approach. For example, FeatureModuleX contains some Material components, the Modules of which are imported in a MaterialComponentsModule (and exported) into SharedModule. SharedModule is imported into AppModule, because AppModule has components that need Material functionality. FeatureModuleX is lazy loaded and so must also import SharedModule.

I consider this question answered and closing the issue, in case you have more arguments please open a new issue, thank you for your contribution!


the deprovision flow in civo cloud doesn't accommodate volumes

Which version of kubefirst are you using?
2.3.5
Which cloud provider?
Civo
Which DNS?
Cloud ones (default)
Which installation type?
UI (Console app)
Which distributed Git provider?
GitHub
Did you use a fork of gitops-template?
No
Which Operating System?
macOS
What is the issue?
following the deprovision docs against a civo github stack, i wasn't able to delete my network because of dangling volumes attached to the network.

Plan: 0 to add, 0 to change, 3 to destroy.

  Terraform will destroy all your managed infrastructure, as shown above.
  There is no undo. Only 'yes' will be accepted to confirm.

  Enter a value: yes

civo_kubernetes_cluster.kubefirst: Destroying... [id=8eea4232-8f9d-421c-8a1b-2d53c329a7f3]
civo_kubernetes_cluster.kubefirst: Destruction complete after 1s
civo_firewall.kubefirst: Destroying... [id=e3ebe482-8bb1-4774-8487-0c25a1dc6856]
civo_firewall.kubefirst: Destruction complete after 4s
civo_network.kubefirst: Destroying... [id=d5390551-ffa4-483b-9adc-bf55b23bc602]
╷
│ Error: error waiting for network (d5390551-ffa4-483b-9adc-bf55b23bc602) to be deleted: DatabaseNetworkInUseByVolumes: Failed to delete the network because it's in use by volumes, Network "cust-management-96eda81a-982dec6a9c39" is used by "pvc-3f0a2af7-96af-48c3-98c0-211e52f286c1,pvc-71928a42-f789-46e4-b870-b2911a8434c5,pvc-9434821c-9b9e-4363-9396-4ebb884822ec,pvc-9f00136d-7081-4b11-b1df-05a690890741,pvc-a5fc80e5-b33c-43e5-807d-45203b7a5a44,pvc-bb32de35-a096-4d1b-8ad9-112d261724c5,pvc-c59c408b-f33b-419a-8408-80a366ea1cd0,pvc-f08e5930-e8ed-4f71-bc01-658b2a0cb437" volumes. Please delete the volumes first in order to delete the network.
│
│
╵
➜  civo git:(main) terraform destroy
civo_network.kubefirst: Refreshing state... [id=d5390551-ffa4-483b-9adc-bf55b23bc602]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  - destroy

Terraform will perform the following actions:

  # civo_network.kubefirst will be destroyed
  - resource "civo_network" "kubefirst" {
      - default = false -> null
      - id      = "d5390551-ffa4-483b-9adc-bf55b23bc602" -> null
      - label   = "management" -> null
      - name    = "cust-management-96eda81a-982dec6a9c39" -> null
      - region  = "FRA1" -> null
    }

Plan: 0 to add, 0 to change, 1 to destroy.

  Terraform will destroy all your managed infrastructure, as shown above.
  There is no undo. Only 'yes' will be accepted to confirm.

  Enter a value: yes

civo_network.kubefirst: Destroying... [id=d5390551-ffa4-483b-9adc-bf55b23bc602]
╷
│ Error: error waiting for network (d5390551-ffa4-483b-9adc-bf55b23bc602) to be deleted: DatabaseNetworkInUseByVolumes: Failed to delete the network because it's in use by volumes, Network "cust-management-96eda81a-982dec6a9c39" is used by "pvc-3f0a2af7-96af-48c3-98c0-211e52f286c1,pvc-71928a42-f789-46e4-b870-b2911a8434c5,pvc-9434821c-9b9e-4363-9396-4ebb884822ec,pvc-9f00136d-7081-4b11-b1df-05a690890741,pvc-a5fc80e5-b33c-43e5-807d-45203b7a5a44,pvc-bb32de35-a096-4d1b-8ad9-112d261724c5,pvc-c59c408b-f33b-419a-8408-80a366ea1cd0,pvc-f08e5930-e8ed-4f71-bc01-658b2a0cb437" volumes. Please delete the volumes first in order to delete the network.

Code of Conduct

[X] I agree to follow this project's Code of Conduct

using Civo CLI v1.0.69 if you encounter this error, you can delete detached volumes with the following:
civo region current fra1
civo volume ls --fields id -o custom | while read line ; do civo volume rm "$line" --yes ; done

this will delete all volumes in your configured region, which may extend beyond the cluster of concern. please use carefully.


The vorbis decoder hangs on Decode()

If it helps, here's the file I'm trying to play:
oggfile.zip
MP3 files work fine with the same code.
Original issue: https://github.com/faiface/beep/issues/52

I get the following error:
ogg/vorbis: ogg: missing capture pattern

So I think the file is malformed.


Add MUI Theme Variables

Add fonts, platette colors, and etc.
Source from figma design, https://www.figma.com/file/nAhWBN06v6hMCHDlIT9mzk/KARMADA?node-id=6%3A134.
Also tweak eslint and prettier setup a bit to make it allow used variable and "auto" end of line. That  makes testing much easier.
What type of PR is this?
/kind design

What this PR does / why we need it:
This PR adds theme variables in theme.js for globally styling our MUI components.
Which issue(s) this PR fixes:
None. This PR adds content to theme.js, and should not have any breaking change.
Special notes for your reviewer:
If your have conflict or your design is changed, we can work this out together. Changing global theme may change default style of some components, there is always some trade-off.
Does this PR introduce a user-facing change?:
"None"

Signed-off-by: @ada2468<EMAIL_ADDRESS>

cc @chinmaym07 can you help to take a look?


dependencies on conda-forge

There are still some light dependencies from conda-forge. Keep this as a record.

[x] humanize
[x] prettytable
[x] tifffile
[ ] attrs

let's still keep this as a record.

We currently don't use anything from conda-forge directly. Closing.


Update JSONClient dependency to 0.5.0

To take advantage of the OAuth1JSONClient superclass.

PR #6.

Merged.


Malicious Link Clicked

Fixes #
Forked malware rule and included the phish classification
Proposed Changes

where classification_s =~ "phish" or classification_s =~ "malware"

@secops-and-hops When you have some time would you please make the suggested changes as per my previous comment and submit a PR. Thanks again for all your work here.

Thanks Anthony for submitting this .
It feels like it might be a good idea to have a separate query for phishing instead of merging them in one. Wondering, if you could please create separate query for phish. Couple of other minor suggestions:
description : In the description section we should probably start with “This query identifies”. Please see query style guide here: https://github.com/Azure/Azure-Sentinel/wiki/Query-Style-Guide
severity: We normally have Low, Medium and High severity detections in Azure Sentinel . Might be helpful to avoid ‘Informational’
This is how we think about the severity level of the detections:
• Low – Could potentially cause noise or is a detection that would need additional detections to raise the overall severity.
• Medium – These are generally quiet but may require some additional investigation to verify the impact of the attack.
• High – These are high confidence, rare detections that are generally guaranteed to indicate compromise or a high level impact of an attack.
Thanks again.

@secops-and-hops - Please have a look at the feedback and make changes when you have a chance.  We are happy to get this included in the repo.  Thanks, Shain


SimpleRNN Error

The SimpleRNN layer seem to require a tensor3 input of shape (batch_size, time_steps, features). However, Keras as a whole seems to expect only matrix inputs/outputs.
model = Sequential()
#model.add(Dense(20, 5, init='uniform', activation='tanh'))
model.add(SimpleRNN(5, 20, activation='sigmoid'))

sgd = SGD(lr=0.1, decay=1e-6, momentum=0.9, nesterov=True)
model.compile(loss='mse', optimizer='sgd')

/usr/local/lib/python2.7/dist-packages/keras/layers/recurrent.pyc in output(self, train)
     47         X = self.get_input(train) # shape: (nb_samples, time (padded with zeros at the end), input_dim)
     48         # new shape: (time, nb_samples, input_dim) -> because theano.scan iterates over main dimension
---> 49         X = X.dimshuffle((1,0,2))
     50
     51         x = T.dot(X, self.W) + self.b

/usr/local/lib/python2.7/dist-packages/theano/tensor/var.pyc in dimshuffle(self, *pattern)
    332             pattern = pattern[0]
    333         op = theano.tensor.basic.DimShuffle(list(self.type.broadcastable),
--> 334                                             pattern)
    335         return op(self)
    336

/usr/local/lib/python2.7/dist-packages/theano/tensor/elemwise.pyc in __init__(self, input_broadcastable, new_order, inplace)
    139                     raise ValueError(("new_order[%d] is %d, but the input "
    140                         "only has %d axes.") %
--> 141                         (i, j, len(input_broadcastable)))
    142                 if j in new_order[(i + 1):]:
    143                     raise ValueError((

ValueError: new_order[2] is 2, but the input only has 2 axes.```

In a Keras model, the only requirements on the input are those of the initial layer.
What does your input look like? I suspect you might be working with a 2D input. The imdb_lstm.py script in the examples folder provides a demo of how to build RNNs. You can replace the LSTM layer in that example with a SimpleRNN layer and it will work.

This is extent of the code - no inputs defined. The compilation fails.
I haven't delved deep enough into the implementation yet, but I'm guessing Theano finding that the input will be a fmatrix, so the dimshuffle((1,0,2)) is invalid. I'm guessing this will require a reshape before-hand, depending on how the batches are being handled. Something along the line of
X = X.reshape((batch_size, X.shape[0]/batch_size, X.shape[1])).dimshuffle((1,0,2))

or
X = X.reshape((X.shape[0]/batch_size, batch_size, X.shape[1]))

I can't recall if Theano handles the second snippet intelligently, resulting in equivalent outputs from both of the above snippets.

You were right, there was in fact an issue were recurrent layers had their default input shape set to matrix whereas they were expecting 3D inputs. This was not immediately apparent because the tests and example script did not implement any model where the recurrent layer came first.
The issue is fixed. Thanks for pointing this out!

Awesome, thanks for the update.


Quest/ANC - Add Family Profile

Description of Feature

Add a configurable Family profile page on Quest to be able to view a Family profile
Add configuration in  the Quest application to be able to have the following 3 sections

Household tasks based on the Family CarePlan
Family members section with 'Add' button
Upcoming services section  - List of Tasks to be conducted  (Up next)

On click see all , view an expanded list view of all upcoming services

Newly Registered

Family Members added

CarePlan Added

To be discussed

How do we generate and assign household tasks
Will a newly created family have a default CarePlan tagged to a Family  or we have an empty profile till the $apply operation is run against the Patient with an applicable PlanDefinition

Can there be a single profile page, that is configured for patients or families?


Change License to a compatible one that would allow easier use in open source projects.

MODNET is an impressive work in solving image and video matting problems.
This work has a non-commercial license. Could you please provide a more convenient license like MIT or Apache license that allows for commercial use.  This would make it easier for this work to be used in open source projects.  Non-commercial License  work is not compatible for use in open source projects.  You can read more about the incompatibility of non-commercial work from here. I am unable to make use of this work in my open source project because of the non-commercial license.

Hi, thanks for your attention.
We will discuss this problem and give you a response under this issus ASAP.

Hi, thanks for your attention.
We will discuss this problem and give you a response under this issus ASAP.

@ZHKKKe Thank you very much for your response.

@ZHKKKe Thank you very much for your response.

@ayoolaolafenwa
We have discussed this issue.
Due to the relevant license of the company, we will change the license to an open source license after the paper is accepted by a conference or journal.
I will @ you  under this question after the paper is accepted.

@ayoolaolafenwa
We have discussed this issue.
Due to the relevant license of the company, we will change the license to an open source license after the paper is accepted by a conference or journal.
I will @ you  under this question after the paper is accepted.

@ZHKKKe please tell me also when this will become open source.

This project was fantastic.
I followed the discussion above @ZHKKKe, however I would be thankful if you can provide me with a more precise date by which the conference or journal should give you the final response.
p.s obviously it will be accepted since not only  it includes novel ideas, but also it works perfect.

@Saleh-Gholam-Zadeh
The decision will be given next week... So you may know which it is.
But I have to say that this work may be rejected. I am not sure now.

Good luck bro.

Good Luck!

Hi, all. Bad news... The paper is rejected by CVPR. So the license will not be changed at this time. Please wait for further annotation about it.

Any particular reason why it was rejected?
What are your future plans? Do you plan to apply to another conference and if accepted change the license?

Hi, all. Bad news... The paper is rejected by CVPR. So the license will not be changed at this time. Please wait for further annotation about it.

I am sorry to hear that and you could try submitting the work to another conference as suggested by @Vozf.

@Vozf @ayoolaolafenwa
Thanks, both.
We are not yet able to see the meta reviews now.
Of course, we will submit it to another conference/journal after revision, and the license will be changed after it is accepted.

@Vozf
Hi, we have submitted the paper.
However, the final decision will be released in August.
Therefore, the license will take some time to change. Thanks.

@ZHKKKe any update on the paper or the license changes?

Good luck! Any updates on this?

Hi, all.
Our paper is accepted by AAAI 2022. The license of our github project has been changed to the Apache License 2.0 license.
I would be happy if you could share your APP based on MODNet with me.

Congratulations on acceptance and thanks for the license

Congrats！

Great! Congratulations @ZHKKKe

@ZHKKKe congrats to you and your team mate!


chore: Test flow

Description
no need fro review only a test

Please include a summary of the change and which issue is fixed.
List any external dependencies that are required for this change.

[ ] PR title and description are to the point and understandable
[ ] I have performed a self-review of my own code'

Please select version type the purposed change:

[ ] major
[ ] minor
[ ] patch
[ ] none

External Relations

[ ] database migration

Changeset

looks like things are working :)


Update a8c-ci-toolkit Buildkite plugin to new name and latest version

What

Update the bash-cache Buildkite plugin name to a8c-ci-toolkit
Update the a8c-ci-toolkit plugin to version 2.15.0

Testing
Ensure that CI is green and that all checks passed.

Note: feel free to add @apps-infrastructure-team as reviewer instead of just me for future similar PRs on other repos 😉

Note: feel free to add @apps-infrastructure-team as reviewer instead of just me for future similar PRs on other repos 😉

Sure, that is what I initially tried but for some reason, I cannot @apps-infrastructure-team as it doesn't show up in the reviewer list.


[Feature Request]: Java Properties (OptiFine) support

Is your feature request related to a problem?
There's no way to have connected textures that are compatible with most packs unless you have OptiFine. However, its rewrites make it a pain to use with other mods.
Solution(s)
The custom CTM used in Athena should have support for OptiFine's format
Describe alternatives you've considered
A mod called Continuity does that on fabric, so it is possible. However, it relies heavily on the Indigo renderer, so it's stuck on fabric and quilt.
Mod Version
4.0.0
Mod Loader Version
1.21: 21.0.43-BETA
Mod Loader
NeoForge
Additional context
Honestly, I'm just asking because there's no proper OF CTM support on NeoForge.

We have no plans to support the Optifine format. if you'd like to use optifine resource packs on neoforge, please use
Connectedness


Fix more clippy lints after updating rust toolchain

Which issue does this PR close?

Closes https://github.com/apache/arrow-rs/issues/3011.
Rationale for this change

Appears some got missed in https://github.com/apache/arrow-rs/pull/3010
What changes are included in this PR?

Are there any user-facing changes?

Weird, the CI was passed in that PR.

The CI for pull requests runs based on what files are changed, to try to avoid building up a backlog


read_gtf throws an error

Hi! Thank you for the library, seems like it really cool)
But unfortunately it doesn't work in my case. I've created separate environment with python 3.7 in conda and install pyranges using   default command from README. Everything went ok, but that sample throws an error
import pyranges

gtf_path = 'Chinese_treeshrew_TS2.0.annotation/Chinese_treeshrew_TS2.0.gtf'
gtf = pyranges.read_gtf(gtf_path)

And here is an error
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/a.ilin/anaconda3/envs/genomic/lib/python3.7/site-packages/pyranges/readers.py", line 301, in read_gtf
    gr = read_gtf_full(f, as_df, nrows, _skiprows, duplicate_attr)
  File "/home/a.ilin/anaconda3/envs/genomic/lib/python3.7/site-packages/pyranges/readers.py", line 331, in read_gtf_full
    extra = _to_rows(df.Attribute)
  File "/home/a.ilin/anaconda3/envs/genomic/lib/python3.7/site-packages/pyranges/readers.py", line 358, in to_rows
    for kv in l[:-1].split("; ")]})
  File "/home/a.ilin/anaconda3/envs/genomic/lib/python3.7/site-packages/pyranges/readers.py", line 356, in <dictcomp>
    for k, v in [kv.replace('"', '').split(None, 1)

Perhaps something wrong with my gtf file? Here is a link to the archive with it
http://www.treeshrewdb.org/data/Chinese_treeshrew_TS2.0.annotation.zip
Thank you very much!
Best,
Alexander
P.S.: Call for charity in README is awesome

Would you be able to share the file with me and I’ll find out why it fails :) Thanks for all the kind words :)

I now see you posted the link. Will try today if my daughter falls asleep early enough. Otherwise I’ll do it Monday

Your last column is malformed. Look up this page for how it is supposed to look :)
I am adding a boolean argument full to read_gtf/read_gff3 so that you can read your file without the last column.

See this page: http://m.ensembl.org/info/website/upload/gff.html :)

Hi!)
Thank you very much, I've requested for the proper one, and now it works
Have a nice day)

I am happy to hear that :)