Backport to branch-2, HBASE-26320 Implement a separate thread pool for the LogCleaner (#3712)

This avoids starvation when the archive directory is large and takes a long time
to iterate through.
Signed-off-by: Duo Zhang<EMAIL_ADDRESS>Signed-off-by: Anoop Sam John<EMAIL_ADDRESS>Signed-off-by: Pankaj<EMAIL_ADDRESS>

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
1m 17s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+1 :green_heart:
hbaseanti
0m  0s
Patch does not have any anti-patterns.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

_ branch-2 Compile Tests _

+1 :green_heart:
mvninstall
2m 59s
branch-2 passed

+1 :green_heart:
compile
3m  0s
branch-2 passed

+1 :green_heart:
checkstyle
0m 35s
branch-2 passed

+1 :green_heart:
spotless
0m 55s
branch has no errors when running spotless:check.

+1 :green_heart:
spotbugs
1m 42s
branch-2 passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
2m 37s
the patch passed

+1 :green_heart:
compile
2m 54s
the patch passed

+1 :green_heart:
javac
2m 54s
the patch passed

-0 :warning:
checkstyle
0m 33s
hbase-server: The patch generated 1 new + 11 unchanged - 0 fixed = 12 total (was 11)

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

+1 :green_heart:
hadoopcheck
8m 51s
Patch does not cause any errors with Hadoop 3.1.2 3.2.1.

-1 :x:
spotless
0m 40s
patch has 66 errors when running spotless:check, run spotless:apply to fix.

+1 :green_heart:
spotbugs
1m 36s
the patch passed

_ Other Tests _

+1 :green_heart:
asflicense
0m 11s
The patch does not generate ASF License warnings.

33m 32s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4460/1/artifact/yetus-general-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/4460

Optional Tests
dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile

uname
Linux c4403f1498ee 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
branch-2 / 4260e3afb7

Default Java
AdoptOpenJDK-1.8.0_282-b08

checkstyle
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4460/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt

spotless
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4460/1/artifact/yetus-general-check/output/patch-spotless.txt

Max. process+thread count
60 (vs. ulimit of 12500)

modules
C: hbase-server U: hbase-server

Console output
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4460/1/console

versions
git=2.17.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
4m 13s
Docker mode activated.

-0 :warning:
yetus
0m  6s
Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck

_ Prechecks _

_ branch-2 Compile Tests _

+1 :green_heart:
mvninstall
3m 36s
branch-2 passed

+1 :green_heart:
compile
0m 35s
branch-2 passed

+1 :green_heart:
shadedjars
3m 56s
branch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 24s
branch-2 passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
2m  6s
the patch passed

+1 :green_heart:
compile
0m 34s
the patch passed

+1 :green_heart:
javac
0m 34s
the patch passed

+1 :green_heart:
shadedjars
3m 55s
patch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 20s
the patch passed

_ Other Tests _

+1 :green_heart:
unit
186m 31s
hbase-server in the patch passed.

208m 46s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4460/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/4460

Optional Tests
javac javadoc unit shadedjars compile

uname
Linux b0033796e914 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
branch-2 / 4260e3afb7

Default Java
AdoptOpenJDK-1.8.0_282-b08

Test Results
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4460/1/testReport/

Max. process+thread count
2519 (vs. ulimit of 12500)

modules
C: hbase-server U: hbase-server

Console output
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4460/1/console

versions
git=2.17.1 maven=3.6.3

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
2m 43s
Docker mode activated.

-0 :warning:
yetus
0m  5s
Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck

_ Prechecks _

_ branch-2 Compile Tests _

+1 :green_heart:
mvninstall
2m 51s
branch-2 passed

+1 :green_heart:
compile
0m 47s
branch-2 passed

+1 :green_heart:
shadedjars
3m 50s
branch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 29s
branch-2 passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
2m 34s
the patch passed

+1 :green_heart:
compile
0m 47s
the patch passed

+1 :green_heart:
javac
0m 47s
the patch passed

+1 :green_heart:
shadedjars
3m 49s
patch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 26s
the patch passed

_ Other Tests _

+1 :green_heart:
unit
197m  7s
hbase-server in the patch passed.

217m 16s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4460/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/4460

Optional Tests
javac javadoc unit shadedjars compile

uname
Linux 5bae01cb4212 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
branch-2 / 4260e3afb7

Default Java
AdoptOpenJDK-11.0.10+9

Test Results
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4460/1/testReport/

Max. process+thread count
2557 (vs. ulimit of 12500)

modules
C: hbase-server U: hbase-server

Console output
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4460/1/console

versions
git=2.17.1 maven=3.6.3

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
1m 14s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+1 :green_heart:
hbaseanti
0m  0s
Patch does not have any anti-patterns.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

_ branch-2 Compile Tests _

+1 :green_heart:
mvninstall
2m 23s
branch-2 passed

+1 :green_heart:
compile
2m 16s
branch-2 passed

+1 :green_heart:
checkstyle
0m 32s
branch-2 passed

+1 :green_heart:
spotless
0m 44s
branch has no errors when running spotless:check.

+1 :green_heart:
spotbugs
1m 20s
branch-2 passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
2m 13s
the patch passed

+1 :green_heart:
compile
2m 59s
the patch passed

+1 :green_heart:
javac
2m 59s
the patch passed

-0 :warning:
checkstyle
0m 33s
hbase-server: The patch generated 1 new + 11 unchanged - 0 fixed = 12 total (was 11)

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

+1 :green_heart:
hadoopcheck
9m 21s
Patch does not cause any errors with Hadoop 3.1.2 3.2.1.

+1 :green_heart:
spotless
0m 50s
patch has no errors when running spotless:check.

+1 :green_heart:
spotbugs
1m 58s
the patch passed

_ Other Tests _

+1 :green_heart:
asflicense
0m 10s
The patch does not generate ASF License warnings.

32m 39s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4460/2/artifact/yetus-general-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/4460

Optional Tests
dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile

uname
Linux 8bc55c8e6cf1 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
branch-2 / 4260e3afb7

Default Java
AdoptOpenJDK-1.8.0_282-b08

checkstyle
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4460/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt

Max. process+thread count
69 (vs. ulimit of 12500)

modules
C: hbase-server U: hbase-server

Console output
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4460/2/console

versions
git=2.17.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
1m 20s
Docker mode activated.

-0 :warning:
yetus
0m  5s
Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck

_ Prechecks _

_ branch-2 Compile Tests _

+1 :green_heart:
mvninstall
2m 34s
branch-2 passed

+1 :green_heart:
compile
0m 46s
branch-2 passed

+1 :green_heart:
shadedjars
3m 49s
branch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 27s
branch-2 passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
2m 33s
the patch passed

+1 :green_heart:
compile
0m 47s
the patch passed

+1 :green_heart:
javac
0m 47s
the patch passed

+1 :green_heart:
shadedjars
3m 52s
patch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 27s
the patch passed

_ Other Tests _

+1 :green_heart:
unit
223m 56s
hbase-server in the patch passed.

243m 42s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4460/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/4460

Optional Tests
javac javadoc unit shadedjars compile

uname
Linux 22edf2510b01 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
branch-2 / 4260e3afb7

Default Java
AdoptOpenJDK-11.0.10+9

Test Results
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4460/2/testReport/

Max. process+thread count
2487 (vs. ulimit of 12500)

modules
C: hbase-server U: hbase-server

Console output
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4460/2/console

versions
git=2.17.1 maven=3.6.3

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.


No padding for some pages

Some pages have layout without any padding on the left / right side.
Example:

Pages:

/apps/category/* - https://beta.flathub.org/apps/category/AudioVideo
/apps/collection/project-group/* - https://beta.flathub.org/apps/collection/project-group/KDE
/apps/collection/ - https://beta.flathub.org/apps/collection/recently-updated

@razzeee 1 fixed, 2 and 3 unfixed?

I fixed 1 earlier today. I think 2 and 3 did not make it through the build yet - or maybe our deployment has problems.


Added track loading using the Lavalink rest API

I've tested everything in here fully using Lavalink v<IP_ADDRESS> and everything works completely fine. Would merge in to the branch I made on my fork but I can't merge in to a non-existent branch from what I can see, feel free to suggest changes if you wish!

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Deployed Kubernetes resource differs from its manifest in Helm output

Output of helm version:
version.BuildInfo{Version:"v3.15.4", GitCommit:"fa9efb07d9d8debbb4306d72af76a383895aa8c4", GitTreeState:"clean", GoVersion:"go1.22.6"}
Output of kubectl version:
Client Version: v1.29.0
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.29.2
Cloud Provider/Platform (AKS, GKE, Minikube etc.):
AKS
We deployed an application from Helm chart using “helm install” command and realized an issue in a deployed Kubernetes resource (missing labels we expected to have there).
We ran “helm template”, “helm install --dry-run” commands and then deployed with “helm install --debug”.
Then we checked deployed Kubernetes resource in the cluster and it was different from the manifest from the screen outputs.
Here is a test example illustrating the issue described above:
lib-chart/Chart.yaml
apiVersion: v2
description: Test Library Helm Chart
name: lib-chart
version: 0.0.1
type: library

lib-chart/templates/_labels.tpl
{{- define "lib-chart.labels" -}}
label1: {{ .Values.global.label1 }}
label2: {{ .Values.global.label2 }}
{{- end }}

app-chart/Chart.yaml
apiVersion: v2
name: app-chart
version: 0.0.1
dependencies:
  - name: lib-chart
    version: 0.0.1
    repository: "file://../lib-chart"

app-chart/values.yaml
(a value for label2 is missing intentionally)
global:
  label1: val1

app-chart/templates/ConfigMap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: configmap
  labels:
    {{- include "lib-chart.labels" . | nindent 4 }}
data:
  mykey: myvalue

Deployment using Helm
Helm-Issue/app-chart>helm upgrade --install test --debug -n test-ns --create-namespace --values=./values.yaml .

history.go:56: [debug] getting history for release test
Release "test" does not exist. Installing it now.
install.go:214: [debug] Original chart version: ""
install.go:231: [debug] CHART PATH: .../Helm-Issue/app-chart

client.go:142: [debug] creating 1 resource(s)
client.go:142: [debug] creating 1 resource(s)
NAME: test
LAST DEPLOYED: Fri Sep  6 08:36:43 2024
NAMESPACE: test-ns
STATUS: deployed
REVISION: 1
TEST SUITE: None
USER-SUPPLIED VALUES:
global:
  label1: val1

COMPUTED VALUES:
global:
  label1: val1
lib-chart:
  global:
    label1: val1

HOOKS:
MANIFEST:
---
# Source: app-chart/templates/ConfigMap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: configmap
  labels:
    label1: val1
    label2:
data:
  mykey: myvalue

Live Manifest
apiVersion: v1
data:
  mykey: myvalue
kind: ConfigMap
metadata:
  annotations:
    meta.helm.sh/release-name: test
    meta.helm.sh/release-namespace: test-ns
  creationTimestamp: "2024-09-06T06:36:44Z"
  labels:
    app.kubernetes.io/managed-by: Helm
  name: configmap
  namespace: test-ns

As you can see, both labels (label1 and label2) present in output-manifest are missing in the live-manifest!
When installing output/rendered manifest using kubectl apply, then both the labels are present in the live manifest.
I.e. deployment using Helm results in differences between an expected state and a live state in a cluster!
This should not happen and we would consider this an issue that needs to be fixed!?
We would be grateful for your feedback, opinions and advice and look forward to it.

It's because your configmap looks like:
apiVersion: v1
kind: ConfigMap
metadata:
  name: configmap
  labels:
    label1: val1
    label2:
data:
  mykey: myvalue

This is not valid YAML.

@yardenshoham thanks for your answer.
Yes, I can imagine it is a wrong yaml.
But then there are two questions:

Why do we not get a (warning/error) message from helm?
Even “helm install –debug” wrote out to do the same as kubectl does (see second question below).
The error (missing value for label in values.yaml) was difficult to find and a message could help here.

Why kubectl deploys this manifest (live manifest looks exactly like this in the cluster)?
It means, there are no concerns from kubectl and from Kubernetes API !?

I don't see why label1: val1 is being removed. label2: nil (implicit nil is valid yaml, fwiw) is most likely being dropped by Kubernetes. As to why, I suspect:
https://github.com/helm/helm/issues/13053#issuecomment-2346238985


Custom ModelCollection still not recognized/kept after method

Larastan Version: v2.9.9
Laravel Version: v11.23.5

Description
I have a custom eloquent collection with a custom method that I made a Phpstan plugin for, and that works. And then I call unique() on that collection, and then Larastan thinks it's an EloquentCollection instead of a ModelCollection:
Laravel code where the issue was found
\PHPStan\dumpType($this->logs); // Dumped type: array<int, App\Models\MiscLog>

$logs = ModelCollection::make($this->logs);
\PHPStan\dumpType($logs); // ✅ Dumped type: ModelCollection<int, App\Models\MiscLog>

\PHPStan\dumpType($logs->relation('organization')); // ✅ Dumped type: ModelCollection<int, App\Models\Organization>

\PHPStan\dumpType($logs->relation('organization')->unique()); // ❌ Dumped type: Illuminate\Database\Eloquent\Collection<int, App\Models\Organization>

Phpstan understands ->relation() (because of that custom plugin I made), but it doesn't understand ->unique() (which is Larastan's or Laravel's 'responsibility'?). I feel like unique() should work.

I thought this was fixed for a lot of Laravel collection methods recently. Maybe it's still the same problem as #1927?

Any advice on this? Still doesn't work in Laravel 11 and Larastan 2.9.9.
Laravel's unique() has hint @return static<int, TModel>, and with that it doesn't work for me. But if I change that to just @return static it does work 😮 Phpstan understands the collection class AND the generic types inside:
Dumped type: App\Base\ModelCollection<int, App\Models\Organization>

instead of
Dumped type: Illuminate\Database\Eloquent\Collection<int, App\Models\Organization>

Maybe because EloquentCollection::unique() overwrites Collection::unique() with a different generic name?? TModel instead of TValue. I have no idea.

I can achive the same as unique() with keyBy('id'), but that's hacky.

Fixed in the framework: https://github.com/laravel/framework/pull/53239

Fixed in the framework: laravel/framework#53239

to be fixed ...

Fixed with f97d92ee

I'm glad it's fixed, but why did it fail? The @return made sense, didn't it?

@rudiedirkx Generic static type is not really supported by PHPStan. It's a weird concept as of right now. So we have some workaround for it in Larastan, and I just added unique method to that list. So @return made sense, yes.


AMBARI-24387 - Support YARN Application timeout feature in Ambari Capaacity Scheduler View

What changes were proposed in this pull request?
AMBARI-24387 - Support YARN Application timeout feature in Ambari Capacity Scheduler View
At high level, we need a mechanism to set timeout values for each leaf queue from Capacity Scheduler view. The sample configuration that should reflect in the backend is
yarn.scheduler.capacity.root.default.maximum-application-lifetime= values in seconds
yarn.scheduler.capacity.root.default.default-application-lifetime=values in seconds
How was this patch tested?
Capacity Scheduler View to Support above two configurations related to application lifetime ( which is a feature of Hadoop-3.0 yarn feature).
Verified in UI
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:01 min
[INFO] Finished at: 2018-08-03T17:17:10+05:30
[INFO] Final Memory: 46M/349M
[INFO] ------------------------------------------------------------------------
(Please explain how this patch was tested. Ex: unit tests, manual tests)
(If this patch involves UI changes, please attach a screen-shot; otherwise, remove this)
Please review Ambari Contributing Guide before opening a pull request.

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/Ambari-Github-PullRequest-Builder/3442/
Test PASSed.

LGTM


Is it possible to customize Keras Tokenizer language to take into account French peculiarities

Hello,
I'm trying to build a sentiment analysis model in french using Keras on top of Tensorflow.
However, Keras doesn't handle french tokenization (unlike Spacy for instance)
The Tokenizer object only handles english words.
I tried to plug Keras to Spacy but there's no clear and understandable documentation on that.
Basically this is what I want to perform in french:
from keras.preprocessing.text import Tokenizer
tokenizer = Tokenizer(num_words=MAX_NB_WORDS)

tokenizer.fit_on_texts(data['clean_text'].head(n)) # here's where the english tokenization takes place.  there seems no way to easily adapt it to french

train_sequences = tokenizer.texts_to_sequences(data['clean_text'].iloc[:n])
test_sequences = tokenizer.texts_to_sequences(data['clean_text'].iloc[n:])

Any help would be appreciated.

Sorry for the very untimely reply, but I've been running into the same issue with trying to tokenize/segment Chinese text. You can supply the Tokenizer object with a custom analyzer function to return a list of tokens according to your specified language.
For example:
def my_tokenize_function(text):
    # tokenize text according to your language
    token_list = ...
    return token_list

tokenizer = Tokenizer(num_words=MAX_NB_WORDS, analyzer=my_tokenize_function)

Sorry for the very untimely reply, but I've been running into the same issue with trying to tokenize/segment Chinese text. You can supply the Tokenizer object with a custom analyzer function to return a list of tokens according to your specified language.
For example:
def my_tokenize_function(text):
    # tokenize text according to your language
    token_list = ...
    return token_list

tokenizer = Tokenizer(num_words=MAX_NB_WORDS, analyzer=my_tokenize_function)

can you give me your full function?
I am kinda lost doing the same thing with Chinese language


Fix nacos removeServiceInstancesChangedListener

What is the purpose of the change
Brief changelog
Verifying this change

Checklist

[x] Make sure there is a GitHub_issue field for the change (usually before you start working on it). Trivial changes like typos do not require a GitHub issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.
[ ] Each commit in the pull request should have a meaningful subject line and body.
[ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
[ ] Check if is necessary to patch to Dubbo 3 if you are work on Dubbo 2.7
[ ] Write necessary unit-test to verify your logic correction, more mock a little better when cross module dependency exist. If the new feature or significant change is committed, please remember to add sample in dubbo samples project.
[ ] Add some description to dubbo-website project if you are requesting to add a feature.
[ ] GitHub Actions works fine on your own branch.
[ ] If this contribution is large, please follow the Software Donation Guide.

Codecov Report

Merging #10440 (9b554ba) into 3.1 (bed1137) will decrease coverage by 0.20%.
The diff coverage is 33.33%.

@@             Coverage Diff              @@
##                3.1   #10440      +/-   ##
============================================
- Coverage     65.56%   65.36%   -0.21%
  Complexity      319      319
============================================
  Files          1253     1253
  Lines         54514    54583      +69
  Branches       8241     8256      +15
============================================
- Hits          35744    35680      -64
- Misses        14886    14998     +112
- Partials       3884     3905      +21

Impacted Files
Coverage Δ

...ubbo/registry/nacos/NacosNamingServiceWrapper.java
45.45% <0.00%> (-4.55%)
:arrow_down:

...he/dubbo/registry/nacos/NacosServiceDiscovery.java
46.05% <35.00%> (-10.47%)
:arrow_down:

...nt/metadata/ServiceInstanceHostPortCustomizer.java
65.78% <0.00%> (-21.06%)
:arrow_down:

...che/dubbo/registry/client/NopServiceDiscovery.java
44.44% <0.00%> (-11.12%)
:arrow_down:

...va/org/apache/dubbo/remoting/TimeoutException.java
22.22% <0.00%> (-11.12%)
:arrow_down:

...bo/rpc/protocol/dubbo/DecodeableRpcInvocation.java
70.19% <0.00%> (-10.58%)
:arrow_down:

...dubbo/remoting/exchange/support/DefaultFuture.java
80.17% <0.00%> (-6.90%)
:arrow_down:

...ava/org/apache/dubbo/rpc/proxy/InvocationUtil.java
84.84% <0.00%> (-6.07%)
:arrow_down:

.../java/org/apache/dubbo/config/ReferenceConfig.java
74.17% <0.00%> (-5.91%)
:arrow_down:

...n/java/org/apache/dubbo/config/ConsumerConfig.java
86.48% <0.00%> (-4.69%)
:arrow_down:

... and 35 more

:mega: Codecov can now indicate which changes are the most critical in Pull Requests. Learn more


[BUG] - Upload Failed

Description
I've tried to upload .txt files and .pdf files to the Files > Files Collection Page. It gives me upload error.

Reproduction steps
1. Go to 'Files > File Collecctions'
2. Try to upload file
3. See "upload error"

Screenshots
![DESCRIPTION](LINK.png)

Logs
No response
Browsers
No response
OS
No response
Additional information
No response

Sorry, uploaded wrong image. Here is the correct one showing the "upload error"

@ecceeecs sorry for the late reply. Could you check your terminal output? Which installation method has you used? Can you try the Docker install method or Online install here https://cinnamon.github.io/kotaemon/online_install/


Upload a file again but with new filename

If a file is being uploaded to hatrac with a new filename, but the file already exists, the hatrac content disposition is not being updated to reflect the new filename. The filename is being set properly in ermrest in this case.
Currently if chaise detects the file exists already, the job is marked complete and the next file is uploaded.
The content disposition in hatrac should be consistent with the filename information that we store in ermrest.

#1949 is not directly related but is talking about how we handle filenames.

Changes were merged in ermrestJS as part of PR #1009.


Feature request: tdclient as shared library

Hello!
Can anyone help, how to build tdclient as shared library?  Can anyone post correct CMake example?

It is not supported. The suggested way is to use static library and to build TdLib as a part of your project.
There are much more problems than benefits when using Td as a shared library.
While TdJson has simple plain C interface which is easy to export as shared library API, Td itself has a complicated C++ interface.
For example, you won't be able to use a built shared library with a different c++ standard library.
On windows, it will be problematic to destroy our objects with the right allocator.
There are also may be problems if you will try to use compiler different from the one used to build the library.
Different versions of TdLib are not ABI compatible. They are generally not even API compatible.
TdLib API contains all these classes, which may be (and will be) changed in future. It also complicates correct usage of a shared library.

how to build tdjson static?
can it be portable like build on one system and copy executable to another system and run there?

@korangu You can't create portable tdjson static, because static lib requires other static tdlibs and stdc++ libs.
You can create tdjson shared with -static-libgcc -static-libstdc++  flags. It will include std library and library will bee portable.

ok thanks  let me try give an update soon


Kore cannot simplify #Ceil(f) for f(Int)[function, no-evaluators]

When given a K definition:
module SIMPLIFY
  imports INT
  syntax Int ::= f ( Int ) [function, no-evaluators]
endmodule

and asked to simplify #Ceil(f), Kore returns the term without change, while we may have expected #Top.
Modifying the definition so that f is total produces #Top.
Is this the expected behavior? If so, should we replicate it in Booster?
state-ceil-f.simplify
{
  "format": "KORE",
  "version": 1,
  "term": {
    "tag": "App",
    "name": "inj",
    "sorts": [
      {
        "tag": "SortApp",
        "name": "SortInt",
        "args": []
      },
      {
        "tag": "SortApp",
        "name": "SortKItem",
        "args": []
      }
    ],
    "args": [
      {
        "tag": "Ceil",
        "argSort": {
          "tag": "SortApp",
          "name": "SortInt",
          "args": []
        },
        "sort": {
          "tag": "SortApp",
          "name": "SortInt",
          "args": []
        },
        "arg":
          {
            "tag": "App",
            "name": "Lblf'LParUndsRParUnds'SIMPLIFY'Unds'Int'Unds'Int",
            "sorts": [],
            "args": [
              {
                "tag": "DV",
                "sort": {
                  "tag": "SortApp",
                  "name": "SortInt",
                  "args": []
                },
                "value": "34"
              }
            ]
          }
      }
    ]
  }
}

Discovered while running booster/test/rpc-integration/test-simplify/ with kore-rpc-dev, and observing the state-symbolic-first.simplify produces a wrong response.

Well, the function is indeed not total in this case. I was confused by the larger example. Closing.


⚠ Cioos National CKAN has degraded performance

In ac1b586, Cioos National CKAN (https://catalogue.cioos.ca/) experienced degraded performance:

HTTP code: 200
Response time: 19200 ms

Resolved: Cioos National CKAN performance has improved in 07f10ae after 12 minutes.


Multi-author

Hello !
It would be great to be able to specify the author for each post so we could have several writers on a single blog !

great idea!

Post meta:
Author:hjue

The code in template:
{{post.author}}

Awesome thanks ! :)

Is it me or it doesn't work in archive page ?


Microsoft.DataProtection swagger changes

MSFT employees can try out our new experience at OpenAPI Hub  - one location for using our validation tools and finding your workflow.

Changelog
Please ensure to add changelog with this PR by answering the following questions.

What's the purpose of the update?

[ ] new service onboarding
[ ] new API version
[ ] update existing version for new feature
[x] update existing version to fix swagger quality issue in s360
[ ] Other, please clarify

When you are targeting to deploy new service/feature to public regions? Please provide date, or month to public if date is not available yet.
When you expect to publish swagger? Please provide date, or month to public if date is not available yet.
If it's an update to existing version,  please select SDKs of specific language and CLIs that require refresh after swagger is published.

[ ] SDK of .NET (need service team to ensure code readiness)
[ ] SDK of Python
[ ] SDK of Java
[ ] SDK of Js
[ ] SDK of Go
[ ] PowerShell
[ ] CLI
[ ] Terraform
[ ] No, no need to refresh for updates in this PR

Contribution checklist:

[x] I commit to follow the Breaking Change Policy of "no breaking changes"
[x] I have reviewed the documentation for the workflow.
[x] Validation tools were run on swagger spec(s) and errors have all been fixed in this PR. How to fix?

If any further question about AME onboarding or validation tools, please view the FAQ.
ARM API Review Checklist

[x] Ensure to check this box if one of the following scenarios meet updates in the PR, so that label “WaitForARMFeedback” will be added automatically to involve ARM API Review. Failure to comply may result in delays for manifest application. Note this does not apply to data plane APIs, all “removals” and “adding a new property” no more require ARM API review.

Adding new API(s)
Adding a new API version
Adding a new service

[x] Please ensure you've reviewed following guidelines including ARM resource provider contract and REST guidelines. Estimated time (4 hours). This is required before you can request review from ARM API Review board.

[x] If you are blocked on ARM review and want to get the PR merged with urgency, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.

Breaking Change Review Checklist
If there are following updates in the PR, ensure to request an approval from Breaking Change Review Board as defined in the Breaking Change Policy.

[ ] Removing API(s) in stable version
[ ] Removing properties in stable version
[ ] Removing API version(s) in stable version
[ ] Updating API in stable or public preview version with Breaking Change Validation errors
[ ] Updating API(s) in public preview over 1 year (refer to Retirement of Previews)

Action: to initiate an evaluation of the breaking change, create a new intake using the template for breaking changes. Addition details on the process and office hours are on the Breaking change Wiki.
Please follow the link to find more details on PR review process.

We are not changing the API version as we are just addressing early phase validation bugs assigned to us and no clients currently use this API version. So it is not a breaking change.

We are not changing the API version as we are just addressing early phase validation bugs assigned to us and no clients currently use this API version. So it is not a breaking change.

Thanks for confirming, but since this swagger was published in public repo there could be customers using this version. Please check the ARM calls pattern for this API version to confirm if there is no caller coming in and re-confirm.

Please help fix the ModelValidation error before it can be merged.

Please help fix the ModelValidation error before it can be merged.

I have fixed this. Please review.


output.environment don't work

Bug report

What is the current behavior?
my webpack.config.js as below, I want to remove arrowFunction and forof and so on from output. But it seems don't work.
refer to the webpack document
const path = require('path');

/** @type {import("webpack").Configuration} */
module.exports = {
  mode: 'production',

  entry: './src/index.js',

  output: {
    path: path.resolve(__dirname, '../dist'),
    filename: './js/[name].[id].[contenthash:5].js',
    environment: {
      arrowFunction: false,
      bigIntLiteral: false,
      const: false,
      destructuring: false,
      dynamicImport: false,
      forOf: false,
      module: false,
    },
    clean: true,
  },
};

this is me source code
const show = () => {
  const info = {
    bigInt: 10086n,
    str: '10086',
  };
  const { bigInt, str } = info;
  console.log(bigInt);

  for (let i of [1, 2, 3, 4, 5]) {
    console.log(i);
  }
};
show();

and, this is output
(()=>{const{bigInt:o,str:n}={bigInt:10086n,str:"10086"};console.log(o);for(let o of[1,2,3,4,5])console.log(o)})();

arrowFunction and const and everything is still there.
If the current behavior is a bug, please provide the steps to reproduce.

What is the expected behavior?
I want to remove or transfrom  ES-features above.
They are not needed in the output because the runtime environment may not support them

Other relevant information:
webpack version:   5.52.0
Node.js version:   v12.22.5
Operating System:  MacOs 11.5.2
Additional tools:

Expected, we don't rewrite your code, you need babel-loader


Announcing Pylance: Fast, feature-rich language support for Python in Visual Studio Code

We are excited to announce Pylance, our fast and feature-rich language support for Python! Pylance is available today in the Visual Studio Code marketplace.
Please see the announcement for more details: https://devblogs.microsoft.com/python/announcing-pylance-fast-feature-rich-language-support-for-python-in-visual-studio-code/
If you are currently using VS Code with the Microsoft language server, we encourage you to give Pylance a try!
Please note that Pylance bugs should be filed at https://github.com/microsoft/pylance-release/issues
Thanks!

Are there plans for open sourcing pylance? It seems like a bit of a step back for the language server ecosystem to have the new and improved successor be closed off.

after installing pylance, and setting it as default language server, I get this problem in my setting.json,
Value is not accepted. Valid values: "Jedi", "Microsoft", "None".(1)
my code version is:
1.46.1
cd9ea6488829f560dc949a8b2fb789f3cdc05f5d
x64

@jaguarest please see https://github.com/microsoft/pylance-release/issues/16

So I take it that this repo will eventually be deprecated as work moves to improving Pylance?

Awesome! Great Job!  😁

@huguesv will the MPLS (this repository) continue to be maintained in the future?
I ask because I use language servers with Neovim's built-in LSP client implementation, not VSCode. It looks like Pylance won't be open-sourced, which will make building/packaging/running it difficult on many platforms. If that is the case, many users of MPLS who use LSP clients besides VSCode will have to migrate over to palantir/python-language-server (slow), muffinmad/anakin-language-server, or pappasam/jedi-language-server (limited functionality, but can be combined with diagnostic-langserver for diagnostics and formatting).

Hey @Seirdy
Our team's plan is to transition users in VS Code over to Pylance. After that, this repository will no longer be maintained, at which point you would have to migrate over to another open-source option.

What an amazing decision. The idea of LSPs was created by Microsoft. I've watched YouTube videos of Microsoft employees lauding their achievements in this space. MPLS was the clear winner for Python against the existing options.

Depending on your use case, you might also be interested in checking out Pyright
(which powers the Pylance language server) or forking this repo.

Thanks for the heads-up, @savannahostrowski
Having some documentation on using Pyright as an editor-agnostic language server and/or seeing more support in other LSP clients (such as neovim/nvim-lsp#303) to aid in migration before this repository is archived would be appreciated.

I'm not a fan of migrating from an Open Source solution to a closed-source one

Hmm, thatk sounds like https://de.wikipedia.org/wiki/Embrace,_Extend_and_Extinguish .


Fix the wording in the REP faucet

super minor, new wording is
Get test net REP, it will be sent to your connected wallet
the button Send REP should say Get REP

@JohnDanz do you agree with this change, ran it by design. Currently it's confusing.

fixed


nvidia_x11: 390.77 -> 390.87

Motivation for this change
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[x] Fits CONTRIBUTING.md.

http://rglinuxtech.com/?p=2391 proposes a solution for 4.19-rc1 though I personally haven't had success with it.
patches = lib.optional (lib.versionAtLeast "4.19" (kernel.version or "0")) ./drm_mode_connector.patch;

with the following patch
diff --git a/kernel/nvidia-drm/nvidia-drm-connector.c b/kernel/nvidia-drm/nvidia-drm-connector.c
--- a/kernel/nvidia-drm/nvidia-drm-connector.c
+++ b/kernel/nvidia-drm/nvidia-drm-connector.c
@@ -226,7 +226,7 @@ static int nv_drm_connector_get_modes(struct drm_connector *connector)

     if (nv_connector->edid != NULL) {
-        drm_mode_connector_update_edid_property(
+        drm_connector_update_edid_property(
             connector, nv_connector->edid);
     }

diff --git a/kernel/nvidia-drm/nvidia-drm-encoder.c b/kernel/nvidia-drm/nvidia-drm-encoder.c
--- a/kernel/nvidia-drm/nvidia-drm-encoder.c
+++ b/kernel/nvidia-drm/nvidia-drm-encoder.c
@@ -216,7 +216,7 @@ nv_drm_add_encoder(struct drm_device *dev, NvKmsKapiDisplay hDisplay)

     /* Attach encoder and connector */

-    ret = drm_mode_connector_attach_encoder(connector, encoder);
+    ret = drm_connector_attach_encoder(connector, encoder);

     if (ret != 0) {
         NV_DRM_DEV_LOG_ERR(

cc maintainer @vcunat

Compiles fine on 4.19.0-rc2 though haven't actually tested this version yet.

I'm sorry this has been lying here for so long.

@vcunat this is really weird: github says  :  "xeji merged commit af3fdb4.." above when it's clearly you who merged this.  Do you see this too?

That's currently a very common GitHub error.  It used to be rarer and I've reported it once or twice to their support, some year or two ago, but...

Thanks. That really got me scared for a moment...


iOS. onSubmitEditing callback doesn't fire sometimes

Hey @markuswind. So, the issue is that onSubmitEditing callback sometimes doesn't fire. It works perfect on Android but on iOS I face this bug on every 3-4 Done click.
I'm using v1.1.1.

Hey, I think that's because it's not done scrolling? The Picker inside the PickerKeyboard won't fire onValueChange unless it stopped scrolling.
This is standard behaviour in http://facebook.github.io/react-native/releases/0.49/docs/picker.html#picker

It happens even when I click the Done button immediately without scrolling.

Also, I noticed that it works as expected if I wait 2-3 second after modal opening. Hmmm... Maybe it is the native Picker component problem. So I'll close this issue for now. Thanks!

Unfortunately I have faced this issue many times. Fixed it by adding this._value variable in PickerKeyboard constructor outside the state object. Like this:
class PickerKeyboard extends Component {
  constructor(props) {
    super(props);

    // refs
    this.customKeyboard = null;

    // initial state
    this.state = {
      value: props.value,
      visible: false,
    };

    this._value = null; // <- HERE
  }

...

And then i passed this _value variable into onSubmitPress instead of this.state.value:
onSubmitPress() {
    this.setVisible(false);

    let onSubmit = this.props.onSubmit;
    onSubmit && onSubmit(this._value);
}

So, looks like sometimes state doesn't update when we click on Done button.

Hmm ok, and how are you updating the this._value property?

  onValueChange(value) {
    this._value = value;
    ...
  }

I edited my previous comment. Looks like this attempt doesn't work too. Don't know what to do

Finally, I added this strings to the PcikerKeyboard component:
  shouldComponentUpdate(prevProps, prevState) {
    if (prevState.visible != this.state.visible) {
      if (this.picker) {
        this.onSubmitPress(this.picker.props.selectedValue);
      }
    }

    return true;
  }

And it works so far!

Alright! I don't think I'll add this, seems like this will also trigger when you're not actually pressing the done button. Perhaps you're not handling the value correctly in your parent view?
I'll close this issue for now.

Yeap, for now it triggers on all close events too. But I'll fix it.

Perhaps you're not handling the value correctly in your parent view?

I'm just using this package. So the parent view is the SelectInput.ios.js component

In case if somebody meets this problem here is the link to my hack:
https://github.com/KirillPd/react-native-select-input-ios/blob/master/src/components/PickerKeyboard.js


Refactor skipvalidations to include create

Issue #5865:
Description of changes:
--skip-validations flag was initally only done for upgrade preflights. Refactored the code to have skip validations for create cluster.
Testing:
Tested creating a cluster
./eksctl-anywhere create cluster -f ./config.yaml

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

/approve


Block caremed.de financialanalystinfo.com mailtransfere.shop restitution.eu.com

Lists in use:

Unified Hosts
Fakenews
Gambling
Porn
Social

Domains:
caremed.de
financialanalystinfo.com
mailtransfere.shop
restitution.eu.com

Details:
Spam
https://www.virustotal.com/gui/domain/caremed.de

https://www.virustotal.com/gui/domain/financialanalystinfo.com

https://mxtoolbox.com/SuperTool.aspx?action=blacklist%3Afinancialanalystinfo.com&run=toolpage

https://www.virustotal.com/gui/domain/mailtransfere.shop

https://www.virustotal.com/gui/domain/restitution.eu.com

caremed.de - German Work and Travel insurance
financialanalystinfo.com - tracking domain used for processing email/marketing statistics such as views/clicks/optouts
mailtransfere.shop - DEAD domain
restitution.eu.com - only SOA Record, no A/AAAA, whitelisted on https://otx.alienvault.com/indicator/domain/restitution.eu.com

financialanalystinfo.com - tracking domain used for processing email/marketing statistics such as views/clicks/optouts

So this is the only one that has the potential to be included. However, in blocking that TLD you remove the ability to unsubscribe at their site. The ideal solution would be find out what sub domains, if any, are in use with the emails they are present in. This requires input from those that encounter it.


No implementation found for method getSupportedEmojis on channel emoji_picker_flutter

[ERROR:flutter/runtime/dart_vm_initializer.cc(41)] Unhandled Exception: MissingPluginException(No implementation found for method getSupportedEmojis on channel emoji_picker_flutter)
May I ask how to solve this problem?
`import 'package:emoji_picker_flutter/emoji_picker_flutter.dart';
import 'package:flutter/foundation.dart' as foundation;
import 'package:flutter/material.dart';
class EmojiPop extends StatelessWidget {
final TextEditingController _controller = TextEditingController();
EmojiPop({super.key});
@override
_onBackspacePressed() {
_controller
..text = _controller.text.characters.toString()
..selection = TextSelection.fromPosition(
TextPosition(offset: _controller.text.length));
}
@override
Widget build(BuildContext context) {
return EmojiPicker(
textEditingController: _controller,
onBackspacePressed: _onBackspacePressed,
config: Config(
columns: 7,
// Issue: https://github.com/flutter/flutter/issues/28894
emojiSizeMax: 32 * (foundation.defaultTargetPlatform == TargetPlatform.iOS ? 1.30 : 1.0),
verticalSpacing: 0,
horizontalSpacing: 0,
gridPadding: EdgeInsets.zero,
initCategory: Category.RECENT,
bgColor: const Color(0xFFF2F2F2),
indicatorColor: Colors.blue,
iconColor: Colors.grey,
iconColorSelected: Colors.blue,
backspaceColor: Colors.blue,
skinToneDialogBgColor: Colors.white,
skinToneIndicatorColor: Colors.grey,
enableSkinTones: true,
recentTabBehavior: RecentTabBehavior.RECENT,
recentsLimit: 28,
replaceEmojiOnLimitExceed: false,
noRecents: const Text(
'No Recents',
style: TextStyle(fontSize: 20, color: Colors.black26),
textAlign: TextAlign.center,
),
loadingIndicator: const SizedBox.shrink(),
tabIndicatorAnimDuration: kTabScrollDuration,
categoryIcons: const CategoryIcons(),
buttonMode: ButtonMode.MATERIAL,
checkPlatformCompatibility: true,
),
);
}
}
`

Which version?
We had this issue before and normally it was caused by cache. Please try flutter clean and see if it helps.

Please open another issue if the issue persists 🙏


Implement clip planes

Implements clipping planes in vertex shaders so that clipDist = dot(clipPlane, vertexPos). This fixes some broken water reflections in Half-Life 2.
The first three patches are minor fixes around uniform buffers.

Looks good!


Form foundation

This PR added FormContainerWidget, FormSubmitButtonWidget and a (basic) FormTextInputWidget.
The data is not yet submitted (to be done in a different PR). Also validations will be added at a later point in time.
See https://docs.google.com/document/d/1Dbr7Y-ODFsQ4YSyYtgfyWM74YwkZTCovSz4xstvkVzc/edit# for documentation of the API.

REVIEW (c228520)

REVIEW (87d70af)
Added some more design fixes by Alex G.

REVIEW (8208a26)
Regarding the wording for editors/visitors I suggest that @dcsaszar, Andreas and I will talk on Friday in a Video chat.


Update with changes for <IP_ADDRESS>

@frankmalin please review and merge

@frankmalin please review and merge


AspNetCore 2.1-preview2 - System.Text.DecoderFallbackException

Hi,
I recently upgraded my code from asp.net core 2.0 to asp.net core 2.1-preview2. Since we upgraded we are encountering a crash in the production (for some requests) which never occurred before for the same code:
System.Text.DecoderFallbackException · Unable to translate bytes [C7] at index 829 from specified code page to Unicode.
System.Text.DecoderExceptionFallbackBuffer.Throw(byte[] bytesUnknown, int index)
System.Text.DecoderExceptionFallbackBuffer.Fallback(byte[] bytesUnknown, int index)
System.Text.DecoderFallbackBuffer.InternalFallback(byte[] bytes, Byte* pBytes, ref Char* chars)
System.Text.UTF8Encoding.GetChars(Byte* bytes, int byteCount, Char* chars, int charCount, DecoderNLS baseDecoder)
System.Text.DecoderNLS.GetChars(byte[] bytes, int byteIndex, int byteCount, char[] chars, int charIndex, bool flush)
System.Text.DecoderNLS.GetChars(byte[] bytes, int byteIndex, int byteCount, char[] chars, int charIndex)
Microsoft.AspNetCore.WebUtilities.HttpRequestStreamReader.ReadIntoBuffer()
Microsoft.AspNetCore.WebUtilities.HttpRequestStreamReader.Read(char[] buffer, int index, int count)
Newtonsoft.Json.JsonTextReader.ReadData(bool append, int charsRequired)
Newtonsoft.Json.JsonTextReader.ParseValue()
Newtonsoft.Json.JsonTextReader.Read()
Newtonsoft.Json.JsonReader.ReadForType(JsonContract contract, bool hasConverter)
Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, bool checkAdditionalContent)
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
Microsoft.AspNetCore.Mvc.Formatters.JsonInputFormatter+<ReadRequestBodyAsync>d__17.MoveNext()
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
System.Runtime.CompilerServices.TaskAwaiter<TResult>.GetResult()
Microsoft.AspNetCore.Mvc.ModelBinding.Binders.BodyModelBinder+<BindModelAsync>d__7.MoveNext()
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
Microsoft.AspNetCore.Mvc.ModelBinding.ParameterBinder+<BindModelAsync>d__11.MoveNext()
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
System.Runtime.CompilerServices.TaskAwaiter<TResult>.GetResult()
Microsoft.AspNetCore.Mvc.Internal.ControllerBinderDelegateProvider+<>c__DisplayClass0_0+<<CreateBinderDelegate>g__Bind|0>d.MoveNext()
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker+<InvokeInnerFilterAsync>d__13.MoveNext()
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker+<InvokeNextExceptionFilterAsync>d__24.MoveNext()

It seems request parsing is failing which wasn't happening before. Can't even log the request model as crash occurs before the controller action is called.

Based on the callstack, I wonder if this could be reproduced outside asp.net core too. You could try with the same content as in your failed request's body and in a standalone .net core console app and using json.net (version 11.0.2, used in asp.net core 2.1.0-preview2-final). If you can still repro it, then you should probably file a bug here: https://github.com/dotnet/corefx/issues

@kichalla, unfortunately, I don't have the exact request body which is causing it to crash. I need to log it before the json is parsed and I don't know whether I can do that in any filter or middleware.

Try this: Write an exception filter and decorate it on the action which fails and in the filter log the request body
A quick example:
    public class TestExceptionFilterAttribute : Attribute, IExceptionFilter
    {
        public void OnException(ExceptionContext context)
        {
            var requestBody = context.HttpContext.Request.Body;
            if (requestBody.CanSeek)
            {
                // rewind it as modelbinding step would have read the body already
                requestBody.Seek(0L, SeekOrigin.Begin);

                //you need not close this stream as underlying layer (which created it) would it for you.
                var streamReader = new StreamReader(requestBody);
                var content = streamReader.ReadToEnd();
                var logger = context.HttpContext.RequestServices.GetRequiredService<ILogger<TestExceptionFilterAttribute>>();
                logger.LogError(content);
            }
        }
    }

By default, JsonInputFormatter buffers the request body before deserliazation it a instance of the model type to bind to, unless you explicitly set MvcOptions's SuppressInputFormatterBuffering to a different value.

@kichalla Thanks, I tried your code snipped and following key value pair seems to be causing the issue:
...
"hostname":"��������-PC",
...

@adnan-kamili The characters in your earlier post are messed up. Is it possible for you to share the whole request data so that we could reproduce at it at our end?

E.g. a wireshark trace

@karelz Any ideas on why this would be happening?

So it sounds like everything here was behaving appropriately when presented with invalid bytes from the client.

Any ideas on why this would be happening?

@kichalla No idea, I think we need isolated repro. Latest reply hints it may be machine name encoding problem? The stack doesn't suggest anything like that though.

Thanks @karelz
Regarding

The stack doesn't suggest anything like that though.

I am not sure about your question here, but the hostname (with invalid characters) is present in the body of the request and decoding it is failing.

Closing this issue as there's no action item on our side.

@kichalla

but the hostname (with invalid characters) is present in the body of the request and decoding it is failing.

Ah, that was not 100% clear to me.
How did the invalid characters get there? Should they be encoded? Is it a problem on server or on client? Anything we can/should fix in CoreFX?

Based on https://github.com/aspnet/Home/issues/3105#issuecomment-385252992, https://github.com/aspnet/Home/issues/3105#issuecomment-385492323 and the callstack, the scenario is that there's a C++ client posting json data(having invalid characters) to asp.net core app and we failed to decode these characters in the server.
AFAIK, we need not fix anything at our end.

@kichalla We received a new error in production, can the following be due to the code we added for logging request body by reading it from the stream:
Reading the request body timed out due to data arriving too slowly. See MinRequestBodyDataRate.
Microsoft.AspNetCore.Server.Kestrel.Core.BadHttpRequestException: Reading the request body timed out due to data arriving too slowly. See MinRequestBodyDataRate.
   at Microsoft.AspNetCore.Server.Kestrel.Core.BadHttpRequestException.Throw(RequestRejectionReason reason)
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.Http1MessageBody.PumpAsync()
   at System.IO.Pipelines.PipeCompletion.ThrowLatchedException()
   at System.IO.Pipelines.Pipe.GetReadResult(ReadResult& result)
   at System.IO.Pipelines.Pipe.GetReadAsyncResult()
   at System.IO.Pipelines.Pipe.DefaultPipeReader.GetResult(Int16 token)
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.MessageBody.ReadAsync(Memory`1 buffer, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpRequestStream.ReadAsyncInternal(Memory`1 buffer, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.WebUtilities.FileBufferingReadStream.ReadAsync(Byte[] buffer, Int32 offset, Int32 count, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.WebUtilities.StreamHelperExtensions.DrainAsync(Stream stream, ArrayPool`1 bytePool, Nullable`1 limit, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Mvc.Formatters.JsonInputFormatter.ReadRequestBodyAsync(InputFormatterContext context, Encoding encoding)
   at Microsoft.AspNetCore.Mvc.ModelBinding.Binders.BodyModelBinder.BindModelAsync(ModelBindingContext bindingContext)
   at Microsoft.AspNetCore.Mvc.ModelBinding.ParameterBinder.BindModelAsync(ActionContext actionContext, IModelBinder modelBinder, IValueProvider valueProvider, ParameterDescriptor parameter, ModelMetadata metadata, Object value)
   at Microsoft.AspNetCore.Mvc.Internal.ControllerBinderDelegateProvider.<>c__DisplayClass0_0.<<CreateBinderDelegate>g__Bind|0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.InvokeInnerFilterAsync()
   at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeNextExceptionFilterAsync()
[12:17:20 ERR] Connection id "0HLDG3CS3KJNF", Request id "0HLDG3CS3KJNF:00000001": An unhandled exception was thrown by the application.
Microsoft.AspNetCore.Server.Kestrel.Core.BadHttpRequestException: Reading the request body timed out due to data arriving too slowly. See MinRequestBodyDataRate.
   at Microsoft.AspNetCore.Server.Kestrel.Core.BadHttpRequestException.Throw(RequestRejectionReason reason)
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.Http1MessageBody.PumpAsync()
   at System.IO.Pipelines.PipeCompletion.ThrowLatchedException()
   at System.IO.Pipelines.Pipe.GetReadResult(ReadResult& result)
   at System.IO.Pipelines.Pipe.GetReadAsyncResult()
   at System.IO.Pipelines.Pipe.DefaultPipeReader.GetResult(Int16 token)
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.MessageBody.ReadAsync(Memory`1 buffer, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpRequestStream.ReadAsyncInternal(Memory`1 buffer, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.WebUtilities.FileBufferingReadStream.ReadAsync(Byte[] buffer, Int32 offset, Int32 count, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.WebUtilities.StreamHelperExtensions.DrainAsync(Stream stream, ArrayPool`1 bytePool, Nullable`1 limit, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Mvc.Formatters.JsonInputFormatter.ReadRequestBodyAsync(InputFormatterContext context, Encoding encoding)
   at Microsoft.AspNetCore.Mvc.ModelBinding.Binders.BodyModelBinder.BindModelAsync(ModelBindingContext bindingContext)
   at Microsoft.AspNetCore.Mvc.ModelBinding.ParameterBinder.BindModelAsync(ActionContext actionContext, IModelBinder modelBinder, IValueProvider valueProvider, ParameterDescriptor parameter, ModelMetadata metadata, Object value)
   at Microsoft.AspNetCore.Mvc.Internal.ControllerBinderDelegateProvider.<>c__DisplayClass0_0.<<CreateBinderDelegate>g__Bind|0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.InvokeInnerFilterAsync()
   at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeNextExceptionFilterAsync()
   at System.IO.Pipelines.PipeCompletion.ThrowLatchedException()
   at System.IO.Pipelines.Pipe.GetReadResult(ReadResult& result)
   at System.IO.Pipelines.Pipe.ReadAsync(CancellationToken token)
   at System.IO.Pipelines.Pipe.DefaultPipeReader.ReadAsync(CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.MessageBody.ReadAsync(Memory`1 buffer, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpRequestStream.ReadAsyncInternal(Memory`1 buffer, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpRequestStream.Read(Byte[] buffer, Int32 offset, Int32 count)
   at Microsoft.AspNetCore.WebUtilities.FileBufferingReadStream.Read(Byte[] buffer, Int32 offset, Int32 count)
   at System.IO.StreamReader.ReadBuffer()
   at System.IO.StreamReader.ReadToEnd()
   at ProdWebApi.Filters.CustomExceptionFilterAttribute.OnException(ExceptionContext context) in /app/Filters/CustomExceptionFilterAttribute.cs:line 48
   at Microsoft.AspNetCore.Mvc.Filters.ExceptionFilterAttribute.OnExceptionAsync(ExceptionContext context)
   at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
   at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeNextResourceFilter()
   at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.Rethrow(ResourceExecutedContext context)
   at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
   at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeFilterPipelineAsync()
   at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeAsync()
   at Microsoft.AspNetCore.Builder.RouterMiddleware.Invoke(HttpContext httpContext)
   at Microsoft.AspNetCore.Builder.Extensions.MapMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Diagnostics.StatusCodePagesMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
   at AspNetCoreRateLimit.IpRateLimitMiddleware.Invoke(HttpContext httpContext)
   at Microsoft.AspNetCore.Cors.Infrastructure.CorsMiddleware.Invoke(HttpContext context)
   at ProdWebApi.Middlewares.CustomResponseHeadersMiddleware.Invoke(HttpContext context) in /app/Middlewares/CustomResponseHeadersMiddleware.cs:line 38
   at ProdWebApi.Middlewares.ForceHttpsMiddleware.Invoke(HttpContext context) in /app/Middlewares/ForceHttpsMiddleware.cs:line 34
   at Bugsnag.AspNet.Core.Middleware.Invoke(HttpContext context, IClient client)
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)

Based on the call stack, the original exception is happening when the JsonInputFormatter is trying to buffer the request body.
Reading the request body timed out due to data arriving too slowly. See MinRequestBodyDataRate.
This exception is being caught in the exception filter but it's still trying to read the request body (from the request stream and not the buffered one) since CanSeek is always true.
@Tratcher can CanSeek be set to True only when all the request body is buffered successfully into FileBufferingReadStream?

@kichalla how would that help here? I see DrainAsync throwing. Where else is the request body read?

Its being read in the exception filter here: https://github.com/aspnet/Home/issues/3105#issuecomment-385250402

@kichalla
Is it possible to change the default fallback strategy if decoding fails?


AttributeError: aenter with aiobotocore 2.5.3

Hello, not sure this is a aiobotocore issue or fsspec issue. But basically what I've reported here: https://github.com/aio-libs/aiobotocore/issues/1031.
A code that was working with aiobotocore 2.5.2 broke this morning when aiobotocore 2.5.3 shipped. The interesting bit is that we can reproduce only on Docker and that when erroring, it suggests AttributeError: __aenter__. Did you mean: '_details'? that seems related to an fsspec object.
Reverting to 2.5.2 works.
A reproduction is here: https://github.com/aio-libs/aiobotocore/issues/1031#issuecomment-1667899504 (but only on Docker)

(moved issue to s3fs)
Notice that s3fs has an exact pin on aiobotocore.
For sure, we want to work out what changed upstream, and make the next release depend on the latest available package.

Thanks for being so responsive! This bit my group also, and the bug reports were super helpful.
FWIW, thehesiod fixed the issue about 45 minutes ago and posted a new version.

Yep! Saw that. I guess we can close this then! Thanks everyone for the amazing reactivity!


[FEATURE]: Translate Rich Text Blocks

Is your feature request related to a problem? Please describe.
When using the new Rich Text (Blocks) field, the content is not translated.
Describe the solution you'd like
Rich Text (Blocks) fields should be translated to the target locale.
Describe alternatives you've considered
For now, this field type is unusable if automated translation is required. Instead, we must continue to use Rich Text (Markdown) text fields.
Additional context
We use Deepl to translate our content. Given the structure of Blocks objects in the Strapi API, it may be difficult to send to the translation API and get back a meaningful response that respects grammar, etc.
Sorry if this issue has already been raised, but I didn't find any previous GH issues.

Any updates?

Hi @jsadoski-rockhall and @boardmain,
Thank you for your interest in this plugin and the feature proposal!
This is definitely an important feature that should be implemented in this plugin. However currently, we as maintainers have more pressing issues on the internal side and are also working voluntarily. If you require this feature, we would be very happy if you would try to implement it yourself and create a pull request. We will give you feedback on problems you may encounter.
From my understanding, the easiest way to implement this feature would be to create a rendering from the blocks syntax to html or xml, then translate that on the provider, and then convert it back to the blocks syntax.
There already exists an html renderer for the blocks syntax, so that may be used and only the translation back to the blocks syntax is required.
An issue that may definitely arise is that the rendering to html is (at least in the referenced html renderer) a lossy conversion, as not all properties are used in the html. I am not quite sure how this can be handled yet.
Another problem may arise when the document is too large to be translated in one request. Currently, there is also no good handling for splitting html documents, but this problem should definitely considered.

@sargreal
hi im doing some test
adding a custom provider ( deepl modified )
but if i try to add for example to the strapi-plugin-translate configurator inside plugins.js
translatedFieldTypes: [
'string',
{ type: 'blocks', format: 'jsonb' },
{ type: 'text', format: 'plain' },
{ type: 'richtext', format: 'markdown' },
'component',
'dynamiczone',
],
it is ignored by the plugin and i can't Handle this type in the custom provider
it is ignored into getTranslateFields
in the "getFieldTypeConfig(type) function"
when it look for the block type it never continue
function getFieldTypeConfig(type) {
  const { translatedFieldTypes } = strapi.config.get('plugin.translate')

	console.log('TYPE: ->', type);
	console.log(translatedFieldTypes);
  return translatedFieldTypes.find((t) => t === type || t.type === type)
}

i see:
TYPE: -> blocks
[
  { type: 'string', format: 'plain' },
  { type: 'text', format: 'plain' },
  { type: 'richtext', format: 'markdown' },
  'component',
  'dynamiczone'
]

can you help to modify the response of  strapi.config.get('plugin.translate') ? if it isn't handled by the plugins.js config gile

@boardmain
  translatedFieldTypes: [
    'string',
    { type: 'blocks', format: 'jsonb' },
    { type: 'text', format: 'plain' },
    { type: 'richtext', format: 'markdown' },
    'component',
    'dynamiczone',
  ],

So first of all, there seems to be something wrong with your setup, since I get an expected Error Error regarding translate config: unhandled format jsonb for translated field blocks when using your config. If that error isn't thrown and you have not customized the config parsing, the config you are providing is not considered at all.
I believe the format parsing should completely be implemented on the plugin side, as the providers should only provide the translation function. The execution of the format conversion is only done on the provider side, since we cannot be sure which formats are supported by a provider (although that could also be stated in the provider's declaration as well. That would be a breaking change however).
Anyways, I have tried today a little bit at implementing this feature in a rudimentary state and will open a PR in the next days where you can have a look and maybe build from there.

@sargreal yes i had an error in my plugins.js config file, now i have correctly the "Error regarding translate config"

While this. update will be fixed, if anyone need it
i published this package
strapi-provider-translate-deepljsonb
it take all text from paragraph, translate it with deepl and writeback to object
simply install it with strapi-provider-translate-deepljsonb  and replace deepl with deepljson i the plugin.js config
it is not perfect, but for now is working

deepljsonb

Hello. I have tried your solution, but nothing changes :(


[11.x] Add port configuration for Beanstalkd driver

This PR adds the port parameter to the Beanstalkd driver configuration in config/queue.php.
Motivation
Including the port explicitly improves readability and aligns the Beanstalkd configuration style with other drivers, which also define both host and port. The port value defaults to null, allowing the Pheanstalk library to use its default port (11300). This ensures full backward compatibility while giving users the flexibility to configure the port via the BEANSTALKD_QUEUE_PORT environment variable if needed.

You can add this to your own application if you need it. 👍


the node property panels (basic and rotation) always unfold

when I fold/collapse the two panels in sciview, then call sciview.addNode(node, false, someParentNode), the panels unfold/expand fully... dishonoring their current visibility state
side note: noticed that the content of these two panels is always refreshed, even when activePublish = false as above; but the panel with the scene tree is not updated (as I wished with the false)

The side note should be addressed by: https://github.com/scenerygraphics/sciview/pull/594

@xulman what is a minimal test for this?

I wonder if this is the same issue: https://github.com/scenerygraphics/sciview/issues/480


Unlink bound path on uds drop

This allows a user to reuse a bound path after the uds was dropped
without having to do any cleanup.

Is this something we want to do? In other words are there any use cases in which we don't always want to unlink?
Also this adds an allocation and makes the types larger, which wouldn't make this zero cost anymore.

@kleimkuhler what do you think?

Note that unlink doesnt do anything if another open socket refers to the same fd. So I couldn't think of a use case where you wouldn't want such RAII semantics.
There might be a way to avoid the allocation by retrieving the local_address of the bound socket in the Drop implementation. But the socket needs to be closed before unlink is called, and I could not figure out how to do that.

So I couldn't think of a use case where you wouldn't want such RAII semantics.

I think we should think the other way around, i.e. why would we want such semantics, as doing nothing is always faster. In Mio we try to match the standard library went it comes to API and as far as I can tell it doesn't unlink anything either, does it?

it doesn't unlink anything either, does it

No, otherwise this wouldn't be needed.
I can understand the concern for the additional allocation, but AFAIK dropping a socket will always be part of a cold path, so paying an additional syscall for convenience seems like a decent trade.

I think we should think the other way around, i.e. why would we want such semantics, as doing nothing is always faster.

One could argue that calling close on a socket falls in the same category. Not calling close on a socket would leak the fd, but its always faster. In this situation, no calling unlink of the bound path leaks the file, but its always faster.

Ideally this should be done in the drop impl of the std UnixDatagram, but I think this is considered a breaking change, so I'm not confident it would be merged.

I think the fact that mio tries to match std is in an important thing to keep in mind here. Also as @Thomasdezeeuw mentioned I think we should keep these types zero-cost.
That being said, I wasn't aware that this tends to be such a difficult problem when using UDS. After looking around online for how others have dealt with this, it seems like there is no great solution.
@jean-airoldie I like that we have the ability to use a Drop impl here and it should be doable after some quick playing around with it. The only issue that comes up is the IntoRawFd impl which becomes something along the lines of:
impl IntoRawFd for UnixListener {
    fn into_raw_fd(mut self) -> RawFd {
        let fd = mem::replace(&mut self.inner, unsafe { mem::uninitialized() });
        mem::forget(self);
        fd.into_raw_fd()
    }
}

That was just to make my changes compile; there may still need to be path cleanup there?
I also wouldn't worry about this being a breaking change, because 0.7 already has a host of breaking changes!
So I think this is something we should consider if it can be a zero-cost solution. I think it should also be documented very clearly because this is not standard behavior.

@jean-airoldie Thanks for the PR. I see how this could arguably be a good thing. Could you provide an argument for why Mio should deviate from std in this case? Perhaps, the first step would be to propose the change to std.

For my personal use case, I'm not using mio directly so I solved the problem from my end using a similar approach. So I don't depend on this getting merged.
I tend to agree that the right solution would be a change in the std. I've never contributed directly to the std but I assume that it would take quite a while for the change to merged and then deployed. However, I'm wondering whether this is enough of a usability problem that it would be worth it to fix it here also so it propagates to tokio faster (assuming such a changed would get merged by std).
My general argument for this that the current behavior of leaking the file makes the lifecycle of a bound UnixDatagram non-idempotent. If you look at the tests in mio involving unix domain sockets, they all involve using a temp dir so that its not an issue. Since this leaked file cannot be reused by a socket, Its completely useless from the user's perspective. I would also argue that its rather unexpected behavior.
I didn't think about mem::replace with uninitialized memory, it seems very obvious now that I see it @kleimkuhler. I'll rewrite the PR.

I'd like also point out the lack of SO_REUSEADDR for AF_UNIX sockets, so its not like you could do anything with that leaked file anyway.

Have any issues been opened to include the behaviour in std lib?

Fair point, I'll open an issue.

The net::UnixDatagram::from_raw_fd(-1) approach doesn't seem to be working for macos.

So I opened an issue in the std lib and the issue was closed with the conclusion is that:

We generally want to adhere to the underlying system primitives as closely as possible so that things behave as expected to people who know how the underlying system primitives behave.

So in my understanding, the reasoning was more to leave the posix API more or less untouched in order to maximize its familiarity.

Personally I think we should follow what the standard library is doing, adding documentation (and perhaps a method) that manual removal of the socket is required.

How would such a method work? The only reason this is an issue in the first place is that you have to close the socket before being able to unlink it. I guess that method could return a guard that unlinks the path when drop, but that's pretty weird. You would still have to make sure that the socket is dropped before the guard.
From what I understand, in the kernel, people deal with this problem by calling unlink on each path before binding it, which seems like a huge pain for the users.
Let's say there was a problem in the posix spec, which I would argue that this is, should mio fix it for the users? Or should we let every user fix the issue themselves so that we maintain familiarity? We still don't have any example where this behavior would be desirable.

How would such a method work? The only reason this is an issue in the first place is that you have to close the socket before being able to unlink it. I guess that method could return a guard that unlinks the path when drop, but that's pretty weird. You would still have to make sure that the socket is dropped before the guard.

I was thinking a method like: fn unlink(self) { } which would close itself and then unlink, essentially what this change proposes to do by default when dropped.

Let's say there was a problem in the posix spec, which I would argue that this is, should mio fix it for the users? Or should we let every user fix the issue themselves so that we maintain familiarity? We still don't have any example where this behavior would be desirable.

I think Mio is complex enough as is, I don't we should deviate from the behaviour from std lib to make it even more complex. But that is my opinion, I don't know if everyone agrees with that.

I was thinking a method like: fn unlink(self) { } which would close itself and then unlink, essentially what this change proposes to do by default when dropped.

I would be satisfied by that. Its kinda weird but it would do the trick.

Alright so turns out that you can straight up delete the socket file of a bound socket which will silently delete the socket without returning an error. This means this mechanism can't work because its possible that you duplicated the socket.

The only solution I can think of would involve reference counting the number of socket dups but its not a zero-cost abstraction, so it wouldn't fit in mio.

This behavior isn't consistent with the unlink method man:

unlink() deletes a name from the filesystem.  If that name was the last link to a file and no processes have the file open, the file is deleted and the space it was using is made available for reuse.

Yea I see cloning the listener in echo_remote and unlinking the original one leads to a NotFound error. That is unfortunate.. I imagine the man is lacking additional detail on what exactly "no processes have the file open" means. I assume since try_clone is just dup under the hood, duplicating the file descriptor is not enough to keep the file from being deleted. Maybe opening another file description would do the trick, but definitely out of the scope of what mio should do at this point.

Yeah that's clearly out of the scope of mio, I'll close the issue. Thanks for your time.
I'll probably keep track of the refcount via a atomic counter living in shared memory, then unlink the path when the counter would reach 0.

Is there any documentation we should add based on this?


fix: stop saving snapshot files to wrong directory during tests

Overview
by not setting the testing directory during integration tests, we end up saving the LOCK and LOG files from the snapshot directory to permanent directories instead of the temp ones created during the test.
this can be testing by running the integration tests locally without this change, and then with this change.
Checklist

[x] New and updated code has appropriate documentation
[x] New and updated code has new and/or updated testing
[x] Required CI checks are passing
[x] Visual proof for any user facing features like CLI or documentation updates
[x] Linked issues closed with keywords

Codecov Report

Merging #1255 (4c2c816) into main (d315948) will increase coverage by 0.04%.
The diff coverage is 50.58%.

@@            Coverage Diff             @@
##             main    #1255      +/-   ##
==========================================
+ Coverage   48.81%   48.86%   +0.04%
==========================================
  Files          74       74
  Lines        4179     4347     +168
==========================================
+ Hits         2040     2124      +84
- Misses       1962     2043      +81
- Partials      177      180       +3

Impacted Files
Coverage Δ

app/app.go
6.23% <0.00%> (-0.02%)
:arrow_down:

pkg/prove/querier.go
21.73% <30.76%> (+21.73%)
:arrow_up:

pkg/prove/proof.go
75.84% <63.10%> (-10.08%)
:arrow_down:

testutil/testnode/full_node.go
61.36% <100.00%> (+0.29%)
:arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Thanks! I was wondering why I had a dirty git dir at testutil/testnode/data/ after running make test locally.


Option to choose network interface

For example ping has the -I option:
$ ping <IP_ADDRESS> -I enp5s0
PING <IP_ADDRESS> (<IP_ADDRESS>) from x.x.x.x enp5s0: 56(84) bytes of data.
64 bytes from <IP_ADDRESS>: icmp_seq=1 ttl=49 time=40.5 ms

$ ping <IP_ADDRESS> -I tun0
PING <IP_ADDRESS> (<IP_ADDRESS>) from y.y.y.y tun0: 56(84) bytes of data.
64 bytes from <IP_ADDRESS>: icmp_seq=1 ttl=49 time=61.4 ms

Is there a similar option to select specific interface for speed test? If not, does anyone else think it would be a good idea to implement?

Not something I'm interested in having here. I would recommend trying to get it added to https://github.com/sivel/speedtest-cli, which has more options.


add macOS installer

While it's not too involved to install command line apps by hand on macOS, it would be nice to have a .pkg installer. A Homebrew Formulae would also be nice, though it probably has to be done outside of the official Taps.

Task:

[ ] Refactor Actions workflows
[ ] Add macOS installer
[ ] Add Homebrew Formulae

v0.1.0 and v0.2.0 has been added to a Homebrew Tap. Future releases will be added to the Tap as well. And eventually, we'll add an Actions workflow to automatically update the Formula in the Tap.


Add a parameter to top level parser and controls for vendor-specific …

…metadata

+1 from me.

Would we support copying from vendor metadata to user defined metadata, as part of the P4 program?

AI Andy: Add a section to PNA spec and pna.p4 header file that explicitly describes all things that vendors are expected to customize in their product's version of the pna.p4 header file.

As I do understand this addition, I prefer that also a detailed "when and how is this supposed to be used" text somewhere in the PNA document to prevent the per browser vendor HTML/CSS properties charade in the browser world...

Alan: What about more fine-grained control, e.g.

1 in and 1 inout struct type for mainparser
1 in and 1 inout struct type for maincontrol
1 in and 1 inout struct type for deparser

Andy: For our P4 compiler, we were thinking of even mixing fields that were read-only vs. ones that were read-write mixed together in the same struct, and have the compiler back end enforce the read-only restrictions on fields that were read-only.  This makes less good use of the front-end checking for direction 'in' parameters, but enables us to across different compiler releases change a field from read-only to read-write, WITHOUT having to move the field between structs, which could be an imposition on customer P4 programs.

I would also suggest 3 inout structures for each parser/control/deparser.
From a mechanics point of view, would we expect cross-platform PNA code to look something like:
#include <pna.p4>
#ifdef VENDOR_A
#include <pna_vendor_a.p4>
#endif
#ifdef VENDOR_B
#include <pna_vendor_b.p4>
#endif

// ...later in some code....
#ifdef VENDOR_A
    vmeta.foo = 42;
#endif

@thomascalvert-xlnx My thinking was that each vendor would have their own customized version of the pna.p4 include file.  There is already a section in psa.p4 and pna.p4 include files that explicitly says that vendors are expected to customize the bit widths of certain typedef's for things like port ids, multicast group ids, etc.
I will create a separate issue in Github to track the even more explicit documentation of where we expect vendors to modify pna.p4 for themselves, vs. where we do not expect them to.

Here is a new Github issue to track the idea of documenting precisely how pna.p4 include files are expected to differ across different targets, vs. which parts we expect to remain the same.


Remove leading slash on Projects API

I removed leading slash on Projects API
reasons

unify with others
constructing url does not work when in GitHub Enterprise environment

LGTM.
Awaiting second LGTM before merging.

Thanks for catching and fixing this. It was an unintended oversight in #467.
LGTM.

Thanks, @shurcooL. Merging.


misleading diff for include matcher

Code:
[{:foo => 1}].should include({:foo => 1}, {:bar => 2})

Error:
Failure/Error: [{:foo => 1}].should include({:foo => 1}, {:bar => 2})
  expected [{:foo=>1}] to include {:foo=>1} and {:bar=>2}
  Diff:
  @@ -1,2 +1,2 @@
  -{:foo=>1}
  +[{:foo=>1}]

I would expect {:bar => 2} to be mentioned in the diff. For larger arrays the current behaviour can make the failure very hard to spot.

What version are you using?

From my Gemfile.lock:
rspec (2.7.0)
  rspec-core (~> 2.7.0)
  rspec-expectations (~> 2.7.0)
  rspec-mocks (~> 2.7.0)
rspec-core (2.7.1)
rspec-expectations (2.7.0)
  diff-lcs (~> 1.1.2)
rspec-mocks (2.7.0)

rspec -v reports 2.7.1.

Odd. This is what I see:
$ gem list rspec

*** LOCAL GEMS ***

rspec (2.7.0)
rspec-core (2.7.1)
rspec-expectations (2.7.0)
rspec-mocks (2.7.0)

$ rspec example_spec.rb
F

Failures:

  1) something does something
     Failure/Error: [{:foo => 1}].should include({:foo => 1}, {:bar => 2})
       expected [{:foo=>1}] to include {:foo=>1} and {:bar=>2}
       Diff:
       @@ -1,2 +1,2 @@
       -[{:foo=>1}, {:bar=>2}]
       +[{:foo=>1}]
     # ./example_spec.rb:3:in `block (2 levels) in <top (required)>'

What version of diff-lcs? I've got 1.1.3.


archive.org s3 api Support please

Please set bucket-name, s3-access-key, and s3-secret-key in config.php added s3 api from archive.org/account/s3.php and changed the bucket url in config.php to .s3.us.archive.org any help thanks
billyprice1

I don't understand what you're referring to here. Can you explain?

How do i intagre this with archive.org s3 api https://archive.org/help/abouts3.txt here is an example of  the s3 for arcive.org noobkilervip.s3.us.archive.org
https://archive.org/~vmb/abouts3.html
Thanks
Billyprice1

Hmm, I'm not sure it would work with that S3-like API since this tool expects to be able to get a full list of all the files in an S3 bucket before rendering a page. The archive.org dataset is so big that even if they exposed a full list of files it would be too big to process.
You'd have to troubleshoot the the setup of this yourself to be sure. If you run into more specific issues you can reopen this issue or create new ones. GL!

Thanks i work on the end point i just found out that it works on http://ibilly-noobkilervip.codeanyapp.com/s3/ with the same api i just rewiere you code and publish it on github to help others and souce you if thats ok

Of course. It's MIT licensed so very open: https://github.com/powdahound/s3browser/blob/master/LICENSE


Fix some texts

Description
Fix text
Checklist

[ ] I have read the contribution guidelines
[ ] I have coordinated with the docs team<EMAIL_ADDRESS>if this PR deletes files or changes any file names or file extensions.
[ ] This notebook or file is added to the CODEOWNERS file, pointing to the author or the author's team.

This PR appears to contain an unresolved merge conflict.
#122 appears to contain a similar changeset to this PR, so I'm assuming that PR supersedes this one.


Sample/experiment mocking library using GraphQL Tools Mocking

Sample/experiment mocking library using GraphQL Tools Mocking

This thing looks good so far.
Some questions / things to consider:

[ ]  1. how do we live update (e.g. between tests or to test different data scenarios when manually testing the app?)
[ ]  2. can we serialize/deserialize the mock store?

1. live update
imagine we have the server running, we need to reset the store between tests + then add test-specific data; what does that look like?
Similarly, what's our strategy for handling sets of collections of data? e.g if a user wants to switch the store to use "minimal data" vs "lots of data" without having to specify a mock for each type in the schema, what does this look like?
2. serialize/deserialize
There are two reasons to think about this:

performance -- can we achieve the ergonomics of a factory pattern but with the performance of pre-built fixture files?
non-web platforms -- assume our android users want a native experience where they build mocks using java builder APIs -- can they serialize to some json format that we can load for them to use in testing?

@hjdivad There are two pieces to this:

The underlying mocking library that can be used directly by consumers (e.g., as part of a Node/browser test runner)
The mock server piece; this will be built on top of the library itself, and the server will leverage it to respond to GraphQL requests with optional fixture data, some communication strategy from consumer to server, etc.

This PR is really only intended to be a demonstration/example of what (1) might look like; there will be future iteration to demo what (2) may look like.


option to use an existing certificate

We would find it useful to be able to pass in the self_link of an existing certificate to lb-http.  This would allow us to create the certificate outside of terraform to avoid having key material in the tfstate file.
I've implemented this feature in our fork: https://github.com/fastly/terraform-google-lb-http/pull/1
To my understanding, Fastly already has a corporate CLA, and I've been added to it.  If this all looks good to you, I'll create a PR.

@lexelby Maybe make a pull request from that Fastly repo to this repo?

Sure, I'd be happy to. But CONTRIBUTING.md says to open an issue and wait for a maintainer to ask me to submit a PR if they like the idea.

Ah, my bad.  Thanks for sharing this.

@lexelby if you don't mind me asking, how do you get a reference to the self_link of an existing certificate?

@eltimn Like this:
gcloud compute ssl-certificates list --project <project ID> --uri

Excellent. Thank you!

@danisla @danawillow Code for this is working and tested in our environment.  How can I help you folks to get this moving?  My employer (Fastly) has signed a CLA that includes me, so that shouldn't be a stumbling block.

This is great, please create a PR and I'll get it merged.

@danisla created: https://github.com/GoogleCloudPlatform/terraform-google-lb-http/pull/19
Interestingly, #18 seems to also have the same bring-your-own-cert functionality.

Fixed with #16


Auth flow improvements

Pull Request Description
Fixes #7447

Change "login to X" to "sign up or login to X"
Trigger validation on every input
Fixes #7449
Change "offline" to "not signed in"
Add "login" button to cloud backend

Important Notes
Feel free to suggest improvements to the appearance.
Checklist
Please ensure that the following checklist has been satisfied before submitting the PR:

[x] The documentation has been updated, if necessary.
[ ] Screenshots/screencasts have been attached, if there are any visual changes. For interactive or animated visual changes, a screencast is preferred.
[x] All code follows the
Scala,
Java,
and
Rust
style guides. In case you are using a language not listed above, follow the Rust style guide.
All code has been tested:

[x] Unit tests have been written where possible.
[x] If GUI codebase was changed, the GUI was tested when built using ./run ide build.

@jdunkerley you may want to test out this PR

The text changes and new log in link look great.
The validation on every character typed is too aggressive, I think something like:

on defocus of a text box
after a brief timeout following a keypress (so we don't validate as the user types everything but only when they pause).

The message for the confirm box should be updated as well. Instead of "Please match the format requested" should just be "Passwords do not match.".

@jdunkerley i think there's no way to avoid the "please match the format requested" message, because that is a system notification generated by chrome. we could have custom error dialogs in HTML instead - what do you think?
as for the validation stuff - yeah that's a good idea, will do.

No need to change the format for the confirm we can just live with that error.


Can add the StaticCustomWithFilters function?

Please forgive my request because the project(static file server) needs ip whitelist and add Cache-Control header.
Atreugo's static file functions are not middlewares and filter.
I am going to write a StaticCustom function with filter, but find i can't use the router.ServeFilesCustom().

Yes, the static files hasn't middlewares and filters.
I will add it in a few days.

I've just added Filters option inside StaticFS.
Now static files will process the global middlewares and allow filters ;)

Thanks!
Yesterday I have implemented the demand with RequestHandlerPathWithFilters.
server.RequestHandlerPathWithFilters("GET", "/default/*filepath", staticFile, filters)
func staticFile(ctx *fasthttp.RequestCtx) {
	ctx.Response.Header.SetCanonical([]byte("Cache-Control"), []byte("public, max-age=3600"))
	fs := &fasthttp.FS{
		Root:               "/data/wwwroot/default",
		IndexNames:         []string{"index.m3u8"},
		GenerateIndexPages: false,
		Compress: true,
		AcceptByteRange: false,
		PathNotFound: noFile,
	}
	fsHandler := fs.NewRequestHandler()
	fsHandler(ctx)
}

This method of RequestHandlerPathWithFilters is very useful. Can use this method to achieve upload
server.RequestHandlerPathWithFilters("POST", "/upload", UploadHandler, filters)
// UploadHandler is here
func UploadHandler(ctx *fasthttp.RequestCtx) {
	data, err := ctx.MultipartForm()
	if err != nil {
		ctx.SetStatusCode(500)
		fmt.Println("get upload file error:", err)
		return
	}
	fileObj := data.File["uploadfile"][0]
	err = fasthttp.SaveMultipartFile(fileObj, fileObj.Filename)
	if err != nil {
		ctx.SetStatusCode(500)
		fmt.Println("save upload file error:", err)
		return
	}
	ctx.Write([]byte("save file successfully!"))
}

I recommend you to use StaticCustom with filters, It's easier to use and avoid extra allocation, increasing performance

See  this example: https://github.com/savsgio/atreugo/tree/master/examples/static_files

avoid extra allocation, increasing performance
I am a newbie, In the follow code, the Path is:
server.RequestHandlerPathWithFilters("GET", "/default/*filepath", staticFile, filters)
But in the StaticCustom method, the Path is:
r.RequestHandlerPathWithFilters("GET", url+"/*filepath", ffs.NewRequestHandler(), fs.Filters)
My understanding should be the same, Where is the difference?

I have already seen https://github.com/savsgio/atreugo/tree/master/examples/static_files
This StaticCustom method is  also finally passed the RequestHandlerPathWithFilters method add to router

Here, you are creating a new FS (extra allocations, it's better preallocating after and use the request handler of FS) on every requests:
func staticFile(ctx *fasthttp.RequestCtx) {
	ctx.Response.Header.SetCanonical([]byte("Cache-Control"), []byte("public, max-age=3600"))
	fs := &fasthttp.FS{
		Root:               "/static",
		IndexNames:         []string{"index.html"},
		GenerateIndexPages: false,
		Compress: true,
		AcceptByteRange: false,
		PathNotFound: noFile,
	}
	fsHandler := fs.NewRequestHandler()
	fsHandler(ctx)
}

You should move this two lines outside.
fs := &fasthttp.FS{
		Root:               "/static",
		IndexNames:         []string{"index.html"},
		GenerateIndexPages: false,
		Compress: true,
		AcceptByteRange: false,
		PathNotFound: noFile,
	}
	fsHandler := fs.NewRequestHandler()

In StaticCustom you create a new FS, but just be added the fsHandler in view path.
Something like this:
fs := &fasthttp.FS{
	Root:               "/static",
	IndexNames:         []string{"index.html"},
	GenerateIndexPages: false,
	Compress: true,
	AcceptByteRange: false,
	PathNotFound: noFile,
}
fsHandler := fs.NewRequestHandler()

func staticFile(ctx *fasthttp.RequestCtx) {
	ctx.Response.Header.SetCanonical([]byte("Cache-Control"), []byte("public, max-age=3600"))
	fsHandler(ctx)
}

In Atreugo
filters := atreugo.Filters{
	After: []atreugo.Middleware{func(ctx *atreugo.RequestCtx) (int, error){
                    	ctx.Response.Header.SetCanonical([]byte("Cache-Control"), []byte("public, max-age=3600"))

              }
         },
}

server.StaticCustom("/mypath", &StaticFS{
        Filters: filters,
	Root:               "/static",
	IndexNames:         []string{"index.html"},
	GenerateIndexPages: false,
	Compress: true,
	AcceptByteRange: false,
	PathNotFound: noFile,
})

I understand, I made a mistake that should not be made.
Thank you very much!


Adding generator meta tag to provide version feedback

Might be helpful for

troubleshooting some issues
following the spread of cleaver over internet :D
indicating which version of cleaver to rollback to in case current version yields different output than expected

The way it's done means that each theme would need to output {{version}} thingy on their own.
Using generator name from spec for this.

Great idea! Wanna merge this?

Sure!

🎉 🎉 🎉 🎉


Modified the package name from "com.example.budgy" to "com.budgy". Re…

…factored reference to the package name.

#15


STATUS_PASSWORD_MUST_CHANGE is a valid password

Show password as not successful but the password is !
CME doesn't consider as valid a password that need to be changed.
MSF does as you can find here: https://github.com/rapid7/metasploit-framework/blob/e04cf24b47434448179575d4cabbf2781054f372/lib/metasploit/framework/login_scanner/smb.rb#L38
To Reproduce
Steps to reproduce the behavior:

pip env run cme smb -u user.lst -p password.lst
"Starting enumerate"
The error message is different but still shown as an error (RED)
CME continue
The must change password is missed by the user

Expected behavior
Whenever CME encounter a login attempt with a status 'STATUS_PASSWORD_MUST_CHANGE', it should display it as green or orange at least.
Crackmapexec info

OS: Kali
Version: Latest
Installed using git

It might be worth checking if it applies for only this status or others in this list:

"STATUS_ACCOUNT_DISABLED"
"STATUS_ACCOUNT_EXPIRED"
"STATUS_ACCOUNT_RESTRICTION"
"STATUS_INVALID_LOGON_HOURS"
"STATUS_INVALID_WORKSTATION"
"STATUS_LOGON_TYPE_NOT_GRANTED"
"STATUS_PASSWORD_EXPIRED"
"STATUS_PASSWORD_MUST_CHANGE"

Interesting ! I will check this :)

Hello guys, what do you think of this ?

J'ai aussi ajouter le status STATUS_ACCESS_DENIED

Look awesome ! Nice