V2 - contributor role should have view access to CLA end points similar to other non cla-manager roles in cla-service.yaml

Background
As per the details provided in Access column for Corporate Contributor persona in below PRD, similar to other non cla-manager roles, view access policies should be applied for contributor role also in cla-service.yaml
https://confluence.linuxfoundation.org/display/PROD/EasyCLA+PRD+for+Version+2

Updated ACS CLI permissions to allow the following policies:

View Templates
Assign CLA Manager Designee
Create CLA Manager Request
View CLA Enabled/Summary Pages
View Company Project CLAs

which are now applied to the following roles:

user
community-program-manager
lf-executive
contributor
contact
lead
donar
lf-training

all the other CLA roles (CLA Manager, CLA designee, CLA Signatory, Program Manager, etc.)

@nirupamav ready for review

@dealako
Contributor should have ready only access to all the endpoints in Project Dashboard and Foundation dashboard
ViewCompanyCLAManagers
ViewCompanyActiveCLAs
ViewCompanyProjectContributors
ViewProjectCompanySignatures
Below is the reference
https://confluence.linuxfoundation.org/display/PROD/Corporate+Console+for+EasyCLA+V2

@nirupamav based on the configuration, the above-mentioned profiles should apply to Contributors:

ViewCompanyCLAManagers
ViewCompanyActiveCLAs
ViewCompanyProjectContributors
ViewProjectCompanySignatures

See the acs-cli configuration.

Verified that contributor has access to below endpoints
https://api-gw.dev.platform.linuxfoundation.org/cla-service/v4/api-docs#operation/getCompanyProjectActiveCla
https://api-gw.dev.platform.linuxfoundation.org/cla-service/v4/api-docs#operation/getCompanyProjectClaManagers
https://api-gw.dev.platform.linuxfoundation.org/cla-service/v4/api-docs#operation/getCompanyProjectEvents
https://api-gw.dev.platform.linuxfoundation.org/cla-service/v4/api-docs#operation/listCompanyProjectMetrics
https://api-gw.dev.platform.linuxfoundation.org/cla-service/v4/api-docs#operation/getProjectCompanySignatures
https://api-gw.dev.platform.linuxfoundation.org/cla-service/v4/api-docs#operation/getCompanyProjectContributors
https://api-gw.dev.platform.linuxfoundation.org/cla-service/v4/api-docs#operation/createCLAManagerDesignee
https://api-gw.dev.platform.linuxfoundation.org/cla-service/v4/api-docs#operation/createCLAManagerRequest


MkdirAll and RemoveAll are missing from filesystem abstraction

It's important to keep the filesystem interface as small as possible, therefor both convenience methods are part of vfs and use Remove/Mkdir of the given filesystem:
https://godoc.org/github.com/blang/vfs#MkdirAll
https://godoc.org/github.com/blang/vfs#RemoveAll

Doesn't this kind of defeat the purpose?  If I can't substitute os.xxx calls with an implementer of vfs.Filesystem how does this help?  So you thing this:
someFS := vfs.OS()
...
someFS.Mkdir(xxx)
...
vfs.MkdirAll(someFS, xxx)
as opposed to:
os.Mkdir(xxx)
...
os.MkdirAll(xx)
Is proper?  You've changed the interface (MkdirAll doesn't take a filesystem parameter and are now directly affecting the code to use something incompatible.  I don't see the benefit here.

Thanks for this, i thought you missed these methods in the main package.
Yep i'm aware of this issue and i discussed this already with others and it's basically about the balance between a small filesystem interface and compliance. Since the stdlib is not at all designed for creating a good abstract filesystem interface, no replacement like vfs will be a simple drop-in replacement i'm afraid.
The consens to create a good filesystem abstraction is to have an interface as small as possible to support multiple backends like memfs, zipfs and others but also provide the convencience methods like MkdirAll everybody is used to. It's a big problem to decide which methods should be included, for example Link, Symlink, Chmod are not included in the current interface but maybe should. On the other hand, most filesystems (if we are talking about abstract filesystems) do not support/need this features.
The current goal of this project is to provide a stable (but small) interface to the most common filesystem operations and support the most common and useful filesystems.
I'm already working on / thinking about more convenience for the user. One example might be to create a bigger interface in front of interfaces like BaseFilesystem, Linker, Chmoder (example names) to support every possible call through one interface, with any unsupported operation returning an error:
vfs.Create(fs,...)
vfs.OpenFile(fs,...)
vfs.Chmod(fs, ...)
vfs.Link(fs,...)

If you're interested please send further feedback

If the intent is to abstract Go's filesystem calls (which aren't part of an official interface unfortunately - quite a lot isn't.. :<) then you need to wrap all of the file calls, not just a couple.  Otherwise it defeats the purpose.   I should be able to have a single filesystem variable that is either a vfs.OS() or vfs/memFS instance depending on whether I'm testing or not.  I could define that as a global or pass it around using it as my new 'filesystem' interface.
If I have to change 'some' os.xxxx calls to instead call vfs.xxxx then the library isn't an abstraction at all, it's its own new library with its own interface to use instead of the stdlib.
At that point, it's no longer serving it's intended (IMO) purpose.

I don't exactly get your point.
At any point in time you only need one variable, let's call it fs.
fs might be an vfs.OsFS or an vfs.memfs.MemFS ( but never os as in stdlib/os, reason for confusion?)
There's no difference between how you interact with these filesystems. The only difference is, that if you wan't to create recursive directories or use other convenience methods like Create, you can't call fs.MkdirAll() but have to call vfs.MkdirAll(fs, ...). But these calls are the same on every filesystem implementing the interface vfs.Filesystem. The stdlib package os does not implement any interface and therefor you need to use vfs.OsFS.
So you have a single variable containing your filesystem, which can be passed around.
Maybe you think of the advantage to use os as your filesystem variable instead of the import namespace os from stdlib, that's a interesting point but would not work either way since you need os.O_RDWR (e.g) constants from this package.

If I want to change my code to use vfs, then I have to change 'some' os.xxx calls to use 'someVar.xxxx' with the same method signature as os.xxx - ie os.Mkdir.
(I would expect to do that for ALL os filesystem calls)
I also have to change 'some' os.xxxx calls to instead be vfs.xxxx(someVar, ...), ie: os.MkdirAll
That is not a proper abstraction of the os package filesystem calls.
So given my first message, I expect to translate this:
os.Mkdir(xxx)
os.MkdirAll(xxx)
to this:
someFS := vfs.OS()
...
someFS.Mkdir(xxx)
someFS.MkdirAll(xxx)
NOT to this:
someFS := vfs.OS()
...
someFS.Mkdir(xxx)
vfs.MkdirAll(someFS, xxx)
With that version, I think this is the worst of all worlds.  The library just created more work for me, not less and has just created an entirely new api with new method signatures.

Sry to hear that.
I honestly don't think that using the 3 calls to vfs.X currently is that much of a dealbreaker. I think more important are missing methods like Chmod but your experience might differ.
As i've written in my previous comment there might be a way around if i will introduce a bigger more complete interface. That would solve your problem only slightly since all calls will follow the format vfs.X(fs,...).
But it's pretty sure that the vfs.Filesystem interface will not include further convenience methods in the future, so someFS.MkdirAll will not exist.
All methods transfered to the vfs package are "convenience" methods which will be the same on every filesystem.

Yes, your prior comment would be something valuable,
I guess I just don't understand your real goal here with what you currently have.
Trying to have a bare minimum interface is not the same as abstracting an existing interface,  Clearly not all implementations will have equivalents but then they're either no-ops or errors when you call them.  To instead define a 'new' interface that is the lowest common denominator, I just don't really understand that.  I guess I'm coming at it purely from a testing perspective with wanting to test apis that make filesystem calls and not have them use the real filesystem at test time.  So basically mocking it out but with the ability to also verify that I can 'read' back what was supposed to be written.

It simple, vfs does not abstract the os.X Filesystem but a filesystem in general. Yap, i guess everthing has been discussed so far.  You can watch this repo to get notified if something gets changed

Gotcha.  It sounds like spf13/afero is more like what I'm looking for.  Thanks for your time!

This is very helpful thread for people comparing github.com/blang/vfs and github.com/spf13/afero. Thank you both! 👍


Fixed incorrect hardcoded RenderTexture identifier

The RTHandle's texture identifier is hard-coded to _GrabbedTexture, and cannot actually be altered using the "Grabbed Texture Name" field. Was there a reason for this? Fixed it anyways, multiple grab passes can be added now.

thanks!


Data model constant not initializing in production

This is the source of the papertrail related error messages.

Still flaking out in development, gets reset to just 'Identifier' some times.  Tests not capturing issue.


Bump requirements files && codecov action to 1.5.2

TL;DR
Bump requirements and version of codecov-action
Type

[x] Bug Fix
[ ] Feature
[ ] Plugin

Are all requirements met?

[ ] Code completed
[ ] Smoke tested
[ ] Unit tests added
[ ] Code documentation added
[ ] Any pending items have an associated Issue

Complete description
The codecov github action version 1.5.2 fixed a bug that shows up in flytekit PRs very frequently. This PR bumps requirements across the board and also also the version of the codecov github action.
Tracking Issue
https://github.com/flyteorg/flyte/issues/
Follow-up issue
NA
OR
https://github.com/flyteorg/flyte/issues/

great expectations still failing but we can revert later right?  back to 0.15.6


Wrong h4 heading in issue note

Hello, @mrliptontea
h4 heading in issue note is something wrong.
Could you please fix it?

Hi! Thanks for spotting that. This definitely looks good but can you check/confirm if markup hasn't changed in Redmine 2.6+?

Hi, @mrliptontea
I have confirmed app/views/issues/_history.html.erb between Redmine 2.6 and the latest one.
There seems to be no change in markup tag hierarchy and order.
https://gist.github.com/taqueci/4d6b7ccaf0e1bcf2d1f8d4f19ec09ab2/revisions

Good job! If you could push a compiled CSS too (so theme works out of the box), that'd great and I could  merge it then.


Fix windows error when using ExecProvider

Popen expect a list of strings for non-shell calls and a string for shell calls.
Solution: run subprocess.Popen() with shell = True .
Related error: WindowsError: [Error 2] The system cannot find the file specified
Related issue: https://bugs.python.org/issue17023

/cc @micw523


Add a copy method

closes https://github.com/pavanramkumar/pyglmnet/issues/59
I am not entirely sure if @property is the right approach to addressing @hugoguh 's second issue. So, I won't make that change until we're sure what's the right way to do it.
travis failure is unrelated to PR

ok, let’s discuss whether we might want a copy method?

Possible uses: user might want to try several alphas, or adjust range of lambdas.
Cons:
the user can just do the following if they want to add a copy:
(in this case for instance if shuts down verbose)
model = GLM(distr='poisson', verbose=True, alpha=0.05, reg_lambda = 1)
model.fit(Xr,yr)
model_another = GLM(distr=model.distr, verbose=False, alpha=model.alpha, reg_lambda = model.reg_lambda)

note also that in this way model_another doesn’t copy model.fit_ which happens when using model_another = model[:]
Pros: a copy method would make syntax simpler (any speed issues also?).

Now independently of having a copy method. What really shouldn’t be allowed is to change alpha, distr or reg_lambda of a model without deleting the .fit_.
Either don’t allow model.alpha = 0.05, or when it happens delete model.fit_


Features file or roadmap

Would be great to have some file that has features that are in progress or planned to be added for the project. I remember there was one a long time ago when I first found this project, but I do not see it anymore.
There are some interesting templates that could be added to this project like for example adding the t3-app with trpc, or a setup with firebase and trpc that in the components page lets you set up cron jobs files, or function triggers.

Hi, are you talking about the project section ?
At this moment there is only the upcoming-features (where for example being able to pick a specific template is mentionned!) but for sure having a proper roadmap is important and I can work on this very soon.


scylla_node: set default smp count to 2

This is how we run dtests by default
so it gets really confusing for users who don't provide --smp in SCYLLA_EXT_OPTS

If they care about smp, they should set it in the jvm_args

With this change, I'd like to remove the default setting of smp and memory in
https://github.com/scylladb/scylla-dtest/blob/7f666e8bc85efa37a6820eb25db8692405ef7794/scripts/run_test.sh#L98
and
https://github.com/scylladb/scylla-pkg/blob/66e6297ad43f6e707b967f3718d714aaa59a7708/scripts/jenkins-pipelines/shell_scripts/dtest.sh#L430

If they care about smp, they should set it in the jvm_args or SCYLLA_EXT_OPTS

yeah but if they don't, we are changing the default under "their" feet, might break some tests.
But I'm o.k. with letting it break, the fixes in that case should be easy, to pin it back in the test code, and not CCM


Kubevela not updating Kubernetes objects

Describe the bug

The bug observed is in two parts:
Part one:
Changes done to a helm chart for postgres in my Github repo reflects in the kubevela application (/Application/my_env/my_app auto sync is active i.e. label: app.oam.dev/source-of-truth=from-k8s-resource), but does not get deployed to my Kubernetes cluster (/local/postgresql/my_env/my_app). Using the deploy button in Kubevela to propagate the change works just the first time. Currently, no further updates can be done to the postgres pods even when the Deploy button is hit again.
TL;DR: Updates made to the Git repo helm chart always show up in the kubevela application, but does not get deployed to the cluster.
Part two:
The pods did not get automatically rescheduled after hitting deploy the first time (from Part one above) and I had to delete the pods before they got rescheduled with the new changes.
To Reproduce

Setup a GitHub repo and make a helm chart with the template below in it:

apiVersion: core.oam.dev/v1beta1
kind: Application
metadata:
  name: run-database
  namespace: run-staging
  annotations:
    app.oam.dev/publishVersion: {{ .Chart.AppVersion }}
    "helm.sh/resource-policy": keep
spec:
  components:
    - type: "postgres-cluster"
      name: run-database
      properties:
        replicas: 3
        volume:
          size: "500Gi"
        resources:
          requests:
            cpu: "4000m"
            memory: "16Gi"
          limits:
            cpu: "4000m"
            memory: "24Gi"
        postgresql:
          parameters:
            max_connections: "1000"
            superuser_reserved_connections: "5"
            huge_pages: "try"
            max_wal_size: "6GB"
            max_locks_per_transaction: "256"
            max_pred_locks_per_transaction: "256"
            shared_buffers: "2GB"
            wal_buffers: "256MB"
            work_mem: "32MB"
            maintenance_work_mem: "512MB"
        ports:
          - expose: true
            port: 5432
            protocol: TCP
        preparedDatabases:
          {{- range $index, $network := .Values.networks }}
          sync_{{ $network }}:
            defaultUsers: true

          {{- end }}
      traits:
        - type: annotations
          properties:
            "helm.sh/resource-policy": keep

Deploy it as a Kubevela application with autosync active i.e. label: app.oam.dev/source-of-truth=from-k8s-resource should be present on it.
Update the memory and cpu requests/limits in the template from step 1 above.
Expected behavior

The Postgres database cluster should get deployed and its pods automatically rescheduled with the new memory and cpu requests/limits specification.
KubeVela Version
CLI Version: UNKNOWN
Core Version: 1.8.0
GitRevision: UNKNOWN
GolangVersion: go1.19.6

Cluster information

AWS EKS
Kubernetes version 1.23

What's the status of this application now?

If you set the publishVersion annotaion, the application controller would not apply the resource to latest status automatically.


Specified log4net version 2.0.3 in nuspec

The csproj references the dll version 1.2.13 of log4net using nuget package versione 2.0.3, but the nuget dependencies point to the last version which is at the moment 2.0.4 using the dll 1.2.14.
Specified the log4net version in nuspec to avoid possible assembly binding errors.

Per PR #107, Log4Net is no longer a dependency.


Added custom array example with documentation.

Added example code with a comprehensive Readme that covers the bulk of the custom array concepts and how they relate to each other.
In response to discussion: https://github.com/diesel-rs/diesel/discussions/4157
However, the Readme needs review and the tests need a fix for a serialization error.

Quick update:

Tests are fixed
Readme got edited and checked
Added the clippy lint check to the GH action so it runs together with all the others on CI

Only thing I could not figure out was how to add the tests to CI.
Should be straight forward to review this PR.

Only thing I could not figure out was how to add the tests to CI.

The tests are automatically picked up: https://github.com/diesel-rs/diesel/actions/runs/10411801607/job/28836874448?pr=4169#step:27:356. I'm not sure why they fail but I can have a look at them next week if you want.

@weiznich Thank you so much for the substantial commentary. T
he bulk of your comments are relatively easy to fix. I still trying to figure out the details about how Diesel handles custom schema, so your comments here are invaluable and I most likely will update the Readme  once I updated the code so that everything resides in the custom schema. Still learning...
I have fixed the failing tests. Apparently, the CI picks up the test automatically and I simply wasn't running a migration prior to test execution. Once I added an embedded migration, the test seems to pass for now.
On that topic, I also added a section about Integration testing to the Readme to complete the document in terms of covering all the important topics.
Moving the content to the Website would be great!
I look into all the details over the weekend and update the PR

Update:

Changed dotenv to dotenvy; (see comment)
Moved Endpoint impl into type definition file. (See comment)
Switched example code and tests from PGConnectionPool to single PGConnection. (See comment)
Moved all custom types into custom schema smdb. See comment
Added a DatabaseError to PgProtocolType in from_sql implementation to convey that the DB contains something that isn't expected. See comment.

Updating the Readme next.

Okay, the Readme and tests are all updated with all suggestions being resolved.
The remaining two conflicts are:
.github/workflows/ci.yml
diesel/src/pg/query_builder/copy/copy_to.rs
The first one is probably that I have added the example to the ci config.
The second refers to a suppressed linter I have added to make clippy on CI pass.
Please check if either one is okay, and if not, just change whatever needs to be changed.
Lastly, I really don't know why so many CI jobs got stuck in limbo. I've read on HN that GH had a major outage recently, but really no idea what's happening there.
Either way, please double check the Readme and if there are no major objections, feel free to merge anytime.

Thanks for the update :+1: I've rebased the PR to resove the merge conflicts. I'm now going over the changes to perform another review.

Thanks a lot for the consistency checks as I didn't had the time to do it. Overall, I already committed the bulk of corrections.
I want to take a closer look at your suggestion to encapsulate tests into a transaction. I get the idea of basically piggy backing on the two phase commit mechanism inherent to transactions. I just have to write and run some sample tests to see how this works out in practice.
I think if that works out, it's sensible to update the test suite and the Readme.
Background is, I'm building and testing my Rust mono repo with Bazel and my CI runs already hundreds of tests in parallel. However,  for DB integration tests at scale, the aborted transaction trick may work out and speeds things up further.

Thanks a lot for the consistency checks as I didn't had the time to do it. Overall, I already committed the bulk of corrections.
I want to take a closer look at your suggestion to encapsulate tests into a transaction. I get the idea of basically piggy backing on the two phase commit mechanism inherent to transactions. I just have to write and run some sample tests to see how this works out in practice.
I think if that works out, it's sensible to update the test suite and the Readme.
Background is, I'm building and testing my Rust mono repo with Bazel and my CI runs already hundreds of tests in parallel. However,  for DB integration tests at scale, the aborted transaction trick may work out and speeds things up further.

@weiznich
Update summary:

Readme was updated with all those minor corrections you have flagged
Updated run_db_migration in lib by removing unnecessary check
Added revert_db_migration to lib; it's used to cleanup the DB after sequential testing.
Added parallel test suite using test transaction, as you laid out in a previous comment.
Updated Readme section on testing.

Just one question: Does the test transaction aborts automatically when it goes out of scope?
I did not find any abort or rollback method, but somehow all tests run flawlessly and I wonder why?
Either way, please review these changes and if there
is nothing in the way, feel free to merge whenever ready.

Thanks for the update. I will try to do a final review (and likely some additional minor adjustments based on that) this week. Otherwise I think this is fine now.

Just one question: Does the test transaction aborts automatically when it goes out of scope?
I did not find any abort or rollback method, but somehow all tests run flawlessly and I wonder why?

It relays on the fact that you drop the connection itself at the end of the test. This will also drop any uncommited changes for that connection.

@weiznich
Thank you for the detailed corrections and the links to the docs in the Readme. Also, thank you for getting the parallel tests in shape. While things worked locally on my box, apparently on CI each test really needs an unique connection.
It was my first try with the  dangling transaction test style so there is stil a bit of learning ahead. Anyways, just want to thank you for your time and effort to get this PR done.

While things worked locally on my box, apparently on CI each test really needs an unique connection.

The problem there was that you likely had already applied the migrations manually. The tests run against an empty database and postgres do not allow to modify the schema from more than one connection at once. That's worked around by having the schema modification itself in the test transaction as well. So it doesn't get commited and there are no problem with concurrent access.
Thanks again for writing this up. I think that's now a good starting point for at least two guides: One for custom types and one for testing. Now I only need to find the capacity to do the last bit of polishing to put it on the web page.

As for the guide, I don't have the capacity to do that as my project progresses, but it might be an option to open an issue with a call for contribution with some guidance and some links to an example. This would be a great first PR for new contributors.

I'm already really grateful that you wrote this PR. I will find the time at some point for the necessary work to turn this into a full guide, I just wanted to manage expectations that this likely won't be the case tomorrow.


Added description of XMLAPI: corp/Standings.xml.aspx

As a side note: Do you prefer a pull request per single api call/formula/isolated subject, or do you prefer a "pack of api calls"? (for example a 5 "corporation" api changes)

You can do a single pull request with many endpoints, but it would be highly preferred if you could make each endpoint a separate commit.

As polarina said it's either a PR per endpoint or one PR with each endpoint as a separate commit. And the reason is that it makes code reviewing easier.

Does this PR need any more work? Can we merge it?

Looks good. Sorry about that, it got lost in my notes to review it. :)


feat(dbbrowser): Optimize the visualization of MySQL table structure design,supporting strings enclosed in single quotes ('').

What type of PR is this?
type-bug
dbbrowser
What this PR does / why we need it:
Optimize the visualization of MySQL table structure design, supporting strings enclosed in single quotes ('').
Which issue(s) this PR fixes:
Fixes #1317
Special notes for your reviewer:
have not frontend change need。
Additional documentation e.g., usage docs, etc.:
none

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.冯浩 seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


DOC: add responsive.css

What's wrong with the existing documentation
Styles specifically for different screen sizes and devices.
Add ScreenShots
No response
Record

[X] I agree to follow this project's Code of Conduct
[X] I'm a GSSOC'24 contributor
[X] I want to work on this issue

please assign me as soon as possible i want to complete the task

please assign me the issue


Problem with grpc in external function

Sorry this sentence is from google translate.
I have a problem with grpc using external functions.
The content of the proto file is as follows：
string name = 3373707;
int32 size = 3530753;
double resolution = 526288726;
string xUnit = 113463836;
string yUnit = 114387357;
string zUnit = 115310878;
repeated double xValue = 240178216;
repeated double yValue = 211549065;
message ListOfListsOfDoubles {
repeated ListOfDoubles listOfDoubles = 1;
}
message ListOfDoubles {
repeated double doubles=1;
}
ListOfListsOfDoubles ZValue = 9;
ListOfDoubles ZValueDoubles = 10;
ListOfListsOfDoubles DValue = 11;
ListOfDoubles DValueDoubles = 12;
The returned data is as follows:
{
"code": 200,
"data": {
"ZValue": {
},
"ZValueDoubles": {
},
"DValue": {
},
"DValueDoubles": {
}
},
"success": true,
"message": "Test request succeeded"
}
As long as there is no ListOfDoubles in ZValue, ekuiper will report the following error:
goroutine 16 [running]:
runtime/debug.Stack()
C:/Program Files/Go/src/runtime/debug/stack.go:24 +0x65
runtime/debug.PrintStack()
C:/Program Files/Go/src/runtime/debug/stack.go:16 +0x19
github.com/lf-edge/ekuiper/pkg/infra.SafeRun.func1()
E:/Go/ekuiper/pkg/infra/saferun.go:32 +0x45
panic({0x17ee9c0, 0x1ad7ad0})
C:/Program Files/Go/src/runtime/panic.go:838 +0x207
reflect.Value.Index({0x17d8ac0?, 0x2136700?, 0x0?}, 0x0?)
C:/Program Files/Go/src/reflect/value.go:1366 +0x16d
github.com/lf-edge/ekuiper/pkg/cast.ToTypedSlice({0x17d8ac0, 0x2136700?}, 0xc00024cc80, {0x1965bb8, 0x3}, 0x6d?)
E:/Go/ekuiper/pkg/cast/cast.go:831 +0x133
github.com/lf-edge/ekuiper/internal/service.decodeField({0x17d8ac0?, 0x2136700?}, 0xc0003a5900?, 0x8?)
E:/Go/ekuiper/internal/service/schema.go:574 +0x3a5
github.com/lf-edge/ekuiper/internal/service.decodeMessageField({0x17d8ac0?, 0x2136700?}, 0xc0003a5900, 0x2c?, 0x0?)
E:/Go/ekuiper/internal/service/schema.go:519 +0x33
github.com/lf-edge/ekuiper/internal/service.decodeMessage(0x1adebd0?, 0xc000425950)
E:/Go/ekuiper/internal/service/schema.go:512 +0xef
github.com/lf-edge/ekuiper/internal/service.decodeSubMessage({0x1964220?, 0xc0004d2c30?}, 0x1964220?, 0x1?)
E:/Go/ekuiper/internal/service/schema.go:620 +0x205
github.com/lf-edge/ekuiper/internal/service.decodeField({0x1964220?, 0xc0004d2c30?}, 0xc0003a5680?, 0xa8?)
E:/Go/ekuiper/internal/service/schema.go:578 +0x3b0
github.com/lf-edge/ekuiper/internal/service.decodeMessageField({0x1964220?, 0xc0004d2c30?}, 0xc0003a5680, 0x17?, 0x68?)
E:/Go/ekuiper/internal/service/schema.go:519 +0x33
github.com/lf-edge/ekuiper/internal/service.decodeMessage(0x1adebd0?, 0xc000425860)
E:/Go/ekuiper/internal/service/schema.go:512 +0xef
github.com/lf-edge/ekuiper/internal/service.decodeSubMessage({0x1964220?, 0xc0004d2c00?}, 0x1964220?, 0x0?)
E:/Go/ekuiper/internal/service/schema.go:620 +0x205
github.com/lf-edge/ekuiper/internal/service.decodeField({0x1964220?, 0xc0004d2c00?}, 0xc00040c000?, 0x48?)
E:/Go/ekuiper/internal/service/schema.go:578 +0x3b0
github.com/lf-edge/ekuiper/internal/service.decodeMessageField({0x1964220?, 0xc0004d2c00?}, 0xc00040c000, 0x20?, 0xfb?)
E:/Go/ekuiper/internal/service/schema.go:519 +0x33
github.com/lf-edge/ekuiper/internal/service.decodeMessage(0xd67778?, 0xc000425c20)
E:/Go/ekuiper/internal/service/schema.go:512 +0xef
github.com/lf-edge/ekuiper/internal/service.(*wrappedProtoDescriptor).ConvertReturnMessage(0x1adebd0?, {0xc000576e4c?, 0x4?}, 0xc0004afc30?)
E:/Go/ekuiper/internal/service/schema.go:280 +0x33
github.com/lf-edge/ekuiper/internal/service.(*grpcExecutor).InvokeFunction(0xc00006ddc0, {0x1?, 0xc00024d638?}, {0xc000576e4c, 0x4}, {0xc0004afc30, 0x1, 0x1})
E:/Go/ekuiper/internal/service/executors.go:182 +0x714
github.com/lf-edge/ekuiper/internal/service.(*ExternalFunc).Exec(0x182dd80?, {0xc0004afc30?, 0xc0004b2dc0?, 0xc0002b40c0?}, {0x1ae8cd8?, 0xc0000045e8?})
E:/Go/ekuiper/internal/service/externalFunc.go:31 +0x50
github.com/lf-edge/ekuiper/internal/xsql.ExecFunc({0xc0003b5768, 0x4}, {0x1aded50, 0xc0004b2dc0}, {0xc0004afc30, 0x1, 0x1}, {0x1ae8cd8, 0xc0000045e8})
E:/Go/ekuiper/internal/xsql/func_invoker.go:53 +0x125
github.com/lf-edge/ekuiper/internal/xsql.(*FunctionValuer).Call(0x1849d20?, {0xc0003b5768, 0x4}, 0x4?, {0xc0004afc30, 0x1, 0x1})
E:/Go/ekuiper/internal/xsql/funcValuer.go:60 +0xbf
github.com/lf-edge/ekuiper/internal/xsql.multiValuer.Call({0xc000735440?, 0x3, 0x0?}, {0xc0003b5768, 0x4}, 0x1862760?, {0xc0004afc30, 0x1, 0x1})
E:/Go/ekuiper/internal/xsql/valuer.go:346 +0x118
github.com/lf-edge/ekuiper/internal/xsql.(*ValuerEval).Eval(0xc0000045a0, {0x1adc978?, 0xc000172b40?})
E:/Go/ekuiper/internal/xsql/valuer.go:511 +0xaef
github.com/lf-edge/ekuiper/internal/topo/operator.project({0xc000356280, 0x1, 0xc000735380?}, 0x1ad8e80?)
E:/Go/ekuiper/internal/topo/operator/project_operator.go:122 +0xef
github.com/lf-edge/ekuiper/internal/topo/operator.(*ProjectOp).Apply(0xc0004b2340, {0x1ae8a80?, 0xc0002b40c0?}, {0x1906640?, 0xc000735380}, 0x21d68f83d01?, 0x0?)
E:/Go/ekuiper/internal/topo/operator/project_operator.go:45 +0x21d
github.com/lf-edge/ekuiper/internal/topo/node.(*UnaryOperator).doOp(0xc000356400, {0x1ae8a80, 0xc0002b4000}, 0xc000621810?)
E:/Go/ekuiper/internal/topo/node/operations.go:130 +0x508
github.com/lf-edge/ekuiper/internal/topo/node.(*UnaryOperator).Exec.func1.1()
E:/Go/ekuiper/internal/topo/node/operations.go:87 +0x50
github.com/lf-edge/ekuiper/pkg/infra.SafeRun(0x0?)
E:/Go/ekuiper/pkg/infra/saferun.go:43 +0x62
github.com/lf-edge/ekuiper/internal/topo/node.(*UnaryOperator).Exec.func1()
E:/Go/ekuiper/internal/topo/node/operations.go:86 +0x7c
created by github.com/lf-edge/ekuiper/internal/topo/node.(*UnaryOperator).Exec
E:/Go/ekuiper/internal/topo/node/operations.go:85 +0x19b
How should this situation be handled
thanks

@VeddySama The error message is basically saying that the receiving data mismatch the protobuf definition. If something is not required, you can put an optional decorator in your proto file like the doc https://developers.google.com/protocol-buffers/docs/proto?hl=en#simple

I tried to add optional to ZValue/ZValueDoubles/DValue/DValueDoubles but it doesn't seem to work. If you don't transmit these, you will still get an error, and the documentation says that optional and repeated can have zero times.

Could you please attach your proto file and a sample input json or map that can trigger this problem?

proto.zip
json:
{"p":{"a":1}}

Thank you, we'll have a look

PR here https://github.com/lf-edge/ekuiper/pull/1315

thanks

PR here #1315

sorry,I just pulled the updated code of 1.6.0, and there is still a problem. Maybe what I described before is not clear. This problem occurs when grpc calls the external service and receives external service data, that is, when DecodeMessage is used.

Could you please attach the error log?

Could you please attach the error log?

This is my /etc/services file
services.zip

@VeddySama This is another problem actually. It seems your return message does not match the proto definition. Do you have a sample output hex string? Maybe you can try to decode it here to check if the protobuf/message can match.

@VeddySama This is another problem actually. It seems your return message does not match the proto definition. Do you have a sample output hex string? Maybe you can try to decode it here to check if the protobuf/message can match.

There is something wrong with my return data, thank you for your help


Vector2 x and y blocks

Adds the x of (Vector2) and y of (Vector2) blocks. The blocks are not ordered in the block picker
Helping with the issue https://github.com/endlessm/godot-block-coding/issues/248#issue-2575803628

Using the (string1)+(string2) block adds numbers instead of concatenating. This is probably a feature of the GDScript + operator. A separate concatenation block or a number to string block may need to be added

Thank you so much for your pull request @DoomTas3r! We're actually in the middle of a little game jam ahead of GodotCon and literally needed to add these exact blocks :laughing:
It looks good to us. All good if we go ahead and merge it?

I squashed and merged this into main just now, but I'll leave this PR open for the moment in case there was something else you intended to change :)


Authentication and Authorization disappears in Django 1.9.6

Hi,
I resently upgraded from django 1.8.11 to 1.9.6. But now my "Authentication and Authorization" module in admin has disappeard. uncommenting the "admin.site = AdminSitePlus()" in my url lets it reappear again. Also downgrading to django 1.8.11 makes it appear again.
So I guess it is due to this app in combination with django 1.9.6. No idea what happens, though.

Consolidating all of these in to #42


Replace behat/mink-extension with friends-of-behat/mink-extension

Hey, thanks for this library! I had some problems upgrading my project to Symfony 5.3 / PHP 8. These were fixed by replacing behat/mink-extension with friends-of-behat/mink-extension.
Advantages:

This is the fork of MinkExtension library that supports PHP ^7.2 and supports Symfony ^4.4 and ^5.0.

You are right, also I can no longer reproduce the composer error I had before. So I will close this issue.

Cool, thanks for getting back to me on this :+1:


Release mode crash

When building in release mode, ReadProcessMemory() or GCHandle.Alloc() cause an AccessViolationException
Debug build works fine. Only happens in release. Strange....

Perhaps those kernel calls need admin access? Try running the prod exe under admin privs.

Ah, so it's not release mode or privileges, but platform. I had my debug build set to 'Any CPU' (which defaulted to x64), and Release build set to 'x86'. When I set it to x64 / Any CPU it runs fines.
So it only works in 64-bit build. 32-bit is broken.
My best guess at this moment is the is a size mismatch. Something is trying to read/write a 64-bit address into a 32-bits, or vice versa. Probably the IntPtr or Marshal, maybe buffer size. Will look into more later.

Perhaps some of the P/invoke calls are only for x64 definitions and need separate wrappings per cpu build.


Update install-kubectl.md to use tabs for the different Operating Systems

Resolves #26096
Main changes:

I have moved the OS-specific sections into separate files, which are referenced for each top-level tab per-OS to make the tabs clearer in each section, and also to allow re-use of the Google Cloud SDK instructions.  I tried with it all in the one .md file and the multi-level tabs weren't very clear.
Add a "ToC" list at the top of each OS tab
Changed the gcloud instructions from a H2 header to H3 within the tab for each OS so it's included in the ToC list, as opposed to being a H2 header after the per-OS instructions
Added included/_index.md and I hope I've added the correct information (taken from content/en/docs/reference/setup-tools/kubeadm/generated/_index.md)

Deploy preview: https://deploy-preview-26682--kubernetes-io-master-staging.netlify.app/docs/tasks/tools/install-kubectl/

I like the way you have structured this @tomkivlin!
I have a suggestion around the files in the  content/en/docs/tasks/tools/included/ folder: We may want to include front matter with title, description, and toc_hide to help explain what these files are and how they're used.

I like the way you have structured this @tomkivlin!
I have a suggestion around the files in the  content/en/docs/tasks/tools/included/ folder: we may want to include front-matter with title, description, and toc_hide to help explain what these files are and how they're used. I'm not sure if that would affect the way these files are being included into the main page -- we wouldn't want the front-matter showing up in the copy.

I'll try it out and see - will try the {{% toc %}} shortcode as well - hadn't seen that one before.

@nate-double-u regarding the toc shortcode, I have given it a go, and there is a consequence of using it, which is that the (currently) H3 headers within the included pages would need to become H2 headers.  I've done the following so you can see:

Linux tab is using H2 headers and the toc shortcode (which displays the list properly but the headers maybe look weird for being that big)
macOS tab is using the static list with H3 headers as before
Windows tab is using original H3 headers with the toc shortcode, showing how the list isn't displayed properly

Let me know what you think. Thanks!

@nate-double-u regarding the toc shortcode, I have given it a go, and there is a consequence of using it, which is that the (currently) H3 headers within the included pages would need to become H2 headers. I've done the following so you can see:
* Linux tab is using H2 headers and the `toc` shortcode (which displays the list properly but the headers maybe look weird for being that big)
* macOS tab is using the static list with H3 headers as before
* Windows tab is using original H3 headers with the `toc` shortcode, showing how the list isn't displayed properly

Let me know what you think. Thanks!

Thanks for giving it a go!
I agree that the Linux tab looks weird, and sorta breaks the hierarchy structure of the page. As for the windows tab, I think the toc shortcode is expecting any H3 to be preceded by something in an H2 or H1, so it's adding empty bullets. We'd likely need to make a new shortcode for this application (or make the existing one more complex and able to handle different types of pages).
So I think for now, the macOS style static list is best. Maybe we can open an issue regarding the toc shortcode, as generally I think automating ToCs is a better than having to hand code them, but in this case we don't have much choice.

So I think for now, the macOS style static list is best. Maybe we can open an issue regarding the toc shortcode, as generally I think automating ToCs is a better than having to hand code them, but in this case we don't have much choice.

I agree, thanks.  I have created #26706 for further discussion on that.
(As an aside, I didn't hand code but used the Markdown All In One extension in vscode which makes life a little simpler.)

Removed the use of toc shortcode in https://github.com/kubernetes/website/pull/26682/commits/8497311c27024367c60a08572157523b78b4d588

For addressing https://github.com/kubernetes/website/issues/26096, I think this looks good to me.
@tengqm, the bigger question may be: should we be addressing https://github.com/kubernetes/website/issues/26096 at all?
/lgtm
/label tide/merge-method-squash

I think breaking the huge page into smaller ones while keeping each page focused on one aspect is the right thing to do.
I'd put a weak LGTM here for I'm not a big fan of nested tab pages.

@nate-double-u, the tab component isn't a part of core Docsy so it's lower risk to update if we decide want to do it.

Hi @tomkivlin . Thanks for opening this pull request.
I will look at the changes in more detail. I think the tab component does not help with the page layout, especially
when there is a top tab with a sub tab component (Linux tab).
There are a few places where the text looks broken up because of the list and note shortcode combinations.
I wonder if this page should include the gcloud kubectl installation instructions?
The overall page reads like a page in page because the "page" per tab does not use the right hand TOC for navigation.
Would you verify the gcloud link in the Windows and macOS inner TOCs?
I think the in page TOC helps to navigate through the tab page, but adds another layer to maintain (links).

Would you verify the gcloud link in the Windows and macOS inner TOCs?

Thanks for spotting that - I've made them each unique in https://github.com/kubernetes/website/pull/26682/commits/6564dc4808314fe801b337974b21847a57bd11b3 and checked here.

There are a few places where the text looks broken up because of the list and note shortcode combinations.
I wonder if this page should include the gcloud kubectl installation instructions?

I tend to agree about the list/note part, but I have left the text as it was before.  I've not changed the main bulk of content in this PR.
Re. gcloud instructions, again I haven't added them in this PR, but I agree that there are probably a number of other provider-specific methods we could include, should we choose.  e.g aks, eks.

@tengqm I am cool with all those suggestions, but just to clarify I haven't modified those texts in this PR (just moved to different files).  Not sure of the norm in k/website and whether I should deal with them in this PR, or raise issues to have them looked at separately?

Not sure of the norm in k/website and whether I should deal with them in this PR, or raise issues to have them looked at separately?

These kinds of small change requests seem common in k/website PRs, I think it's a "we're looking at this file anyway, let's see what else we can cleanup" thing. I'd include them in this PR.

Hi @tomkivlin .
I think the changes look okay.
This PR has possibly diverged from the original issue. Also, there is no longer a single page.
Including the common snippets is an interesting approach.
@nate-double-u, Since you worked on this page recently and logged the issue , what do you think about the changes? Is the issue resolved?

/lgtm

This PR has possibly diverged from the original issue. Also, there is no longer a single page.

@nate-double-u, Since you worked on this page recently and logged the issue , what do you think about the changes? Is the issue resolved?

The goal of the issue was to makes the instructions clearer for people looking for one operating system's installer, and I think this does it. I'm OK that it's been made into separate pages rather than using the tabs suggestion.
/lgtm

@tomkivlin . Great job!
/approve

@tomkivlin . Great job!

Thanks for all the help and support. Only my 2nd/3rd PR so good to have the input.


imu measurement rate and rosbag play rate

Hello @gaowenliang ,
My imu works on 50 hz, if I record the values on my rosbag I can see that recording happens not exactly at 50hz, so in order to work on this algorithm shoud I play rosbag with rosbag play -r mybagfile.bag anyway, or should I directly play it without indicating the frequency?
Also, does this algorithm work on only 200 hz or some other frequencies as well, if so should I give the frequency of my IMU as an input to the system somewhere?
Thanks,
Metin

@mmp52 Regardless of the frequency of the imu, ros sends an imu message with a timestamp.The code is processed based on this timestamp. Directly play it without indicating the frequency.

@tianxinyu0509 Thank you very much!


Messages randomly not delivered

Hi!
I'm using elasticmq with Go and currently this is my pooling implementation:
func listenForLocalSQSEvents(urls []string) {
	ctx := context.Background()
	sess := session.Must(session.NewSessionWithOptions(session.Options{
		Config: aws.Config{
			CredentialsChainVerboseErrors: aws.Bool(true),
			Endpoint:                      aws.String(env.GetString("ELASTICMQ_URI")),
			Region:                        aws.String("ELASTICMQ_REGION"),
		},
	}))
	svc := sqs.New(sess)
	for _, url := range urls {
			go func(url string) {
				for {
					log.Printf("requesting at: %v", time.Now())
					msgResult, err := svc.ReceiveMessage(&sqs.ReceiveMessageInput{
						AttributeNames: []*string{
							aws.String(sqs.MessageSystemAttributeNameSentTimestamp),
						},
						MessageAttributeNames: []*string{
							aws.String(sqs.QueueAttributeNameAll),
						},
						QueueUrl:            aws.String(url),
						MaxNumberOfMessages: aws.Int64(10),
						WaitTimeSeconds:     aws.Int64(5),
					})
					if err != nil {
						panic(err)
					}
					log.Printf("msgResult: %+v", msgResult.String())
					if err := eventsLambda.Consume(ctx, msgResult); err != nil {
						log.Printf("local sqs consumer error: %v", err)
						continue
					}
				}
			}(v)
	}
}

The problem is, some messages are never delivered and some are delivered with no delay at all.
Using the UI I noticed that the messages which are never delivered increase the counter on the Approximate number of not visible Messages column.

Hi @nicolasassi, if you want us to investigate the issue, please provide an application reproducing the problem, preferably in a form of GH repo, that we'd be able to run locally.

I've experience similar behaviour, I have a hunch this is related to the dedletter queue functionality and the way messages ids are being assigned.
I've noticed that the messages are pushed directly to the deadletter queue.
Was not quite able to confirm the root cause.
Removing the deadletter queue from the configuration or raising maxReceiveCount seems to adress the issue so it's not breaking our functional tests.

@micossow If you could help me with a logback configuration that would log client calls and internals to the console output I could try to get more insight into this.
So far, my configuration only outputs initialisation logs. on 1.3.14:
`
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
    <layout class="ch.qos.logback.classic.PatternLayout">
        <Pattern>
            %d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n
        </Pattern>
    </layout>
</appender>

<logger name="com.mkyong" level="debug" additivity="false">
    <appender-ref ref="CONSOLE"/>
</logger>

<root level="error">
    <appender-ref ref="CONSOLE"/>
</root>

`

Thanks I got more log data with this.
With the following queue config:
`local-test-notification {
defaultVisibilityTimeout = 0 seconds
delay = 0 seconds
receiveMessageWait = 0 seconds
    deadLettersQueue {
        name = "local-test-notification-deadletter"
        maxReceiveCount = 3
    }
}

local-test-notification-deadletter {
    defaultVisibilityTimeout = 10 seconds
    delay = 0 seconds
    receiveMessageWait = 0 seconds
}`

sqs  | 14:32:58.022 [elasticmq-akka.actor.default-dispatcher-12] DEBUG o.elasticmq.actor.queue.QueueActor - local-test-notification: Sent message with id 0abdd44f-ad6f-4c1a-8b7d-2fd965d411f1 sqs  | 14:32:58.023 [elasticmq-akka.actor.default-dispatcher-12] DEBUG o.elasticmq.actor.queue.QueueActor - local-test-notification: Receiving message 0abdd44f-ad6f-4c1a-8b7d-2fd965d411f1 sqs  | 14:32:58.024 [elasticmq-akka.actor.default-dispatcher-12] DEBUG o.elasticmq.actor.queue.QueueActor - local-test-notification: Awaiting messages: replying to sequence 0 with 1 messages. sqs  | 14:32:58.025 [elasticmq-akka.actor.default-dispatcher-12] DEBUG o.elasticmq.actor.queue.QueueActor - local-test-notification: Receiving message 0abdd44f-ad6f-4c1a-8b7d-2fd965d411f1 sqs  | 14:32:58.026 [elasticmq-akka.actor.default-dispatcher-12] DEBUG o.elasticmq.actor.queue.QueueActor - local-test-notification: Awaiting messages: replying to sequence 1 with 1 messages. sqs  | 14:32:58.035 [elasticmq-akka.actor.default-dispatcher-12] DEBUG o.elasticmq.actor.queue.QueueActor - local-test-notification: Receiving message 0abdd44f-ad6f-4c1a-8b7d-2fd965d411f1 sqs  | 14:32:58.039 [elasticmq-akka.actor.default-dispatcher-12] DEBUG o.elasticmq.actor.queue.QueueActor - local-test-notification: Awaiting messages: replying to sequence 2 with 1 messages. sqs  | 14:32:58.059 [elasticmq-akka.actor.default-dispatcher-12] DEBUG o.elasticmq.actor.queue.QueueActor - local-test-notification: send message InternalMessage(0abdd44f-ad6f-4c1a-8b7d-2fd965d411f1,ArrayBuffer(0abdd44f-ad6f-4c1a-8b7d-2fd965d411f1#bcac6d6b-4661-426f-ad81-4c1c1ec59c72, 0abdd44f-ad6f-4c1a-8b7d-2fd965d411f1#9e7393fe-224e-4b7b-9e7a-358ec72b3ec6, 0abdd44f-ad6f-4c1a-8b7d-2fd965d411f1#e0764a3d-3e1c-4384-8a03-e60ac5912aa7),1680273178035,O:36:\"Symfony\\Component\\Messenger\\Envelope\":2:{s:44:\"\0Symfony\\Component\\Messenger\\Envelope\0stamps\";a:1:{s:46:\"Symfony\\Component\\Messenger\\Stamp\\BusNameStamp\";a:1:{i:0;O:46:\"Symfony\\Component\\Messenger\\Stamp\\BusNameStamp\":1:{s:55:\"\0Symfony\\Component\\Messenger\\Stamp\\BusNameStamp\0busName\";s:21:\"messenger.bus.default\";}}}s:45:\"\0Symfony\\Component\\Messenger\\Envelope\0message\";O:63:\"Message\\Notification\":1:{s:83:\"\0Message\\Notification\0id\";s:36:\"973261f1-7cdc-467d-9f02-97812bb49e67\";}},Map(),2023-03-31T14:32:58.022Z,0,OnDateTimeReceived(2023-03-31T14:32:58.023Z),3,false,None,None,None,None) to dead letters actor Some(Actor[akka://elasticmq/user/$a/$c#2062594922]) sqs  | 14:32:58.070 [elasticmq-akka.actor.default-dispatcher-8] DEBUG o.elasticmq.actor.queue.QueueActor - Moved message (0abdd44f-ad6f-4c1a-8b7d-2fd965d411f1) to local-test-notification-deadletter
The message seems to be moved directly to the deadletter queue, but I still do not see the reason.

Well, that's how the SQS works. If you receive the message maxReceiveCount times, but don't delete it using DeleteMessage, it gets moved to dead letter queue.
SQS messages do not get deleted automatically.

Thanks @micossow there is definetly some eagger and unsolicited fetching from happening on the client library side that I have to track down.


add CONTRIBUTING.md

add contributing file for new contributors to contribute to your project

Hey, @starlightknown  I'm new to open source and will like to contribute to this project. can you guide me on what needs to be done?

Hey, @starlightknown I'm new to open source and will like to contribute to this project. can you guide me on what needs to be done?

Hey @slayer321  good to have you hear. You can look at example templates from CONTRIBUTING.md online . Fork the repo, add the CONTRIBUTING.md file, raise a PR , I will them review and accept your PR. If accepted you get your green square! (i.e your changes are accepted ) Hope this helps!

Understood, working on it.


Added missing builld components to RCD [empty]

About the PR

Added missing builds to the RCD[Empty]. This matches with the rest of the RCDs
Why / Balance

fixes #28278
Technical details

Media

[ x ] I have added screenshots/videos to this PR showcasing its changes ingame, or this PR does not require an ingame showcase

Breaking changes

Changelog

Yeah, it doesn't need to have the AvailablePrototypes declared twice. It already parents the original RCD.

That's why they pay you guys the big bucks :godo:


Modify UTXO and state designs for transparent coinbase output checks

Motivation
Zebra needs a design for tracking the source of transparent coinbase UTXOs, so we can validate:

the coinbase maturity rule, and
the mandatory transparent coinbase shielding rule.

Scope
Validate spending transparent outputs of coinbase transactions. (If the miner or funding stream receiver choose to receive the ZEC using a transparent address).
Zebra already ignores the genesis coinbase outputs, so they are out of scope.
Specifications

A transaction with one or more transparent inputs from coinbase transactions MUST have no transparent outputs (i.e.tx_out_count MUST be 0). Inputs from coinbase transactions include Founders’ Reward outputs and funding stream outputs.

 Note that transparent outputs of coinbase transactions include Founders’ Reward outputs and transparent funding stream outputs.

https://zips.z.cash/protocol/protocol.pdf#txnencodingandconsensus
Solution
Closes #1970.
Script verification RFC
Data structures

Add a transparent spends flag and spend height in an AwaitSpendableUtxo request
Add a SpendPools enum
Create a TransparentOutputOrigin with a coinbase flag and height, for internal state use
Lookup the coinbase flag and height

Note: the finalized state format does not change, because it already stores all the information we need.
Validation

Verify the spends flag and spend height
Explain why we can verify requests in parallel
Leave some optimisations for later

State RFC

Change to using AwaitSpendableUtxo

Review
@jvff and @conradoplg can do the initial review on this design, because everyone else is working on the pool balances design.
@conradoplg can you check my arguments for transaction ID uniqueness?
We need all IDs to be unique in a single chain. And we need some limited uniqueness across any chain.
Reviewer Checklist

[ ] Design implements Specs
[ ] Transaction IDs are unique where required

Follow Up Work
Open a ticket for the optional optimisations, but don't schedule it in a sprint.

Maybe line 39 should also replace unspent transaction outputs with unspent transparent outputs? Or maybe just remove the long form and leave UTXO, because it is already defined in the Definitions section.

Fixed in 20373b4

Is this one ready to go or do we still need to add some documentation?

Is this one ready to go or do we still need to add some documentation?

I needed to change the design to match the current implementation https://github.com/ZcashFoundation/zebra/pull/2413#discussion_r662761742
And add a better explanation of why the design works https://github.com/ZcashFoundation/zebra/pull/2413#discussion_r661667491
Once we've had a final review, we should be good to merge.

@jvff can you do a final review here?
The design just needs a quick review to make sure the latest changes make sense.
We can pick up any minor issues when we do the implementation.

I also added some missing arguments to the queue function in 4ec9bb8f.
I don't think that should block merging either.


Downloaded files are locked

I'm trying to move finished torrents to another directory, but the folder/files are still locked after download has been completed.
I'm using this code:
public static async Task<String> DownloadTorrent(String torrentFile)
{
	try
	{
		var parser = new BencodeNET.Parsing.BencodeParser();
		BencodeNET.Torrents.Torrent torrent = parser.Parse<BencodeNET.Torrents.Torrent>(torrentFile);

		using (Leak.Client.Swarm.SwarmClient client = new Leak.Client.Swarm.SwarmClient())
		{
			Leak.Client.Notification notification = null;
			var hash = Leak.Common.FileHash.Parse(torrent.OriginalInfoHash);

			List<String> allTrackers = new List<string>();
			foreach (var tr in torrent.Trackers)
				allTrackers.AddRange(tr);

			var session = await client.ConnectAsync(hash, allTrackers.ToArray());

			session.Download(@"C:\Torrents\InProgress");

			do
			{
				notification = await session.NextAsync();
				if (notification is Leak.Client.Notifications.DataChangedNotification datNot)
				{
					Console.WriteLine($"Downloading ({Math.Round((((double)datNot.Completed / (double)torrent.NumberOfPieces) * 100), 2)}%)");
				}
			}
			while (notification.Type != Leak.Client.NotificationType.DataCompleted);
		}

		if (System.IO.Directory.Exists($"C:\\Torrents\\InProgress\\{torrent.OriginalInfoHash}"))
		{
			System.IO.Directory.Move($"C:\\Torrents\\InProgress\\{torrent.OriginalInfoHash}", $"C:\\Torrents\\Finished\\{torrent.OriginalInfoHash}");
		}

		return $"C:\\Torrents\\Finished\\{torrent.OriginalInfoHash}";
	}
	catch(Exception ex)
	{
		System.Diagnostics.Debug.WriteLine(ex);
	}

	return null;
}

I always get a System.IO.IOException when trying to move the directory from InProgress to Finished: Access to the path 'C:\\Torrents\\InProgress\\[[REDACTED HASH]]' is denied.
If I restart the application and add code to move it before it tries to download them it moves the directory successfully.
I'm guessing either the SwarmClient or SwarmSession isn't releasing their resources properly.

How do you successfully download a file? I always get stuck at notification = await session.NextAsync(); and get an exception
System.TypeInitializationException:“The type initializer for 'Leak.Meta.Store.HashExtensions' threw an exception.”
What do you think this might be for?


using "find" or "copy" with section in "pre-comment" doesn't work

Describe the bug
When selecting a word in a "pre-comment" shown in the "decompiled code view" and using "ctrl-f" to search that word in the remaining decompiled code, the behavior of the respective "Decompiler Find Text" dialog is absolutely braindead:
Rather than using the actually selected word, the complete content of the comment is used as the "Find:" text.. it should be obvious that this is an absolutely useless "feature"! The same problem seems to also affect the regular copy/paste functionality - which also copies the complete comment rather than copying the explicitly selecteted word.
Adding insult to injury the layout of the "Decompiler Find Text" becomes unusable when an "overly long" text (like some random pre-comment) is among the texts of its "find" history; It seems that if one of the texts in the respective "Find:" input/dropdown field exceeds the width of the dialogs default width, then the height of the input/dropdown field collapses to 3-4 pixels - rendering is absolutely useless (this situation can be worked around by manually making the dialog wider .. but UI usability wise this is just bad).
Expected behavior
When some subset of a comment has been specifically selected, then operations like "search" and "copy" should obviously use that selected subset and NOT the complete comment.
There should never be a situation where the "Decompiler Find Text"'s "Find:" field collapses to some unusable form.
Environment (please complete the following information):

OS: Win10
Ghidra Version: 10.1.1

This may be a regression.  I will take a look.

I am seeing this with the Find... action, but not yet with the Copy/Paste actions.
Do you have a particular set of instructions to reproduce the issue for Copy/Paste?

Do you have a particular set of instructions to reproduce the issue for Copy/Paste?

my mistake.. I had pressed copy after CRTL-F and the focus had already switched to the "Find:" field, i.e. I had copied the text from the find-field instead of from the original location..

OK.   I see the issue.  I'm pretty sure this has never worked.   I'm going to try to get a simple fix in place for this.

Just out of curiousity.. I've noticed that the various bug-reposts that I have posted during the past 1-2 weeks seem to land in 2 different buckets.. there are the UI usability bugs ones (like this one) where I am getting a response very quickly (thanks for that!) but then there are the more "serious" problems (e.g. flaws of the decompiler) where there seems to be no reaction whatsoever..
is there some dedicated team looking into respective bug reports or is this some kind of bottleneck situation where the one person familiar with a specific sub-system is just busy with something else?

is there some dedicated team looking into respective bug reports

Yes, we are all dedicated :wink:
Our trend for responding to issues has been to more immediately address critical items.  Along with those, we also are able to address items that are relatively easy to fix, such as the UI items you've mentioned.   Many of the outstanding issues we still have are those that are nuanced and require a more in depth analysis to triage.  Unfortunately, the work required to address those items ends up slowing down the entire process.  I envision a future with an improved ticket processing, including faster response times.


Update GCN 4.2.1 to GDK 4.3.7

Add here your WMS ID: 11495
We have rebranded Graal Cloud Native (GCN) to Graal Development Kit for Micronaut (GDK).
General requirements

Information in the workshop is adequate and updated
Code is correct and working
Links are correct
Make sure you enter the help email link in your manifest.json
Please make sure WMS URLs are updated as needed after your PR is approved

Checklist - Refer to the QA document for the complete list
Please confirm that the following is completed before submitting your PR

[YES ] All filenames are lower case (including folders, images, files, etc.)
[YES ] Filenames are descriptive
[YES ] Your workshop folder structure should be similar to the one used in the sample workshop (https://github.com/oracle-livelabs/common/tree/main/sample-livelabs-templates/sample-workshop)
[YES ] Are you using multiple versions (desktop/, sandbox/, tenancy/)? Make sure that each of them contains a manifest.json and an index.html
[YES ] Image references in markdown contain an alternate text

Thank you, @carmenberdant1.


Dont try download the map if player already has it

So I was doing some online testing of my coop missions. Let's forgtet for a bit about the other issue with coop replays.
But then when I have a local copy of my coop replay, which has all the information needed. Example:
{"uid": 9654895, "recorder": "speed2", "featured_mod": "coop", "launched_at":<PHONE_NUMBER>.579191, "complete": true, "state": "PLAYING", "num_players": 2, "max_players": 4, "title": "speednoob only", "host": "FunkOff", "mapname": "scca_coop_a06", "map_file_path": "maps/scca_coop_a06.zip", "teams": {"1": ["FunkOff", "speed2"]}, "featured_mod_versions": {"1": 55, "2": 55, "3": 49, "4": 49, "5": 49, "6": 49, "7": 51, "8": 49, "9": 50, "10": 50, "11": 50, "12": 50, "13": 50, "14": 50, "15": 49, "16": 49, "17": 50, "18": 51, "19": 50, "20": 50, "21": 51, "22": 49}, "sim_mods": {}, "password_protected": false, "visibility": "PUBLIC", "command": "game_info", "game_end":<PHONE_NUMBER>.2083693}

the client can't actually play it
java.util.concurrent.CompletionException: java.io.FileNotFoundException: http://content.faforever.com/faf/vault/maps/scca_coop_a06.zip
	at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(Unknown Source)
	at java.base/java.util.concurrent.CompletableFuture.completeThrowable(Unknown Source)
	at java.base/java.util.concurrent.CompletableFuture$UniAccept.tryFire(Unknown Source)
	at java.base/java.util.concurrent.CompletableFuture.postComplete(Unknown Source)
	at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(Unknown Source)
	at com.faforever.client.task.CompletableTask.lambda$new$1(CompletableTask.java:17)
	at javafx.base/com.sun.javafx.event.CompositeEventHandler.dispatchBubblingEvent(Unknown Source)
	at javafx.base/com.sun.javafx.event.EventHandlerManager.dispatchBubblingEvent(Unknown Source)
	at javafx.base/com.sun.javafx.event.EventHandlerManager.dispatchBubblingEvent(Unknown Source)
	at javafx.base/com.sun.javafx.event.BasicEventDispatcher.dispatchEvent(Unknown Source)
	at javafx.base/com.sun.javafx.event.EventDispatchChainImpl.dispatchEvent(Unknown Source)
	at javafx.base/com.sun.javafx.event.EventUtil.fireEventImpl(Unknown Source)
	at javafx.base/com.sun.javafx.event.EventUtil.fireEvent(Unknown Source)
	at javafx.base/javafx.event.Event.fireEvent(Unknown Source)
	at javafx.graphics/javafx.concurrent.EventHelper.fireEvent(Unknown Source)
	at javafx.graphics/javafx.concurrent.Task.fireEvent(Unknown Source)
	at javafx.graphics/javafx.concurrent.Task.setState(Unknown Source)
	at javafx.graphics/javafx.concurrent.Task$TaskCallable.lambda$call$2(Unknown Source)
	at javafx.graphics/com.sun.javafx.application.PlatformImpl.lambda$runLater$9(Unknown Source)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at javafx.graphics/com.sun.javafx.application.PlatformImpl.lambda$runLater$10(Unknown Source)
	at javafx.graphics/com.sun.glass.ui.InvokeLaterDispatcher$Future.run(Unknown Source)
	at javafx.graphics/com.sun.glass.ui.win.WinApplication._runLoop(Native Method)
	at javafx.graphics/com.sun.glass.ui.win.WinApplication.lambda$runLoop$3(Unknown Source)
	at java.base/java.lang.Thread.run(Unknown Source)
Caused by: java.io.FileNotFoundException: http://content.faforever.com/faf/vault/maps/scca_coop_a06.zip
	at java.base/jdk.internal.reflect.GeneratedConstructorAccessor101.newInstance(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
	at java.base/java.lang.reflect.Constructor.newInstance(Unknown Source)
	at java.base/sun.net.www.protocol.http.HttpURLConnection$10.run(Unknown Source)
	at java.base/sun.net.www.protocol.http.HttpURLConnection$10.run(Unknown Source)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getChainedException(Unknown Source)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(Unknown Source)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
	at com.faforever.client.map.DownloadMapTask.call(DownloadMapTask.java:55)
	at com.faforever.client.map.DownloadMapTask.call(DownloadMapTask.java:22)
	at javafx.graphics/javafx.concurrent.Task$TaskCallable.call(Unknown Source)
	at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	... 1 more
Caused by: java.io.FileNotFoundException: http://content.faforever.com/faf/vault/maps/scca_coop_a06.zip
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(Unknown Source)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getHeaderField(Unknown Source)
	at java.base/java.net.URLConnection.getHeaderFieldLong(Unknown Source)
	at java.base/java.net.URLConnection.getContentLengthLong(Unknown Source)
	at java.base/java.net.URLConnection.getContentLength(Unknown Source)
	at com.faforever.client.map.DownloadMapTask.call(DownloadMapTask.java:51)
	... 6 more

It complains that the map isn't in the vault, correct?
Well its not there (since its my testing version), but the map is in my folder, so it should look there first before even trying to download it.

Your replay file has an entry for "map_file_path" which I can't find in the code anywhere.  Is your file located in .faforever/Maps/scca_coop_a06 or .faforever/Maps/maps/scca_coop_a06 ?  Is it a directory or is it a zip file?

map folder name is scca_coop_a06
its placed in the maps folder C:\Users%USERNAME%\Documents\My Games\Gas Powered Games\Supreme Commander Forged Alliance\Maps
as another 800 maps and missions, not zipped
https://puu.sh/DtQNt/2c210acbc9.png

Are you running on Linux or Windows?  Linux file names are case sensitive so "SCCA_Coop_A06" won't match "scca_coop_a06".

windows

Here's what I think is going on: ReplayService calls GameService which asks MapService for a list of installed maps.  MapService only returns maps with a type of 'skirmish'.  So when you run a coop or any other mod type, the client won't see locally stored maps since they're not 'skirmish' type.  To modify this, MapService needs to return maps that correspond to the type of game you're playing.  But, replay files only contain a featured_mod attribute and the maps have a type attribute but the two values don't match up.  e.g. featured_mod = 'coop' but replay type='campaign_coop'.  So we'd have to build some type of matching between the two values.

For displaying the maps when starting a game only skrimish maps should be displayed because u dont want to host coop maps ;) keep that in mind

ok so just ask user whether to start anyway I guess...


remove "Code generated by tzgen" from the tz package comment

The document of the tz package says "Code generated by tzgen. DO NOT EDIT. Package model contains the types.", but it is not useful for users of the tz package.

<EMAIL_ADDRESS>

Code generated by tzgen. DO NOT EDIT. Package model contains the types.
Code generated by tzgen. DO NOT EDIT. Package model contains the types.
Code generated by tzgen. DO NOT EDIT. Package model contains the types.
Code generated by tzgen. DO NOT EDIT. Package model contains the types.
Code generated by tzgen. DO NOT EDIT. Package model contains the types.
(snip)

@shogo82148 thanks!


Cannot make install

When I run cmake .., everything is perfect. Then I run `sudo make install and I get this:
~/lemonbuddy/build on 3ad583b
➔ sudo make install
[  2%] Linking CXX static library libxpp.a
Traceback (most recent call last):
  File "/home/marcel/lemonbuddy/lib/xpp/generators/cpp_client.py", line 3162, in <module>
    from xcbgen.state import Module
ImportError: No module named xcbgen.state
make[2]: *** [lib/xpp/CMakeFiles/xpp.dir/build.make:70: lib/xpp/libxpp.a] Error 1
make[1]: *** [CMakeFiles/Makefile2:206: lib/xpp/CMakeFiles/xpp.dir/all] Error 2
make: *** [Makefile:128: all] Error 2

I have to recompile because the package in the aur does not seem to support i3 (am I correct?). Anyway it doesn't work.
uname -a:
Linux Hermes 4.6.4-1-ARCH #1 SMP PREEMPT Mon Jul 11 19:12:32 CEST 2016 x86_64 GNU/Linux

the package(s) in the aur support i3; you need the dependencies installed.

@ashleymcdonald I have jsoncpp libsigc++ and the i3 example still does not work.

when compiling from one of the aur versions (that install fine i assume) does it say yes to i3 support early in the process ?
I'm personally a bspwm user myself so i have not used the i3 modules but support for i3 is definitely in the aur versions.
I'm guessing you will have to wait for @jaagr to chime in.
Good luck

ps. your original error looks like you dont have libxcb

Hi @Marcel-Robitaille
Sorry for the late respons, but I've been on vacation the past few weeks. As @ashleymcdonald mentioned it's an xcb issue. I'm not sure what package manager you are using but you will need to install the equivilant of the libxcb pacman package or apt-get's libxcb1-dev.

I no longer need to compile my own version. The aur package has support for i3. The issue must lie elsewhere.
I am still getting the issue when trying to sudo make install even with pacman's libxcb installed (version 1.12-1).

What distro are you using? Try installing xcb-proto.

xcb-proto is installed and is version 1.12-2.

Could you post the output of this command:
$ find /usr/lib -name xcbgen -type d

Sorry, I was away from this computer for a few days. Anyway:
~
➔ find /usr/lib -name xcbgen -type d
/usr/lib/python3.5/site-packages/xcbgen

Ok, then I know what's causing the issue. I'll patch the build routine.

Just for future reference:
The lib/xpp/CMakeLists.txt searches in /usr/lib/python2.7/site-packages whereas (on arch at least) the relevant files are in the /usr/lib/python3.5/site-packages folder
The PKGBUILD for lemonbuddy in the AUR actually corrects the paths using
sed 's/python2.7/python3.5/g' -i lib/xpp/CMakeLists.txt

So if the AUR package builds for you, then you can just use the build instructions from there and it should work for you.


Uncaught TypeError: Cannot read properties of undefined (reading 'core') at  node_modules\ng2-completer\bundles\ng2-completer.umd.js:4:96

I am getting an error saying "Uncaught TypeError: Cannot read properties of undefined (reading 'core') at  node_modules\ng2-completer\bundles\ng2-completer.umd.js:4:96 at node_modules\ng2-completer\bundles\ng2-completer.umd.js:5:2".
Can anyone help me in this.

Me too, does anyone have a solution ?


Add Markdoc, .mdoc

Language name
Markdoc - https://markdoc.dev
URL of example repository
https://github.com/withastro/astro/blob/main/examples/with-markdoc/src/content/docs/intro.mdoc
https://github.com/markdoc/docs/blob/main/pages/index.md
URL of syntax highlighting grammar
https://github.com/markdoc/language-server/tree/main/syntaxes
Most popular extensions
.mdoc, .md
Detected language
Roff

Most popular extensions: .mdoc, .md

Ugh, this project picked two file extensions already in-use by existing markup languages (Roff and Markdown). Assuming that Markdoc achieves sufficient usage on GitHub, adding support for it is going to be a pain-in-the-arse… 😞

If I understand correctly, Markdoc boils down to Markdown plus an extended syntax like {% this %}? If so I imagine a lot of repositories that use Markdown with other template tools that use this Jinja-like syntax can get misidentified as Markdoc very easily.
I noticed that HTML+Django (alias of Jinja) does not have the .html extension, even though it's commonly used (see nearly every Django project). I assume this is intentional since the syntax would be too similar to tell them apart. Perhaps Markdoc should be treated similarly, and not have the .md extension registered for it?

This is why every language should specify itself at the top of the doc like XHTML xD


[!] Unable to find a specification for Crashlytics (= 3.14.0)

Crashlytics (legacy)

The solution is shown in your screenshot: pod repo update
https://github.com/dpa99c/cordova-plugin-firebasex#out-of-date-pods


Add #[builder(crate = "parol_runtime::derive_builder")]

This is related to #23.
I've tried to extract boilerplates into parol_runtime, but we still need pub use parol_runtime::derive_builder in lib.rs or main.rs because derive_builder does not support re-exporting.
Now we can remove it, thanks to https://github.com/colin-kiegel/rust-derive-builder/pull/275.
This is a draft because it uses not released version of derive_builder.

@ryo33 Very good.
So, we have to wait for https://github.com/colin-kiegel/rust-derive-builder/pull/275 to be merged.
Best case would be that they publish a new version of derive_builder.
As I see from the comments the maintainers are currently evaluating your pull request, which is fine.

@jsinger67 Now I think it's ready to merge.

Works perfectly 💯
Again, thanks a lot!


Discrepancies between icefall and sherpa

Hello !
I exported a streaming zipformer (pruned_transducer_stateless7_multi) in onnx. Using sherpa-onnx I have different (quite large) wer than using icefall (I didn't had that issue between when using pruned_transducer_stateless3 and sherpa). Do you have an idea where it might comes from ?

Have you used onnx_pretrained.py to verify that the exported model is able to decode waves?

@csukuangfj Yes, I can decode files (and just double check with onnx_pretrained.py), in general it seems like I have 1.5-2% absolute wer than using icefall (approximate estimation). Might be related to https://github.com/k2-fsa/icefall/issues/1006 (second issue)
Thank you !

@csukuangfj I test the specific file (which is problematic) using onnx_pretrained.py
onnx_pretrained.py and decode.py give the same results
sherpa-onnx give worst (much worst) wer. Hence, I think it might be a sherpa-onnx issue, let me open an issue there to track it

@ezerhouni  Is it possible to share your model?
If the modified_beam_search in sherpa-onnx gives incorrect results, it should not be difficult to debug it.
I suggest you set num_active_paths to 1, dump the output of the joiner, and compare it with the greedy search.  If you can share the model, we can debug that.

@csukuangfj Thank you for your help, I will try to see if we can share the model.
From a quick experiment, greedy search and modified beam search with num_active_paths=1 produces what is expected but modified beam search with num_active_paths=4 does not. Icefall (decode.py) with beam search of 4 does generate the proper output.

If num_active_paths=1 generates the correct results, then I suggest that you use num_active_paths=2 for debugging using gdb.

@csukuangfj We are still working on it to try to see where it might comes from. In the mean time, I think I could reproduce the issue:
Model :

https://huggingface.co/marcoyang/icefall-libri-giga-pruned-transducer-stateless7-streaming-2023-04-04 using pretrained.pt with pretrained.sh, I get the following transcript :
hyp : THERE ARE SEVERAL NON GOVERNMENTAL ORGANIZATIONS IN SOME SCHEMIST
Converting the above model to ONNX and using sherpa-onnx:
hyp : THERE ARE SEVERAL NON GOVERNMENTAL ORGANIZATIONS IN SOME SCHEME MOST
(I modified the padding to 0.64s in sherpa-onnx)
Both are with modified_beam_search with beam of 4

The wav file is common-voice-en-19703943.wav (will try to upload it)
I will add more details later

Could you please add more details?

Hi @csukuangfj, I tried to make a reproducible example using the model given by @ezerhouni.
I first prepared a sample audio and cloned the model
# get audio
wget -q https://datasets-server.huggingface.co/assets/mozilla-foundation/common_voice_13_0/--/en/test/0/audio/audio.mp3
ffmpeg -i audio.mp3 -ar 16000 common_voice_en_27710027.wav

# clone model
cd icefall/egs/librispeech/ASR/
mkdir tmp
cd tmp
git clone https://huggingface.co/marcoyang/icefall-libri-giga-pruned-transducer-stateless7-streaming-2023-04-04
cd ..

Then, running inference on Icefall, with modified beam search:
2023-06-28 08:56:44,792 INFO [pretrained.py:242] device: cuda:0
2023-06-28 08:56:44,792 INFO [pretrained.py:244] Creating model
2023-06-28 08:56:45,479 INFO [zipformer.py:405] At encoder stack 4, which has downsampling_factor=2, we will combine the outputs of layers 1 and 3, with downsampling_factors=2 and 8.
2023-06-28 08:56:45,491 INFO [pretrained.py:248] Number of model parameters: 70369391
2023-06-28 08:56:48,306 INFO [pretrained.py:256] Constructing Fbank computer
2023-06-28 08:56:48,307 INFO [pretrained.py:266] Reading sound files: ['/root/common_voice_en_27710027.wav']
2023-06-28 08:56:48,308 INFO [pretrained.py:272] Decoding started
2023-06-28 08:56:49,553 INFO [pretrained.py:287] Using modified_beam_search
2023-06-28 08:56:49,879 INFO [pretrained.py:346]
/root/common_voice_en_27710027.wav:
JOE KITSON DISAPPROVED OF FILMS AND BUSTER ALSO HAD RESERVATIONS ABOUT THE MEDIA

2023-06-28 08:56:49,880 INFO [pretrained.py:348] Decoding Done

Then, converting the model to ONNX.
Done!
/root/common_voice_en_27710027.wav
JOE KITON DISAPPROVED OF FILMS AND BUSTER ALSO HAD RESERVATIONS ABOUT THE MEDIA
----------
num_threads: 1
decoding_method: modified_beam_search
Wave duration: 6.300 s
Elapsed time: 1.047 s
Real time factor (RTF): 1.047/6.300 = 0.166

So there's just a slight substitution from KITSON -> KITON.
I think I've seen worse deterioration in results, but that's using my own models, which I can't share. Also, on most devices (iOS, Windows desktop), we often have to speak loudly for the recognizer to work. I'm not sure how to reproduce this.
Looking forward to having this thoroughly investigated. Cheers!

@w11wo @ezerhouni
Thanks for posting the detailed instructions for reproducing.
I am able to reproduce the results you posted.
However, one thing I want to mention is that pretrained.py uses model.encoder.forward(), while sherpa-onnx uses
model.encoder.streaming_forward().
I also tested it with onnx_pretrained.py from icefall and it produces the same result with sherpa-onnx for the
given wave common_voice_en_27710027.wav.
Have you ever encountered a case that onnx_pretrained.py decodes a different result from sherpa-onnx?

cc @yaozengwei

@csukuangfj Sorry I forgot to add more details (I wanted to share the audio file). In my small experiment, I am seeing the same results between onnx_pretrained.py and sherpa-onnx, if I find some time, I will try to do some more experiment this week

As for resampling, could you use the wave after resampling to test icefall and sherpa-onnx? In this way, we don't need
to care about the sampling issue.
If we find the root cause is resampling, we can change how resampling is implemented in sherpa-onnx.

@csukuangfj
Thanks for the response. Yes, I find the same results between onnx_pretrained.py and pretrained.py like you said.
Regarding the difference between model.encoder.forward() and model.encoder.streaming_forward(), does that also apply to sherpa-ncnn and/or other architectures like conv-emformer-transducer-stateless2? Because I also see a drop in performance in sherpa-ncnn and when using that architecture.

@csukuangfj
I have found a case where onnx_pretrained.py and pretrained.py doesn't return the same result (deletion occurs); nor does sherpa-onnx (but it's slightly better). This is using the exact same model and setup as the above. Note that this file has a particularly low volume, which might be related to the normalization issue.
These are my inference results:
pretrained.py:
THE BUNGALOW WAS PLEASANTLY SITUATED NEAR THE SHOR

onnx_pretrained.py:
# completely empty!

sherpa-onnx:
NEAR THE SHOR

You can find the audio file here: timit_test0.wav.zip
Hope this helps!

You can find the audio file here: timit_test0.wav.zip

I re-tested the same audio test file with the latest Zipformer model:

icefall: Zengwei/icefall-asr-librispeech-streaming-zipformer-2023-05-17
sherpa-onnx: csukuangfj/sherpa-onnx-streaming-zipformer-en-2023-06-26

And it seems to have significantly improved the performance, as suggested in this thread!
zipformer/pretrained.py:
THE BUNGALOW WAS PLEASANTLY SITUATED NEAR THE SHORE

zipformer/onnx_pretrained-streaming.py:
THE BUNGALOW WAS PLEASANTLY SITUATED NEAR THE SHOR

sherpa-onnx (with 0.5s tail padding):
THE BUNGALOW WAS PLEASANTLY SITUATED NEAR THE SHOR


Makefile: remove AUTHORS from default target

While it is generally good practice to look before jumping, many
developers enjoy running make without reading the Makefile and
indiscriminately use git add -A. This behavior, coupled with
generating AUTHORS in the default target result in garbage in various
PRs and extra round trips to remove said garbage.
Signed-off-by: Stephen J Day<EMAIL_ADDRESS>

Job: Distribution-PRs FAILED:

---
ice 'registryv2' failed to build: The command '/bin/sh -c apt-get update &&     apt-get install -y librados-dev apache2-utils &&     rm -rf /var/lib/apt/lists/*' returned a non-zero code: 100

2016/02/22 21:27:03 http: response.WriteHeader on hijacked connection
+ docker rm -f -v 708b2816c25de43b4002358154fa9d49b13af8e73f438b13fd8421f1c4e22d16
708b2816c25de43b4002358154fa9d49b13af8e73f438b13fd8421f1c4e22d16
+ rm -rf /docker-versions.AVeAG
Build step 'Execute shell' marked build as failure
[PostBuildScript] - Execution post build scripts.
[Distribution-PRs] $ /bin/sh -xe /tmp/hudson2552405925157785028.sh
+ set +x

---

Current coverage is 53.50%

Merging #1481 into master will decrease coverage by -7.36% as of 7230e9d

@@            master   #1481   diff @@
======================================
  Files          124     124
  Stmts        10441   10441
  Branches       841     737   -104
  Methods          0       0
======================================
- Hit           6355    5586   -769
+ Partial        841     737   -104
- Missed        3245    4118   +873

Review entire Coverage Diff as of 7230e9d

Uncovered Suggestions

+0.19% via .../driver/s3-aws/s3.go#240...259
+0.17% via ...ge/driver/gcs/gcs.go#471...488
+0.16% via ...ge/driver/oss/oss.go#150...166
See 7 more...

Powered by Codecov. Updated on successful CI builds.

LGTM

LGTM


Add BrokerOperatorResult and RollbackStats

Related to #1376
Add BrokerOperatorResult and RollbackStats features to the project.

rocketmq-broker/src/processor/admin_broker_processor.rs

Add BrokerOperatorResult and RollbackStats to the imports.

rocketmq-tools/src/admin/mq_admin_ext.rs

Implement BrokerOperatorResult and RollbackStats in the MQAdminExt trait.
Update method signatures to include BrokerOperatorResult and RollbackStats.

rocketmq-tools/src/admin/mq_admin_ext_async.rs

Implement BrokerOperatorResult and RollbackStats in the MQAdminExtLocal trait.
Update method signatures to include BrokerOperatorResult and RollbackStats.

Summary by CodeRabbit

New Features

Enhanced broker operations with new methods for deleting topics and resetting offsets.
Added support for rollback statistics in broker commands.

Bug Fixes

Activated previously commented-out methods for better functionality in broker management.

Documentation

Updated method signatures in the MQAdminExt and MQAdminExtLocal traits for clarity and usability.

@vishwamartur This is not just about removing the comments from the trait. You need to convert the Java file from the Java version into a Rust struct and place it in the corresponding API directory. You will need to create this directory yourself. The example I provided is just a reference.


Promise.all is not resolve type promise in intellisense

Promise.all should be able to take array of Promise<T> and resolve array of T
But it not work as expect

As you can see pairs should be number[]
Which cause from a problem in this file

So I fix it like this

@sandersn can you take a look.

@Thaina, I can't repro your fix. In fact, it seems like the problem arises from the constructor call, so I don't see how changing the type of all will help inference to PromiseConstructor.new.
The reason that the compiler infers {} for T in new Promise((resolve, reject) => { resolve(1) }) is that type inference comes only from arguments -- (resolve, reject) => { resolve(1) } in this case. But this function argument is contextually typed, so it tries to get type for its parameters from the function it was called from (new Promise).
So when the compiler requests the contextual type of resolve, it sees that it's (value?: T | PromiseLike<T>) => void. It detects the type variable T, and since it's contextually typed, knows that it has no chance to infer it. So it sets T={} and continues.

@sandersn That's not my point
I mean even that Promise.all not return Promise<number[]> it should return Promise<any[]> And when I call then(pairs) pairs should be number[] or at least any[]
But it cam out it is Promise<{}>
What I suspect is, PromiseLike<T> is not Promise<T>

The cause of this is discussed at #5254.

I mean even that Promise.all not return Promise<number[]> it should return Promise<any[]>

I'm curious as to why you think so. Can you elaborate?

@DanielRosenwasser  No it's not related in anyway. I only talk about Promise.all not a constructor of Promise. Just look at the Promise.all().then in my screenshot and you would see that the type of parameter named pairs is Promise which should not. It could be anything but Promise

I think I understand a bit better, but now I can't repro the problem. I typed in almost exactly the code from your screenshot:
Promise.all([new Promise((resolve, reject) => { resolve(2) })])
       .then(pairs => pairs[0]);

The type of pairs is {}[] for me, not PromiseLike<{}>[]. I tried this from master and the playground (1.8.x). What version of TypeScript are you using?
Note that {}[] is still a useless type, but at least it's not still wrapped in a Promise. Based on the limitation I described above, you'll need a type parameter annotation on new Promise<number> to get the correct type:
Promise.all([new Promise<number>((resolve, reject) => { resolve(2) })])
       .then(pairs => pairs[0]);

Now pairs: number[].

This is a dupe of #7566. #7566 is fixed in 2.0. I'm not sure why it works in the playground.

Never mind, I pasted in my own definition of Promise to the playground since it doesn't support ES6.


adding multiple images at once

### Tasks
- [ ] drag and drop multiple images
- [ ] file picker

I want to try this out!
But will definitely need some help as go about it, as I'm trying to contribute to this project for the first time but am an using the site for a long time

Hey @Agastya18, as you might have seen that I've already asked to work on this one and have started exploring the code base around this issue 😉

Hey @codingmickey, sure thing. I'd start with one or the other, as either drag&drop or file picker will lead you through most of what needs to be done.
For drag&drop, start with handleAppOnDrop() in the src/components/App.tsx file, modifying the getFileFromEvent() to getFilesFromEvent() (though I'm not sure how we should handle the fileHandle), and go from there.
For file picker, start with onImageAction() in the same file, adding multiple: true to the fileOpen() call, and again see where it leads you.

@Agastya18 we'll need to find some other issue for you!

Thanks a lot for this information, this helped me start.. 2 questions that I have are:

Once the images are uploaded should I distribute them on the layout? or stack one on top of the other and let the user manage that
This would be solved by creating a simple for loop, how to optimize it?

Thanks a lot for this information, this helped me start.. 2 questions that I have are:
Once the images are uploaded should I distribute them on the layout? or stack one on top of the other and let the user manage that
This would be solved by creating a simple for loop, how to optimize it?
@dwelle

Is this still in progress by codingmickey? If not, I would like to take this up @ad1992

Oops, forgot about this one. I'll be taking a look at the open PR by @codingmickey soon.

I'm waiting for this & adding image to library feature, both are going to be very helpful in my workflow.

I've raised a PR if you don't mind you can help me in reviewing it?

Is this still in progress by codingmickey? If not, I would like to take this up @ad1992

I've raised a PR if you don't mind you can help me in reviewing it?

Is this still in progress by codingmickey? If not, I would like to take this up @ad1992

I've raised a PR if you don't mind you can help me in reviewing it?

Sure

That PR seems to do what I want aside from automatic arrangement of the images, which I imagine can be done in another pull request. So is it going to be merged any time soon?

That PR seems to do what I want aside from automatic arrangement of the images, which I imagine can be done in another pull request. So is it going to be merged any time soon?
Am also waiting as you 🥲😂

tlDraw automatically arrange the uploaded images in horizontal manner, which you could also try here. Ideally you would want to arrange them in a pack, so it doesn't get too wide & make images far too small to fit the canvas.
Was thinking about such.. maybe will raise another PR whenever I get time!


Put Python algorithms in classes

Python functions should be defined in classes called Matrices and Envelope.

Hello there!
I'm new to contributing to open source projects so please bear with me.
Is this as simple as wrapping the python files with the python class syntax?
If so, solve.py needs to be part of class Matrices and wrap.py needs to be part of class Envelope right (just making sure)?
Thanks for your patience.

Hi.
Python class is new to me, so if you have ideas and suggestions, feel free to comment them.
Yes, everything can be wrapped in the Envelope class, except for the functions unwrapEnvelope(), solveForwardSubstitution() and solveBackwardSubstitution(): they go in the Matrices class because they return a matrix and a vector.
I also believe the functions evaluateSolution() and evaluateMatrices() - from the test/test.py file - can be wrapped in the Matrices class, but that can wait.

Sounds good, I'll wrap em' and send em' your way.

They're ready, do I make a new branch and push it to that?

Yes, that is it. Please make sure you added Signed-off-by: name <email> in the commit message.

I cant push, access denied. Do you need to add me as a contributor or something first?

Try GitHub - Creating a pull request.

Should've gone through. 👍
Would be awesome if you let me know if that was what you needed or if im on the wrong track.

Thank you for your PR. I'm reviewing it and there is a few points to solve.

Your classes are missing the method __init__() to initialize new instances of envelopes and matrices. Because of that, I think we can replace the method initEnvelope() with __init__() like this:

-    def initEnvelope(self):
-        return {
-            'diagg': [],
-            'enve': [],
-            'enveCol': [],
-            'enveLin': []
-        }
-
+    def __init__(self):
+        self.diagg = []
+        self.enve =[]
+        self.enveCol = []
+        self.enveLin = []

This allows the other methods to access the defined lists. Let me know how far you pretend to go on this issue.

Your commit message is too long because you missed a break line. You need to amend it with the git command git -s --amend and remove the Signed-off-by: name <email> from the title.
The author's name of the commit is showing as [your. If you are the author of the commit, it is better if you use the same name for author and Signed-off-by.

By the way, this issue affects the next major release 2.x.x, so it will take a while to push it to the main branch.

For some reason it looks like the commit is going through from another github account that I use, but I've set the correct account (TrojanCodes) which should be the sole author from now on.
Regarding 1 I can create init methods but will this in turn require the code to be refactored as well? How simple would the refactoring be?
Also I'm not quite sure what to add for the __init__() in Matrices.py since the methods themselves return what is required.
Thanks for your continued guidance.


v2.4

Codecov Report

Merging #64 (f22ca97) into master (96f0466) will not change coverage.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##            master       #64   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files           39        39
  Lines          494       494
  Branches       110       110
=========================================
  Hits           494       494

Impacted Files
Coverage Δ

src/plugins/harmonies.ts
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 96f0466...f22ca97. Read the comment docs.


  Attempting to load in same window doesn't help.
Steps to Reproduce Issue
app.asana.com click "Use google account"
Specifications

Version of Nativefier: v7.0.1
Version of Node.js: v6.9.1
OS: Windows 7

Having similar enough issues with Duolingo to report it here.  When able to finally enter username and password in the google login window, the title of the window changed to "Connecting..." and it stayed white and blank for five minutes.  When i tried to reload, Google gave a 405 error and "that's all we know" message.
Getting to the point of making enough progress to even having that error was even weirder:

Clicking the "Log in with Google" link opens in my default browser.
Right-clicking the "Log in with Google" link and choosing the "Open in new window" option does nothing.
Right-clicking another random link on the page and choosing the "Open in new window" option worked.
From this new window, clicking on "Log in with Google" opens a new Nativefier window and i was able to enter my Google username and password.

Then the above happened— reloading any of the open Windows did not log me in.  But repeating step four just now to double-check that's what i did caused the apparently successful login to complete, and expanded the new window into a full size Nativefier window with me logged in.
So, uh, it's possible to log in to Duolingo using Google accounts with Nativefier, but much harder than it should be!  Thanks for the work, and hope this bug report / crazy workaround is useful to a developer of Nativefier or other overly-persistent user like myself!

@mlncn workaround didn't work for me, but I've found another one — you can set a password for your Asana account even if you login with Google. To do that just use "Forgot your password?" link on login page, enter your Google account email (the one you log in to Asana with), and Asana will send recovery link to this email, click on it and set your password, then use it with your Google account email to log in to Asana with nativefied app.

Duplicate of more general #164, closing this one.

This is a narrow-cast solution which wokred for me and keeps non-google-login links opening externally for asana:
nativefier --name "Asana" --internal-urls "accounts\.google\.com.*" https://app.asana.com/

This used to work up until today but has stopped working for me. Asana redirects to browser for login.


Prevent error when the node-sass error does not contain a file

Fixes #156

Makes sense. Thx for pointing it out!


fix typo in Line-225

from derived _table to derived_table

@ToMakeSense : Thanks for your contribution! The author(s) have been notified to review your proposed change.

Looks good on Preview; merging.


multiple conflicting drivers gives warning in synth and did not stop the flow

Prompt
For mpw7 tapeout, I wrongly connected output of a module inside the top module to one of the bits of the io_in input signal. This gave warning in synth stage stating:
"13. Executing CHECK pass (checking for obvious problems).
Checking module user_project_wrapper...
Warning: multiple conflicting drivers for user_project_wrapper.\io_in [16]:
port tdo_o[0] of cell core_region_i (mba_core_region)
module input io_in[16]"
The tool did not stop until LVS stage
This causes waste of time since multiple drive should not be acceptable in RTL to continue on flow
Proposal
Openlane should give error instead of warning in synth stage for multiple conflicting driver issue

Definitely a good enhancement


What o365 group guid do we use in Step 4

hello,
I have reached the section where I'm inputting the admin group guid. I was wondering which o365 group I should be putting here?  as well, it appears that the name field is required -- is there a specific name we have to input?
tks

@lawrenceong001 This O365Group can be any group you create that contains the members of admins. The name field should contain the value "Settings"

Hi there folks (@lawrenceong001 @desmay @junterpfe) are you able to advise how you got Step 4 completed? I have input my GroupID, however there is no admin button. Perhaps it's something on the backend with my O365 group creation?