[FLINK-4173] Use shade-plugin in flink-metrics

replaced assembly-plugin with shade-plugin in graphite/ganglia
added shade-plugin to statsD
removed assembly-plugin in dropwizard

Thanks for your contribution @zentol. I've looked at your changes and they look good modulo some minor comments.
Have you tried your changes out on a cluster? I think we should do this before merging.

I've made the following changes:

use "append" instead of "override" for the shade configuration
include the missing org.acplt:oncrpc dependency

Could you recap why do we needs to build fat jars here at all? Simply having the transitive dependencies does not work?

It would work but is more work for the user and is less prone to version mismatches. Not using a fat-jar means that the users have to provide the dependencies and put them in the /libs folder themselves. For ganglia for example this would mean copying 5 jars. And there is no guarantee that the version against which the reporter was compiled is identical to the one provided at runtime.


Enable "Processes listening on ports" by default

Description
ROX_PROCESSES_LISTENING_ON_PORT is considered 'true' unless explicitly disabled.
Checklist

[ ] Investigated and inspected CI test results
[ ] Evaluated and added CHANGELOG entry if required
[ ] Documented user facing changes (create PR based on openshift/openshift-docs and merge into rhacs-docs)

Testing Performed
Integration tests covering this feature already exist. We should check in the CI if they were run.

cc @gavin-stackrox


workflow: update to pnpm 7

This PR includes:

included packageManager on package.json
added Contribution entry on README.md
added CONTRINUTING.md

Once merger you should publish 0.20.1


Streamline/Rationalize HPE NonStop Configuration

Using inheritance, the NonStop configuration entries become quite small, and we don't copy values all over the place.
Fixes #12858

@rsbecker, it would be nice if you could test this and confirm that my guesses are correct enough.

@rsbecker, it would be nice if you could test this and confirm that my guesses are correct enough.

Thanks. I'm going to give it a whirl.

@rsbecker, it would be nice if you could test this and confirm that my guesses are correct enough.

@levitte I think something is not right in the Makefile. I'm getting:
c99  -I. -Iinclude -Iapps/include  ARRAY(0x81724c0)  -DOPENSSL_PIC -DOPENSSLDIR=...

which kills the shell pretty fast. This is coming from this block in the generated Makefile:
CNF_CFLAGS=ARRAY(0x81724c0)
CNF_CXXFLAGS=
CNF_LDFLAGS=ARRAY(0x8171e30)
CNF_EX_LIBS=ARRAY(0x8171d50)

Note also that this change impacts PR #12928 and PR #12923. I can put together a PR to handle that once this is resolved.

Okie, I'll have a look at that.

Okie, I'll have a look at that.

Thanks. Also note that our build is broken (reported #12937) by commit 6600baa9bb6e59be91692791a6251c172a099a65, so that's not helping.

Okie, I'll have a look at that.

Thanks. Also note that our build is broken (reported #12937) by commit 6600baa, so that's not helping.

It might not be that commit, specifically, but the build was fine at commit 28833f1465.

I've fixed so there are no more ARRAY in the resulting Makefile.

I've fixed so there are no more ARRAY in the resulting Makefile.

Build is proceeding. I'll report back. It probably will be blocked by #12937

I'm not seeing the OPENSSL_SYS_TANDEM define. A sample of what is being generated is, after configuring with nonstop-nse:
c99  -I. -Iinclude -Iproviders/common/include -Iproviders/implementations/include  -g -O0 -Wextensions -Wnowarn=203,220,272,734,770,1506 -Wbuild_neutral_library -Wverbose -Wtarget=tns/e -Wsystype=oss -Wilp32  -DOPENSSL_PIC -DOPENSSLDIR="\"/usr/local-ssl3.0/ssl\"" -DENGINESDIR="\"/usr/local-ssl3.0/lib/engines-3\"" -DMODULESDIR="\"/usr/local-ssl3.0/lib/ossl-modules\"" -DOPENSSL_BUILDING_OPENSSL -DOPENSSL_VPROC= -D_XOPEN_SOURCE -D_XOPEN_SOURCE_EXTENDED=1 -D_TANDEM_SOURCE -DB_ENDIAN -D_TANDEM_ARCH=2 -DOPENSSL_TANDEM_FLOSS -D__TANDEM -I/usr/local/include   -c -o crypto/asn1/libcrypto-lib-f_string.o crypto/asn1/f_string.c

Most config macros are collected in include/openssl/configuration.h in the build tree.  You'll find it there.

Most config macros are collected in include/openssl/configuration.h in the build tree. You'll find it there.

My grep didn't find it. Tried again in configuration.h:
# ifndef OPENSSL_SYS_tandem
#  define OPENSSL_SYS_tandem 1

Uhmmm...  so what's the difference between configuration.h and configuration.h?

Uhmmm... so what's the difference between configuration.h and configuration.h?

The issue is that tandem should be uppercase, right? like OPENSSL_SYS_TANDEM not OPENSSL_SYS_tandem. I think.

@levitte would setting sys_id => 'TANDEM', fix this or would this be something the perl scripts need to handle (upshifting).

@levitte would setting sys_id => 'TANDEM', fix this or would this be something the perl scripts need to handle (upshifting).

With this change, the build is proceeding nicely so far. I still need to work around the libcrypto.so link issue (#12797), but that's a separate issue. There are two pull requests (#12928, #12923) that will either need to be modified, updated, or have a new PR issued once this is merged. I can handle that or those two can go first and we can include the changes in this PR.

@levitte would setting sys_id => 'TANDEM', fix this or would this be something the perl scripts need to handle (upshifting).

Yes, that's the answer of course...
(I haven't had to deal with new platforms in ages, I've obviously forgotten some details)

Argh. I posted results in the wrong PR. OK, here goes.

-Wshared is needed for DLL links and is not being picked up.
-Weld="-export_all" and/or -Wxld="-export_all", depending on the platform, are needed for DLL links.

I turned off dynamic-engine so that I don't have to fix this for every DLL in the universe ;). The build is continuing.

@levitte  Something else, really just a nit: The __TANDEM define is automatically generated by the compilers. Is it actually needed in the configuration?
Also, just an FYI, while there are still a few MIPS machines out there, they are completely out of support and we are not building/testing any of those.

Would this be the fix for my DLL issue, as a sample, adding the shared_ldflag?
    'nonstop-archenv-x86_64-oss' => {
        template         => 1,
        defines          => ['_TANDEM_ARCH=3'],
        cflags           => '-Wtarget=tns/x -Wsystype=oss',
        lflags           => '-Wxld="-set systype oss"',
        shared_ldflag    => '-Wshared -Wxld="-export_all"',
        shared_target    => 'nonstop-x86-oss',
    },

FYI: I have tested the above and it seems to make the difference.

The build finally completed. The following updates:
     # Itanium + oss:
     'nonstop-archenv-itanium-oss' => {
         template         => 1,
         defines          => ['_TANDEM_ARCH=2'],
         cflags           => '-Wtarget=tns/e -Wsystype=oss',
         lflags           => '-Weld="-set systype oss"',
+        shared_ldflag    => '-Wshared -Weld="-export_all"',
         shared_target    => 'nonstop-itanium-oss',
     },
     # x86_64 + oss:
     'nonstop-archenv-x86_64-oss' => {
         template         => 1,
         defines          => ['_TANDEM_ARCH=3'],
         cflags           => '-Wtarget=tns/x -Wsystype=oss',
         lflags           => '-Wxld="-set systype oss"',
+        shared_ldflag    => '-Wshared -Wxld="-export_all"',
         shared_target    => 'nonstop-x86-oss',
     },

and we should be good to merge.

Hmmm, the shared_target was supposed to refer to entries in Conffigurations/shared-info.pl (I just misspelt them), but you're right, the way it currently looks, that's an unnecessary indirection.

Also, just an FYI, while there are still a few MIPS machines out there, they are completely out of support and we are not building/testing any of those.

Would you rather I simply removed those templates, then?

@levitte  Something else, really just a nit: The __TANDEM define is automatically generated by the compilers. Is it actually needed in the configuration?

Well, no.  If it's always generated automatically, then there's no need for us to add it.

Okie, I've made changes as requested.  Considering what you say about multi-hour builds, I suppose another test will take another day :confounded:

Considering the positive feedback, I feel confident enough to take this PR out of WIP

Okie, I've made changes as requested. Considering what you say about multi-hour builds, I suppose another test will take another day 😖

That and I have to get 1.1.1h out the door.

We have 1.1.1h ported at tested (the unthreaded version). The 3.0.0 at 8e3a64fdb6 with your changes looks much cleaner. Both .so files built properly (still subject to #12797). Thank you. This can probably move to approved - I don't think my approval matters, but I've given it.

@openssl/committers, this can go to a final review

This pull request is ready to merge

Merged
c60330cb0efcde5fea80113c110499d352b2abca Configuration: Streamline NonStop entries
37fe90ad17fcaaeb33d01c660ed1c12db723d833 Configure: Show 'enable' and 'disable' config attributes
cdb5129e5c5fd8ad678c5efb1e87c91595d907b4 Use OPENSSL_SYS_TANDEM instead of OPENSSL_SYSNAME_TANDEM


moto setting AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY on import

Here: https://github.com/spulec/moto/blob/master/moto/core/models.py#L26
These should only be set if you call a mock method, otherwise just importing moto can break various other calls which are not supposed to be mocked

Just curious:  what's your use-case where you import moto but also make actual calls to AWS?

IIRC we have a secretTool which connects to dynamodb to retrieve secrets.  I need to get the secret as I just want to mock S3 and not an API call which requires a secret.

another instance: I need to connect to our secret store to retrieve a SSL cert required to mock a SSL server.

Hmm yeah, I worried that edge case might happen.. is there a chance you can restructure your code to separate the imports? Or include a  os.environ.setdefault("AWS_ACCESS_KEY_ID", "") (i.e. with empty string) before importing moto? IIRC then boto3 should try the other credential providers

ya I have a work-around for now, but ideally this should not be done on import

This might also bite you in other case. Assume you have a project with unit and integration tests.
For unit tests you want to mock everything outside of your code so you use moto.
For integration tests you might want to hit the actual AWS resources.
Now if you use pytest and use @pytest.mark.integration to distinguish which test is unit and which one is integration and you run them like this:
python -m pytest -m "not integration" # for unit tests only
python -m pytest -m integration # for integration tests only
You'll end up with running integration tests with moto populated:
os.environ.setdefault("AWS_ACCESS_KEY_ID", "foobar_key")
os.environ.setdefault("AWS_SECRET_ACCESS_KEY", "foobar_secret")

Which obviously is not what you want. I think that's because you're running scan for the annotations for all tests and you then hit the import for unit tests and you end up with this env setting code being executed.
A workaround here is to keep the tests in separate directory ("/integration" in my case) and then run it with:
python -m pytest tests/integration
(alternatively add '-m integration' to be extra safe that no unit tests in this package will be executed.)
Nevertheless, this is yet another workaround and if we'd have an option to control if it would be great. Something like MOTO_DISALLOW_AWS_CREDENTIALS env_var or switch?.
By default -- not to break backward compatibility -- it might be turned off.
Problem is that for boto3 env properties are very high in the hierarchy. Higher are only in-code values which I don't want to hard-code for security and configuration reasons.

Can someone give the PR here a try and let me know if it solves the issue? https://github.com/spulec/moto/pull/2285

@spulec isn't that the same thing effectively?  After you patch a particular service, anyone who requests os.environ['AWS_ACCESS_KEY_ID'] for example will get the fake key.  It should be that only requests to mocked services should get the mock key.  The way I achieved this locally was to explicitly set the access_key_id, etc in the create_client call.

Well, this moved from happening on import to only happening when the moto context is activated so that seemed like an improvement.
I understand what you are saying though. Given that boto/boto3 requires some form of authentication, I'm not sure what other options we have. It seems too painful to force everyone to change the client instantiations. We might be able to mock credentials via instance metadata, but that has been a struggle in the past.

can't you instead mock the create_client method like I was suggesting and override the credentials there?  I think that would be marginally better than setting environment variables.

That assume the client is created after the mocking is activated, yeah? I don't know how popular it is, but I have seen a number of people that setup their clients at the module level.

I think it's counter-intuitive to create a client, then say you want to
enable mocking s3, and then have that retroactively affect clients you've
already created.  Perhaps this warrants a major version change.
On Mon, Jul 15, 2019 at 6:47 PM Steve Pulec<EMAIL_ADDRESS>wrote:

That assume the client is created after the mocking is activated, yeah? I
don't know how popular it is, but I have seen a number of people that setup
their clients at the module level.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/spulec/moto/issues/2058?email_source=notifications&email_token=AA6Q77L42FXTCSGRP3L2C33P7USDVA5CNFSM4GXJL2IKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZ7OEHQ#issuecomment-511631902,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AA6Q77JVEUWZ7FZ4WBFDSQTP7USDVANCNFSM4GXJL2IA
.

Hmmmm, I'm not sure. Given that the clients are lazy (at least in boto3), I don't have a good feel for how people think about them.
Just to make sure I understand: is there a problem with the current approach? If it is only that a user reading the environment variables will see something different while the mock is activated, that doesn't seem like a huge deal to me (though please let me know if there is a use-case I'm not considering with this).

Here's the scenario, before or after I enable s3 moto I create a dynamodb client to get secrets, i then enable s3 Moto, now my secrets calls fail. I need this secret because it contains the password for the mock DB we create. This is marginally better in that after import, but before enabling s3 mocking I can pull the password, but ideally mocking s3 should not affect dynamodb calls.

Ah, I see.
Because of various issues we've had in the past, I get extra nervous about allowing any sort of real connections to AWS when a user has any service activated. Some more here. I have even caught myself a few times where I forget to add the decorator for a particular service and am thankful that Moto throws an exception. I don't keep real credentials available, but it is clear that much of our userbase does.
As far as defaults go, I'm tempted to lean toward anything AWS not working. If there is a way we could allow an explicit override of that, I would be in favor though.

Perhaps then there should be an explicit global moto context  to inform users what's happening

I'm not sure I understand. Are you talking something like https://github.com/spulec/moto/issues/2076 or something different?

Since we fixed this to no longer happen on import, I am going to close this issue. If someone wants to take a shot at making this even less intrusive, I'm open to ideas.


Assessment3

Metric
Value

alert_status
:x: ERROR

bugs
6

code_smells
53

reliability_rating
D (at least 1 Critical Bug)Note: A being best, E being worst

security_rating
A (0 Vulnerabilities)Note: A being best, E being worst

vulnerabilities
0

Metric
Value

alert_status
:x: ERROR

bugs
6

code_smells
53

reliability_rating
D (at least 1 Critical Bug)Note: A being best, E being worst

security_rating
A (0 Vulnerabilities)Note: A being best, E being worst

vulnerabilities
0


 I'm asking for a way to achieve something, but I think there are many ways it could be done and I don't know which is most reasonable. In essence, when looking at a histogram of events (like durations of goroutines), I like to filter the histogram to interesting subareas (like examining the long tail in more detail). Then I find crazy outliers that I want to examine on the timeline, but it's hard to find those goroutines on the timeline from the histogram.
This would be easier if any (or several) of the following were possible:

The histogram had a way to open a table of the raw data that was synchronized with its current view, such that narrowing the histogram narrowed the data in that table
The histogram added a context menu item for bars representing only a single data element that let me jump to that element
The "Goroutines" tab (the one next to the histogram) let me sort the data by duration (this is a less general and less cool solution, though perhaps easier).

Making the histogram present a filtered view of the dataset it represents seems like the most generally useful approach, as it can be applied regardless of the dataset being visualized.

aea47c657c7c0180e7754c3b5fbfa002b1608f03 adds an option to filter the goroutine list by the selected range in the histogram.


Allow publishers to edit paymentCollector

Sometimes, a publisher will decide that entire revenue for a service should be sent to a different address (not the publisher's one).
Let's allow this in UI as well (maybe in Edit Metadata).
It's already in ocean.js (https://github.com/oceanprotocol/ocean.js/blob/main/src/contracts/Datatoken.ts#L418), so this is mostly design
Caller must be:  NFT.owner() or have feeManager rights

We need to add information on the paymntCollector on the docs before having the UI. So people understand what it's all about: https://github.com/oceanprotocol/docs/issues/1150


Clone and resolve dependencies locally before deployment

I would like to achieve the following workflow for deployment:

Check remote server
Create remote release
Clone repository to local directory
Resolve dependencies in local directory
Send result to remote server (preferably via rsync)
Perform migrations on remote server

Can this exact order be achieved with Rocketeer? It seems dependencies are always resolved after deployment.
Background: in a perfect world I don't need git, composer or similar on the remote server for a deployment.

+1
I also need to clone the project locally before uploading to a remote server.

The cloning part can be achieved with the LocalClone strategy, a speciality here is the local dependency resolving. With the current structure the only option I see is a custom deploy strategy which performs clone, resolves dependencies and then uploads to the remote server. The original dependencies strategy needs to be disabled here.

And I did exactly that now. With cc0656c it is now actually possible to disable strategies.
Now I really don't need neither Git nor Composer on the servers. :-)


add philips tv to stable

Automated adapter checker
ioBroker.philips-tv

:thumbsup: No errors found
Add comment "RE-CHECK!" to start check anew


Grammer and added end links

@eriklindeman
IMPORTANT: When the changes are ready for publication, add a #sign-off comment to signal that the PR is ready for the review team to merge.
#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team

#sign-off


Update README.md to include TEST_PROJECT_ID

Creating pull request for the correct branch, 'readme'. TEST_PROJECT_ID is required during the build phase, otherwise the PubSub integration test in cmdline-pull will fail. Add it to the README file for the cmdline-pull sample.

Takashi-san (@tmatsuo),
I have updated the README file but CI keeps failing at the openssl line, when attempting a pull request. The update fixes a key issue in the documentation of cmdline-pull sample, for people who are generally unfamiliar with Google Cloud samples.
Any ideas what's going on here?
It would be really appreciated if you can please look into the above.
Thanks!

@hitstergtd Thanks for the pull request. Nevermind on the travis failure. Since we're using secret env vars, PRs from other repo can not utilize those secrets and fails. I will fix this situation later.

@tmatsuo,
No problem at all and thank you for merging my PR, much appreciated. I'll keep it in mind for the future.


Changed session file extension, default session config name is now as…

… proposed

Not merging! :)


Investigate if the if test should escape the value

Probably not, adding the task to double check

function render (__o) {
  if (__e(__o.foo)) {
}

Looks good. We don't need escape conditions and it was already fixed.

Can we please add few specs for this to the output.js file to double check?


Add scroll in Web editor Loader page

Godot version:
3.2.4 beta 4(Web Editor)
OS/device including version:

Ubuntu 20.10, Mozilla Firefox
Screen 1366x768
Issue description:
Couldn't scroll in Loader page in Web Editor
Steps to reproduce:
Open Web Editor and try scroll

I'm working on the HTML5 editor page and downsized the logo so that scrolling shouldn't be necessary anymore, even on 1366×768 or mobile displays.


The EMG and EEG data for 512Hz l download can not be load by np.load

this is the .npy,  just contain:/annex/objects/MD5-s36978768--33f28c453964139a5852eca457887b83 in it

Due to size limitations, EEG and EMG epochs with a sample rate of 512hz are temporarily placed in the G-node database (https://gin.g-node.org/NCClab/EEGdenoiseNet).


Demo

A demo notebook with actual data that walks users through the process.

Use resting-state analysis as example application

Closing this, as we already have 3 demo/tutorials


"403 Invalid API Token: caveat must be an array" when using hatch publish

Describe the bug
I receive the following error when using hatch publish for a package that has been published fine many times before:
Error uploading to repository: https://upload.pypi.org/legacy/ - Client error '403 Invalid API Token: caveat must be an array' for url 'https://upload.pypi.org/legacy/'

I have not changed anything to do with the publishing (HATCH_INDEX_USER and HATCH_INDEX_AUTH are the same and I haven't touched pyproject.toml.
Expected behavior
Package is published
To Reproduce
Publish a package using hatch in a GitHub Actions workflow
My Platform
I'm running the publish workflow in GitHub Actions. You can see it here if it's any help:
https://github.com/OCHA-DAP/hdx-python-utilities/blob/main/.github/workflows/publish.yaml

I am getting this same error using twine

Thanks, I'm looking into this. Please avoid adding "+1" or "me too" comments and 👍 the top-level comment if you're affected instead.

This should now be resolved.


Add material for 2nd tutorial

I'll add the solution to the tutorial questions to the book_model.json file after the tutorial has taken place

Thanks for the help with the AoG package, @greimel! I have changed the code for all figures following your suggestions


add waitToHide

@DavertMik: I have added the feature that I proposed in https://github.com/Codeception/CodeceptJS/issues/51

Thanks!


When you zoom into map (and therefore certain points are off screen) they should still be weighed in and not discarded...

I am using this to overlay a uiimage of heatmap over mapkit. Let's assume for a minute we are talking about a mkmapview of a city. I have points (busyness) for how many ppl are in certain locations...therefore the heatmap of those locations should be denser (brighter).  This works when all the points are visible on the main screen...but when you zoom in on a certain area, the busiest points of the city are no longer taken into account when calculating brightness of blobs.
ie: if you have a lot of ppl on top left of map, it is bright yellow with a red background, whereas a lower populated area in center of map is just red.  This is as it should be, BUT if i zoom in to center of map, the top left "weight" of the top left coordinates is not taken into account and now center of map is bright yellow. For purposes of brightness of each blob, each weight should always be taken into account. Changing map rect (or zoomlevel) should not change brightness of each section of heatmap.
I have no idea which section is in control of this however it SEEMS to be between 1.0 and 1.5 that is causing this (right after you copy the nsarray into array.
Ideas on a fix?

Try playing with the weightsAdjustmentEnabled and groupingEnabled values.

This does not achieve the effect of "remembering" the thrown away values from Step 1 "Filter out range of points"...but i will work on it and post my findings.
Thanks!


Repository structure revision: sdk/[x] instead of sdk/[service]

In our recent conversations with the Azure Spring Boot team we've been looking at what it might look like to pull the Azure Spring Boot modules (into the azure-sdk-for-java repo). The Azure Spring Boot modules cover a cross section of existing service areas as well as some that don't fit neatly into the service folders we have today.
I would like to propose that we loosen the definition of what a folder means under the ```sdk`` folder to include supporting libraries for particular frameworks such as Spring Boot. Whilst this issue came up in the context of the Java mono-repo I think it is worth getting a consensus on this for all our repos so when we have similar situations come up in other repos we don't have to debate the pros and cons again.
Applying this to the spring boot libraries, we would end up with this structure:
sdk/
  springboot/
    azure-spring-boot-bom
    azure-spring-boot-parent
    azure-spring-boot
    azure-servicebus-jms-spring-boot-starter
    ...

Alternatively, if there was a clear connection between a service directory and the spring boot library we could go with this structure:
sdk/
  springboot/
    azure-spring-boot-bom
    azure-spring-boot-parent
    azure-spring-boot
  servicebus/
    azure-servicebus-jms-spring-boot-starter
    ...

I prefer the first option because its more likely that the spring boot libraries are going to co-evolve together. Building them together will result in less developer friction - and they are more likely to need to ship together when something like the version of spring boot they are built against changes.

Adding @weshaggard @JonathanGiles @joshfree @hemanttanwar @kurtzeborn @JimSuplizio to the issue so we can discuss and settle this when folks start filtering back from the holiday break.

@mitchdenny is the springboot any more then a set of dependency libraries? For example does it also include dramatically different toolsets? If it is just another dependency I think I would prefer that we go with your second option and have the libraries still tied to the service as opposed to the framework. I see this no different from the azure-core dependency. In fact it would be ideal if we could hide most of the springboot dependency in some flavor of an azure-core library.
As for loosening the meaning of "service" I'm not too concerned with that as the most important thing for me is that we keep the structure consistent as "sdk//" and hopefully the "" ties together the set of packages in some way. If we felt that springboot was the largest tying factor then I'd be fine with naming that but at this point I still think the service calls are probably the largest tying factor so I'd lean towards keeping your second suggestion.

+1 on service libraries ending up in the service folders. Core libraries for Springboot could follow Azure Core pattern.

I would agree with @weshaggard for structure.
I like to clarify few points.

Most of springboot modules will reside in sdk/springboot  : This will allow developer/users of Spring Boot community to point to one place where  everything related to "springboot " reside.

From Engineering toolset point of view, its no different and it can reuse existing tools.

We will try to keep Azure.core  framework agnostic as much as possible.
Azure.core is working at protocol layer like HTTP and I with to keep it that way.
sdk/springboot will have its own equavalent of azure-core but more specific to springboot

Case by case basis we can discuss which sprongboot library need Azure.core like pattern.

Most of the springboot could work with sdk/<something>/<package but following two might have one extra level and  we might have to restructure these

a.  sdk/springboot/spring-integration-azure/spring-integration-servicebus  @ https://github.com/microsoft/spring-cloud-azure/tree/master/spring-integration-azure/spring-integration-servicebus
b. 'spring-cloud-azure-servicebus-stream-binder-core' @ https://github.com/microsoft/spring-cloud-azure/tree/master/spring-cloud-azure-stream-binder/spring-cloud-azure-servicebus-stream-binder-core

Why would each module live in the service folder it supports? Does the springboot community discover the modules through source browsing or is there another listing of modules? What if there was a springboot folder with all the base support modules with a README that linked to the service folder implementations for each service. One of the key things I would like to see happen here is that the spring modules build on top of the Track 2 libraries and get built and tested in the pipelines for each of those services . When those libraries are updated in a way that needs spring module updates, the engineer would be able to do all the work in the context of their service work.

My initial instinct was to have an /sdk/springboot/ directory structure containing all spring-related code. I've since convinced myself that the better approach is to have the spring libraries co-exist with the service library they are built on top of.
The reason for my initial thinking was discoverability - having all spring code in a single directory meant it was easy to have a single readme and a single place for users to find all available functionality that we offer. As Peter notes, and as was noted in the meeting the other day, this is not the primary way that spring developers discover libraries (they would normally use https://start.spring.io/). Additionally, just because we do decide to co-locate the spring code with the service library does not mean that the readme in /sdk/springboot cannot be maintained with links out to each library, so I don't think we are losing much at all.
The stronger argument to keeping the spring boot code with the service library is to ensure all builds are taking into account the spring code, so that we instantly see build failures and can react immediately.
In short - I agree with the second approach in Mitch's description above.

So to close this out I think we are coalescing around the following approach:

Were a sprint boot module has a primary relationship with a particular Azure service, that module will be co-located with those modules.
Spring boot modules that are not associated with any particular service will exist in sdk/springboot (including any parents POMs, BOMs etc).

I'd like to propose an intermediate step here. Initially put everything in sdk/springboot and then as part of making the libraries use track 2 versions of the SDK we move them in to the appropriate service directory. This would allow us to get the code itself migrated as is into the springboot directory and start standing up the engineering system elements around it and the slowly we peel off the libraries as appropriate.

This plan sounds good to me although I think we should ask ourselves whether we should just be doing the track 2 part initially. Basically, leave track 1 where it is and as things move to track 2 they go into the central repo.

We had a follow on conversation with the spring team and initially it will be easier to put them all into sdk/spring with the intention to break them out when and if we can. There are some relatively tight couplings that will make this complex in the short term.
Additionally they also maintain different versions of the modules on branches to support different versions of Spring Boot/Spring Cloud which adds some complexity about how we bring these two worlds together.
All that said I think that this issue gives us our answer in terms of desired direction. I'm going to close the issue down and when its the right time to break modules out of sdk/spring/ we'll do that.

Sorry for reopen this issue, but I feel we still need to revisit this decision. I'd prefer to keep the Spring modules under sdk/spring:

It’s easy for Spring developers to visit our repo and browse code

We will redirect developers to /sdk/spring from the old repos

It’s more convenient for daily development as most modules are managed by Shanghai team

I guess you've been in the repository for a while now, how have you found integrating changes from various other sdk/[service] folders. Has it been challenging or have you found the inner-loop bearable?
When I opened this thread I indicated my preference of having everything in the spring folder because I assumed that those libraries would co-evolve. But looking forward to when you've fully cut over to track 2 dependencies are they more likely to revolve with the service areas they integrate with or their other peers in the set of spring libraries?

Once the work to get to track 2 is all complete, what is the most likely workflow that will happen? Is it changes across many spring modules or is it updating specific modules based on changes in the underlying service libraries?

Another way to look at what @mitchdenny and @Petermarcu are saying is to think about how these will likely ship. Will they ship as a full set of spring libraries? If so then maybe having them in the sdk/spring folder is the correct answer. Or will they generally need to be shipped when the individual service ships? If so then they should go under the sdk\service folder instead. In general we have been working under the assumption most things should group under the service as opposed to the technology because you will generally need to update all the libraries for a service at the time the service changes, so they are on the shipping schedule matching the service and not the technology they are built on top of.
Another prospective on why we group under the service folder is that folks coming to our repository are generally going to go into the sdk folder and find the service they want to use and then figure the library of the service they want to use.

I'm closing down this issue. I think were we've landed for now is that all spring components stay in the sdk/spring folder (with the exception of the BOM). The reason for this is that all of the auto configuration logic exists within azure-sprint-boot and the various starters are just (mostly) codeless wrappers around that logic.
Unless we are going to tear apart azure-spring-boot into smaller modules and put it in under each service directory. The reasoning is that if we change KeyVault, in order to update the Spring Boot starter we need to update azure-spring-boot anyway, so it would create a circular dependency between the sdk/spring directory and the sdk/keyvault directory.
If folks think I'm wrong on my intuition around closing this down feel free to open it up again :)

I'm going to re-open this until I can fully understand what is happening here.
@jialindai based on your comments at https://github.com/Azure/azure-sdk/pull/1155#discussion_r428489545 it would appear that we are supposed to have multiple spring libraries that contain implementation for each service, but that seems to contradict what @mitchdenny is claiming that all the implementation lives in azure-spring-boot. As I stated in my comments in the other thread I'm not a spring expert but some things just don't seem to add up for me about how the implementation is factored. It is either in one library, which violates the guidelines in https://github.com/Azure/azure-sdk/pull/1155 or it is split by service in which case I'd still argue that we split the implementations out into the service folders.

I got an e-mail from @saragluna on another question related to these discussions. This time it was about spring-data-cosmosdb and where we should put it. The spring-data-cosmosdb module is not a Spring Boot module, it is a Spring Data module.
We really have two options for where we place it. The first is in sdk/cosmosdb, and the other is in sdk/spring. Where you place it has more to do with whether it co-evolves with cosmosdb (that is you want to immediately know if changes in cosmos break the Spring module), or whether its more important to be stabilized alongside the rest of the Spring Boot modules.
Regardless of which route we go the module will have to socialize well with the engineering system and the other modules that it co-habitates with.
My gut tells me that this library should co-evolve with cosmosdb, and as part of this migration effort we should update it to be track 2 compliant (for a Spring Data definition of track 2 compliance).

Also adding @kushagraThapar to this issue since I've now looped in the spring-data-cosmosdb thread which he was added to - which is related to this PR.

Thanks for the discussion and sorry for my late response.
The Spring modules can be roughly divided into following categories, and each of these categories is a separate release unit:

Azure Spring auto configuration

Owned by Azure Spring team
The code was in azure-spring-boot repo
It includes auto configuration code for services (key vault, aad, cosmos, service bus, storage)

Cosmos Spring module

Owned by cosmos team
The code is in spring-data-cosmosdb repo
It integrate cosmos SQL API with Spring data API

Gremlin Spring module

Owned by Azure Spring team
The code is in spring-data-gremlin repo
It integrate cosmos gremlin API with Spring data API

App config Spring module

Owned by App config team
The code is in spring-cloud-azure repo
It integrate app config with Spring config API

Messaging Spring module

Owned by Azure Spring team
The code is in spring-cloud-azure repo
It integrate service bus, event hub with Spring Cloud stream API

When considering where to put the code, one category should be considered together.
I'd prefer to keep the categories owned by Azure Spring team under /sdk/spring. Because different categories are released separately, this requires different pipelines under /sdk/spring.
For cosmos Spring module and App config Spring module, it's open to me, I'd like to get inputs from @kushagraThapar and @mrm9084.

We don't need separate pipelines to publish separate packages. Whilst it is generally true that we would expect modules co-located under sdk/spring to ship together at roughly the same time, the engineering system provides the capability to release a subset of the modules (just by approving the package you want to release).
The folder structure isn't really about ownership, its more about co-evolution. So if you want to be in a world where your customers who are using the cosmos spring module can expect that when a new version of cosmos modules are released that there will be a corresponding update to the cosmos-spring module - then it makes sense for them to be co-located, after all I would imagine that cosmos modules would be the major internal dependency for the cosmos-spring module.
Does that make sense?
The benefit of the arrangement is that you can then work on adding unit tests which validate that the updates to cosmos immediately work with your existing spring integration code. If you placed this code under sdk/spring you wouldn't pick this up until after cosmos had shipped an update and you decided to update the dependency.
Long after you have the opportunity to provide feedback on the Cosmos changes as they come through. The same applies to all other integrations that you've got.

I agree with @mitchdenny and @jialindai  , cosmos spring module (spring-data-cosmosdb) should be placed within sdk/cosmos directory.
azure-cosmos module is the major and only internal dependency on spring-data-cosmosdb. Every time there is a azure-cosmos release, it is highly likely that spring-data-cosmsodb module also needs to be updated and released. If placing them together also provides the benefit of testing spring-data-cosmosdb module immediately after any changes are made in azure-cosmos module, then that would be the best way going forward.
Also, azure-spring modules have a release cadence which does not completely inline with spring-data-cosmosdb module's release cadence. spring-data-cosmosdb will be released more frequently based on customer feedback, hotfixes, new changes in azure-cosmos, and so on.
If spring-data-cosmosdb is placed within azure-spring, then managing those extra releases might just be extra work which can be avoided if spring-data-cosmosdb is to be placed with sdk/cosmos.

@kushagraThapar and I had a chat earlier today and he was able to clarify a few things for me. We then chatted about the best way to proceed.
From the previous posts it seems that we're all in agreement that spring-data-cosmosdb will go into the sdk/cosmos directory. The reason is that there's a tight coupling between azure-cosmos and spring-data-cosmosdb. There is, however, a catch to this: spring-data-cosmosdb is tightly coupled with the v3 azure-cosmos and work will need to get done, by cosmos, for spring-data-cosmosdb to target the v4 version of azure-cosmos. This means that Spring will need to continue targeting the released version of spring-data-cosmos, even after this has been moved into the repository, until this work has been done.
Here's the roadmap of what needs to get done:
What needs to get done now:

Move the source for spring-data-cosmosdb into sdk/cosmos. (Proabably should rename spring-data-cosmosdb to azure-spring-data-cosmosdb but that could be done afterwards) It should have azure-client-sdk-parent as its parent, it should be added version_client.txt and should also have an entry in the sdk/cosmos/ci.yml file. If it doesn't compile there should be just enough work done to get it compiling/passing in the pipelines. It should continue to have com.microsoft.azure:azure-cosmos as its dependency (andt here's already an entry in external_dependencies, which was previously added by the Spring team) until it has been made v4.
As previously mentioned, Spring starters and samples will continue to target the released versions of com.microsoft.azure:spring-data-cosmosdb and com.microsoft.azure:azure-cosmos (again, both already have entries in external_dependencies)

What will need to get done during spring-data-cosmosdb v4 work:

spring-data-cosmosdb will start using the "current" azure-cosmos version as it's dependency

What will need to get done after the spring-data-cosmosdb v4 work is considered good enough for Spring to start picking up:

Since the v4 version of spring-data-cosmosdb won't be released, an unreleased_ dependency should be declared in version_client.txt. This dependency is what Spring will use until spring-data-cosmosdb v4 has been shipped.
Once the v4 version of spring-data-cosmosdb has been shipped, only then will the unreleased_ dependency be removed from version_client.txt and Spring will then start using the dependency version of Spring.

The original intent behind this issue was to build some consensus on how we should approach placing modules in the source tree structure. We used Spring as an example in the Java repository but the intent of the issue is try to get some consistent guidance on how we can structure things within our repositories. I'd like to tie a bow on this issue with some guidance for teams that struggle with this in the future. I'm thinking some general statements like the following:
Where should your module be located in the source tree?
Most of the azure-sdk-for-[language] repositories contain an sdk/ folder within the root of the repository. Under that folder you will find many more folders, typically aligned to each specific Azure service - for example sdk/cosmos/ and sdk/eventhubs/.
If your module/library/package is the primary client library that is used to access a service for a particular language then it must be located under the appropriately named sdk/[service] folder.
In addition to the primary client libraries that must be located under the sdk/[service] folder, any libraries which interface application frameworks (e.g. Spring Data, ASP.NET, etc) with a particular service should also co-locate their libraries within the sdk/[service] folder. This is because the primary client library within the service folder represents a internal dependency within the SDK that we would want to validate and test on each incoming PR.
In limited circumstances, libraries for application frameworks MAY be located outside of the sdk/[service] folder structure and instead may occupy a location determined by the name of the application framework (e.g. sdk/spring/). This is only appropriate IF the module contains common infrastructure for application framework integration or where all integration with specific services are co-located within a single module. Note that there needs to be a good justification for factoring code this way since it can introduce inefficiencies in the developer inner loop and make some classes of problems more difficult to detect.

@mitchdenny I'm fine with updating the structure guidance similar to what you called out but can you do that update in our repo layout policy doc https://github.com/Azure/azure-sdk/blob/master/docs/policies/repostructure.md?
However from my prospective this issue is more about trying to decide on whether or not we should allow spring to be an exception to this guidance. From what I've heard so far my stance is still the same which is I feel anything that has a dependency and needs to change with the service should be in the service folder and if it is application framework specific without any real service implementation then it is OK to go under the spring folder. I think of the spring folder much like the core folder in that there are some common pieces needed to work on spring but there are no concrete service implementations included as part of it. It sounds like we have agreement for cosmos and I think we should follow the same model for the other services.

Just merged the change into the policy doc. Should we leave this issue open until Azure Spring Boot does the work to break up azure-spring-boot? I'm not sure were all there yet (I agree with you in principle though, I just think the horse has bolted).

@mitchdenny @jialindai For App Configuration there was hope to unlink our release from those of the other Spring modules. We are actively working on adding new features to the library. In addition we have 4 libraries currently that fall under App Configuration, spring-cloud-azure-appconfiguration-config, spring-cloud-azure-appconfiguration-config-web, spring-cloud-azure-feature-management, and spring-clould-azure-feature-manatement-web.
Also, for our packages their is no dependency on the other Azure Spring packages, and only the two appconfiguration packages have a dependency on from the azure-sdk-for-java, appconfiguration, key vault secrets, and identity.


Migrate Doc Editor/Backend to Emerald Cluster

Title of ticket:
Description
Summarize issue
For Production deployment , we need Emerald Cluster with SDN to be ready( FYI @lmullane ), till then for DEV, TEST we can use existing Silver OCP. ***** In case if Emerald SDN release get delayed from November 2022, then we need to set up on premise deployment . this will be a different effort !!!!! - as informed ***** - this will get decided as we go - OCTOBER 20th.???
cc: @m-prodan
DOD

[ ] OCP templates for all instances
[ ] OCP N/W yamls
[ ]  SDN N/W  set ups, policies,
[ ]  Communication with existing Silver OCP
[ ] deployments - DEV, TEST and PRod

As discussed @abin-aot, please fill out DOD and put an estimate.

Abin will start this one, likely mid-sprint. Unlikely to complete by sprint end

Started and moving back to Product backlog, due to changes required again using IStore requests etc. Informed
PO and Dev team , https://chat.developer.gov.bc.ca/channel/nsx-early-adopters/thread/y9kqTbE3wN86nRGMZ?jump=y9kqTbE3wN86nRGMZ


Register page doesn't work

I have followed the tutorial for running in docker, and when I navigate to http://localhost:3000/signup, I am unable to sign up. Is the identity function missing now?
Is there any patch for the self-hosted version so that I don't need to manually access prisma studio? Or at least can the user data injection be done with script?

I'm also having deployment problems. The schema.prisma is causing errors and I can't get beyond this.

@Kfelts can you try docker log <containerID>? But I don't think we have the same error.

Thanks for the feedback y'all!
Logs and screenshots would be immensely helpful if you could provide some here

@krumware Thank you for replying! So I have the following config(I haven't changed anything apartfrom copy and paste from repo):
.env
# Set this value to 'agree' to accept our license:
# LICENSE: https://github.com/calendso/calendso/blob/main/LICENSE
#
# Summary of terms:
# - The codebase has to stay open source, whether it was modified or not
# - You can not repackage or sell the codebase
# - Acquire a commercial license to remove these terms by emailing<EMAIL_ADDRESS>NEXT_PUBLIC_LICENSE_CONSENT=
LICENSE=

BASE_URL=http://localhost:3000
NEXT_PUBLIC_APP_URL=http://localhost:3000

POSTGRES_USER=unicorn_user
POSTGRES_PASSWORD=magical_password
POSTGRES_DB=calendso
DATABASE_HOST=database:5432
DATABASE_URL=postgresql://${POSTGRES_USER}:${POSTGRES_PASSWORD}@${DATABASE_HOST}/${POSTGRES_DB}
GOOGLE_API_CREDENTIALS={}

# Remove this var if you don't want Calendso to collect anonymous usage
NEXT_PUBLIC_TELEMETRY_KEY=js.2pvs2bbpqq1zxna97wcml.oi2jzirnbj1ev4tc57c5r

# Used for the Office 365 / Outlook.com Calendar integration
MS_GRAPH_CLIENT_ID=
MS_GRAPH_CLIENT_SECRET=

# Used for the Zoom integration
ZOOM_CLIENT_ID=
ZOOM_CLIENT_SECRET=

# E-mail settings
# Configures the global From: header whilst sending emails.
<EMAIL_ADDRESS>
# Configure SMTP settings (@see https://nodemailer.com/smtp/).
EMAIL_SERVER_HOST=smtp.example.com
EMAIL_SERVER_PORT=587
EMAIL_SERVER_USER=email_user
EMAIL_SERVER_PASSWORD=email_password

# Encryption key that will be used to encrypt CalDAV credentials, choose a random string, for example with `dd if=/dev/urandom bs=1K count=1 | md5sum`
CALENDSO_ENCRYPTION_KEY=

# It is highly recommended that the JWT secret must be overridden and very unique
JWT_SECRET=secret

NODE_ENV=production

version: '3.1'

volumes:
  database-data:

networks:
  stack:
    external: false

services:
  database:
    image: postgres:14.1
    restart: always
    volumes:
      - database-data:/var/lib/postgresql/data/
    env_file: .env
    networks:
      - stack

  calendso:
    image: calendso/calendso:latest
    restart: always
    networks:
      - stack
    ports:
      - 3000:3000
    env_file: .env
    depends_on:
      - database

Reproduce:

Go to http://localhost:3000/signup
see the following error

Log in docker:
[nix-shell:~/calendso]$ docker-compose up
Starting calendso_database_1 ... done
Starting calendso_calendso_1 ... done
Attaching to calendso_database_1, calendso_calendso_1
calendso_1  | + /app/scripts/wait-for-it.sh database:5432 -- echo database is up
calendso_1  | nc command is missing!
calendso_1  | + npx prisma migrate deploy
database_1  |
database_1  | PostgreSQL Database directory appears to contain a database; Skipping initialization
database_1  |
database_1  | 2022-01-06 15:32:26.381 UTC [1] LOG:  starting PostgreSQL 14.1 (Debian 14.1-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
database_1  | 2022-01-06 15:32:26.381 UTC [1] LOG:  listening on IPv4 address "<IP_ADDRESS>", port 5432
database_1  | 2022-01-06 15:32:26.381 UTC [1] LOG:  listening on IPv6 address "::", port 5432
database_1  | 2022-01-06 15:32:26.405 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
database_1  | 2022-01-06 15:32:26.436 UTC [26] LOG:  database system was interrupted; last known up at 2022-01-06 15:24:45 UTC
database_1  | 2022-01-06 15:32:26.584 UTC [26] LOG:  database system was not properly shut down; automatic recovery in progress
database_1  | 2022-01-06 15:32:26.593 UTC [26] LOG:  redo starts at 0/181A8C0
database_1  | 2022-01-06 15:32:26.593 UTC [26] LOG:  invalid record length at 0/181A8F8: wanted 24, got 0
database_1  | 2022-01-06 15:32:26.593 UTC [26] LOG:  redo done at 0/181A8C0 system usage: CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s
database_1  | 2022-01-06 15:32:26.655 UTC [1] LOG:  database system is ready to accept connections
calendso_1  | Prisma schema loaded from prisma/schema.prisma
calendso_1  | Datasource "db": PostgreSQL database "calendso", schema "public" at "database:5432"
calendso_1  |
calendso_1  | 53 migrations found in prisma/migrations
calendso_1  |
calendso_1  | No pending migrations to apply.
calendso_1  | ┌─────────────────────────────────────────────────────────┐
calendso_1  | │  Update available 3.0.2 -> 3.7.0                        │
calendso_1  | │  Run the following to update                            │
calendso_1  | │    npm i --save-dev prisma@latest                       │
calendso_1  | │    npm i @prisma/client@latest                          │
calendso_1  | └─────────────────────────────────────────────────────────┘
calendso_1  | + yarn start
calendso_1  | yarn run v1.22.15
calendso_1  | $ next start
calendso_1  | ready - started server on <IP_ADDRESS>:3000, url: http://localhost:3000
calendso_1  | prisma:query SELECT "public"."users"."id", "public"."users"."username", "public"."users"."email", "public"."users"."name", "public"."users"."bio", "public"."users"."avatar", "public"."users"."theme", "public"."users"."plan" FROM "public"."users" WHERE "public"."users"."username" = $1 LIMIT $2 OFFSET $3
calendso_1  | prisma:query SELECT "public"."users"."id", "public"."users"."username", "public"."users"."email", "public"."users"."name", "public"."users"."bio", "public"."users"."avatar", "public"."users"."theme", "public"."users"."plan" FROM "public"."users" WHERE "public"."users"."username" = $1 LIMIT $2 OFFSET $3
calendso_1  | prisma:query SELECT "public"."users"."id", "public"."users"."username", "public"."users"."email", "public"."users"."name", "public"."users"."bio", "public"."users"."avatar", "public"."users"."theme", "public"."users"."plan" FROM "public"."users" WHERE "public"."users"."username" = $1 LIMIT $2 OFFSET $3

Awesome, thanks for the detail!
I'm talking with the cal.com team on slack about it

Thanks. This is a really promising project, good job!

hey there, account creation is not part of the open core, yet. you need to use prisma studio if you self host. the majority of the self-host users have an existing user base already or have their own account creation and sync it up with the posgres of calendso.

Hello guys,
Do you know how limit registration to a specific domain ?


Add tables github_search_code|commits|issues|labels|repos|topics|users

https://docs.github.com/en/rest/reference/search

I have created separate issues for all the mentioned tables.


Improve byte-order handling

We probably need an enum for this, and some kind of schema enforcement / handling too.

This is fixed


[SPARK-9598][SQL] do not expose generic getter in internal row

Merged build triggered.

I'll update toSeq in another PR.

Merged build triggered.

Merged build started.

Merged build started.

Test build #39714 has started for   PR 7932 at commit c3887eb.

Test build #39715 has started for   PR 7932 at commit c60de4c.

Test build #39714 has finished for   PR 7932 at commit c3887eb.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test build #39715 has finished for   PR 7932 at commit c60de4c.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test build #1331 has started for   PR 7932 at commit c60de4c.

Test build #1331 has finished for   PR 7932 at commit c60de4c.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Test build #1334 has started for   PR 7932 at commit c60de4c.

Test build #1334 has finished for   PR 7932 at commit c60de4c.

This patch passes all tests.
This patch does not merge cleanly.
This patch adds no public classes.

Merging this first. Please send a followup pr to move the default implementation to a separate base.


1-Click App on Digital Ocean

[ ] Apply as a Vendor. For this ticket create your own account and document the process.
[ ] Create a 1-Click App
[ ] Add a button to launch the 1-Click App on the Readme

Hello sir!
I would like to solve this task!
Thank you!

Hello sir! I would like to solve this task! Thank you!

Hii @ArthrowAbstract are you working on this ?


Support Solidity 0.8.1 Unchecked

Hello,
In the extension I have Solidity 0.8.1 selected as the compiler but I get syntax errors for unchecked blocks.  Anyway to fix this?
unchecked is new syntax in Solidity 0.8.0
Thanks
Nick

@juanfranblanco any plans for this?

Sorry @mudgen I missed this! How do you get the syntax errors? I assume is not highlighted but the compiler should not complain.

The Linter complains.  See this image:

@juanfranblanco  Is there a way to make it not complain about this?

Many thanks @mudgen, I have tested with the latest solhint I have raised an issue with them https://github.com/protofire/solhint/issues/280.

@juanfranblanco No answer from Solhint project. I wish this could be fixed somehow.

I assume this is a problem with the antlr solidity parser. So it will be just something that needs to be added.

Actually the antr parser has been upgraded https://github.com/solidity-parser/antlr/blob/a4d87bb8fa99e23101694e2f05bfa5e656777437/Solidity.g4 it just needs to be updated in solhint

@juanfranblanco Can you do it?   Maybe no one else will.

@mudgen no, truly. I don't know what consequences will have in the linter, although not many. You can disable solhint in vscode for the time being.

@juanfranblanco Do you know can fix this?

Closing this as the awesome solhint team have updated the parser.. upgraded in latest


Web api - conventional routing instead of attribute routing

At the moment the Web Api project template contains a ValuesController which uses the following attribute for routing - [Route("api/[controller]")]. For prototyping purposes, I was trying to get rid of this attribute route and replace it with a MapRoute call in Startup.cs without much luck. The following code does match the first "Get" action, but not the others.
routes.MapRoute(
    name: "default",
    template: "api/{controller}/{action}",
    defaults: new { action = "Get" });

Any help would be highly appreciated. Thanks!

There's no reason why you can't use traditional routing (MapRoute) instead of attribute routing for an API.
We really strongly recommend using attribute routing for APIs because it gives you fine grained control over your URL surface. You also can run into scaling problems if you write many routes using MapRoute to try and accomplish the same thing.
================
You should take a look at https://docs.microsoft.com/en-us/aspnet/core/mvc/controllers/routing if you haven't already. If that doesn't answer your questions then please provide more information about what you're trying to do and we can try to help.

@rynowak, thanks for getting back to me on this. Yes, I have found the documentation and I have been reading up on this yet, I was not able to come up with a MapRoute statement which could replace the [Route("api/[controller]")] attribute route. I was thinking that I might have to use MapGet but I could not get that to work. Any code samples would be highly appreciated.
Let me try to give you some context as I might be approaching the issue in the wrong way. I am working on an ASP.NET Core application (targeting .NET Framework) that leverages areas and will be used by separate teams. The scope of the team that is using the application is in the URL (ex: /team1/Area/Home/Index, /team2/Area/Home/Index). I have a settings file which defines which area is available for which team and based on that setting file I was thinking of explicitly setting up the routes of my application using MapRoute instead of writing validation code and redirect to a 404 in case an area is not available to a particular team.
The other option that I have been exploring which is mentioned in the documentation that you have linked above is to:

Use an attribute route for all Web Apis - [Route("api/[area]/[controller]")]
Use the application model to update/add a routing attribute for each team based on the settings file

Thanks again for your guidance!

There isn't really a way to use MapRoute(...) to say the same thing as [Route("api/[controller]")] and MapRoute("api/{controller}/{action}") when we're talking about routing. There really isn't an equivalency here, they are made of different stuff.
[Route("api/[controller]")] is token substitution, the name of the controller (let's say ProductsController is the class name) gets substituted in to make "api/Products". Additionally, any attribute routes that appear of on the action methods get appended.
So when you write:
[Route("api/[controller]")]
public class ProductsController : Controller
{
    [HttpGet("{id}")]
    public IActionResult GetProduct(int id) { }
}

There's an entry created in the attribute routing tree that links api/Products/{id} back to your ProductsController.GetProduct method.

One the other hand conventional routing is a totally different mechanism. When you do define a route like MapRoute("api/{controller}/{action}/{id?}") you've defined a pattern for matching URL paths. So a URL path like api/Products/GetProduct/7 will be matched and then we'll look up the route data { controller = Products, action = GetProduct } in a table and find your ProductsController.GetProduct.
Now what will happen if you want to write a route that's specific to a given controller or action? you can do this with defaults.
MapRoute("api/Products/{id?}", defaults: new { controller = "Products", action = "GetProduct"})

You've create a route now that will always and only match the ProductsController.GetProduct method. When this pattern matches, the same set of route values are produced by both of these examples.
However -- this still isn't the same as [Route("api/[controller]")] because you have to create a route for each action. This leads to the scalability problem I pointed out in my original answer.

If your needs are as simple as disabling a few areas, then I think the simplest thing to do would be to build your application using attribute routing, and then use application model to remove the controllers that you don't want to work at startup time based on configuration.

@rynowak, thanks for your guidance. Will go with the attribute route + application model option.

@lszomoru Will below code work for you by "https://localhost:44387/api/values/getbyid/1"?
Rourting
            app.UseMvc(routes =>
            {
                routes.MapRoute(
                    name: "default",
                    template: "{controller=Home}/{action=Index}/{id?}");
                routes.MapRoute(
                    name: "api",
                    template: "api/{controller=Values}/{action=GetAll}/{id?}");
            });

ValuesController
  //[Route("api/[controller]")]
   public class ValuesController : Controller
   {
       // GET: api/values
       //[HttpGet]
       public IEnumerable<string> GetAll()
       {
           return new string[] { "value1", "value2" };
       }
       // GET api/values/5
       //[HttpGet("{id}")]
       public string GetById(int id)
       {
           return "value";
       }

       //// POST api/values
       //[HttpPost]
       //public void Post([FromBody]string value)
       //{
       //}

       //// PUT api/values/5
       //[HttpPut("{id}")]
       //public void Put(int id, [FromBody]string value)
       //{
       //}

       //// DELETE api/values/5
       //[HttpDelete("{id}")]
       //public void Delete(int id)
       //{
       //}
   }

Note, remove all of the Attribute route.


wip informe

Codecov Report

Merging #44 (dec6919) into master (5e84df6) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master      #44   +/-   ##
=========================================
  Coverage     97.94%   97.94%
  Complexity      109      109
=========================================
  Files            20       20
  Lines           195      195
  Branches         10       10
=========================================
  Hits            191      191
  Misses            2        2
  Partials          2        2

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5e84df6...dec6919. Read the comment docs.


Avoid processing http source if host doesn't include GitHub

This should "fix": https://github.com/xing/XNGPodsSynchronizer/pull/12#issuecomment-309778508, well basically it will give us the old behaviour: for http sources that are not hosted in GitHub.com we will skip the URL parsing and the repository won't be mirrored.
Ideally, it would be great if we could find a way to get a git repo (for mirroring) with the Pod name from somewhere else besides the podspec, but I don't believe this is possible.
Alternatively, we could create our own git repository versioning the file hosted in the http source (for binary dependencies), a bit crazy but I think it'll be cool and will properly "mirror" the contents.

@pietbrauer please try this.

Coverage decreased (-0.4%) to 79.894% when pulling 91d853bd0183f011f67f7a24b8c3f02f678ff97e on fix_http_source_parsing into 1a1ff9e5219ed15effb11c06d40736b449a10ef0 on master.


Feat/responsive

Resolves:

[X] #47

:tada: This PR is included in version 1.0.0 :tada:
The release is available on:

Chrome Web Store
GitHub release

Your semantic-release bot :package::rocket:


Wiki needs better instruction on referencing locally-built nuget packages

When building the current BotBuilder project libraries, nupkg files are created. Due to the workflow of VS 2017, these can't be put in a post-build step to copy to a central location (eg: /build).
Users aren't directed to take those nuget packages and put them in a common spot, nor how to create a nuget package source for use in their own bot.
Suggest adding direction to do relevant steps (manually copy from location x, y, z to here, then go to VS and add nuget package source to that location) or simply run the following operations from a VS 2017 command line in the bot builder repo root directory:
for /R "%cd%\libraries" %%f in (*.nupkg) do xcopy /Y "%%f" "%cd%\build"

nuget sources Remove -Name bfv4-local
nuget sources Add -Name bfv4-local -Source %__startPath%\build

Restart VS after running the above commands, and local nuget packages are available as dependencies to any bot solution

+1

+1

@johnataylor @Kaiqb

To add to this, the locally build issues that folks are seeing today are a leftover from a publishing bug that @tomlm has (we think) fixed as of earlier today. Packages seem to be correctly published to the standard NuGet feed.

Verified that the NuGet packages are currently available from https://api.nuget.org/v3/index.json. Will update the topic this morning.

Topic updated.


Integrate into x-transformers

Hi!
Thank you so much for implementing this. My lab has had the set transformer on the radar for a bit since we have some permutation invariant data we are working with... Do you have an example of how one might utilize this with x-transformers? My initial idea was something like:
import torch
from x_transformers import TransformerWrapper, Encoder
from isab_pytorch import ISAB

model = TransformerWrapper(
    num_tokens = 20000,
    max_seq_len = 1024,
    attn_layers = ISAB(
        dim = 512,
        heads = 8,
        num_latents = 128,
        latent_self_attend = True
    )
).cuda()

This is just based on reading Vaswani2017, Lee2019, and poking around the code. I see that Encoder is just inheriting AttentionLayers, so I wasn't sure if it could be a direct swap.
This is way outside my area of expertise, so I thought I would ask here first before chasing myself down a rabbit hole.
Thanky you!

hey! curious what kind of data you are using it for
would actually recommend https://github.com/lucidrains/perceiver-pytorch#experimental , which would be doing what you want, with all the tiny details that made perceiver work

Oh nice I didn't know that. Thank you! I will promptly check this out.
Specifically, we are interested in genomic region and gene transcription data. So, a "bag of genes" or a "bag of transcription factor binding sites". One can think of it as a sentence, but there's no inherent order - so positional encodings arnt necessary nor do they make any sense for this context.
In the past with other language models we've fudged this by randomly shuffling the "bag of genes" to simulate context by would be nice to not have to do that.

@nleroy917 that is indeed a valid use case! let me know how your experiments with the ISAB-perceiver fairs and maybe i'll put a bit more polish into it

@lucidrains small update! (we can move this discussion over to the Perceiver repo if necessary since this is slightly outside the ISAB scope.)
I read the perciever paper and agree it would be a nice fit here. I had a question, however. I noticed the .experimental Perceiver implementation requires the specification of number of input channels and frequency bands. this doesn't necessarily apply to transcription or region data since it truly is just a list of genes or regions. It can directly be thought of as a sentence with no order. So, there is really only one input channel, and only one frequency_band. Its just ["gene1", "gene2", ...] (We literally represent the data as strings).
I was looking at the PercieverLM and was wondering if I could just remove the positional embeddings entirely:
class PerceiverLM(nn.Module):
    def __init__(
        self,
        *,
        dim,
        num_tokens,
        max_seq_len,
        **kwargs
    ):
        super().__init__()
        self.token_emb = nn.Embedding(num_tokens, dim)
        # self.pos_emb = nn.Embedding(max_seq_len, dim)
        # remove since data is permutation invariant

        self.perceiver_io = PerceiverIO(
            dim = dim,
            queries_dim = dim,
            logits_dim = num_tokens,
            **kwargs
        )

    def forward(
        self,
        x,
        mask = None
    ):
        n, device = x.shape[1], x.device
        x = self.token_emb(x)

        # pos_emb = self.pos_emb(torch.arange(n, device = device))
        # pos_emb = rearrange(pos_emb, 'n d -> () n d')
        # x = x + pos_emb
        # remove since data is permutation invariant

        logits = self.perceiver_io(x, mask = mask, queries = x)
        return logits

I wasn't able to elucidate the exact mechanism implemented by Perciever - I am still mostly working off my knowledge from attention is all you need. The training task is going to be BERT-like.
Let me know if you have any suggestions. Thank you again for the insight!

@nleroy917 yup, you could, but each nucleotide will still need positions?

@nleroy917 as an aside, have you read https://arxiv.org/abs/2306.15794

@nleroy917 yea, actually the more i think about it, it is still better to use regular language modeling (or perhaps the new hyena dna if you need to capture long distance interactions) for the nucleotides or k-mers within each gene. then you can apply perceiver, or a regular transformer followed by pooling for some invariant prediction, on the set of genes

@nleroy917 are you doing supervised or unsupervised learning?

Hi @lucidrains
The task would be unsupervised; masked language modeling to be specific with various downstream fine-tuning tasks. So, BERT.
To be more clear, I am interested in replication of the results of this paper: https://www.nature.com/articles/s41586-023-06139-9.epdf?sharing_token=u_5LUGVkd3A8zR-f73lU59RgN0jAjWel9jnR3ZoTv0N2UB4yyXENUK50s6uqjXH69sDxh4Z3J4plYCKlVME-W2WSuRiS96vx6t5ex2-krVDS46JkoVvAvJyWtYXIyj74pDWn_DutZq1oAlDaxfvBpUfSKDdBPJ8SKlTId8uT47M%3D
They take a "bag of genes" approach but use the standard transformers library from hugging face (which doesn't use any experimental/cutting-edge attention as far as I am aware), and I would be interested in assessing performance of a more appropriate architecture, like a set transformer.
So, litearlly, the "sentences" are lists of Gene IDs ["ENS0001, "ENS0002, ... , ENS00100"], which get tokenized into specific genes (no byte pair encoding, etc, its akin to splitting on white space). Vocab size of ~25,000. Hope that makes sense.
TL;DR I'm interested in a genomic approach when the modality isn't raw sequence information, rather features extracted from a genomic experiment that have no inherent order.
Pseudo-code:
gene_tokenizer = GeneTokenizer()
data = read_scRNA_dataset("/path/to/matrix.h5ad")

model = SetTransformer(...)

for cell in data:
    gene_tokens = gene_tokenizer(cell)
    # ["ENS0001", "ENS0002", ... ]

    gene_token_ids, mask = gene_tokenizer.encode_tokens(gene_tokens)
    # gene_token_ids = [102, 1034, 4562, ....]
    # mask = [1, 1, 1, ... ]

    out = model(gene_token_ids, mask=mask)

ah i see, i actually think self-attention will be hard to beat, even if it does not account for permutation invariance
a lot of the exotic types of attention don't work out. in fact, i've tried ISAB architecture for a contracting project and it underperformed regular self-attention. however, if you are trying to innovate for longer context, i think that is worth exploring
what is the average number of tokens per gene, and how many genes typically?

Wow, that's interesting. I appreciate that insight and anecdote. We've used Word2Vec in the past, and it can learn great associations even though the data is permutation invariant.

what is the average number of tokens per gene, and how many genes typically?

I've yet to read the paper in its entirety, but I think its one token mapped to one gene, and they reported ~25,000 genes in the paper. I'm curious how you think self-attention might scale?
I know that a big area right now is scaling up context, but assuming a context of ~2048, how might self-attention respond to a vocab of 250,000? Or a million? Does the memory complexity just explode or does the model struggle to learn?

@nleroy917 memory is no longer an issue, with recent advances like flash attention, however, 25k is definitely nearing the upper limit for sequence length (still paying the quadratic compute)
without reading the paper, is geneformer is already doing self-attention on 25k tokens? i don't think that is possible with naive self attention. for 250k to 1 million, then i think you are onto something and would try ISAB / perceiver

i guess we should both read the geneformer paper first haha
also saw this some time ago which may be relevant

@nleroy917 that's interesting to carry out MLM on the gene ids. without knowing the domain that well, wouldn't you want to concat meta information? (which chromosome it is on, length, etc)

Sorry if that was confusing.
To be specific: geneformer used a context size of 2048 (max_seq_length), while their number of unique tokens 25,000.
I was inquiring about scaling up the vocab to ~1 million. I.e. your input embeddings are a matrix of dimensionality 1 million rows by ~100. That paper looks interesting and I'll check it out. I appreciate all the insight its super helpful and giving me a clearer picture here.

@nleroy917 if you are referring to the vocabulary size (number of unique genes), 1 million is fine
if you are using vocab in place of the input (which is gene_token_ids in your example above, being passed into model), then you are right that ISAB would be the best approach. flash attention can go up to about 16-32k

Sorry if that was confusing.
To be specific: geneformer used a context size of 2048 (max_seq_length), while their number of unique tokens 25,000.
I was inquiring about scaling up the vocab to ~1 million. I.e. your input embeddings are a matrix of dimensionality 1 million rows by ~100. That paper looks interesting and I'll check it out. I appreciate all the insight its super helpful and giving me a clearer picture here.

ok i think i understand, you'd like to basically shoot for longer context than geneformer, which is capped at 2048
if you are working with gene ids, your input exceeds 16k unique genes, and no nucleotide sequences being passed in, then agreed PerceiverLM should be tried. i think you'll also have to think about how to approach set prediction. maybe DETR paper has some clues

@lucidrains small update! (we can move this discussion over to the Perceiver repo if necessary since this is slightly outside the ISAB scope.)
I read the perciever paper and agree it would be a nice fit here. I had a question, however. I noticed the .experimental Perceiver implementation requires the specification of number of input channels and frequency bands. this doesn't necessarily apply to transcription or region data since it truly is just a list of genes or regions. It can directly be thought of as a sentence with no order. So, there is really only one input channel, and only one frequency_band. Its just ["gene1", "gene2", ...] (We literally represent the data as strings).
I was looking at the PercieverLM and was wondering if I could just remove the positional embeddings entirely:
class PerceiverLM(nn.Module):
    def __init__(
        self,
        *,
        dim,
        num_tokens,
        max_seq_len,
        **kwargs
    ):
        super().__init__()
        self.token_emb = nn.Embedding(num_tokens, dim)
        # self.pos_emb = nn.Embedding(max_seq_len, dim)
        # remove since data is permutation invariant

        self.perceiver_io = PerceiverIO(
            dim = dim,
            queries_dim = dim,
            logits_dim = num_tokens,
            **kwargs
        )

    def forward(
        self,
        x,
        mask = None
    ):
        n, device = x.shape[1], x.device
        x = self.token_emb(x)

        # pos_emb = self.pos_emb(torch.arange(n, device = device))
        # pos_emb = rearrange(pos_emb, 'n d -> () n d')
        # x = x + pos_emb
        # remove since data is permutation invariant

        logits = self.perceiver_io(x, mask = mask, queries = x)
        return logits

I wasn't able to elucidate the exact mechanism implemented by Perciever - I am still mostly working off my knowledge from attention is all you need. The training task is going to be BERT-like.
Let me know if you have any suggestions. Thank you again for the insight!

in which case, yea, you are right that removing the positional information should be fine. i misunderstood and thought you had the nucleotide sequences for each gene basically flattened as the input

in which case, yea, you are right that removing the positional information should be fine. i misunderstood and thought you had the nucleotide sequences for each gene basically flattened as the input

Yeah, there's actually no sequence information. It's even simpler than that... just a list of genes. The issue we've always thought of is that a "bag of genes" has no inherent order. You can shuffle it and its still the same, no information lost.

@nleroy917 ok, let's start with being able to turn off the fourier encodings

Omg.
Thank you for that. So, to recap you still think self-attention is superior?

it is if you are at 8 to 16k
realistically, you won't scale up to longer contexts without some struggle. you'll likely need some curriculum learning
I think perhaps first you should test your hypothesis that more context would help by substituting the self attention in geneformer with flash attention and fine tuning to 4096 or 8192. If you see a signal, then shoot for 25k with perceiver and see how many latents are required to maintain expressiveness

here is another paper that used something like isab https://github.com/lucidrains/recurrent-interface-network-pytorch for generative modeling, for reference. I can count on one hand the number of papers that use this, so you are breaking new grounds

I think perhaps first you should test your hypothesis that more context would help by substituting the self attention in geneformer with flash attention and fine tuning to 4096 or 8192. If you see a signal, then shoot for 25k with perceiver and see how many latents are required to maintain expressiveness

That makes a ton of sense. I'll start here. This discussion cleared up lots of areas of confusion and gave me lots of starting points, so thank you. Appreciate all the work you do! I'll try to provide updates as they come (I work slow since much of this is outside my area of expertise 🥲)

yea no problem! keep me updated


add space.dispose()

Hi William,
Here you go for space.dispose() as discussed in https://github.com/williamngan/react-pts-canvas/issues/17. I've added it both to CanvasSpace and DomSpace as both of them have an event listener registered on the window. Hope this is good. Let me know if anything else is required.
Note that the docs need to be regenerated.
Cheers

Thank you @tibotiber !


Button: prop type componentClass is invalid

Greetings, I'm in the process of adding some react components to my Flask based web app and have been trying to bootstrap the react components to match my other elements. I don't know why, but when I include: script src="/static/js/react-bootstrap-bower/react-bootstrap.js". I get the error:
'Invariant Violation: Button: prop type componentClass is invalid; it must be a function, usually from React.PropTypes.'
and then an error:
'Uncaught SyntaxError: Unexpected token <'
which i expect is just something I did, but the location was VM277941:3 which I was uncertain on how to locate. I pulled everything from this repo and everything is up to date as of 1/16/2015. Thans

This repo is solely for the release of bower. Please submit your issue to the react-bootstrap source repo at https://github.com/react-bootstrap/react-bootstrap.


added issue_template.md

part of #20

hi, i just wanted to know like how can i claim this PR for hacktoberfest? Its my first time participating in Hacktoberfest so could you please tell the same?

@Shatakshi0805  it is an automatic process. You have to sign up on the site: https://hacktoberfest.digitalocean.com/
Then the hacktober site will fetch your PRs with Github API.
The rules are:
A valid pull request is one that is accepted by the maintainer of the project you submit it to.

A pull request is considered approved once it has an overall approving review from maintainers, or has been merged by maintainers, or has been given the 'hacktoberfest-accepted' label.
The pull request must contain commits you made yourself.

Infos: https://hacktoberfest.digitalocean.com/faq

okay, thanks for informing.


Permission denied (publickey).

mldl@mldlUB1604:/media/mldl/data1t/ub16_prj/LightLDA$ sudo bash build.sh
[sudo] password for mldl:
Cloning into 'multiverso'...
The authenticity of host 'github.com (<IP_ADDRESS>)' can't be established.
RSA key fingerprint is SHA256:nThbg6kXUpJWGl7E1IGOCspRomTxdCARLviKw6E5SY8.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'github.com,<IP_ADDRESS>' (RSA) to the list of known hosts.
Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
build.sh: line 5: cd: multiverso: No such file or directory
build.sh: line 6: cd: third_party: No such file or directory
sh: 0: Can't open install.sh
Makefile:6: *** Makefile.config not found. See Makefile.config.example..  Stop.
make: *** No targets specified and no makefile found.  Stop.
mldl@mldlUB1604:/media/mldl/data1t/ub16_prj/LightLDA$

I have faced the same difficulty as above. Has any solution been found/proposed. Appreciated if any input could be provided.

This is a github issue. You shall config the publickey of your github account. But the following way should also work.
Try to change the first line of build.sh from
git clone -b multiverso-initial git@github.com:Microsoft/multiverso.git
to
git clone -b multiverso-initial https://github.com/Microsoft/multiverso


Enterprise Templates Artefacts (Checklist, Training & Registration, Visitor Registration Goal Tracking)

Enterprise Templates
Checklist
Training & Registration
Visitor Registration
Goal Tracking.

@microsoft-github-policy-service agree company="Microsoft"


⚠ Bofrost.de has degraded performance

In 98edb85, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 1416 ms

Resolved: Bofrost.de performance has improved in 258bf7e after 55 minutes.


GUI feature not working in multiple user accounts

Environment
Windows build number: 21370.1
Your Distribution version: Ubuntu 20.04.2 LTS
GUI app support: 1.0.17
Steps to reproduce

Create a new user in wsl
Test gui app in default user, it will work fine
Then switch to the new user and try launching any gui app
An error will appear

WSL logs:

versions.txt
WSLg ( x86_64 ): 1.0.17+3.Branch.master.Sha.a526dfd5ad03d126bb2d8c528f6c3563e86a40da
Mariner: VERSION="1.0.20210224"
FreeRDP: e4a2fc2053bd8c5f99455fcd08ffee7e5591567a
weston: fd961f5cd116c9358d82ce94d139c1578e21bd00
pulseaudio: 2f0f0b8c3872780f15e275fc12899f4564f01bd5
mesa:
pulseaudio.log
W: [pulseaudio] authkey.c: Failed to open cookie file '/home/wslg/.config/pulse/cookie': No such file or directory
W: [pulseaudio] authkey.c: Failed to load authentication key '/home/wslg/.config/pulse/cookie': No such file or directory
W: [pulseaudio] authkey.c: Failed to open cookie file '/home/wslg/.pulse-cookie': No such file or directory
W: [pulseaudio] authkey.c: Failed to load authentication key '/home/wslg/.pulse-cookie': No such file or directory
E: [pulseaudio] module-console-kit.c: GetSessionsForUnixUser() call failed: org.freedesktop.DBus.Error.ServiceUnknown: The name org.freedesktop.ConsoleKit was not provided by any .service files
E: [pulseaudio] module.c: Failed to load module "module-console-kit" (argument: ""): initialization failed.
E: [rdp-sink] module-rdp-sink.c: RDP Sink - Trying to connect to /mnt/wslg/PulseAudioRDPSink
E: [rdp-sink] module-rdp-sink.c: Connected failed
E: [rdp-sink] module-rdp-sink.c: RDP Sink - Trying to connect to /mnt/wslg/PulseAudioRDPSink
E: [rdp-sink] module-rdp-sink.c: RDP Sink - Connected to fd 12

Expected behavior
GUI apps should work fine without any issues no matter which user account we are using.
Actual behavior

Can you try su trojanx without the -, that works fine for me. The - resets the user environment variable for the new user... so DISPLAY that was setup in WSL INIT gets wiped and you won't be able to connect to the X server.
You can restore functionality by restoring these environment variables:
export DISPLAY=:0
export XDG_RUNTIME_DIR=/mnt/wslg/runtime-dir
WAYLAND_DISPLAY=wayland-0
export PULSE_SERVER=/mnt/wslg/PulseServer

Thanks a lot, your solution worked for me. This issue can be closed.
From: Steve @.>
Sent: 01 May 2021 12:31 PM
To: @.>
Cc: Iftekharuddin @.>; @.>
Subject: Re: [microsoft/wslg] GUI feature not working in multiple user accounts (#138)
Can you try su trojanx without the -, that works fine for me. The - resets the user environment variable for the new user... so DISPLAY that was setup in WSL INIT gets wiped and you won't be able to connect to the X server.
You can restore functionality by restoring these environment variables:
export DISPLAY=:0
export XDG_RUNTIME_DIR=/mnt/wslg/runtime-dir
WAYLAND_DISPLAY=wayland-0
export PULSE_SERVER=/mnt/wslg/PulseServer
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmicrosoft%2Fwslg%2Fissues%2F138%23issuecomment-830567573&data=04|01||12624e351ee8495589f608d90c6ee17d|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637554492647559155|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=u0JeUmdKJpa46GjzAM8MDRnY%2FGaVUwpdI3glcbbxYDo%3D&reserved=0, or unsubscribehttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAGH5CUN77FOYTSXZR6HGVO3TLORKXANCNFSM4357KO6A&data=04|01||12624e351ee8495589f608d90c6ee17d|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637554492647569162|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=lNuCCQjSlBxIQckdp2fOms%2B9JwkzWm42OhU2fttovH0%3D&reserved=0.


StaticFile.fromFile IllegalArgumentException with large files

The following code results in an IllegalArgumentException when the file is longer than Int.MaxValue
case req@GET -> Root / "file" =>
      StaticFile.fromFile(new File("massive-file.txt"), Some(req)).getOrElseF(NotFound())

And the exceptionis
logtype="application" transaction_id="" req_time="2017-11-15T10:53:34.195+11:00" loglevel="ERROR" threadname="scala-execution-context-global-73" classname="o.h.b.p.Stage" linenumber="Http1ServerStage.scala:226" message="Error writing body"
java.lang.IllegalArgumentException: null
	at java.nio.ByteBuffer.allocate(ByteBuffer.java:334)
	at fs2.io.file.FileHandle$$anon$2.read(FileHandle.scala:160)
	at org.http4s.StaticFile$._readAllFromFileHandle0$1(StaticFile.scala:150)
	at org.http4s.StaticFile$.readAllFromFileHandle$1(StaticFile.scala:145)
	at org.http4s.StaticFile$.$anonfun$fileToBody$7(StaticFile.scala:160)
	at org.http4s.StaticFile$.$anonfun$fileToBody$7$adapted(StaticFile.scala:160)

Some background as to how / why I discovered this. I was looking into streaming of large responses and noticed that the JVM CPU usage is very high while doing so.  I wasn't expecting this as I expect that Streaming of a large response is largely IO.  To narrow down to the least possible application code I decided just to stream a large file.  CPU usage is still higher than what I'd expect.  I'm still investigating.

Is this the code that has to unzip the file from the jar?

Nope.  The file is just being read directly.

That exception happens when you allocate a negative buffer size.  I bet the .toInt is the villain of this story.

I think that (end - offset) is to make sure we don't try to read beyond the end of the file.  I think if we min that against 0, we'll be fine.
I'm also don't see anything that prevents trying to read past the end of the file if a start and end are given that are out of bounds.
Does anyone want to try a PR?  It looks like this could be fixed starting in release-0.17.x.

I fixed this while looking at #1555. PR coming.


Add movie counter to show link

Add movie counter to show link
In this pull request, I implemented the following

[x] Number of movies in the homepage is correctly displayed in the 'show' link in the navbar

APROVED 🎉
Hello @Braingenius2  👋
Congrats, Your task is complete and looks fantastic 💯! There is nothing else to say other than... it's time to merge it! 🤩
Congratulations! 🎉
Highlights
🚀 You have a descriptive pull request summary
🚀 You have good commit messages
🚀 You used Gitflow properly
🚀 You have no linter errors
Optional suggestions
N/A
Cheers and Happy coding!👏👏👏


fix(simple-oauth2.js): add simple-oauth2 unit-test

@ranrib it is failing tests on node8 and node10

@ranrib it is failing tests on node8 and node10


[Bug]: not able to define existing image.imagePullSecret

Contact Details
No response
What happened?
Hi :)),
it seem's like I am not able to define an existing imagePullSecret. What I did was creating an imagePullSecret first, which I actually use for all my image pulls (which are working), and providing the name of the secret in the version: 4.4.18 neo4j-helm-charts/neo4j-standalone:
image:
  imagePullPolicy: IfNotPresent
  customImage: "neo4j:4.4.17-enterprise"
  imagePullSecrets:
    - "gitlab-pull-secret"`

k get secrets gitlab-pull-secret -o yaml

apiVersion: v1
data:
  .dockerconfigjson: ...
kind: Secret
metadata:
  name: gitlab-pull-secret
  namespace: staging
type: kubernetes.io/dockerconfigjson

I am getting following error:
  Error: Failed to render chart: exit status 1: Error: execution error at (neo4j-standalone/templates/neo4j-imagePullSecret.yaml:27:13): No docker-registry secret exists for imagePullSecret "gitlab-pull-secret" in the cluster.
   Missing imageCredential entry for "gitlab-pull-secret"
  Use --debug flag to render out invalid YAML
  Error: plugin "diff" exited with error

I don't understand why I would need to specify additional imageCredential entries since I am already having a prepared and working imagePullSecret.
PS: I don't need the secret for image.customImage but for an additional initContainer to add a custom plugin which I'd like to copy to an additional volume.
Cheers,
Szop
Chart Name
Standalone
Chart Version
4.4.18
Environment
Microsoft Azure
Relevant log output
No response
Code of Conduct

[X] I agree to follow this project's Code of Conduct

@mrszop i am unable to reproduce your issue. I tried the same and everything seems to be working just fine
values.yaml used
neo4j:
  name: licenses
  acceptLicenseAgreement: "yes"
  edition: enterprise
  password: my-password
volumes:
  data:
    mode: defaultStorageClass
# Override image settings in Neo4j pod
image:
  imagePullPolicy: IfNotPresent
  # set a customImage if you want to use your own docker image
  customImage: harshitsinghvi22/helm-charts:4.4.18-enterprise

  #imagePullSecrets list
  imagePullSecrets:
    - "demo"

imagePullSecret creation command
kubectl create secret docker-registry demo --docker-server=https://index.docker.io/v1/ --docker-username=XXXX --docker-password="XXXXX" --namespace staging

output
 $ helm install standalone neo4j-standalone -f ~/Desktop/sample/imagepullsecret.yaml --namespace staging
NAME: standalone
LAST DEPLOYED: Sun Mar 26 12:12:29 2023
NAMESPACE: staging
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
Thank you for installing neo4j-standalone.

Your release "standalone" has been installed  in namespace "staging".

The neo4j user's password has been set to "my-password".To view the progress of the rollout try:

  $ kubectl --namespace "staging" rollout status --watch --timeout=600s statefulset/standalone

Once rollout is complete you can log in to Neo4j at "neo4j://standalone.staging.svc.cluster.local:7687". Try:

  $ kubectl run --rm -it --namespace "staging" --image "harshitsinghvi22/helm-charts:4.4.18-enterprise" cypher-shell \
     -- cypher-shell -a "neo4j://standalone.staging.svc.cluster.local:7687" -u neo4j -p "my-password"

Graphs are everywhere!

WARNING: Passwords set using 'neo4j.password' will be stored in plain text in the Helm release ConfigMap.
Please consider using 'neo4j.passwordFromSecret' for improved security.

```

From what you have shared , it seems like the imagePullSecret you have created is in staging namespace but maybe you are trying to install the neo4j standalone helm chart in default or some other namespace where the secret does not exists

Hello @harshitsinghvi22,
thank you for testing and the the output. The mistake is on my side, since I am using helmfile and especially helmfile apply to handle all my charts and check first if anything changed before installing. After hours I found out this helm chart is using the lookup function which was originally querying the k8s API when run in dry-run. Now it is disabled (https://github.com/helm/helm/issues/8137) and I was getting a 'nil' result back which failed. Using helmfile sync is a workaround for me now. Sorry for the trouble!
Cheers

@mrszop you can use the disableLookups flag in the values.yaml to disable lookups when performing --dry-run


RenderTexture not working on Canvas with pixi-legacy

Last year I fixed an issue to get generation of render textures working properly for Canvas mode if the 'target' had no parent: https://github.com/pixijs/pixi.js/pull/5030
Checking this test again on v5, it doesn't work at all for the canvas renderer (but does work fine for WebGL still)
Expected Behavior
v4 works: https://jsfiddle.net/tx4g1jma/
Current Behavior
v5 doesn't work: https://jsfiddle.net/themoonrat/wqo7nt82/3/
Environment

pixi.js version: _e.g. 5.0.4
Browser & Version: Latest Chrome & Firefox
OS & Version: Windows 10

Added another display of this on pixi examples; showing the various ways of generating a texture
https://www.pixiplayground.com/#/edit/8n9Jy3acWWJyv9XZRlKm1


Fix config import relative path glob

Summary
Closes #9745
Notes
Previously, resolveImports would apply a glob filter if the path contained any '*', or otherwise convert relative paths to absolute. This meant that it was impossible to specify globs with paths relative to the main config file.
 A test case is added to ensure that this now works as expected.

/retest

/ok-to-test


Question regarding normalization

I would like to apologize in advance if this is the wrong place to ask a question. I have a question regarding the data normalization and the pca feature creation. If your dataset had a shape of (batches,mfcc,samples) how would you apply the transforms I mentioned before. In your code, are you normalizing and extracting pca features for each training example alone ?

Sure, you can ask questions here. I didn't fully understand your question though. Can you ellaborate?

Yes of course, when you normalized the samples, did you normalize each sample individually given its current values or did you check all the available samples to get their mean for example and then normalize?

For PCA, the pre-processing step includes StandardScaler which standardizes the features. For the Mel-features, I didn't explicitly normalize standardize the datamean

feel free to ask any other questions you may have


TypeError: Expected int32, got list containing Tensors of type '_Message' instead.

When I run extract_features.py this error comes up.

Hi @umlen-B, what version of Keras and TensorFlow are you using?
I pushed an update yesterday with a requirements.txt. Probably best to run pip install -r requirements.txt to make sure we're on the same version.


An ability to display local max value for Y axis

https://github.com/patrykandpatrick/vico/assets/46930374/313e27af-9293-4665-8f1d-61bf92a55a5f
Is there a possibility to adjust max value on the Y axis? If at some point I had 12:00:00 it will be inconvenient to see such max value for axis.
The daily value is often between 01:00:00 and 01:00:00 and it would be fantastic if your library support displaying on Y axis the biggest visible value.
P.s. your library is great, thank you

Actually #269 could solve my issue, but not in the best way (the expected behaviour is described in the post and a lot of libraries for web/python can do that)