Out of memory while editing

VS 2015 RC
PTVS Dev 2015-06-19 VS 2015
I had been using that VS session for maybe 2 hours.  I was in the process of adding a decorator to every test in blobservice (file is about 4000 lines long).  In a span of about 60 seconds, it started getting very slow, then unresponsive.  I saved dump file in our bug share under 'hugues-outofmemory', it's 3600MB.

I've added the whole project to that folder.  Unzip that and open the solution.
Open test_storage_blob.py in the editor.
Scroll down to line 455.
Press CTRL-C to copy that whole line to your clipboard.  It's the @record line.
Scroll to the next test (line 470) and CTLR-V to add the decorator to the test.
Repeat this for every test.  Scroll down using the down arrow key.
I checked the memory before I started, and it was around 300MB.  I wasn't even done with half the file, and memory was at 1000MB.

So it's fairly complicated as to how this comes about, but here's my best written description that doesn't involve a whiteboard:
We have two modules:
A.py
from B import h
def f(x): return h(x)

f(1)

B.py
def g(x): return None
def h(x): return g(x)

On the first pass, we create a FunctionAnalysisUnit (henceforth FAU) for each function assuming no types for the parameter (f, g, h). When we analyze the f(1) we create a second FAU for f(int). If there were another call to f, we would create a third FAU because the call chain is different.
(The call chain is the sequence of CallExpression nodes leading to the call. This is what lets us get analysis for parameter sets uniquely.)
We also add a dependency to the return value of f(int), so that whenever it updates we will reanalyze this function (which in this case does not matter, but if we were assigning the result we'd need to do this). The dependency is a set on f(int) that now contains a reference to our module's analysis unit.
When we analyze the f(int) FAU we will create a h(int) FAU for the h function. Because the call has come from A, we will key it off A's version to ensure we drop the FAU when A is reanalyzed. The call chain for h(int) is f(1)->h(x), which gives us a unique analysis for the original call to f. We also add a dependency from f(1)->h(x)'s return value, which is a reference in a set pointing to h(int)'s FAU.
Now when we analyze the h(int) FAU we will create a g(int) FAU with call chain f(1)->h(x)->g(x). (Call chains are typically capped at three long, so calls from g are only distinguished as far back as the h(x), so x is now the union of all calls to f). This call has come from B, so we key it off B's version and add the return value dependency.
Here's where things get tricky: we reanalyze A again without modifying it (or with an innocuous modification).
The entire process above repeats, but at any point where we keyed off A, we will notice that the version has changed and drop the old references. So h no longer has a reference to the old h(int) and will create a new one with the new (unchanged) type of x.
However, two things go wrong when we get to the call to g.
First: the FAU was not keyed off of A, but B whose version has not changed. So we don't drop the old FAU which keeps the old h(int) FAU alive because it is still a dependency of the return value.
Second: we don't recognize that f(1)->h(x)->g(x) is the same call chain because we do reference comparisons on the CallExpression nodes, and reparsing A has changed the identity of the first two nodes. So we add a new FAU without dropping the old one.
Multiply this by 100 calls to h from A and by 100 edits and you have thousands of FAUs being "leaked". As soon as you reanalyze B all the unnecessary FAUs are dropped and memory usage returns to normal.
I don't see any easy way to fix this for 2.2. The extra FAUs are correctly left behind (as in, they are indistinguishable from ones that should be removed), we have no good concept of value comparisons of AST nodes, and no easy way to detect or clear the dependencies. I'm open to suggestions.

Is this a regression compared to 2.1?

I think Dino fixed this, but we need to check.

It's not fixed, but at least now it's in the analyzer process, not devenv.exe, so there's more memory to spare.
Analyzer process is the blue line. This is with the project mentioned in my original comments.

So it turns out that most of this issue had in fact been resolved, and there was just one lingering reference leak that's being resolved in #3332.
However, in the code from #3298 we still see memory growing with each reanalysis, and this time it seems to be due to nested functions being assigned to dynamically generated classes. I'll investigate that over on #3298.


chore: bdd-sbhg-1583952243 to 0.0.1

chore: Promote bdd-sbhg-1583952243 to version 0.0.1

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approvers:
If they are not already assigned, you can assign the PR to them by writing /assign  in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


refactor(sbb-menu): rename opening and closing methods

Use open for opening overlays and close for closing overlays.

Codecov Report

Merging #1456 (e95a7c0) into master (ab917b0) will increase coverage by 0.96%.
The diff coverage is 64.50%.

@@            Coverage Diff             @@
##           master    #1456      +/-   ##
==========================================
+ Coverage   54.85%   55.82%   +0.96%
==========================================
  Files          49       67      +18
  Lines        1659     2603     +944
  Branches      406      695     +289
==========================================
+ Hits          910     1453     +543
- Misses        671     1050     +379
- Partials       78      100      +22

Impacted Files
Coverage Δ

src/components/sbb-clock/sbb-clock.tsx
0.00% <0.00%> (ø)

src/components/sbb-logo/sbb-logo.tsx
0.00% <0.00%> (ø)

src/components/sbb-menu/sbb-menu.tsx
25.89% <ø> (ø)

...ents/sbb-pearl-chain-time/sbb-pearl-chain-time.tsx
100.00% <ø> (ø)

...in-vertical-item/sbb-pearl-chain-vertical-item.tsx
93.75% <ø> (ø)

...-pearl-chain-vertical/sbb-pearl-chain-vertical.tsx
100.00% <ø> (ø)

src/components/sbb-pearl-chain/sbb-pearl-chain.tsx
67.74% <ø> (-3.69%)
:arrow_down:

src/components/sbb-section/sbb-section.tsx
85.00% <ø> (ø)

src/components/sbb-signet/sbb-signet.tsx
0.00% <ø> (ø)

...mponents/sbb-slot-component/sbb-slot-component.tsx
100.00% <ø> (ø)

... and 47 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Cannot perform, deis info, deis create

Hi this issue happened several times, I can log in successfully but I cannot perform any deis comands inside my app directory. I just restarted the platform and the issue still happening. Its happening several times. Does anyone knows how to fix it?
root@james:/u01/deis_apps/app_test/deisBuilder# deis login http://deis.<IP_ADDRESS>.xip.io
username: admin
password:
Logged in as admin
root@james:/u01/deis_apps/app_test/deisBuilder# deis info
Error:
404 NOT FOUND
detail: Not found
All units seems to be working fine.
root@james:/u01/deis_apps/app_test/deisBuilder# deisctl list
UNIT                                    MACHINE                         LOAD    ACTIVE  SUB
deis-builder.service                    6f57887c.../<IP_ADDRESS>       loaded  active  running
deis-controller.service                 88e4817d.../<IP_ADDRESS>       loaded  active  running
deis-database.service                   88e4817d.../<IP_ADDRESS>       loaded  active  running
deis-kube-apiserver.service             6f57887c.../<IP_ADDRESS>       loaded  active  running
deis-kube-controller-manager.service    6f57887c.../<IP_ADDRESS>       loaded  active  running
deis-kube-kubelet.service               6f57887c.../<IP_ADDRESS>       loaded  active  running
deis-kube-kubelet.service               88e4817d.../<IP_ADDRESS>       loaded  active  running
deis-kube-kubelet.service               cd2b9b48.../<IP_ADDRESS>       loaded  active  running
deis-kube-proxy.service                 6f57887c.../<IP_ADDRESS>       loaded  active  running
deis-kube-proxy.service                 88e4817d.../<IP_ADDRESS>       loaded  active  running
deis-kube-proxy.service                 cd2b9b48.../<IP_ADDRESS>       loaded  active  running
deis-kube-scheduler.service             6f57887c.../<IP_ADDRESS>       loaded  active  running
deis-logger.service                     88e4817d.../<IP_ADDRESS>       loaded  active  running
deis-logspout.service                   6f57887c.../<IP_ADDRESS>       loaded  active  running
deis-logspout.service                   88e4817d.../<IP_ADDRESS>       loaded  active  running
deis-logspout.service                   cd2b9b48.../<IP_ADDRESS>       loaded  active  running
deis-publisher.service                  6f57887c.../<IP_ADDRESS>       loaded  active  running
deis-publisher.service                  88e4817d.../<IP_ADDRESS>       loaded  active  running
deis-publisher.service                  cd2b9b48.../<IP_ADDRESS>       loaded  active  running
<EMAIL_ADDRESS>               88e4817d.../<IP_ADDRESS>       loaded  active  running
<EMAIL_ADDRESS>                 6f57887c.../<IP_ADDRESS>       loaded  active  running
<EMAIL_ADDRESS>                 88e4817d.../<IP_ADDRESS>       loaded  active  running
<EMAIL_ADDRESS>                 cd2b9b48.../<IP_ADDRESS>       loaded  active  running
deis-store-daemon.service               6f57887c.../<IP_ADDRESS>       loaded  active  running
deis-store-daemon.service               88e4817d.../<IP_ADDRESS>       loaded  active  running
deis-store-daemon.service               cd2b9b48.../<IP_ADDRESS>       loaded  active  running
<EMAIL_ADDRESS>          6f57887c.../<IP_ADDRESS>       loaded  active  running
deis-store-metadata.service             6f57887c.../<IP_ADDRESS>       loaded  active  running
deis-store-metadata.service             88e4817d.../<IP_ADDRESS>       loaded  active  running
deis-store-metadata.service             cd2b9b48.../<IP_ADDRESS>       loaded  active  running
deis-store-monitor.service              6f57887c.../<IP_ADDRESS>       loaded  active  running
deis-store-monitor.service              88e4817d.../<IP_ADDRESS>       loaded  active  running
deis-store-monitor.service              cd2b9b48.../<IP_ADDRESS>       loaded  active  running
deis-store-volume.service               6f57887c.../<IP_ADDRESS>       loaded  active  running
deis-store-volume.service               88e4817d.../<IP_ADDRESS>       loaded  active  running
deis-store-volume.service               cd2b9b48.../<IP_ADDRESS>       loaded  active  running

404 typically means that the app name was not found. Does it exist when you run deis apps? If not, what's the current status of the database?

Its strange, I had to restart the platform, when the platform started the apps are not showing, I will try to redeploy. When I list conteiners running on coreos, all apps are running, but its not listing from "deis apps c" commands. Question:  When I destroy an app,  the docker images should be erased as well or I have to remove them mannually? I didn't destroy any app that are not showing.
The database seems to be working fine:
deis-database.service - deis-database
Loaded: loaded (/run/fleet/units/deis-database.service; linked-runtime; vendor preset: disabled)
Active: active (running) since Mon 2015-10-05 16:41:57 UTC; 40min ago
Thanks @bacongobbler

@robeferre Also note that the Deis client infers the app name from the directory you're in based on the existence of the deis git remote. You can always specify the app name manually with -a <app>, as in: deis info -a myapp.

The apps still running, I'm using kubernetes as scheduler.
  It doesn't stop existing application containers.  It's normal that when the platform comes back up, the app container are still running.
It sounds to me like the most likely explanation here is that the database has lost some or all of its state.  Are you able to connect directly to the database and poke around the apps table to see if your app is listed there?

@ krancour
I just destroyed all apps that I have created, and the Database table looks empty:
deis=# select * from api_app;
uuid | created | updated | owner_id | id | structure
------+---------+---------+----------+----+-----------
I'm tring to figure out why the docker images related to my destroied apps has not been erased.
-------------
NODE 1
-------------
core@coreos_1 ~ $ docker images
REPOSITORY                           TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
<IP_ADDRESS>:5000/echoed-yearbook   v2                  5cfd8cf1e059        45 hours ago        795.2 MB
<IP_ADDRESS>:5000/rugged-fishbone   v2                  579a94317913        6 days ago          795.2 MB
<IP_ADDRESS>:5000/classy-windmill   v4                  396ba40043ea        6 days ago          795.2 MB
deis/router                          v1.10.1             b54973a4f497        2 weeks ago         35.22 MB
deis/publisher                       v1.10.1             23e3709f1d24        2 weeks ago         17.58 MB
deis/logspout                        v1.10.1             02a870675c74        2 weeks ago         15.47 MB
deis/logger                          v1.10.1             810f7cdfd922        2 weeks ago         14.9 MB
deis/builder                         v1.10.1             7bfb2d74ff7b        2 weeks ago         152.1 MB
deis/store-monitor                   v1.10.1             a035e3e596a2        2 weeks ago         379.6 MB
deis/store-metadata                  v1.10.1             f0127e076fbe        2 weeks ago         379.6 MB
deis/store-gateway                   v1.10.1             7989cbef9846        2 weeks ago         399.5 MB
deis/store-daemon                    v1.10.1             872b993588c7        2 weeks ago         379.6 MB
deis/registry                        v1.10.1             4e5b0f0435db        2 weeks ago         144.4 MB
deis/database                        v1.10.1             d00218251a12        2 weeks ago         180.9 MB
deis/controller                      v1.10.1             35710f0c155e        2 weeks ago         122.7 MB
alpine                               3.1                 8dd8107abd2e        3 weeks ago         5.037 MB
gcr.io/google_containers/pause       0.8.0               2c40b0526b63        6 months ago        241.7 kB
-------------
NODE 2
-------------
core@coreos_2 ~ $ docker images
REPOSITORY                           TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
<IP_ADDRESS>:5000/offset-keypunch   v2                  a2b50ae030d1        2 hours ago         876.5 MB
<IP_ADDRESS>:5000/preset-babushka   v2<PHONE_NUMBER>c7        2 hours ago         876.5 MB
<IP_ADDRESS>:5000/echoed-yearbook   v2                  5cfd8cf1e059        45 hours ago        795.2 MB
<IP_ADDRESS>:5000/classy-windmill   v3                  8e2d30e3a4b7        6 days ago          795.2 MB
<IP_ADDRESS>:5000/rugged-fishbone   v2                  579a94317913        6 days ago          795.2 MB
<IP_ADDRESS>:5000/classy-windmill   v2                  447b6fab235d        6 days ago          795.2 MB
<IP_ADDRESS>:5000/classy-windmill   v4                  396ba40043ea        6 days ago          795.2 MB
deis/router                          v1.10.1             b54973a4f497        2 weeks ago         35.22 MB
deis/publisher                       v1.10.1             23e3709f1d24        2 weeks ago         17.58 MB
deis/logspout                        v1.10.1             02a870675c74        2 weeks ago         15.47 MB
deis/logger                          v1.10.1             810f7cdfd922        2 weeks ago         14.9 MB
deis/builder                         v1.10.1             7bfb2d74ff7b        2 weeks ago         152.1 MB
deis/store-monitor                   v1.10.1             a035e3e596a2        2 weeks ago         379.6 MB
deis/store-metadata                  v1.10.1             f0127e076fbe        2 weeks ago         379.6 MB
deis/store-gateway                   v1.10.1             7989cbef9846        2 weeks ago         399.5 MB
deis/store-daemon                    v1.10.1             872b993588c7        2 weeks ago         379.6 MB
deis/registry                        v1.10.1             4e5b0f0435db        2 weeks ago         144.4 MB
deis/database                        v1.10.1             d00218251a12        2 weeks ago         180.9 MB
deis/controller                      v1.10.1             35710f0c155e        2 weeks ago         122.7 MB
alpine                               3.1                 8dd8107abd2e        3 weeks ago         5.037 MB
gcr.io/google_containers/pause       0.8.0               2c40b0526b63        6 months ago        241.7 kB
<IP_ADDRESS>:5000/wiggly-macaroni   v5                  99fa6fec60e2        13 months ago       637.1 MB
<IP_ADDRESS>:5000/calmer-doghouse   v7                  bc1b067e2ac1        13 months ago       676.8 MB
-------------
NODE 3
-------------
core@coreos_3 ~ $ docker images
REPOSITORY                           TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
<IP_ADDRESS>:5000/preset-babushka   v3                  f3bd71c6e3ef        About an hour ago   876.5 MB
<IP_ADDRESS>:5000/offset-keypunch   v3                  22154771ccaa        2 hours ago         876.5 MB
<IP_ADDRESS>:5000/rugged-fishbone   v6                  9c5b4be17ed0        5 hours ago         795.2 MB
<IP_ADDRESS>:5000/echoed-yearbook   v2                  5cfd8cf1e059        45 hours ago        795.2 MB
<IP_ADDRESS>:5000/nimbus-greenery   v2                  80bb449e5a42        2 days ago          795.2 MB
<IP_ADDRESS>:5000/junior-unicycle   v2                  12d5af12c820        3 days ago          795.2 MB
<IP_ADDRESS>:5000/classy-windmill   v3                  8e2d30e3a4b7        6 days ago          795.2 MB
<IP_ADDRESS>:5000/rugged-fishbone   v2                  579a94317913        6 days ago          795.2 MB
<IP_ADDRESS>:5000/classy-windmill   v4                  396ba40043ea        6 days ago          795.2 MB
<IP_ADDRESS>:5000/classy-windmill   v2                  2c5f811f3da6        6 days ago          795.2 MB
deis/router                          v1.10.1             b54973a4f497        2 weeks ago         35.22 MB
deis/publisher                       v1.10.1             23e3709f1d24        2 weeks ago         17.58 MB
deis/logspout                        v1.10.1             02a870675c74        2 weeks ago         15.47 MB
deis/logger                          v1.10.1             810f7cdfd922        2 weeks ago         14.9 MB
deis/builder                         v1.10.1             7bfb2d74ff7b        2 weeks ago         152.1 MB
deis/store-monitor                   v1.10.1             a035e3e596a2        2 weeks ago         379.6 MB
deis/store-metadata                  v1.10.1             f0127e076fbe        2 weeks ago         379.6 MB
deis/store-daemon                    v1.10.1             872b993588c7        2 weeks ago         379.6 MB
deis/registry                        v1.10.1             4e5b0f0435db        2 weeks ago         144.4 MB
deis/database                        v1.10.1             d00218251a12        2 weeks ago         180.9 MB
deis/controller                      v1.10.1             35710f0c155e        2 weeks ago         122.7 MB
alpine                               3.1                 8dd8107abd2e        3 weeks ago         5.037 MB
gcr.io/google_containers/pause       0.8.0               2c40b0526b63        6 months ago        241.7 kB
deis/slugrunner                      latest              cc4a0915354e        12 months ago       832.5 MB
Some apps PODs conteiners still running even when the app was destoied.
8e297480b0a9        gcr.io/google_containers/pause:0.8.0   "/pause"               27 hours ago        Up 27 hours                                                                            k8s_POD.e4cc795_cheery-pachinko-v2-cmd-iipkv_cheery-pachinko_170d030e-6bf4-11e5-948d-000c29fd65ad_74200f1f
571337e0571d        gcr.io/google_containers/pause:0.8.0   "/pause"               27 hours ago        Up 27 hours                                                                            k8s_POD.e4cc795_august-buttress-v2-cmd-iwb97_august-buttress_16543515-6bf4-11e5-948d-000c29fd65ad_1634b860
1f04a5fe9547        gcr.io/google_containers/pause:0.8.0   "/pause"               27 hours ago        Up 27 hours                                                                            k8s_POD.e4cc795_nimbus-greenery-v2-cmd-7a9hk_nimbus-greenery_184be0d3-6bf4-11e5-948d-000c29fd65ad_826a7d63
9dcab67f8e75        gcr.io/google_containers/pause:0.8.0   "/pause"               27 hours ago        Up 27 hours                                                                            k8s_POD.e4cc795_junior-unicycle-v2-cmd-fmdwj_junior-unicycle_17dc9727-6bf4-11e5-948d-000c29fd65ad_8288af02
c98eadea391d        gcr.io/google_containers/pause:0.8.0   "/pause"               27 hours ago        Up 27 hours                                                                            k8s_POD.e4cc795_oblong-whitecap-v2-cmd-n5ile_oblong-whitecap_18bab326-6bf4-11e5-948d-000c29fd65ad_ee698378

Any reason this issue was closed? I'm having the same problem with a recently created app, but another app that I have running on my cluster responds to these commands just fine

@davidcelis if it doesn't show up in deis apps:list, that means there's no record of it in the database. Would you mind opening a new issue with the following information:

the version of Deis you're running
deis --version -- client version
deisctl journal controller
a reproducible problem case (if possible)

Any info provided would be greatly appreciated. Thanks! :)

Also, triple-confirm that you're running the commands from the app's directory, or you're specifying the app with --app. The client infers what app you're working on based on its name or the existence of a git remote called deis.

I ended up finding #4096 and following steps there to remove the app's process found in fleetctl. There was definitely an entry for the app in the postgresql database. I was able to remove the process and then re-create the app (which, strangely enough, seemed only to add a git a remote and start working again)


APNS server uses hard coded ports

I've seen this repeatedly with wildfly-12.x and eap-7.1.0.GA-redhat-11
11:31:19,396 ERROR [stderr] (Thread-226) Caused by: java.net.BindException: Address already in use (Bind failed)
11:31:19,396 ERROR [stderr] (Thread-226) 	at java.net.PlainSocketImpl.socketBind(Native Method)
11:31:19,396 ERROR [stderr] (Thread-226) 	at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:387)
11:31:19,396 ERROR [stderr] (Thread-226) 	at java.net.ServerSocket.bind(ServerSocket.java:375)
11:31:19,396 ERROR [stderr] (Thread-226) 	at java.net.ServerSocket.<init>(ServerSocket.java:237)
11:31:19,396 ERROR [stderr] (Thread-226) 	at java.net.ServerSocket.<init>(ServerSocket.java:181)
11:31:19,396 ERROR [stderr] (Thread-226) 	at javax.net.ssl.SSLServerSocket.<init>(SSLServerSocket.java:136)
11:31:19,396 ERROR [stderr] (Thread-226) 	at sun.security.ssl.SSLServerSocketImpl.<init>(SSLServerSocketImpl.java:113)
11:31:19,396 ERROR [stderr] (Thread-226) 	at sun.security.ssl.SSLServerSocketFactoryImpl.createServerSocket(SSLServerSocketFactoryImpl.java:79)
11:31:19,396 ERROR [stderr] (Thread-226) 	at com.notnoop.apns.utils.ApnsServerStub$GatewayRunner.run(ApnsServerStub.java:158)

The testsuite hangs for minutes and then continues. Perhaps a timeout on the mock endpoint is also needed
[INFO] Wildfly Camel :: Testsuite :: Standalone :: Basic .. FAILURE [17:38 min]

If the server needs to start on the client side, we have a utility that can transport port numbers in a deployment resource.


Feature: Integration of Yajta coverage

This pull request integrate yajta coverage report to jdbl for test base debloating
The new behavior is the following: A method or class is considered bloated if it appear in neither the jacoco report nor the yajta report.
Summary of changes:

Refactoring of JacocoReportReader: the data structure representing jacoco is still a Map<String, Set>, but it now contains for each observed class, the list of observed methods. (Observed means covered by tests.)
Name of various method accessing this data structured have been renamed (from get unused methods to get used mehots) to reflact better the new content.
the method addYajtaAnalysis can now add observed method for classes that did not appear at all in the jacoco report.
static class initializer of non removed classes can no longer be considered bloated. (If the class is not considered bloated, it must be loaded by test execution, therefor the static class initializer will be called. Hence, if it is labelled as bloated it is a misreporting from both coverage tools.
removal of Constructor is also deactivated, but this more debatable.

Thanks a lot, @nharrand, we have integrated those changes here.


Use fetch+convert samplePage to smart component

@gnurag, added API request using fetch instead of axios and render result into simple table.
Currently, I am just making API request for /api/inventory/v1/hosts.

Codecov Report

:exclamation: No coverage uploaded for pull request base (main@f478338). Click here to learn what that means.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main       #2   +/-   ##
=======================================
  Coverage        ?   16.41%
=======================================
  Files           ?       11
  Lines           ?       67
  Branches        ?        6
=======================================
  Hits            ?       11
  Misses          ?       52
  Partials        ?        4

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f478338...f2567d6. Read the comment docs.

Merged, thanks @kgaikwad!


Update cw1 to the v2 framework

It appears we can no longer split the update to v2 between cw1 and other packages due to dependency hell. Closing.


mimetype is media type

Not sure; we'd be not compliant with the standard. The standard says

If a "charset" parameter is specified, it SHOULD be a required parameter, eliminating the options of specifying a default value.

Plus charset isn't carried in text files, so without it things could go wrong in Android share I suppose.


Improve websocket handling

Should fix issue https://github.com/bjornbytes/lovr-webvr-server/issues/2
Detailed explanation
Websocket worked fine for local development but two issues arise
as soon as the server and the client were on different machine.

The hostname is fixed to localhost, which won't work if the file
are served from say <IP_ADDRESS>

The port is fixed to 8081 which might cause issue if the server
is asked to run on that port and will cause issue if the server
machine didn't open the port 8081.

Both fix require editing the same line in the view and I stumbled
upon both issue at once so I fix both in the same commit. Current
solution for the port issue is to use the HTTP port + 1. Things stay
the same for the default case, solve the issue of starting express
on port 8081 and fix my issue as home server have port 5000 to 5999
open.

Looks good 👍


creating a folder or file

I have problems creating a file, I get the following message.
[@Linux root]#mkdir jc
mkdir: can't create directory 'jc': Read-only file system
[@Linux root]#

The rootfs is squashfs, it is read only in RAM, you can't write to it. (But you can use tmpfs to do modifications, but they will stay only in RAM)

I was able to create a file but it would be in the volatile.
`
[@Linux root]#cd /var/volatile
[@Linux volatile]#cat > test2.sh
!/bin/bash
echo "hola mundo"
[@Linux volatile]#cat test2.sh
!/bin/bash
echo "hola mundo"
[@Linux volatile]#sh test2.sh
hola mundo
[@Linux volatile]#
 Is there any way to save it permanently? Since I want to do driver test and then add with:sudo insmod hello_world.ko`


How to get  picture blob ？i need to upload for mutil

example:
const worksheet = workbook.getWorksheet( 'Sheet1' )
const images = worksheet.getImages()
console.log( worksheet )
console.log( images )

but i need imageFile type to upload image。

@pengcheng433  That's good.
const worksheet = workbook.getWorksheet( 'Sheet1' )
const images = worksheet.getImages()
console.log( worksheet )
console.log( images )
  for (const { imageId } of images) {
    const image = workbook.getImage(imageId);
    console.log('image-->', image);
  }

{
  type: 'image',
  name: 'image1',
  extension: 'png',
  buffer: <Buffer 89 50 4e 47 0d 0a 1a 0a 00 00 00 0d 49 48 44 52 00 00 01 77 00 00 02 9b 08 06 00 00 00 0e ae ca 9a 00 00 00 01 73 52 47 42 00 ae ce 1c e9 00 00 00 04 ... 33633 more bytes>,
  index: 0
}

thanks bro， and yes i also reslove this
const workbook = new ExcelJS.Workbook();
await workbook.xlsx.readFile(filePath);
const worksheet = workbook.getWorksheet("Sheet1");
const images = worksheet.getImages();
console.log("images", images).length;
var image = workbook.getImage(0);
console.log("image", image);
console.log(image.buffer);

@pengcheng433 Interested in keeping an eye on this program, which after all has been largely unmaintained.


SharedIndexInformer not calling event handlers during the first resync

This issue is the root cause for https://github.com/argoproj/argo-workflows/issues/10947
Based on my tests, the issue was introduced between client-go v0.24.0-alpha.1 and v0.24.0-alpha.2.
Seems like when the first resync happens (20 minutes in case of argo-workflows), the event handlers are not being called.
I was able to identify a potential change that introduced this: https://github.com/kubernetes/client-go/commit/54928eef9f824667b23a938188498992d437156a#diff-dbac68d65bbc9814c2336df6ee91f22e807d84a9127a4d34a8b39c5657d0e447L551
Looking at the code, there is small behavior change when evaluating the isSync flag:

resource version was checked only when d.Type == Replaced. Now it's being evaluated on all updates
the d.Type == Sync set the isSync to true
for any other cases: isSync is false

All an in one, there might be a chance of falsely setting isSync to true.
This flag is being used in distribute, which conditionally calls the listeners.
The issue also reproduces with latest client-go version: v0.29.0-alpha.2

any ideas @alexzielenski @howardjohn @lavalamp @liggitt @dims @pohly ?

/remove-lifecycle stale

/reopen

can u reopen @dims ?

@tooptoop4 I think this can easily be fixed if the d.Type is passed to the onUpdate callback and keep the same conditions for isSync as before the refactorization.

can u reopen @dims ?

Can you create an issue in https://github.com/kubernetes/kubernetes/issues instead?
That is where the code is being maintained and thus it might get more attention from relevant developers and SIGs.

/reopen
/transfer kubernetes


ffmpeg decoding fails sometimes

raise CouldntDecodeError(

pydub.exceptions.CouldntDecodeError: Decoding failed. ffmpeg returned error code: 1
shows this issue it was working fine and recently started showing this issue

nevermind i fixed it this issue occurs when you run out of free credit on elevenlabs

how to fix it? i have same issue


Allow closing the current workspace

Is your feature request related to a problem? Please describe.
We need to be able to close the current workspace and load a new one.

The codegen doesn't parse the current ast and only keep the first it parsed. The #92 issue is blocking this feature.


[jnigen] Add getRange method to JArray<JPrimitive> and change the dart equivalent type for JArray<jchar> to int

Close #990
Convert the return type of operator [] of JArray<jchar> to int, to be more consistent with the getRange return type. Similarly operator []= gets an int now
Remove some unused functions in dartjni.c
Use .asTypedList(len).setRange in setRange to be more efficient (~5x faster on my machine)
Close #1097 – Improve the performance ofJArray's operator []= (~70% faster on my machine)

coverage: 91.642% (+0.04%) from 91.607%
when pulling aa789a69240fb2bcab800da22ea9f6d427f4259f on jarray-refactor
into 2393cca5197efa30e0b58febf6a4aea29d542359 on main.

The reason the workflow is failing is https://github.com/microsoft/linux-package-repositories/issues/130, merging this anyways.


New package: AlphaStableDistributions v0.1.0

Registering package: AlphaStableDistributions
Repository: https://github.com/baggepinnen/AlphaStableDistributions.jl
Created by: @baggepinnen
Version: v0.1.0
Commit: d3ce9482f33b960588d26d14ed72369c118d115c
Reviewed by: @baggepinnen
Reference: https://github.com/baggepinnen/AlphaStableDistributions.jl/issues/1

This pr can be closed, I transfered the repo to an org

This pr can be closed, I transfered the repo to an org

Done.


[SPARK-11740][Streaming]Fix the race condition of two checkpoints in a batch

We will do checkpoint when generating a batch and completing a batch. When the processing time of a batch is greater than the batch interval, checkpointing for completing an old batch may run after checkpointing for generating a new batch. If this happens, checkpoint of an old batch actually has the latest information, so we want to recovery from it. This PR will use the latest checkpoint time as the file name, so that we can always recovery from the latest checkpoint file.

Test build #45902 has started for PR 9707 at commit 494696e.

Test build #45902 has finished for PR 9707 at commit 494696e.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/45902/
Test FAILed.

Merged build finished. Test FAILed.

retest this please

Test build #45912 has started for PR 9707 at commit 494696e.

Test build #45912 has finished for PR 9707 at commit 494696e.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/45912/
Test PASSed.

/cc @tdas

LGTM.
I wonder whether we can add a unit test to test this behavior. We could create a CheckpointWriter with a mock JobGenerator, and then test whether the latest checkpoint time is respected.
BTW, I would like to merge this to older branches like 1.5 and 1.4 as well.
@harishreedharan Can you also take a look at this?

LGTM. Thanks @zsxwing !

Added a unit test using mock

Test build #46113 has started for PR 9707 at commit fa580b7.

Test build #46113 has finished for PR 9707 at commit fa580b7.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/46113/
Test PASSed.

Thanks @zsxwing and @harishreedharan. Merging this to master, 1.6 and 1.5


Create custom automation flows

Summary
In addition the the built-in workflows we provide to allow you to automatically triage your items, add items from a repository, and archive when they are no longer needed on your project, we will allow users to create custom automations. This will give flexibility to set other project fields and build workflows custom to a use case.

This issue is being closed as outdated. For more information, please check out this Discussion post. Stay tuned for new additions to our refreshed public roadmap!

We wanted to provide more details on why this issue was removed from the public roadmap. We are aware of gaps in the automation flows for projects. We will continue to ship updates to existing workflows, such as our recent release of auto close when complete and are continuing to investigate how we can add even more flexibility with custom automation flows. However we currently do not have a timeline for when this would be available.


python3Packages.pillow: 9.2.0 -> 9.3.0

Description of changes
https://github.com/python-pillow/Pillow/blob/9.3.0/CHANGES.rst
Updates pillow, fixes pillow-simd build.

Things done

Built on platform(s)

[x] x86_64-linux
[x] aarch64-linux (non-simd)
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.11 Release Notes (or backporting 22.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

@ofborg build python310Packages.pillow python310Packages.pillow-simd

passthru.tests build for me on aarch64-linux, still building for macos...

See https://pillow.readthedocs.io/en/stable/releasenotes/9.3.0.html#security


Decryption adds "" around text

When calling encrypt with text and decrypt with the encrypted result then the decrypted text is surrounded with quotation marks like "text"

Fixed in https://github.com/blockstack/blockstack-android/commit/cd251e0c19fb6f600a18c7b09421be053b309e42


Porkbun as DNS provider

Are you in the right place?

If you are looking for support on how to get your upstream server forwarding, please consider asking the community on Reddit.
If you are writing code changes to contribute and need to ask about the internals of the software, Gitter is the best place to ask.
If you have a feature request for NPM then you are in the right place.
I might be in the right place but this might be a certbot issue

Is your feature request related to a problem? Please describe.
My request is about the providers available in let's encrypt DNS challenges
Describe the solution you'd like
Porkbun offers and API to edit DNS https://porkbun.com/api/json/v3/documentation
Describe alternatives you've considered
I am currently running certbot manually and uploading the certs under the custom option but I am also try to get cloudflare to work
Additional context
I saw the comment here that I should create this request #613 https://github.com/jc21/nginx-proxy-manager/issues/613#issuecomment-723036527

Unfortunately the certbot porkbun dns plugin is not compatible out of the box, but would require changes on their side, to be able to fetch credentials from a file. I could open an issue over there, maybe even suggest some code, but I am not able to test it

The pull request is open. You can already try it out if you use the docker image jc21/nginx-proxy-manager:github-pr-1099


Update to RN 0.25

Any easy way to test it from your side? :)

Thanks, RN 0.25 support has been added in the last version 👍


stop dupes for sharer ids, too

this is a very unlikely event, but also needs to be guarded against.

👍


Help with school bus routing on a map with known bus stop coordinates

I want to generate all the Routes for a school with the following given data:-

Capacity of the busses
Location [lat,lng] of the school
Location [lat,lng] of all the stops.

I need to generate the most optimal routes based on the stops, so was thinking of applying heuristics for the solution (Gillet & Miller Sweep heuristic).
All [Lat, Lng] data are available.

Blue Marker in the map is the school Location
Yellow Markers are the Stops

?
How can I use the VeRyPy for this ?

Dear  Ashar,
definitely you can. What you need to do, is to calculate a distance matrix based on location [lat,lng] information of the school and stops. You can refer to the example in the VeRyPy repostory: https://github.com/yorak/VeRyPy/blob/master/examples/bing_maps_example.py
The example script takes a tab separated file with a list of addresses (one for each row) as an input, but you could modify it to accept coordinates. I'd propose adding a new command line flag to input such file. If you need any help implementing the feature to the bing_maps_example.py file, please let me know. Also, note that you need to get a Bing Maps API key to calculate the distances in the road network: https://www.bingmapsportal.com
Please note that the developer key only allows 50 locations (https://docs.microsoft.com/en-us/bingmaps/rest-services/routes/calculate-a-distance-matrix). From the map it seems you have more? If you have the time, this can be done in pieces - or you can use another map server that can calculate larger distance matrices for you.
Hope this allows you to continue. Please let me know if you hit any roadblocks.

Hi Yorak,
Thank you for the prompt response, I really appreciate it.
Just a few queries before I dive into this:-

The distance matrix has to be the onRoad travel distance right? if yes then I can create my own distance matrix using distance from the normal route and directions APIs increasing the run time of course [ Services like OSRM etc. ]?
Can you explain the format of the .vrp file as if I create my own distance matrix I would have to create the .vrp file and I can't understand the structure of data in that file
The distance matrix calculation has to be from school to all the individual stops only or also between all the stops? one to many or all to all?

Thanks in advance

Hi,
no problem, I'm just happy if I could be of assistance. Regarding your questions:

The distance matrix has to be the onRoad travel distance right? if yes then I can create my own distance matrix using distance from the normal route and directions APIs increasing the run time of course [ Services like OSRM etc. ]?
The distance matrix calculation has to be from school to all the individual stops only or also between all the stops? one to many or all to all?

Yes, the distance matrix is  n+1 x n+1 matrix D, where n is the number of bus stops and the +1 being the school. The values are the distance or time it takes to move from any stop to another (or straight from/to the school). Hence, it contains precalculated distances/travel times for all possible combinations of locations. When computing such a matrix, be sure to use a API that is capable of effectively computing such distances as the matrix size is quadratic to the number of stops and can grow quite large. Here, I'd recommend against rolling out your own solution for the on road distance calculation, because it is another, quite extensive, can of worms (but, if you are interested, please check bidirectional Dijkstra, R-trees, and contraction hierarchies). Hence, if you already use a MAP API, please check if it provides a distance matrix computation service.

Can you explain the format of the .vrp file as if I create my own distance matrix I would have to create the .vrp file and I can't understand the structure of data in that file.

The VRP file is TSPLIB95 format and how the matrix is given is determined by the EDGE_WEIGHT_TYPE : EXPLICIT and EDGE_WEIGHT_FORMAT attributes. If you set the EDGE_WEIGHT_FORMAT : FULL_MATRIX then you can give the full matrix separated by white space, row by row. Note that then you can input asymmetric distances where it might take shorter time to get from A to B than from B to A. However, note that some VeRyPy CVRP algorithms might not fully support such asymmetric distances.
Another option for using the TSPLIB .vrp files is to import the algorithms as a Python module as shown below:
from classic_heuristics.gillet_miller_sweep import gillet_miller_init
from util import sol2routes

# Get this using e.g. OSRM distance matrix API. Be sure to have the school at index 0
full_distance_matrix = ...
# Again, first is the school, and the rest are indexed as the distance matrix and show how
# many pupils are expected to be picked up at each stop (only 4 stops in this example)
pupils_on_each_bus_stop = [0,12,5,3,1]
capacity_of_the_bus = 42 # or whatever
minimize_number_of_buses = False # default optimizes total travel time / length

solution = gillet_miller_init(
    D=full_distance_matrix,
    d=pupils_on_each_bus_stop,
    C=capacity_of_the_bus,
    minimize_K=minimize_number_of_buses )

for route_idx, route in enumerate(sol2routes(solution)):
    print("Route #%d : %s"%(route_idx+1, route))

In ORSM this seems to be called table service:
https://github.com/Project-OSRM/osrm-backend/blob/master/docs/http.md#table-service

Hi,
Thanks for the prompt and detailed info, I will implement this and get back to you.

Hi,
one more thing that occurred to me when thinking about this. I spend too much time in my youth in a school bus. Although they should not, they were often overfull. The issue in modeling this is that in CVRP the C constraint is hard, i.e., it cannot be violated, even a little bit. Also, some classical algorithms allow minimizing K (the number of vehicles), but not all. Hence, after you have managed to compute the distance matrix and produce an initial solution with Gillett-Miller Sweep algorithm, I'd recommend looking into adapting the algorithm SG82-LR3OPT, where the Lagrangian relaxation makes the constraints soft and 3-opt* (inter-route 3-opt) finds the moves. Just be sure to add another relaxed constraint calculation for K here. If your problem instances are fairly large (lets say over 100 bus stops), you might also need to split the problem or improve the performance of LR3OPT.
I just wanted to share the insight.
-Jussi

Hi,
Thanks for the insight it is very helpful.
Although Bus overloading is what I am trying to remove so I don't think this is applicable to my case. If a bus can transport 30 students I only want 30 students assigned to that bus.
Having said that I do want variable C (capacity) for the busses as different sizes of busses can have different capacities of students.
For reference let us say we have 3 busses with [20, 30, 50] students capacity for the same school.
Any idea as to how to handle this case?
Thanks in Advance

Oh, what you are describing is the Heterogeneous Fleet VRP (HFVRP). By default, VeRyPy expects all of the vehicles to be the same, but modifying the code to support multiple different vehicle types is possible. However, please create a separate branch for this. Modifying the code is algorithm specific. I guess the most straightforward approach to make the necessary changes would be to use the API as in my previous example but give the capacity C as a list of capacities, try to run the algorithm, and fix the places where exceptions are thrown.
If you are mostly interested in Gillet & Miller, then, looking at the code you need to:

add a vehicle_idx field to routedata.py
assign an increasing vehicle_idx for each new current_route in sweep.py (i.e., here and here)
If the C is a list (or similar, check with hasattr(C, '__getitem__')), instead of straight comparison, look up the vehicle capacity using the vehicle_idx in the sweep.py capacity constraint check.
Finally, you need to check all permutations of capacity constraints adding yet another for loop layer  to the algorithm. You can use the built-in from itertools import permutations. It is usual that many vehicles have the same capacity, so at the beginning of the function it is advisable to prune the duplicates HFC_permutations = list(set(permutations(C)).

There are some open questions such as what to do if the algorithm runs out of vehicles during the sweep? After you get the basic Sweep to work as implemented by sweep.py, you can look into the improvement procedure as implemented in the file gillet_miller_sweep.py. However, some of the algorithms might be easier to convert to HFVRP such as the LR3OPT I mentioned earlier.

I am not fixated on using the Gillet & Miller I would be happy to try the LR3OPT if it gives better results although running out of Vehicles is a serious limitation for HFVRP and kind of counter-intuitive as it would require back and forth human intervention in the system.
Before I implement the HFVRP I am trying to do the CVRP. Unfortunately, I was not able to build the LKH in windows so now I am working on Linux.
from classic_heuristics.gillet_miller_sweep import gillet_miller_init
from util import sol2routes
# Get this using e.g. OSRM distance matrix / table API. Be sure to have the school at index 0
full_distance_matrix = ...
# Again, first is the school, and the rest are indexed as the distance matrix and show how
# many pupils are expected to be picked up at each stop (only 4 stops in this example)
pupils_on_each_bus_stop = [0,12,5,3,1]
capacity_of_the_bus = 42 # or whatever
minimize_number_of_buses = False # default optimizes total travel time / length

solution = gillet_miller_init(
    D=full_distance_matrix,
    d=pupils_on_each_bus_stop,
    C=capacity_of_the_bus,
    minimize_K=minimize_number_of_buses )

for route_idx, route in enumerate(sol2routes(solution)):
    print("Route #%d : %s"%(route_idx+1, route))

In the above code, the parameter "points" is missing. I added it as follows:-
points = [[24.419954, 54.4416255], [24.440368, 54.59438], [24.423042, 54.525564]]
solution = gillet_miller_init(
    points=points,
    D=full_distance_matrix,
    d=pupils_on_each_bus_stop,
    C=capacity_of_the_bus,
    minimize_K=minimize_number_of_buses)

I am getting the following error:-
Traceback (most recent call last):
  File "test.py", line 35, in <module>
    minimize_K=minimize_number_of_buses)
  File "/home/ubuntu/VeRyPy/classic_heuristics/gillet_miller_sweep.py", line 336, in gillet_miller_init
    intra_route_improvement=_improvement_callback)
  File "/home/ubuntu/VeRyPy/classic_heuristics/sweep.py", line 485, in sweep_init
    callback_data = pcds_callback(D,d,C,L,sweep)
  File "/home/ubuntu/VeRyPy/classic_heuristics/gillet_miller_sweep.py", line 88, in _pack_datastructures_callback
    return (len(D), D,produce_nn_list(D),d,C,L,node_to_pos,pos_to_node, avr)
  File "/home/ubuntu/VeRyPy/util.py", line 38, in produce_nn_list
    NN_D[i] = sorted(enumerate(D[i,:]), key=itemgetter(1))
TypeError: list indices must be integers or slices, not tuple```

How do I have to pass the points array?

Hi, the error suggests that the D (full_distance_matrix) is not a 2D Numpy array of size (1+n) x (1+n). Could you verify this?
During testing you could use "as the crow flies" distances. To compute these, you can use the [haversine formula](VeRyPy. https://github.com/yorak/VeRyPy/blob/0cdd2e4895cbfc09c7e10906e2ec53c28eac230a/cvrp_io.py#L38) that comes with VeRyPy. An usage example below:
import numpy as np
from cvrp_io import _haversine

points = [[24.419954, 54.4416255], [24.440368, 54.59438], [24.423042, 54.525564]]

n_incl_depot = len(points)
geodesic_D = np.zeros( (n_incl_depot, n_incl_depot) )

for i, p1 in enumerate(points):
    for j, p2 in enumerate(points):
        geodesic_D[i,j] = 0.0 if p1==p2 else _haversine(p1,p2)

print(geodesic_D)

Hi,
I passed the exact response that came from the OSRM.
full_distance_matrix = [
    [
        0,
        7562.1,
        23971.5
    ],
    [
        7545.1,
        0,
        18324.4
    ],
    [
        12008.1,
        17728.9,
        0
    ]
]

Hi,
So I ran the Gillet & Miller on a dummy (test) data, basically, one school with 2 stops having 4 and 5 students respectively.
School Loc = [24.419954, 54.4416255]
stops = [[24.459459, 54.321149], [24.440368, 54.59438]]

I ran the following code:-
from classic_heuristics.gillet_miller_sweep import gillet_miller_init
from util import sol2routes
import numpy

points = [[24.419954, 54.4416255], [
    24.459459, 54.321149], [24.440368, 54.59438]]
# Get this using e.g. OSRM distance matrix / table API. Be sure to have the school at index 0
full_distance_matrix = [
    [
        0,
        14997,
        17728.9
    ],
    [
        15737.5,
        0,
        32034.8
    ],
    [
        18324.4,
        33151.2,
        0
    ]
]
full_distance_matrix = numpy.array(full_distance_matrix)

# Again, first is the school, and the rest are indexed as the distance matrix and show how
# many pupils are expected to be picked up at each stop (only 4 stops in this example)
pupils_on_each_bus_stop = [0, 4, 5]
capacity_of_the_bus = 10  # or whatever
minimize_number_of_buses = False  # default optimizes total travel time / length

solution = gillet_miller_init(
    points=points,
    D=full_distance_matrix,
    d=pupils_on_each_bus_stop,
    C=capacity_of_the_bus,
    minimize_K=minimize_number_of_buses)

for route_idx, route in enumerate(sol2routes(solution)):
    print("Route #%d : %s" % (route_idx+1, route))

As you can see the bus capacity I entered 10. I got the following result:-
Route #1 : [0, 1, 2, 0]

I the above image you can see the blue marker is the school and the two stops labeled as 1 and 2.
Result:-
There was one route created containing both the stops as the stop1 + stop2 = 9 student < Bus Capacity (10).
Issue:-
Realistically the distance between the two stops is a lot. one bus should not be assigned to both the stops. I don't think any school in the world will have a bus go to both stops it's not practical. The total distance of the route being 68 KM.
Is there any way to handle this scenario?
I have not fixated on Gillet & Miller any other algorithm that can handle this scenario is also welcomed.
Thanks in advance

I'm very happy to hear that you have been able to resolve the initial issues and get the pipeline from coordinates to an optimized routes works. It is easy to build on that foundation.
While it has already been used in several other research and R&D projects, I'm aware there is room for improvement in getting started with VeRyPy. Unfortunately, I've been, and currently am, working on entirely different topics, and have not been able to find larger blocks of time to work on VeRyPy. I've applied a research grant for that purpose so hopefully I can make some improvements in the near future.
Regarding the issue: that is typical. The way to solve it with VeRyPy is to use the maximum route duration/length constraint L in conjunction with the capacity constraint C. This allows you to avoid such "nonsensical" solutions.
A more granular way to model this would be to per delivery travel time constraints (that is, the travel time is measured for each pickup/delivery) or multicriteria optimization where the pupil travel time is taken into account in the cost function. In fact, this is usually what is desired in bus routing: Without such modified cost function, it may be that a pupil is picked up from a bus stop near the school, only to be transported around the city when other pupils are picked up to before delivering them to the school. In effect the first pupils would've been better off just walking the distance. If we consider VeRyPy, it might be that the simplest way to implement this would be to use the construction heuristics as is and offload the travel duration minimization to improvement heuristics (local search). The delta calculations ( e.g., for 2-opt* ) need to be adapted to also account for the transportation times of the students. To make this performant, you probably need to add some auxiliary data to the RouteData structure. Again, this is not entirely trivial modification which also means it is not CVRP anymore. Use of an specialized git branch for such extension is advisable.
Please note that I pushed some changes to master make sure nobody uses the inter-route 3-opt by accident.

I will try using the maximum route duration/length constraint L and check if it works for my case.
With respect to modeling based per delivery travel time constraints (that is, the travel time is measured for each pickup/delivery) this is basically Duration Matrix (1+n)*(1+n) where n is the number of stops and 1 is the school. [Correct me if I am wrong]
so we will result with a 2D NumPy array similar to the distance matrix for the cost function.
What should be the next step?
NOTE- I am not completely familiar with your code, so I am not sure where travel duration minimization is happening and how to factor in the new cost function.

Yes, I think that by using the L constraint you can already force the routes to different buses to be more balanced. Each experiment you do increases your understanding of the problem and code base which you can leverage in choosing the next step.
Be warned that my intuition says that measuring the total transportation time cost of the pupils becomes a quite bit more complex as one has to consider all the next stops the bus route will take. Hence, the cost of traveling an edge is dependent on who the bus is transporting. Such context dependent travel costs are tricky to compute effectively and the travel time cost is a weighted sum over the selected edges (values in D) before offloading at the school. Drawing this out with pen and paper might help.
Another way I can come up for solving the problem of excessive transportation times is to allow the buses to begin their routes far away from the school. The drawback would be that you need to specify a number of starting locations for the bus and implement a specialized version of your chosen algorithm among the ones VeRyPy implements for such Open VRP variant. If you think about it, making driving when bus is empty next to free as opposed to making the travel costs expensive with pupils on board will create similar (but not the same!) routings to the OVRP alternative.
I see that the major issue for in adapting VeRyPy for the bus routing use is that the computation of cost and constraint checks are not done in an unified way between the different algorithms. By avoiding of using such abstraction, I was able to reach the main goal in writing the algorithms as close to the original descriptions (my eye was on replications, not extendability). Hence, I cannot point a single point where the modifications need to be made, sorry.
Perhaps some more specialized solver would be a better fit to you? If you would like to continue with VeRyPy, my proposal is that you first look what you can do with vanilla VeRyPy. Then, if the solutions given by, e.g., Gillet and Miller, Paessens savings and perhaps Stewart and Golden (LR3OPT) are usable to you when capacity C and maximum route duration L constraints are both specified, you might be able to make an informed decision if investing the time to modify best of these algorithms to heterogeneous fleet and load dependent travel times.

Regarding using the L constraint, I didn't find any documentation hinting at values that L can take on.
Can you help me with the format of how to pass the values?

How to pass duration/length as L.
Unit of measure in which it is expecting the value.

Thanks in advance

Adding L was a bit of an afterthought. Some classical CVRPs had them and I decided to add the support for these constraints. Hence, all construction algorithms (that is, the *_init(..) functions) have the optional parameter L. It uses the same unit as the distance matrix D and specifies the maximum duration/length for each route.  Therefore, this should do it:
solution = gillet_miller_init(
    D=full_travel_time_matrix,
    d=pupils_on_each_bus_stop,
    C=capacity_of_the_bus,
    L=maximum_route_duration)

As you can see, I'd recommend optimizing using travel time instead of distance. By doing so, you can use stopping times. VeRyPy currently supports only one and same stopping time for all stops. However, adding support for a stopping time that is dependent on the number of pupils picked up from a bus stop is trivial. Just add half of the stopping time to outgoing and half to incoming edges of each bus stop. In practice you need to add those to each row/column of the distance/travel time matrix. Then, given that D is in time units and u is a Numpy array of stopping times:
U_rows = np.tile(u/2, (len(u), 1))
U_cols = np.tile(u/2, (len(u), 1)).T
D+=U_rows+U_cols
np.fill_diagonal(D, 0.0)

Now, the stopping times are embedded in the D. Please note that adding half of the stopping time to incoming edges and half to outcoming edges allows keeping symmetric problems symmetric.

Hi,
I ran the Gillet and Miller for my dataset.
PC config:-
CPU - Intel Core i7 - 8 cores
RAM- 32 GB
OS - Ubuntu 18.04

Copy of the code:-

import json
from classic_heuristics.gillet_miller_sweep import gillet_miller_init
from util import sol2routes
import numpy
from loader import Loader
from flask import jsonify

class testFunction:
    def __init__(self):
        pass

    def gilletMiller(self, runFromJson, input=None):
        runFromJson = True

        with open('request.json')as jsonFile:
            data = jsonFile.read()
        options = json.loads(data)
        print("capacity_of_the_bus = ", options["capacity_of_the_bus"])
        points = input["points"] if runFromJson != True else options["points"]

        full_distance_matrix = input["full_distance_matrix"] if runFromJson != True else options["full_distance_matrix"]

        full_distance_matrix = numpy.array(full_distance_matrix)

        pupils_on_each_bus_stop = input["pupils_on_each_bus_stop"] if runFromJson != True else options["pupils_on_each_bus_stop"]

        capacity_of_the_bus = input["capacity_of_the_bus"] if runFromJson != True else options["capacity_of_the_bus"]

        minimize_number_of_buses = False

        loader = Loader("Running Heuristics...",
                        "Heuristics Done!", 0.1).start()
        solution = gillet_miller_init(
            points=points,
            D=full_distance_matrix,
            d=pupils_on_each_bus_stop,
            C=capacity_of_the_bus,
            minimize_K=minimize_number_of_buses)
        loader.stop()

        ROUTES = []

        loader.__init__("Preparing Response", "Response ready!!", timeout=0.1)
        loader.start()

        for route_idx, route in enumerate(sol2routes(solution)):
            print("Route #%d : %s" % (route_idx+1, route))
            ROUTES.append({route_idx+1: route})

        loader.stop()

        with open("response.json", "w") as outfile:
            json.dump(
                jsonify({"success": True, "data": {"routes": ROUTES}}), outfile)

        return ROUTES

if __name__ == "__main__":
    testObj = testFunction()
    ROUTES = testObj.gilletMiller(runFromJson=True, input=None)

Issue:-

The code has been running for 20 hrs now and it is still not complete. Is there any benchmark for the Time Constraint for the process? As you can see I did not yet implement the " L " yet running just the basic algorithm for the preliminary results.
The data set contains 297 entries including all the stops and the school at index 0.

Request.json File:-
request.zip

That seems excessive. As you can see, from the page 103 of the VeRyPy manuscript Gillet & Miller should be able to solve problems up to around 700 customers within 1 hour. In fact, if we consider average case, under 10 minutes (Figure 22: Two phase algorithms, C) even if the comprehensive start customer for the sweep is enabled (as it is by default).
Perhaps it is the excessive logging? It is recommended to run VeRyPy with the python -O test.py, which will entirely turn off the debug output. Also, the verbosity of the logging can be changed using the shared_cli.set_logger_level(level) function.
I'll check this on my machine.

I enabled debug and it seems the algorithms gets stuck right away.
Do a sweep from position 112 (n1) by steps of 1
DEBUG:Sweep node order [1, 152, 113, 169, 275, 70, 51, 130, 184, 23, 16, 9, 49, 128, 28, 280, 143, 76, 121, 129, 98, 78,....
DEBUG:Considering n1 between rays -1.0206, -1.0193
DEBUG:Added n1 to the route set
DEBUG:Considering n152 between rays -1.0193, -1.0171
DEBUG:Added n152 to the route set
DEBUG:Considering n113 between rays -1.0171, -1.0136
DEBUG:Added n113 to the route set
...
DEBUG:Considering n129 between rays -0.9740, -0.9725

Where it gets stuck is calling the routing_algo. Hence, in TSP optimization of all places. This happens using the built-in TSP 3-opt. Even after testing this with thousands of classical problem instances, it seems you have hit some pathological case in TSP optimization.

Ok, found the reason. It seems the internal TSP 3-opt local search does not work correctly on asymmetric distances. I'd assume the delta calculation does not work correctly, which causes the 3-opt TSP to get stuck into infinite improvement loop. Myself, I have tested VeRyPy only with symmetric distances. I know some people have used it also for asymmetric instances, but I have not received any pull requests.
Unfortunately, at the moment I do not have the time to dig deeper. However, there are few options:

use symmetric distances:

for i in range(full_distance_matrix.shape[0]):
            for j in range(i, full_distance_matrix.shape[1]):
                full_distance_matrix[i,j] = full_distance_matrix[j,i]

Fix the current 3-opt local search implementation. When a segment is reversed, the cost of the segment is changed and needs to be recalculated. This also applies to 2-opt. Requires changes to here, here, here and all others moves where the segment is reversed.
Implement or use another 3-opt or entirely different TSP algorithm that supports the VeRyPy tsp_solvers API. For example, IIRC LKH and ACOTSP both support asymmetric distances.

It is possible that I'm able to look into this later, but currently I'm quite busy with other work, so it might take even few weeks.


gemspec: Raise upper limit on Bundler version to 3.0.0

Resolves #14016. :key:
Checklist

[x] I've run bundle exec rspec from the root directory to see all new and existing tests pass
[x] I've followed the fastlane code style and run bundle exec rubocop -a to ensure the code style is valid
[x] I've read the Contribution Guidelines
[x] I've updated the documentation if necessary.

Motivation and Context
Bundler 2.0 was recently released, and those users who want to upgrade cannot do so until this gem removes its requirement on Bundler being less than 2.0.  Moreover, users who are unable to satisfy the RubyGems ≥ 2.5.0 or Ruby ≥ 2.3 requirement imposed by Bundler 2.0 will not be forced to upgrade, I'm pretty sure. This PR just loosens the requirement. I thought about completely removing the upper bound, but decided not to.
Resolves #14016.
Description
This PR is a one-liner change to the Gemspec.

I'm not super up to date, but I read somewhere bundler 3.0 doesn't work with the pre-installed Ruby of macOS. However from what I understand, this should still work for previous setups, right?

The test failure I'm getting is:
expected Exception with "Invalid password passed via 'MATCH_PASSWORD'" but nothing was raised

But I am failing to see how this results from this change. I'll note that the test in question  is setting ENV['MATCH_PASSWORD'] to "invalid", then expecting Match::Encryption::OpenSSL#decrypt_files to throw an error, which it only does if MATCH_PASSWORD is present in ENV… but that doesn't seem to be happening. I'd expect that to happen on multiple platforms. Am I missing something?

That's our one and only flaky test. I restarted the tests, should be fine in a minute.


Examples are not working on master

Description
Failed to run first example.
I just follow the steps at Developer corner section and got an Uncaught TypeError.
Not sure if I'm missing something because I haven't changed anything in the project. So it must be a bug.
Steps to Reproduce
Example:

git clone https://github.com/FNNDSC/ami.git
cd ami
yarn install
yarn example geometries_slice

Expected Results
I expected to see the example running.
Actual Results
controls.trackball.js:17 Uncaught TypeError: Class constructor rt cannot be invoked without 'new'
    at new _class (controls.trackball.js:17)
    at init (geometries_slice.js:137)
    at ./examples/geometries_slice/geometries_slice.js.window.onload (geometries_slice.js:156)

Browsers Affected

[x] All
[ ] Chrome
[ ] Firefox
[ ] Edge
[ ] Safari 9
[ ] Safari 8
[ ] IE 11

Versions

AMI: master
THREEJS: 129

Please fix this

cc @jennydaman  is this worth for outreachy programm

@sherrif10 sorry for the late response. This project is not under our Outreachy effort.


Python 3.6 update

Hi @conda-forge/core, conda-buils fails with a cryptic error on win+npy112:
AttributeError: 'UnsatisfiableError' object has no attribute 'pkgs'
How can I find the cause of this failure ?

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

it seems the pyqt 4.x from conda-forge cannot be co-installed with numpy112, anyway unpined pyqt (uses pyqt 5 from defaults) and it seems to work now (previously I had to pin pyqt4)


Problem with composite foreign keys in SQL Server

SQL Server 2017.
SQL snippet for problem reproduce:
CREATE TABLE [dbo]. [order] (
[id] numeric(8,0) IDENTITY(1,1) NOT NULL,
[type] char(2) NOT NULL,
[currency] char(3) NOT NULL,
[symbol] varchar (12) NOT NULL,
[group] varchar(16) NOT NULL
CONSTRAINT [order$order_pk] PRIMARY KEY CLUSTERED([id]))

CREATE TABLE [dbo].[sec] (
[type] char(2) NOT NULL,
[symbol] varchar(12) NOT NULL,
[code] varchar(9) NULL,
[currency] char(3) NOT NULL,
[country] char(2) NULL,
[issuer] char(2) NULL
CONSTRAINT [sec$sec_pk] PRIMARY KEY CLUSTERED([symbol],[currency],[type]))

ALTER TABLE [dbo].[order] WITH NOCHECK
ADD CONSTRAINT [orderSorder_sec_fk] FOREIGN KEY([symbol], [currency], [type]) REFERENCES [dbo].[sec]([symbol], [currency], [type])
ON DELETE NO ACTION
ON UPDATE NO ACTION

INSERT INTO sec ([type],symbol,[code],currency, country, issuer)
VALUES ('x1','sym1','code 1','USD','US','IS')

INSERT INTO order ([type],currency, symbol,[group]) VALUES ('x1','USD','sym1', 'group1')

Confirmed that the fix is working in the latest build of EA, thanks

Confirmed that the fix is working in the latest build of EA, thanks


Type parameters do not shadow library prefixes in instance creation expressions

Given:
// foo.dart
class Foo {}

And:
import "foo.dart" as T;

class Bar<T> {
  test() {
    new T.Foo(); // <--
  }
}

void main() {
  new Bar().test();
}

I expect an error on the marked line because T should refer to Bar's type parameter, not the import prefix T. Instead, all I get from analyzer is:
hint • Unused import: 'foo.dart' at temp.dart:1:8 • unused_import

If I use T inside test() in some other kind of expression, I get the expected error. I think it's only when used in a new ... expression that it doesn't get the scope right.
I stumbled onto this while converting the language_2/prefix negative tests to be multitests, so this is probably an ancient bug that was never caught because negative tests make it too easy to mistake the true cause of a failure.

This is a bug. There has been some confusion about whether prefixes were statically scoped in places where they needed to be prefixes (which this isn't, but something like new p.C.n() requires p to be a prefix), but the paragraphs saying that "Dart is lexically scoped" are unambiguous.

Analyzer now reports:

error • The prefix 'T' can't be used here because it is shadowed by a local declaration. • a.dart:5:9 • prefix_shadowed_by_local_declaration

Analyzer now reports:

error • The prefix 'T' can't be used here because it is shadowed by a local declaration. • a.dart:5:9 • prefix_shadowed_by_local_declaration


NVD CVE Scraping

Adds scheduled tasks to scrape NVD CVE data and publish the raw data to kafka for processing.

Needs more tests

Needs more tests

Codecov Report

Merging #19 (adc6b1b) into master (7880c6c) will decrease coverage by 13.92%.
The diff coverage is 49.55%.

@@              Coverage Diff              @@
##             master      #19       +/-   ##
=============================================
- Coverage     66.66%   52.73%   -13.93%
- Complexity       16       28       +12
=============================================
  Files            11       17        +6
  Lines            36      146      +110
  Branches          1        3        +2
=============================================
+ Hits             24       77       +53
- Misses           11       68       +57
  Partials          1        1

Impacted Files
Coverage Δ
Complexity Δ

...ava/db/migration/V0_02__EnableCITextExtension.java
100.00% <ø> (ø)
2.00 <0.00> (ø)

...main/java/db/migration/V0_03__CreateTestTable.java
100.00% <ø> (ø)
2.00 <0.00> (ø)

...ndrewkboyd/mitresiphon/MitreSiphonApplication.java
16.66% <0.00%> (-16.67%)
1.00 <0.00> (ø)

...wkboyd/mitresiphon/controllers/TestController.java
100.00% <ø> (ø)
2.00 <0.00> (ø)

...va/xyz/andrewkboyd/mitresiphon/dto/EchoResult.java
50.00% <ø> (ø)
1.00 <0.00> (ø)

.../xyz/andrewkboyd/mitresiphon/tasks/IpRepTasks.java
0.00% <0.00%> (ø)
0.00 <0.00> (?)

...yz/andrewkboyd/mitresiphon/tasks/NVDFetchTask.java
0.00% <0.00%> (ø)
0.00 <0.00> (?)

.../xyz/andrewkboyd/mitresiphon/tasks/TaxiiTasks.java
40.00% <0.00%> (-60.00%)
2.00 <0.00> (-1.00)

...tresiphon/tasks/helpers/HttpModifiedCheckImpl.java
15.00% <15.00%> (ø)
1.00 <1.00> (?)

...yd/mitresiphon/tasks/helpers/NvdHttpFetchImpl.java
27.27% <27.27%> (ø)
1.00 <1.00> (?)

... and 8 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7880c6c...adc6b1b. Read the comment docs.

Codecov Report

Merging #19 (adc6b1b) into master (7880c6c) will decrease coverage by 13.92%.
The diff coverage is 49.55%.

@@              Coverage Diff              @@
##             master      #19       +/-   ##
=============================================
- Coverage     66.66%   52.73%   -13.93%
- Complexity       16       28       +12
=============================================
  Files            11       17        +6
  Lines            36      146      +110
  Branches          1        3        +2
=============================================
+ Hits             24       77       +53
- Misses           11       68       +57
  Partials          1        1

Impacted Files
Coverage Δ
Complexity Δ

...ava/db/migration/V0_02__EnableCITextExtension.java
100.00% <ø> (ø)
2.00 <0.00> (ø)

...main/java/db/migration/V0_03__CreateTestTable.java
100.00% <ø> (ø)
2.00 <0.00> (ø)

...ndrewkboyd/mitresiphon/MitreSiphonApplication.java
16.66% <0.00%> (-16.67%)
1.00 <0.00> (ø)

...wkboyd/mitresiphon/controllers/TestController.java
100.00% <ø> (ø)
2.00 <0.00> (ø)

...va/xyz/andrewkboyd/mitresiphon/dto/EchoResult.java
50.00% <ø> (ø)
1.00 <0.00> (ø)

.../xyz/andrewkboyd/mitresiphon/tasks/IpRepTasks.java
0.00% <0.00%> (ø)
0.00 <0.00> (?)

...yz/andrewkboyd/mitresiphon/tasks/NVDFetchTask.java
0.00% <0.00%> (ø)
0.00 <0.00> (?)

.../xyz/andrewkboyd/mitresiphon/tasks/TaxiiTasks.java
40.00% <0.00%> (-60.00%)
2.00 <0.00> (-1.00)

...tresiphon/tasks/helpers/HttpModifiedCheckImpl.java
15.00% <15.00%> (ø)
1.00 <1.00> (?)

...yd/mitresiphon/tasks/helpers/NvdHttpFetchImpl.java
27.27% <27.27%> (ø)
1.00 <1.00> (?)

... and 8 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7880c6c...adc6b1b. Read the comment docs.

Needs storage of last check time in DB

Needs storage of last check time in DB


fix: Added tree overlap again

What does this PR change?
Solves #692

Fixed tree overlap
Trees spawn with more different scales now, this enables putting smaller trees on smaller portions of terrain, so overall more trees should spawn on single empty parcels

How to test the changes?
🌲 fly around and check the trees :) 🌳
Our Code Review Standards
https://github.com/decentraland/unity-renderer/blob/master/docs/code-review-standards.md

@Kinerius this solves the situation of the trees overlapping but I see an strange issue that I've never seen before:

There are terrain holes bein carved on the North and East borders.


[feature] Filter for multiple tags

Apologies if I'm missing something obvious that's already possible, but I'd like to use the tag filter to filter multiple types of block into certain chests (i.e., in this case types of sand and types of gravel).
When clicking a tag option, previously selected tags are disabled currently:

This is the intended behavior. You can attach multiple pipes to a chest if you want to filter by multiple tags.

Ah, well, regardless of intentional, I wrote this as a feature request to see if the current behavior could be changed to support a new use case :)
My use case is auto-sorting stuff into a nice array of barrels, as should be somewhat visible in the screenshot. This isn't possible with multiple pipes, and even with multiple pipes you're limited to 6 tags (or at most 5 if the container should remain usable), which makes more complex sorting impossible.
I realize this is a Minecraft mod issue list, so I want to explicitly say that I don't expect this to be implemented, I'm just floating the idea and implicitly asking about the reason behind the current design.

The tag filter is intended to filter things like "all ores" and "all ingots", it's not intended to be used for.. "all types of sand", which I personally find a little weird to begin with. How many types of sand could your modpack possibly contain? If it's more than, say, 10, then it's too many types of sand. And if it's 10 or less, you're much better off just putting them all into a filter. If you use the largest filter module and two additional filter increase modifiers, there are more than enough slots for that.
I was hesitant to implement a tag filter in the first place, because tags are not something that should be exposed to the player in such an obvious and direct way (similarly to how I believe that chunks and especially the term "chunk loaders" should not have to be exposed directly to players). All of these things are backend mechanics that have little to nothing to do with the actual gameplay. The tag filter is deliberately limited to inspire players to come up with better solutions than making use of what is essentially an engine feature.


Add support for checking bulk upload status

Added new method and updated the docs.

@pazembrz  can we base this on develop branch to control the release process better?

@pazembrz can we base this on develop branch to control the release process better?

Why I did it based on main I have no idea :man_facepalming:  I will re-base it properly into develop next week.

I had to do some git magic to reset base branch, as I was still having commits from main in my PR, but now it's removed and should be ready to merge.


RUM-1004 Add the SR functional tests for checkboxes and radiobuttons

What does this PR do?
A brief description of the change being made with this pull request.
Motivation
What inspired you to submit this pull request?
Additional Notes
Anything else we should know when reviewing?
Review checklist (to be filled by reviewers)

[ ] Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
[ ] Make sure you discussed the feature or bugfix with the maintaining team in an Issue
[ ] Make sure each commit and the PR mention the Issue number (cf the CONTRIBUTING doc)

Codecov Report

Merging #1609 (a56fcc6) into develop (ea817f1) will increase coverage by 0.09%.
The diff coverage is n/a.

@@             Coverage Diff             @@
##           develop    #1609      +/-   ##
===========================================
+ Coverage    83.63%   83.72%   +0.09%
===========================================
  Files          451      451
  Lines        15567    15567
  Branches      2318     2318
===========================================
+ Hits         13018    13032      +14
+ Misses        1930     1920      -10
+ Partials       619      615       -4

see 20 files with indirect coverage changes
:loudspeaker: Have feedback on the report? Share it here.


BEV camera Setting for CarlaSC dataset.

Hello, could you please provide the BEV camera Setting (for example intrinsic or extrinsic matrix) for CarlaSC dataset?

Hi, thanks for your question. The information on sensors can be found in file Data/Generation/GatherCarlaData.py. The BEV camera in specific is found at lines 96 through 105, which defines the BEV camera as:
    cameras = []
    cam_bp = blueprint_library.find('sensor.camera.rgb')
    cam_bp.set_attribute("image_size_x", str(1920))
    cam_bp.set_attribute("image_size_y", str(1080))
    cam_bp.set_attribute("fov", str(105))
    cam_location = carla.Location(0, 0, 50)
    cam_rotation = carla.Rotation(270, 0, 0)
    cam_transform = carla.Transform(cam_location, cam_rotation)
    cam01 = world.spawn_actor(cam_bp, cam_transform, attach_to=vehicle)
    cam01.listen(camera_queue.put)
    cameras.append(cam01)

More information on the sensor parameters can be found at the CARLA website here: https://carla.readthedocs.io/en/stable/cameras_and_sensors/#camera-semantic-segmentation


Flipped output

Hey,
Nice code!
The output is flipped. You probably have noticed. I fixed that with a simple np.flip.
im1 = np.flip(im1, 0)
im2 = np.flip(im2, 0)

im1.T might give the same effect. I don't modify the image orientations, it's just what comes from imageio.imread(), perhaps the most popular library for that task.


Large JSON response preview performance is poor

When an HTTP response body is very large, the performance of the preview tab is very bad, causing the browser to hang while rendering. If the performance of the JSON rendering cannot be improved, it might be a good idea to warn the user before starting the render of very large response bodies, or provide a configurable maximum size to render, reverting to the plain text view like the response tab if the body is too large.

I've done this before (requested too much and the browser hangs), but I never thought of it as a problem with RESTClient.  Instead you can select "Download response body" and then open it with a text editor or other.

My opinion is that if an extension hangs the browser, it is an issue with the extension. If a user is aware that they are going to be getting a very large response, they can certainly avoid the "preview" tab, but I am often surprised by the size of the response, or simply forget that the response will be large while the "preview" tab is already selected.


feat(ARM): Add multi-arch images for chaos-runner

Signed-off-by: Udit Gaurav<EMAIL_ADDRESS>
What this PR does / why we need it:
Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #
Special notes for your reviewer:
Checklist:

[ ] Fixes #
[ ] Labelled this PR & related issue with documentation tag
[ ] PR messages has document related information
[ ] Labelled this PR & related issue with breaking-changes tag
[ ] PR messages has breaking changes related information
[ ] Labelled this PR & related issue with requires-upgrade tag
[ ] PR messages has upgrade related information
[ ] Commit has unit tests
[ ] Commit has integration tests

@uditgaurav do we want to look at the push stage and check if we need changes in config

Thanks @ksatchit resolved circle ci build for push stage!


unable to run mixtral

i'm getting the following error
~/tmp/mlx-examples/llms/mixtral main* 5m
❯ python3 mixtral.py --model-path mlx_model
[INFO] Loading model from disk.
[INFO] Starting generation...
In the beginning the Universe was created.Traceback (most recent call last):
  File "<redacted>/tmp/mlx-examples/llms/mixtral/mixtral.py", line 324, in <module>
    for token, _ in zip(generate(prompt, model, args.temp), range(args.max_tokens)):
  File "<redacted>/tmp/mlx-examples/llms/mixtral/mixtral.py", line 275, in generate
    logits, cache = model(prompt[None])
                    ^^^^^^^^^^^^^^^^^^^
  File "<redacted>/tmp/mlx-examples/llms/mixtral/mixtral.py", line 213, in __call__
    h, cache[e] = layer(h, mask, cache[e])
                  ^^^^^^^^^^^^^^^^^^^^^^^^
  File "<redacted>/tmp/mlx-examples/llms/mixtral/mixtral.py", line 177, in __call__
    r, cache = self.attention(self.attention_norm(x), mask, cache)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<redacted>/tmp/mlx-examples/llms/mixtral/mixtral.py", line 106, in __call__
    queries = self.rope(queries)
              ^^^^^^^^^^^^^^^^^^
  File "<redacted>/tmp/mlx-examples/llms/mixtral/mixtral.py", line 51, in __call__
    costheta, sintheta = RoPE.create_cos_sin_theta(
                         ^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: type object 'RoPE' has no attribute 'create_cos_sin_theta'

these are the contents of my mlx_model folder
❯ l mlx_model/
.rw-r--r--  354 <redacted> staff 20 Feb 12:06 -N config.json
.rw-r--r-- 493k <redacted> staff 20 Feb 12:01 -N tokenizer.model
.rw-r--r-- 3.6G <redacted> staff 20 Feb 12:01 -N weights.0.npz
.rw-r--r-- 3.6G <redacted> staff 20 Feb 12:02 -N weights.1.npz
.rw-r--r-- 3.6G <redacted> staff 20 Feb 12:02 -N weights.2.npz
.rw-r--r-- 3.6G <redacted> staff 20 Feb 12:03 -N weights.3.npz
.rw-r--r-- 3.6G <redacted> staff 20 Feb 12:04 -N weights.4.npz
.rw-r--r-- 3.6G <redacted> staff 20 Feb 12:04 -N weights.5.npz
.rw-r--r-- 3.6G <redacted> staff 20 Feb 12:05 -N weights.6.npz
.rw-r--r-- 3.6G <redacted> staff 20 Feb 12:06 -N weights.7.npz

i've searched for create_cos_sin_theta both in the examples and in the mlx framework code and i can't find it 🤷♂

Yeah that is a bug introduced when we switched RoPE to the fast implementation. No need for creating cos and sin theta anymore.
I would advise using the default RoPE and passing the base as a parameter since it is now supported. If you do fix this feel free to PR it, we would appreciate the fix, otherwise we 'll get to it shortly.

@jbrodriguez let us know if you are working on this, ow I will send a fix later today.

Please check PR @angeloskath and @awni is it ok?

i was worked on it https://github.com/ml-explore/mlx-examples/pull/470, but someone beat me to it :)

Merge the @jbrodriguez PR 💪

Love the chivalry and attitude from both of you guys!!!

Closing since @jbrodriguez fixed it at #470 .


Define zos license

MIT
IMO it is the less restrictive license. It gives the rights to anyone to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the software. It just requires that the given license is included in all the copies of the code.
In addition, it clarifies the code does not provide warranty of any kind, and denotes that the authors are not responsible for any problem the code may cause.
GPL 3.0
This license is more extensive and descriptive. It gives the rights to anyone to use, copy and modify the software. However, it does not allow sublicensing. i.e. the main difference would be that you can modified the code without including said license if you don't distribute it. It also clearly explains that there is no warranty for the software.
Apache 2.0
Has a pretty similar philosophy to the MIT. It gives the rights to anyone to reproduce, copy, modify, publish, distribute, sublicense, and distribute and/or sell copies of the software. It states the code does not provide warranty of any kind too
Some differences with the MIT license are that:

it requires any modified file to carry a note stating those changes.
it does not grant permission to use the trademarks or product names of the Licensor.
it provides an express grant of patent rights from contributors to users.

References

https://opensource.org/licenses
https://choosealicense.com/

@elopio would you mind sharing your thoughts/ideas?

Thanks for the summary!
There is a typo here which makes the sentence a bit difficult to understand:

you can modified the code without including said license

The distinction between GPL and LGPL is also relevant.

From what I've seen so far, most solidity projects use MIT (expat license is a better name). But I think that's because most of the solidity code out there are libraries that have to be mixed with other code to be useful. We are just starting to see complex platforms like Zeppelin OS, with services that can be consumed by stand-alone projects. IMO, the expat license is not good enough for this.
The FSF recommends to use Apache 2 instead of expat, because of the patents provision, which is something very important for us to take into account. I'm ok with Apache 2.
Now about GPL, take a look at the aragon case. They chose GPL. Somebody copied a part of the code and released it under MIT: https://twitter.com/izqui9/status/977858436988891136
If we choose MIT/expat/Apache2, we can't copy GPL code. If we choose GPL, we force everybody who copies our code to release under GPL and then we can copy the improvements.
And as @frangio mentions, the GPL/LGPL distinction is very interesting here. We could license the core as GPL, and the libraries as LGPL. Or everything LGPL. But, I'm not so sure how to apply the terms static and dynamic linking, loadable modules and system libraries to blockchain contracts.
See here: https://en.wikipedia.org/wiki/GNU_Lesser_General_Public_License#Differences_from_the_GPL
here: https://www.gnu.org/licenses/why-not-lgpl.html
here: http://linuxmafia.com/faq/Kernel/proprietary-kernel-modules.html
and here: https://www.gnu.org/licenses/gpl-faq.en.html#SystemLibraryException
If we choose MIT/expat/Apache2, we put little restrictions to derivatives so those terms are not so important.
I personally prefer GPL always, because to me, every piece of code released that way helps the revolution. This is a harder fight, it's more complex to enforce, we will lose potential contributors that don't like or can't release their code under GPL, and we will need to solve in a very clear way the concepts of static, dynamic, loadable and system. It could be very beneficial later on the game, but we need to discuss if it will help or complicate our immediate goals.
If we add a nice CLA like the one from FSFE, the specific license we choose is less of a concern, because we will be able to relicense as long as the new license is free too.
But two things. First, I'm very new to this community. There are some communities out there that are averse to GPL and CLAs. We can invite the OpenZeppelin community to voice their opinions here.
Second, I'm not a lawyer. I recommend to involve our legal advisors early. They can tell us how to best protect our project depending on the license we choose.

First of all and experience in open source I would strongly recommend to involve your lawyer/legal advisory on this. In a nutshell LGPL is the most business-friendly one, largely used by companies working w/ open source products/library integrated in a commercial product. If you don't have completely clear your business model (or if you would at least stay open to different business model) You should select the license easier adapting to different business model (read LGPL).
Just my 2C


Objects with an unhashable __name__ property cause an exception

Some objects (unittest.mock._call for instance) have a __name__ property which isn't a string. This causes a predictable:
TypeError: unhashable type: '_Call'
In:
/pytest_involve.py:335: in get_members_by_file
The solution here is probably to check if __name__ is a string. It won't be very common; probably will just be anything using the descriptor api so that getattr returns a value when it shouldn't.

Closed in https://github.com/MisterKeefe/pytest-involve/pull/2


Update Modules.md

I had not tested the calculator object but 1+2*33/11 should make 7, if the calculator does its maths right, of course.....

I ran the code, and it gave 9
It probably doesn't take operator ordering into account. I'm open to changes to the calculator, but as-is this comment is correct


Junit formatter prevents Jenkins picking up results on success

Bug Report

TSLint version: 5.9.1
TypeScript version: 2.7.1
Running TSLint via: CLI

Actual behavior
Running tslint -o lint-results.xml -t junit -p tsconfig.json on a lint-free project results in an xml file with an empty <testsuites> element. While this is acceptable by some consuming tools, others reject it. Unfortunately for me, the latter includes Jenkins. Jenkins hits a non-breaking error trying to consume lint-results.xml, and skips it. Which makes generating the file in the first place pointless.
Before you added the core junit formatter, we could use tslint-junit-formatter, whose last commit before then seems to have been a fix for this very issue. But because tslint prioritises its own core formatters over external ones with the same name, this formatter can no longer be used (it's mostly redundant except for that one fix, anyway).
Expected behavior
A single "success" <testsuite> element is generated inside the <testuites>, and Jenkins is able to consume it (as is Bamboo, which has the same problem, according to the referenced tslint-junit-formatter issue).
I will say I'm happy to submit a PR for this one, if you'll have me!

https://github.com/palantir/tslint/pull/3838 would pave the way for this.

This is the same for Bamboo and probably other CI tools.
tslint-junit-formatter already does this, so it's solution could be used, see:
See https://github.com/GenoLogics-OpenSource/tslint-junit-formatter/issues/8

@DaemonExMachina 	You still want to create a PR for that? 👍

Just added https://github.com/palantir/tslint/pull/4566 - that should fix the problem.

Thanks!


Multi language is not working with relationship data

Multi language is not working with relationship data.
We have created relationship between 2 tables. when we go to edit page then relationship data is not show/translate as per selected language see the screenshots

Please use the issue template.
This can be reopened once the required information are provided.


[contracts] Add forge coverage

Background
We haven't been using coverage reports, so can't evaluate our current unit-test posture or monitor for any degradations. Would be good to put this in main so we can evaluate the audit PR coverage (all PRs have had tests reviewed manually).
Linked Issues & Documentation
N/A
Definition Of Done

Easily generate coverage reports
(optional) Fails on CI if coverage is below some threshold

Tasks

[ ] Coverage reporting available
[ ] Threshold enforced in CI

NOTE: included this in the sprint as it will enforce coverage on inbound PRs for contracts.
May not be appropriate.

Closing, there is an open issue blocking this without refactoring (which is not a high priority at this time).


xarray ufuncs

The documentation warns that the universal functions in xarray.ufuncs should not be used unless compatibility with numpy < 1.13 is required.
Since we only support numpy >= 1.15: is it time to remove that (already deprecated) module?
Since there are also functions that are not true ufuncs (e.g. np.angle and np.median) and need __array_function__ (or something similar, see #3917), we could also keep those and just remove the ones that are dispatched using __array_ufunc__.

Sounds good to me

which suggestion do you mean? Removing the whole module or just the __array_ufunc__ functions?

actually, it seems we might need to wait a bit longer, the deprecation warning says we would remove them only after we stopped supporting numpy < 1.17.

I think we should put that warning (removed after support for 1.17 is dropped) into the documentation, too.

:+1:


Fixed showing current datetime instead of release datetime in package…

Hi guys.
I fixed problem with showing current datetime instead of release datetime in package detail view (issue #79).
Have a nice day.
Peter

Thanks for the pull request!


emit event must be in the update block

@pirapira  emit Incremented(newData) should be in the update field.  Events modify the substate, which affects the blockhash.

I'm not 100% sure this has to be the case. While events cannot be emitted in a STATIC_CALL context, they are not something that changes state like storage writes do.

I see. The program cannot access the registered events, so firing an event can never break an invariant.

Ok, so I close this one :-)


Add SRFI 115

This one is for regular expressions. Will be necessary for FFI implementation of purescript-strings. This one has quite a number of dependencies but it seems to be the only SRFI that deals with regex.
Update:
It appears this one doesn't support traditional regex and instead only works with S-exp regex. This is neat but is a major mismatch for the PS API. Writing a translation would be too expensive, so I'm looking at alternative libraries. There doesn't seem to be any other regex SRFI library, can we add a non-SRFI one?

Closing. Related discussion in #193.


Return change in query name in "TRIGGERS" mode

You could somehow change the json response structure for Grafana in “TRIGGERS” mode. If possible, match the structure mode that is returned in the “Metrics” query model
In the example below I create 2 queries, for demonstration purposes:
As you can see, in “Metrics” mode, 1 “OBJECT” and 3 “Fields” are returned, 1 being time and 2 from the respective queries

The return of the 3 columns, both of which can think about using the “time” masses, then turning to the transformation of GRAFANA into objects. And the query name is established by the host and item name.

The “TRIGGERS” query mode already returns 2 objects and 2 fields each query. Since the name returned is “triggers count” for both queries, making it impossible for me to disassociate the two in a specific view panel.

I ask first for the possibility of being able to set the query return name in “triggers” mode as “$hostname_application”, and if possible change the structure as it is in “METRICS” mode.

So, I'm using your plugin together with the "FLOWCHARTING" visualization plugin, and within this there is the option to map this query in a graph, but I am unable to do more than one mapping because the queries use the same title name, with no possibility of differentiation.
Example: In the "METRICS" query mode I can use the queries normally after setting only one transformation(multi-frame-time-series), which basically transforms 3 "FIEDS" (time, and the two queries) into two "OBJECT" (each object with time and a query) and turns one table into two.

Unlike the "TRIGGERS" query mode option that generates the same names, making it impossible to use the plugin. since I need to differentiate each count query, as the idea is that each one has its mapping

Ok, got it!

It would be interesting if the return had the same structure as the "METRICS" mode, in this case "$host : $application  "

I'm not sure how exactly flowcharting plugin handles data frames, but I anyway want to refactor all returning data to data frames since it's a new standard in Grafana. Hopefully it will work.

I added simple "Count " name, that should be enough.