I/O error with Linux driver

Hello!
I built ODBC driver for Linux (build passed without any errors, master branch). But ODBC driver can perform only one query per connection, second query throws I/O error. For example, this error reproduces in isql:
vagrant@vagrant-ubuntu-trusty-64:~$ isql clickhouse -v
+---------------------------------------+
| Connected!                            |
|                                       |
| sql-statement                         |
| help [tablename]                      |
| quit                                  |
|                                       |
+---------------------------------------+
SQL> show tables
+------+
| name |
+------+
| test1|
| test2|
+------+
SQLRowCount returns 2
2 rows fetched
SQL> show tables
[S1000][unixODBC]I/O error
SQL>

I use this driver in Erlang application and this error there also happens.
I tried to build driver on local machine, on Vagrant and on CircleCI container. And this error is reproduced everywhere.

Maybe fixed here -
https://github.com/yandex/clickhouse-odbc/commit/4a69b7ce2d2d8f85e6c4d9bd58f34065e802f0fe


Using build works, build --prod fails with intl-messageformat

🐞 Bug report

[ ] new
[x] build
[ ] serve
[ ] test
[ ] e2e
[ ] generate
[ ] add
[ ] update
[ ] lint
[ ] xi18n
[ ] run
[ ] config
[ ] help
[ ] version
[ ] doc

Is this a regression?
No
Description
I've added intl-messageformat to the project and copied and pasted their example code. serve and build work fine. Build --prod works fine with Chrome, however build --prod with the optimisation: true fails in IE (and in Chrome if forced to use the es5 files).
I have added commonChunk: false to the angular.json build config. And added the intl-pluralrules polyfill to the polyfills file.
    "intl-messageformat": "^7.6.0",
    "intl-pluralrules": "^1.1.1"

I've created a repo here
https://stackblitz.com/edit/angular-p1cyli
🔬 Minimal Reproduction
ng build --prod
serve the dist project in IE11 and look for the error in the console.
🔥 Exception or Error
IE11

SCRIPT5007: Object expected
main-es5.a51589e5bc4634a245a6.js (1,3570)

🌍 Your Environment

Angular CLI: 8.3.19
Node: 12.13.1
OS: darwin x64
Angular: 8.2.14
... animations, common, compiler, compiler-cli, core, forms
... language-service, platform-browser, platform-browser-dynamic
... router

Package                           Version
-----------------------------------------------------------
@angular-devkit/architect         0.803.19
@angular-devkit/build-angular     0.803.19
@angular-devkit/build-optimizer   0.803.19
@angular-devkit/build-webpack     0.803.19
@angular-devkit/core              8.3.19
@angular-devkit/schematics        8.3.19
@angular/cli                      8.3.19
@ngtools/webpack                  8.3.19
@schematics/angular               8.3.19
@schematics/update                0.803.19
rxjs                              6.4.0
typescript                        3.5.3
webpack                           4.39.2

Anything else relevant?

Looking further at this, including the @formatjs libs directly from their dist builds correctly.
From
import IntlMessageFormat from "intl-messageformat";
import { supportedLocales } from '@formatjs/intl-utils';

to
import IntlMessageFormat from "intl-messageformat/dist/core.js";
import supportedLocales from<EMAIL_ADDRESS>
I will mention this on @formatjs.
I'll levea

Can you check if this is still reproducible with version 9?
You can update using;
ng update @angular/core @angualar/cli --next

Yes, it is still an issue
main-es5.607632b5ecce60f3d54d.js:1 Uncaught TypeError: Sf is not a function
    at main-es5.607632b5ecce60f3d54d.js:1
    at Module.zUnb (main-es5.607632b5ecce60f3d54d.js:1)
    at a (runtime-es5.06dc6a1139ed7db15522.js:1)
    at Object.0 (main-es5.607632b5ecce60f3d54d.js:1)
    at a (runtime-es5.06dc6a1139ed7db15522.js:1)
    at t (runtime-es5.06dc6a1139ed7db15522.js:1)
    at Array.r [as push] (runtime-es5.06dc6a1139ed7db15522.js:1)
    at main-es5.607632b5ecce60f3d54d.js:1
(anonymous) @ main-es5.607632b5ecce60f3d54d.js:1
zUnb @ main-es5.607632b5ecce60f3d54d.js:1
a @ runtime-es5.06dc6a1139ed7db15522.js:1
0 @ main-es5.607632b5ecce60f3d54d.js:1
a @ runtime-es5.06dc6a1139ed7db15522.js:1
t @ runtime-es5.06dc6a1139ed7db15522.js:1
r @ runtime-es5.06dc6a1139ed7db15522.js:1
(anonymous) @ main-es5.607632b5ecce60f3d54d.js:1

just wondering if there is any movement on this?

NG is now using webpack 4.41.4 since NG CLI  8.3.21 and changes some module handling, this issues appears resolved for me so I will close it.


Stop mixing "import" and "require"

Any chance you will clean up the mess of mixing import and require?

There is no reason to use require in the files below...it breaks overall coding practices.
// core/glyphlist.js
17: var getLookupTableFactory = require("./core_utils.js").getLookupTableFactory;

// core/unicode.js
var getLookupTableFactory = require("./core_utils.js").getLookupTableFactory;

There is no reason to use require in the files below...it breaks overall coding practices.

Yes, it is most certainly necessary!
Unless those files are kept ES5-compatible, providing Babel translated builds of the PDF.js library wouldn't work because size of those files; see https://github.com/mozilla/pdf.js/pull/8287#issue-115954313.

There is no reason to use require in the files below...it breaks overall coding practices.

Yes, it is most certainly necessary!
Unless those files are kept ES5-compatible, providing Babel translated builds of the PDF.js library wouldn't work because Babel cannot handle files that large; see https://github.com/mozilla/pdf.js/pull/8287#issue-115954313.

Then I am really sorry you are dealing with this mess - a lot bigger than I first thought :-(


Replaced all hardcoded configuration options (PORT, AUDIO, WPM) with …

…uses of 'config' module.

Thanks for updating all of the commands. I didn't want to do that until it was tested on Mac and I don't get my Mac back until mid-week.


Multi_gpu train issue

I changed the default parameters num_gpus-->8\num_nodes-->8, and keep other default parameters unchanged, then run the train.py with 8_v100, but it always stuck on the device initialization and can't go on the data read process, are there any additional settings required here? 🤔

I tested other multi_number, it's all stuck, ang can successly run when gpus=1


chore(master): release 0.15.1

:robot: I have created a release beep boop
There were no major changes between 0.15.0 and 0.15.1 so below is the change log for the 0.15.0 release.
📈 Extending the built support for ARM64 OS
We have added support for Linux ARM64 and macOS M1 machines.
✍ Code signing
We have implemented code-signing of our binaries for macOS and Windows releases using our Apple Developer account Swarm Association, so now the installation flow is much smoother.
💦 Splash screen
When Swarm Desktop is launching there is a splash screen displayed in order to communicate to the user better the progress of the startup.
🪟 Windows build improvements
The Windows build now has a proper Swarm Desktop icon put on the Desktop and Start menu.
🐝 New Bee 1.6.2 version
We have upgraded to the latest Bee 1.6.2 version.
🏄 Bee Dashboard improvements
We have done a lot of work on fixes and improvements to the Dashboard UI

improved the top-up and upgrade workflows
added postage stamp buy guide
Bee node status data auto-refresh
and much more that you can see in Dashboard's CHANGELOG

Features

add header to onboarding overlay transaction request (#169) (b1a0179)
add token price endpoint (#175) (ea15234)
auto update support (#146) (9e6f869)
bundle static assets (#144) (b50438d)
code signing (#137) (7e5feca)
improved electron-reforger config (#147) (83bcf36)
splash screen (#182) (250af98)
top level error catch (#156) (a7afb4e)
windows installer screen (#203) (6508cf7)

Bug Fixes

auto update logger (#181) (079be52)
copy installer for windows (#183) (4900169)
dmg release dependency (#153) (61d4f90)
installer: install estimate fix (#157) (87fafe6)
remove ellipsis from main text (#174) (f165358)
splash overflow hidden (#202) (7012f88)
squirrel startup test (#178) (df6c5b2)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/ethersphere/swarm-desktop/releases/tag/untagged-412dfe41baa04b92e3bd :sunflower:


web dev club page

The pages are complete UI-wise. Will you be updating the data in a separate PR?

@dinskid Yes ! I will do it in next pr, as it takes only data upgradation so ig this is not a matter of concern anyone can do it.

Cool! The rest of the things are fully functional from what I tested. I'll merge the PR, thanks!


Fix panics related to Walk func and also path handling in default layouts

The stock code in 0.5.0 did not work in 2 cases and would panic.
The first was within a filepath Walker function that upon encountering an error in the walk function would not test for the error and would then panic when trying to use input parameters to get file information.
The second is a bit mysterious.  When using cleanup the go path would use multiple up directory filepath components to navigate to the top of a directory tree.  If the GOPATH was set of a project in a local file system with for example /home/user/project would end up stripping important potions of the path.  This is probably because it is being run in another environment that does not use a directory tree for projects that are being used with trash.  I figure this is some incompatibility between the types of project structures that can be used with go.

Please review carefully as I believe there could be conflicting requirements in relation to the cleanup function between systems such as Circle and Travis CI etc and using it locally within a file system.

@karlmutch Go projects are supposed to have a certain directory structure. I admit we need to be more intelligent than just cd ../../../../src to get to source root. That will be addressed in a later release. Feel free to suggest a fix.
What I'm trying to do is to guess the root package of the project trash is being run against.
For example, rancher/os should reside in /go/src/github.com/rancher/os (if all your Go projects are using /go as GOPATH) or /home/ivan/src/rancher/os/src/github.com/rancher/os (I'm using a dedicated path as GOPATH for just RancherOS). In both cases I can tell the "root" package of the project is "github.com/rancher/os" from the directory structure of the checked out project.
For projects that do not observe the Go Way dir structure, we probably want to use the top level package key (like in trash's own trash.yml) to get the root package.

Splitting into two separate requests.  Not entirely sure how to organize forked repo multiple PR's so will close the request and open two new ones using branches on my personal fork.


Error in PubMed evaluation using run_summarization.py

I am using the script roberta_base.sh to train and test the model on PubMed summarization task. I am able to successfully train the model for multiple steps (5000) but it fails during evaluation time. Below is some of the error string.
I0416 18:16:41.567906<PHONE_NUMBER>30944 error_handling.py:115] evaluation_loop marked as finished
WARNING:tensorflow:Reraising captured error
W0416 18:16:41.568143<PHONE_NUMBER>30944 error_handling.py:149] Reraising captured error
Traceback (most recent call last):
  File "bigbird/summarization/run_summarization.py", line 534, in <module>
    app.run(main)
...
File "/usr/local/lib/python3.7/dist-packages/tensorflow_estimator/python/estimator/tpu/tpu_estimator.py", line 2268, in create_tpu_hostcall
    'dimension, but got scalar {}'.format(dequeue_ops[i][0]))
RuntimeError: All tensors outfed from TPU should preserve batch size dimension, but got scalar Tensor("OutfeedDequeueTuple:0", shape=(), dtype=float32, device=/job:worker/task:0/device:CPU:0)

I am not too familiar with the code and about this error. Searched it online but didn't get much help. Hope you can help.  Evaluation works if the use_tpu is made false and code is run on GPU or CPU. But it takes way longer. Any thoughts on how to resolve this ?

Hi @prathameshk, can I ask how do you finetune the model on your custom dataset? I was thinking replace data_dir by path_contains_tfrecords, but I got errors.

If you haven't already, then check out the HuggingFace implementation of BigBird. That can be easier to use and integrate with your project.


Is this package going to work with materialize 1.0.0 rc1 ?

Hi,
From the example I saw on the site, it seems that this works with materialize 0.9, but will it work with 1.0 rc1 ? Especially considering that there has been major rewrite.
Thanks
Atindra

No, our version is base on 100.2. We will upgrade to version 1.0.0 when it's released.

@scote Any news on update guys? v1.0.0 has been released 6 months ago. We have some plans to to this?

Sadly I am not working on this project anymore.

@scote Sad to hear that. @jfcere still working on this?


slurm: <IP_ADDRESS> -> <IP_ADDRESS>

Motivation for this change
Regular update.
Changelog:
https://github.com/SchedMD/slurm/blob/slurm-18.08/NEWS
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[x] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
nixos/tests/slurm.nix succeeds
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Assured whether relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

@GrahamcOfBorg build nixosTests.slurm


Fix DateTime('UTC') parsing error

also possible fix https://github.com/didww/clickhouse/commit/b8f31fed27f08c74feec123ed4d46a56612380aa
@archan937  can u merge ?

@tatarsky-v what if different timezone than UTC ?


ggplot2 language change, deprecates older code

Describe the bug
If update the app will need to update references to aes_string from ggplot2 as that command is deprecated.
To Reproduce
Steps to reproduce the behavior:

Launch the app locally.

Expected behavior
Should work without errors but this breaks the entire app.
Screenshots

Additional context
If don't have updates to the app then don't have to change.


Use Permission gate for photo library access on iOS

Describe the bug
On the first time using the app and taking a photo with the standard camera the user is presented with permission gates for camera permission and location permission. However, there seems to be no gate for the photo library permission.
To Reproduce
Steps to reproduce the behavior:

Fresh install of the app
Go to + (Start observation)
Use Standard camera
(Notice no native permission alert) Press "Observe Organisms"
Grant camera permission
Take photo
Press the checkmark button
Notice the native permission alert for ""iNaturalistReactNative" Would like to Add to your Photos. This permission has no permission gate?

Expected behavior
There should be a permission gate for the alert of granting photo library permission.
Screenshots

Context (please complete the following information):

Device: iPhone 7
OS: iOS 15.7.9
App version: 0.14.0 + newer commits

@abhasinat, when do you think this permission gate should appear? FWIW, there are three permissions we need in this flow: camera (to take a pic), photo library (to write that pic out to your photo lib), and location (to get the obs coordinates). We ask for camera when you tap the camera button in the add obs modal, and we ask for location on ObsEdit when we start trying to fetch your current location. IMO we should for photo library permission when you tap the green checkmark button on StandardCamera, i.e. when you're ready to leave the camera and move to ObsEdit.

I agree- I think showing the gate on taking a picture sounds great.

@abhasinat is there specific header, body, and button text you want to use for this permission gate? I'm assuming we'll need different text than what we're using on the Gallery permissions granted and denied screens. I'm also assuming we can keep the same background image and icon that we use on the Gallery screen.

Ahh, good catch. I hadn't added a separate priming screen for saving to the gallery permissions. I've added that in Figma now with a new background image we can use.
I don't think this priming screen needs a denied state, since we don't want to reprompt that every time a user takes a photo.


Running multiple specific tests can result in unwanted tests run

Thought this should have been fixed by: https://github.com/Azure/azure-cli/pull/5520
Related: https://github.com/Azure/azure-cli/issues/5519
Specified four tests, every other test in the file/class was run
C:\Users\wilx\azure-cli>

azdev test --test test_storage_file_batch_upload_scenarios test_storage_file_batch_copy test_storage_blob_batch_copy test_storage_file_batch_upload_scenarios
The tests are set to run against current profile latest.

=============
  Run Tests
=============

Modules: storage
Drive test by nosetests
test_storage_blob_batch_copy (azure.cli.command_modules.storage.tests.latest.test_storage_batch_operations.StorageBatchOperationScenarios) ... SKIP: This is a live only test. A live test will bypass all vcrpy components.
test_storage_blob_batch_delete_scenarios (azure.cli.command_modules.storage.tests.latest.test_storage_batch_operations.StorageBatchOperationScenarios) ... SKIP: This is a live only test. A live test will bypass all vcrpy components.
test_storage_blob_batch_download_scenarios (azure.cli.command_modules.storage.tests.latest.test_storage_batch_operations.StorageBatchOperationScenarios) ... SKIP: This is a live only test. A live test will bypass all vcrpy components.
test_storage_blob_batch_upload_scenarios (azure.cli.command_modules.storage.tests.latest.test_storage_batch_operations.StorageBatchOperationScenarios) ... SKIP: This is a live only test. A live test will bypass all vcrpy components.
test_storage_file_batch_copy (azure.cli.command_modules.storage.tests.latest.test_storage_batch_operations.StorageBatchOperationScenarios) ... SKIP: This is a live only test. A live test will bypass all vcrpy components.
test_storage_file_batch_delete_scenarios (azure.cli.command_modules.storage.tests.latest.test_storage_batch_operations.StorageBatchOperationScenarios) ... SKIP: This is a live only test. A live test will bypass all vcrpy components.
test_storage_file_batch_download_scenarios (azure.cli.command_modules.storage.tests.latest.test_storage_batch_operations.StorageBatchOperationScenarios) ... SKIP: This is a live only test. A live test will bypass all vcrpy components.
test_storage_file_batch_upload_scenarios (azure.cli.command_modules.storage.tests.latest.test_storage_batch_operations.StorageBatchOperationScenarios) ... SKIP: This is a live only test. A live test will bypass all vcrpy components.

----------------------------------------------------------------------
Ran 8 tests in 6.549s

OK (SKIP=8)

Environment summary
Install Method (e.g. pip, interactive script, apt-get, Docker, MSI, edge build) / CLI version (az --version) / OS version / Shell Type (e.g. bash, cmd.exe, Bash on Windows)

Update: specifying any more than 1 test causes all the tests to run in that file/class

This will not be a simple fix and seems to be a flaw with nose as a test runner. I'll look at pytest as a runner, but in the meantime I'll push it back to investigate deeper next release.

Upon further investigation, pytest seems to be a poor (i.e. VERY slow) replacement for nose.  If you run the tests with --series then it will play the correct number of tests. There is a tradeoff--at some point the test will run faster playing all tests in parallel than it will playing select tests in series. The fact that most test classes have a single test also mitigates the severity of this.

To summarize the problem:

azdev test is run, targeting multiple specific test methods
the class containing the test methods contain other test methods

In this case, by default, the test infrastructure will run all tests within the test class that contained the targetted test method.
The workaround is to run the command with --series, i.e.:
azdev test test1 test2 --series
If the test class contains many slow tests that you really don't want re-run, this could result in a quicker run. However, if you would only omit a few tests by running in series, it may be faster to simply run all the class test methods in parallel.

@williexu,  @tjprescott, is this issue still valid? It is good to have this addressed if not overly expensive. Maybe we can automatically append the --series ?

I think there's a good chance it is still valid.


SwaggerResponseAttribute - Type of IEnumerable

I just started playing around with this library today, so I may be missing something, but is this attribute compatible with IEnumerable?
If I set it to something like this, in the UI it correctly shows the model/schema:
[SwaggerResponse(System.Net.HttpStatusCode.BadRequest, "Validation Failed", typeof(ValidationResult))]
However, if I try to say it's an enumeration of some kind, the UI shows nothing for the response model:
[SwaggerResponse(System.Net.HttpStatusCode.BadRequest, "Validation Failed", typeof(ValidationResult[]))]
[SwaggerResponse(System.Net.HttpStatusCode.BadRequest, "Validation Failed", typeof(List<ValidationResult>))]
[SwaggerResponse(System.Net.HttpStatusCode.BadRequest, "Validation Failed", typeof(IEnumerable<ValidationResult>))]
If I set the ResponseType to the same enumerable, the "Response Class (Status 200)" section at the top does populate correctly.  I tried to fork the project to determine if these two sections were following different logic, but I couldn't really tell either way.
The project looks great!  And I saw the thread about questioning whether to include this SwaggerResponse attribute, and I think it was a good choice adding it in!  Thanks!
edit
Looking at the data that is getting sent to the browser, it does appear everything is getting setup correctly:

There must just be an issue on the display side

I believe this is actually an issue with the swagger-ui as opposed to SB itself.
First and foremost, SB is a Swagger JSON generator. For convenience though, it does include an embedded version of swagger-ui, which consumes the generated JSON, so that you can seamlessly provide a rich documentation UI with a one-line Nuget install.
Can you view the raw JSON directly (browse to swagger/docs/v1) and check if the responses are being added correctly?. I suspect this to be the case but it would be great if you can confirm.
This means that SB is doing it's job correctly. However, I've noticed the support in swagger-ui for error model descriptions is somewhat limited and I think this may be what you're experiencing. If this is the case, then the swagger-ui repo may be a better place to find some answers or post an issue:)
Let me know ...

Yep, after originally posting I edited it to note it appears the JSON seems to be correct.
Sorry, I didn't realize the actual ui piece was a separate project;  I'll post this scenario over there and we'll see what we can find.
Thanks!


feat: automatically activate a sequence when added

Resolves #35.

@gwhitney, I can merge this PR, but like you said at last week's meeting, if there's anyone else who wants some practice, please feel free to merge it. If anyone needs a primer, this video provides an overview of creating and merging a PR.


Optimization & Performance Evaluation?

Hi,
I was wondering if there is any guide that I could follow in order to optimise the loading time (performance) of map resources and if there is any way I could evaluate/estimate the system resources that the system should have in order for multiple concurrent users that are accessing the endpoints(lets say ~around 500 users).
Currently if I am using the bonn example and I am trying to get the "roads" section of the map, its a bit slow, even in localhost.
Thanks,
Ionut

@istancescu good question. We're actually working on some observability metrics via #714 which will help with run-time evaluation. You could use these along with some load testing tools to estimate the resource allocation at runtime.
How often is your data updating? If the data is not constantly changing (i.e. every second) then the best way to handle scaling is to implement various layers of caching using an object store (i.e. S3) and a CDN. With these layers in place, you can then have tegola handle cutting new tiles on cache misses. I can provide an example architecture if this would work for your situation.

@ARolek the plan is to serve multiple vectors via endpoints. These endpoints might be world-wide and can be used in multiple applications and be consumed with Mapbox GL.
The data will be imported manually into the database so I think it will remain the same but distributed multiple times to multiple consumers.
The whole point would be to overlay the distributed vectors onto some maps by the consumers.
Yes please, an example architecture would be helpful.


auto change hls stream to match live stream

hi all,
atm i have this line
exec /usr/bin/ffmpeg -i rtmp://localhost/live/$name -threads 2 -c:v libx264 -profile:v baseline -b:v 4000K -s 1280x720 -f flv -c:a aac -ac 1 -strict -2 -b:a 128k rtmp://localhost/hls/$name;
but trouble with this is when someone changes the settings on the encoder box of the live stream it breaks my hls stream as i need to then reconfigure the video/audio bitrate and size
could i change it with this line
exec /usr/bin/ffmpeg -i rtmp://localhost/live/$name -threads 2 -c:v libx264 -profile:v baseline -f flv -c:a aac -ac 1 -strict -2 rtmp://localhost/hls/$name;
thanks,
rob

smashed it
exec /usr/bin/ffmpeg -i rtmp://localhost/live/$name -threads 2 -vcodec libx264 -vprofile baseline -acodec aac -strict -2 -f flv rtmp://localhost/hls/$name;


Wait for Pluto to start

In the function which starts the IKE daemon, wait for it to start (up
to 5s) before returning. This helps avoid races when starting the
gateway; if we don't wait, the daemon starts alongside the gateway,
and in some cases we end up trying to set tunnels up before the daemon
is ready, resulting in delays (the tunnels aren't set up until the
next resync round).
Signed-off-by: Stephen Kitt<EMAIL_ADDRESS>

LGTM
Are 5 secs sufficient? or shall we give it some more time?

It's normally very quick, just a process start.

We can always bump it if that's not enough.


Useful Vim Shortcuts!

Hi guys! If you have ever wondered how you can make coding with vim easier, I found a useful page that might come in handy for some of you :)
https://vim.rtorr.com/
Some of the shortcuts I have been using:
ZZ - save and quit
gg=G - auto-indent entire file
:%s/old/new/gc - replace all old with new throughout file with confirmations
dd - delete the line your pointer is on
v - start visual mode, mark lines, then do a command (like y-yank)
y - yank (copy) marked text
p - paste copied text
d - delete (cut) marked text
w - jump forward to the start of a word
b - jump back to the start of a word
There are definitely many more useful ones in the link above so do check them out! Hopefully, this can help some of you who are finding it difficult to code in vim :)

Thanks! My fav shortcut is (number)dd because I always have useless code

I also want to add that the commands under tabs are very useful. Also, to open tabs for all your java files in the directory, you can type vim -p *.java if you don't care about the order of the tabs.

In fact for d and dd it is not delete but cut! What you are doing is cutting the lines so next time you press p it will paste them in.
Also a more useful tip regarding these shortcuts d, v, y is that if you enter the capital version, D, V, Y, it will immediate cut the whole line, enter Visual line mode  and copy the whole line.

Hi guys! If you have ever wondered how you can make coding with vim easier, I found a useful page that might come in handy for some of you :)
https://vim.rtorr.com/
Some of the shortcuts I have been using:
ZZ - save and quit gg=G - auto-indent entire file :%s/old/new/gc - replace all old with new throughout file with confirmations dd - delete the line your pointer is on v - start visual mode, mark lines, then do a command (like y-yank) y - yank (copy) marked text p - paste copied text d - delete marked text w - jump forward to the start of a word b - jump back to the start of a word
There are definitely many more useful ones in the link above so do check them out! Hopefully, this can help some of you who are finding it difficult to code in vim :)

Can also check this cheatsheet which categorized those useful commands:
https://phoenixnap.com/kb/vim-commands-cheat-sheet#:~:text=Vim Commands Cheat Sheet 1 Moving Inside a,Enabling Vim Color Schemes. ... More items...

@Eclipse-Dominator nice it worked last minute 😆

thought this was useful


It is not EnityFramework. After FindAll, query already returned results.
Documentation: https://dapper.phnx47.net/
Example:
var limit = 10u;
var offset = 5u;
var users = await userRepository.SetLimit(limit, offset).FindAllAsync();

Some tests: https://github.com/microorm-dotnet/MicroOrm.Dapper.Repositories/blob/0435a3c2e2085cd003b0764264f42c553f4b2fe3/test/MicroOrm.Dapper.Repositories.Tests/SqlGeneratorTests/MsSqlGeneratorTests.cs#L428-L445


array unions

I ran into this thing in JS land the other day where I needed to join two arrays(search results) but not have any duplicates.  In Ruby I'd usually use union which has a nifty operator that acts like so:
edible = ["potato", "apple", "ramen"]
rations = ["apple", "cereal"]
edible |= rations

this returns ["potato", "apple", "ramen", "cereal"] without the extra apple.
In JS, I can add two arrays and edible.concat(rations) returns ["potato", "apple", "ramen", "apple", "cereal"] but I want to get rid of the duplicate of apple.  I didn't want to use jQuery as I was using a micro library already to avoid bloat and so I googled this. I'd like to take the credit for someone else's thought process but actually I have to give credit to stack overflow which ultimately gave me a solution.
The logic is as so:
var unify = edible.concat(rations.filter(function (item) { return edible.indexOf(item) < 0; }))

It makes use of filter to return only unique items. See here: http://jsfiddle.net/simo/98622/
I could make this into a easify function if you're interested. I would propose a function name of either union or unify for this.

This sounds great. This is the perfect example of the type thing Easify should be simplifying. Please do add this. It can be called both unify and unite. Just alias it.

Cool. Working on this presently.


Transaction containing self-payment not visible on the list of transactions

Transaction that contains payment with source and destination being the same account is not visible in /accounts/{address}/transactions.

https://horizon.stellar.org/accounts/GB6YPGW5JFMMP2QB2USQ33EUWTXVL4ZT5ITUNCY3YKVWOJPP57CANOF3/transactions
https://horizon.stellar.org/operations/3201756385251329

Confirmed.  This occurs since the horizon-importer does not consider the txfeehistory table for finding transaction participants.   An account that pays to itself is only effected in one way,  the tx fee, and so it only shows up in the metadata stored in txfeehistory.


vSphere 6.7 vm html console

I used this script here: https://github.com/vmware/pyvmomi-community-samples/blob/master/samples/generate_html5_console.py to generate link but it doesnt work, see below.
I get this link from script:
http://myvcenter:7331/console/?vmId=vm-429&vmName=myvm&host=myvcenter>&sessionTicket=cst-VCT-5287c10e-1edf-e6e3-abc5-b6020891467e--tp-BE-D6-93-F7-1B-87-AD-4F-6A-4B-A6-11-83-86-3F-EB-4D-0F-CB-7A&thumbprint=BE:D6:93:F7:1B:87:AD:4F:6A:4B:A6:11:83:86:3F:EB:4D:0F:CB:7A
The error I get on browser is: This site can’t be reached
It seems like port 7331 is not even open on vcenter server.
Could this be due to version 6.7 ? if yes, is there a way I can get html console for a VM?

Hi plrnr,
The below modified code is working , please try.
def get_guest_console(service_instance):
search_index = service_instance.content.searchIndex
serverGuid = service_instance.content.about.instanceUuid

vm = search_index.FindByUuid(None, "5016531e-0c73-064e-e9fe-bcee63cd6847", True, True)
content = service_instance.RetrieveContent()
vcenter_data = content.setting
vm_moid = vm._moId
vcenter_settings = vcenter_data.setting
console_port = '9443'

for item in vcenter_settings:
    key = getattr(item, 'key')
    if key == 'VirtualCenter.FQDN':
        vcenter_fqdn = getattr(item, 'value')

session_manager = content.sessionManager
session = session_manager.AcquireCloneTicket()
print(type(session))
vc_cert = ssl.get_server_certificate(("<IP_ADDRESS>", 443))
vc_pem = OpenSSL.crypto.load_certificate(OpenSSL.crypto.FILETYPE_PEM, vc_cert)
vc_fingerprint = vc_pem.digest('sha1')
print(type(vc_fingerprint))
print("console_op","https://" + "<IP_ADDRESS>" + ":" + console_port + "/vsphere-client/webconsole.html?vmId=" + str(vm_moid) + "&vmName=" + vm.name + "&serverGuid=" + serverGuid + "&host=" + vcenter_fqdn + ":443&sessionTicket=" + session + "&thumbprint=" + str(vc_fingerprint) + "&locale=en-US")

def main():
"""
Command-line program for listing the events on a system.
"""
args = cli.get_args()
context = ssl._create_unverified_context()
if args.disable_ssl_verification:
service_instance = SmartConnect(host=args.host,
user=args.user,
pwd=args.password,
port=int(args.port),
sslContext=context)
else:
service_instance = SmartConnect(host=args.host,
user=args.user,
pwd=args.password,
port=int(args.port))
atexit.register(Disconnect, service_instance)
get_guest_console(service_instance)

Start program
if name == "main":
main()
VMuuid and VC ip  not passing command line argument
Thanks,
Shweta

hi Shweta,
Thanks for that! Can you confirm your vCenter version please?

Ok, just tried your script and I do some progress. However,

It takes me to login page to vCenter.
After I login with my vcenter credentails, it does open the console but it says below:
The console has been disconnected. Close this window and re-launch the console to reconnect.

And it just gives me the same message even if I close/re-launch...
Please do let me know what vCenter version do you have?

Hi Plrnr,
Try constructing url as below,
print("console_op","https://" + "<IP_ADDRESS>" +  "/ui/webconsole.html?vmId=" + str(vm_moid) + "&vmName=" + vm.name + "&serverGuid=" + serverGuid + "&host=" + vcenter_fqdn + ":443&sessionTicket=" + session + "&thumbprint=" + str(vc_fingerprint) + "&locale=en-US")

Yes , you should login to vcenter
After constructing the url above, you can able to connect to VM web console
I tried with 6.7 and above  versions

Thanks,
Shweta

what about pre-authenticate url? Can I use the authentication session to forward a request to this generated url?

hi,
I still get the same message: The console has been disconnected. Close this window and re-launch the console to reconnect.
Does the session ticket we get has any expiry time?

Here is the link script generates:
https://myvcsa.local:9443/vsphere-client/webconsole.html?vmId=vm-429&vmName=newvm&serverGuid=35b995f7-4098-40e3-aa30-d6c5d1e86891&host=myvcsa.local:443&sessionTicket=cst-VCT-52b353cf-65dc-f293-0bfe-03f3ce59703c--tp-BE-D6-93-F7-1B-87-AD-4F-6A-4B-A6-11-83-86-3F-EB-4D-0F-CB-7A&thumbprint=BE:D6:93:F7:1B:87:AD:4F:6A:4B:A6:11:83:86:3F:EB:4D:0F:CB:7A&locale=en-US

Here is my code:
content = si.RetrieveContent()
serverGuid = content.about.instanceUuid
vm_moid = vmRef._moId
vcenter_data = content.setting
vcenter_settings = vcenter_data.setting
console_port = '9443'
serverIp = 'myvcsa.local'
httpsPort = 443
for item in vcenter_settings:
    key = getattr(item, 'key')
    if key == 'VirtualCenter.FQDN':
        vcenter_fqdn = getattr(item, 'value')
session_manager = content.sessionManager
sessionTicket = session_manager.AcquireCloneTicket()

ssl._create_default_https_context = ssl._create_unverified_context
vc_cert = ssl.get_server_certificate((serverIp, httpsPort))
vc_pem = OpenSSL.crypto.load_certificate(OpenSSL.crypto.FILETYPE_PEM,
                                         vc_cert)
vc_fingerprint = vc_pem.digest('sha1')
print("console_op", "https://" + serverIp + ":" + console_port + "/vsphere-client/webconsole.html?vmId=" + str(vm_moid) + "&vmName=" + vmName +
      "&serverGuid=" + serverGuid + "&host=" + vcenter_fqdn + ":443&sessionTicket=" + sessionTicket + "&thumbprint=" + str(vc_fingerprint) + "&locale=en-US")

I've been dealing with this same issue @plrnr. Works great in vCenter 6.5, but not vCenter 6.7.
I've noticed a couple of things that might be handy:

The end point /vsphere-client/webconsole.html gives me a login screen, but like you, results in a "console has been disconnected" error
The end point /ui/webconsole.html gives no login prompt. However, if I have vCenter opened in another tab (i.e. already have a valid session), the link works and I get the console.

@aspurohit you sure you didn't have an active session in a different tab while testing?

@willnx
Thanks for responding. Its a relief to know that it is something on 6.7 version. I am going to bring up a 6.5 environment and try.

Confirmed it works on 6.5!

Hi Plrnr/wilnx,

I tried with end point "/vsphere-client/webconsole.html" gives me login page , for 6.5,6.7 and latest
vcenter version. I am facing any error. I could login successfully.
And end point "/ui/webconsole.html" gives me no login prompt. I found this issue earlier also.

Thanks,
Shweta

do you get any solutions？   i  have beening blocked by this issue for months.
for vc6.0--6.5,  inputing username and password in the sso login page and click login then the vm console shows as expected, but vc6.7 ,after login,I get the message: The console has been disconnected. Close this window and re-launch the console to reconnect.

I never was able to figure out a solution on 6.7. I have not tried on the latest 7.0 version.

i checked it works on vsphere 7 with a change in url at the beginning, see below. It does not have a 9443 port and uses "ui" instead of vsphere-client
https://vcsa7-dev.atg.lc/ui/webconsole.html?


[Custom Components] Custom component is re-rendering on every form change

After upgrading Angular to 10, my custom formio component is now refreshing on every form change which is resulting in the component losing it's state and flashing while it loads the previous state from the outer form.
The component is being destroyed and re-inited on every keystroke.
I'll gladly provide any other information needed.

Investigating this more, it seems to work as long as I don't install a newer version of formiojs 4.11.3.

Hello @michealharrington, what version of angular-formio are you using?

The latest I could get to work was
angluar-formio = 4.9.6
formiojs = 4.11.3

Hello @michealharrington, Thanks for providing your version. Form.io offers Support Packages that are for training, support, or development services. The hours can be used in one-hour increments at your convenience over a 90 day period.  We will assign one of our senior developers to work directly with you in online sessions. I would be happy to provide you with our Support Packages Documentation if you are interested.
If you are interested in hearing more about our commercial offerings we would be happy to set up a short call to hear more about your project and see how we can help.

@wag110894 - I sent you a message (on LinkedIn) with my contact info.  I wanted to get more information on your training services.  Would you please get back to me with details when you get a chance?  Thanks!

We're currently addressing a backlog of GitHub issues, and as part of this effort, some inactive issues may be marked as closed. This isn't a dismissal, but a step toward more efficient tracking.
If you feel the issue is still relevant, please re-open and we'll ensure it gets the attention it deserves. Your understanding is appreciated as we work to enhance our open-source responsiveness.


Fix get upload params NPE

Description
This PR an NPE on getUploadTemplateParams for template upload
Fixes: #6071
Types of changes

[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] New feature (non-breaking change which adds functionality)
[x] Bug fix (non-breaking change which fixes an issue)
[ ] Enhancement (improves an existing feature and functionality)
[ ] Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity
Feature/Enhancement Scale

[ ] Major
[x] Minor

Bug Severity

[ ] BLOCKER
[ ] Critical
[ ] Major
[x] Minor
[ ] Trivial

Screenshots (if appropriate):
How Has This Been Tested?

@blueorangutan package

@nvazquez a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_multiplication_x: debian :heavy_multiplication_x: suse15. SL-JID 2815

@blueorangutan package

@nvazquez a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 2818

@blueorangutan test

@nvazquez a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

Trillian test result (tid-3549)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 24687 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr6079-t3549-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py

@blueorangutan test

@nvazquez a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests


Remotely sending commands

I'd like to be able to turn motion detection on and off "remotely," i.e. from other machines on the local area network.  Specifically, I'd like to be able to issue some URL to the Pi that would execute "md 1" and "md 0."  That way, my home security system could turn motion detection off when I disable monitoring in other parts of the system and turn it back on when I enable it.
One thought would be something like this:
http://pi_cam.local/command/md_0
would turn monitoring off.
Is this already built in? If not, how hard would it be to add it?  If you could point me to the right points in the code, I might try adding it myself.

Hi,
That is precisely what the "Motion Detection Start/Stop" button (on the main web page) does.  You should be able to connect to that server from a web browser on a different system elsewhere in your LAN.  So I don't think anything "new" needs to be added.
-- Fred P.

I suppose if you want some dedicated URLs for the tasks, then a simple php file which writes an "md 1" or "md 0" to file "/var/your-cam-path/FIFO" would do the job, or something that issued the bash command, e.g.,
echo "md 1" > /var/your-cam-path/FIFO

The current system uses cmd_pipe.php to issue commands to the pipe so just use a URL with this and md 0 or md 1 as a parameter

Thanks for the responses.  I know that the webui can be used to turn motion capture on and off, but I want to do with without the UI sometimes.  I want to be able to start my home monitoring software and have it directly toggle motion capture on multiple devices.
Both responses looked helpful.  Just to fill out roberttidy's respons, the specific url would be:
http://path.to.camera/html/cmd_pipe.php?cmd=md 0
To turn motion capture off and
http://path.to.camera/html/cmd_pipe.php?cmd=md 1
To turn it on.
Thanks!


issue about the paddlepaddle version and model weight

Thank u for this excellent work!
I tried to install the environment. After many many many failed attempts, I successfully installed and run RNAErnie with
paddlepaddle-gpu          2.5.1.post112            pypi_0    pypi
paddlenlp                 2.5.2                    pypi_0    pypi

which is different from the environment required by the repository.
when I run the test.py in this repository, I get a warning:
[2024-06-25 02:18:05,827] [ WARNING] - Some weights of ErnieModel were not initialized from the model checkpoint at output/BERT,ERNIE,MOTIF,PROMPT/checkpoint_final/ and are newly initialized: ['ernie.embeddings.task_type_embeddings.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.

this problem cause the inference output to be random because paddlepaddle will randomly initialize the weight of ['ernie.embeddings.task_type_embeddings.weight'] since it is not provided by the checkpoint file. I do not know whether this problem is caused by the different paddlenlp or paddlepaddle-gpu version, or u have a custom model file used by pretraining, or just other reasons.
Also, could u provide some result examples for the test.py (e.g., the embedding value for these 3 RNAs), for others to reproduce and confirm the success of installation？

This warning seems caused by version update of paddlepaddle. It should not have influence on model performance since test.py file generate only last layer hidden state. However, I strongly recommand that you use the version in environment.yaml or docker is the simplest way to get started, since paddle not very stable and compatible with older versions. And I will also provide the results of these RNAs ASAP.

This warning seems caused by version update of paddlepaddle. It should not have influence on model performance since test.py file generate only last layer hidden state. However, I strongly recommand that you use the version in environment.yaml or docker is the simplest way to get started, since paddle not very stable and compatible with older versions. And I will also provide the results of these RNAs ASAP.

I failed with environment.yaml, the docker also could not find the gpu. After failing with these two ways, I refered to install paddlepaddle by myself. Anyway, I will go to find other solutions. Thany u for your work.

After reading the source code of paddlenlp, I found this warning could be solved by specify use_task_id=False, regardless of the version of paddlenlp.

That will be great.
And If you failed with paddle-gpu/paddlenlp, and you are using CUDA11.2, you could try the command:
python -m pip install paddlepaddle-gpu==2.3.2.post112 -f https://www.paddlepaddle.org.cn/whl/windows/mkl/avx/stable.html
pip install paddlenlp==2.4.0 -i https://pypi.tuna.tsinghua.edu.cn/simple

It should work.
If you could not find gpu in docker, you could refer to nvidia-docker.
Moreover, the output of test.py should be:

embeddings_cls[:, 0, :20]:
[[-1.77774417,  0.00598652, -0.40756163,  1.48542976,  1.00323224,
-1.12849081, -0.59234452,  0.59155643,  1.02687085,  0.59921616,
-5.34262943, -1.67266476, -0.62758511,  0.09924193, -0.14699396,
-1.84140718, -1.54178190, -0.25275639, -2.94911885,  0.62381172],
[ 0.44317573, -1.52981794, -0.44530281, -0.74898922,  1.12226999,
0.35646021, -1.07076824,  0.54384816, -0.32401377,  0.92292696,
-2.39912391, -0.89087659, -2.03453350,  0.09272160,  0.14290528,
0.66565150,  0.54146516, -0.15687519, -2.35277724, -0.30207837],
[-0.76079893,  0.00487283, -0.85944021,  1.80532372,  0.26026809,
-1.22708309, -0.37417799,  0.68968868,  0.73462498,  0.41082427,
-4.79745197, -1.21026504,  0.31113577,  1.00250030, -0.63518941,
-1.27956653, -2.22530723, -0.24357563, -0.52435875,  1.58814859]]

embeddings_cls[:, 1, :20]:
[[-0.73240942, -0.39978394, -0.76209849, -1.28554451,  0.77304256,
-0.70314878, -1.09901369, -0.90953970,  0.85369569,  0.82643604,
-5.99799013,  0.45907605, -0.34646827, -1.24245358, -0.15599670,
-0.76430464, -0.39999729, -0.03397463, -0.59768838,  0.25285265],
[ 0.75230992, -1.93841732, -0.29124042, -0.98008269,  0.95582294,
0.79164785, -0.23181930, -0.54312563,  0.15678431,  0.11541642,
-4.02349234,  1.09067440, -0.94936466, -0.94698775, -0.64417297,
1.33173978,  1.19598091,  0.60152602, -1.71687996, -0.32354167],
[-1.37551093,  0.53867716,  1.14147651,  0.75911707,  0.60456169,
-0.39371085,  1.09126151,  0.40128496,  0.20201759,  1.76694763,
-1.90801167, -0.30953187, -0.14494842,  0.39712042, -0.91256326,
0.58106613,  0.35296690, -1.34865499, -1.05024314, -0.05232841]]

embeddings_cls[:, 2, :20]:
[[-1.01459241,  0.25639206, -0.52442497, -0.78820735,  0.02778245,
-1.77636433, -1.16884017, -0.69088089,  0.34502584,  0.60756481,
-3.80801296,  1.56891406, -0.84827286, -0.24529281, -0.97400403,
0.21175295, -0.40491876,  0.94469374,  0.15253644,  0.44595960],
[ 0.59894556, -1.16830647, -0.35044086, -1.02457511,  0.47519094,
-0.54397160, -0.82733262, -0.69585860,  0.50984484,  0.70106894,
-4.61783409,  0.89723241, -1.03709316, -1.36343861, -1.03756356,
0.82032251,  0.97853953,  0.10334171, -1.33009422, -0.10655852],
[-0.37394369, -0.39951679, -0.00646931, -0.11586973,  0.48873645,
-0.70540166, -0.06809238, -1.13572860,  1.18095231,  0.46109867,
-5.50498486,  0.67741048,  1.10808492,  0.81550378, -1.02222455,
-1.58559287,  0.10839003, -0.39578721, -0.48898828,  0.96963543]]

That will be great.
And If you failed with paddle-gpu/paddlenlp, and you are using CUDA11.2, you could try the command:
python -m pip install paddlepaddle-gpu==2.3.2.post112 -f https://www.paddlepaddle.org.cn/whl/windows/mkl/avx/stable.html
pip install paddlenlp==2.4.0 -i https://pypi.tuna.tsinghua.edu.cn/simple

It should work.
If you could not find gpu in docker, you could refer to nvidia-docker.
Moreover, the output of test.py should be:

embeddings_cls[:, 0, :20]:
[[-1.77774417,  0.00598652, -0.40756163,  1.48542976,  1.00323224,
-1.12849081, -0.59234452,  0.59155643,  1.02687085,  0.59921616,
-5.34262943, -1.67266476, -0.62758511,  0.09924193, -0.14699396,
-1.84140718, -1.54178190, -0.25275639, -2.94911885,  0.62381172],
[ 0.44317573, -1.52981794, -0.44530281, -0.74898922,  1.12226999,
0.35646021, -1.07076824,  0.54384816, -0.32401377,  0.92292696,
-2.39912391, -0.89087659, -2.03453350,  0.09272160,  0.14290528,
0.66565150,  0.54146516, -0.15687519, -2.35277724, -0.30207837],
[-0.76079893,  0.00487283, -0.85944021,  1.80532372,  0.26026809,
-1.22708309, -0.37417799,  0.68968868,  0.73462498,  0.41082427,
-4.79745197, -1.21026504,  0.31113577,  1.00250030, -0.63518941,
-1.27956653, -2.22530723, -0.24357563, -0.52435875,  1.58814859]]
embeddings_cls[:, 1, :20]:
[[-0.73240942, -0.39978394, -0.76209849, -1.28554451,  0.77304256,
-0.70314878, -1.09901369, -0.90953970,  0.85369569,  0.82643604,
-5.99799013,  0.45907605, -0.34646827, -1.24245358, -0.15599670,
-0.76430464, -0.39999729, -0.03397463, -0.59768838,  0.25285265],
[ 0.75230992, -1.93841732, -0.29124042, -0.98008269,  0.95582294,
0.79164785, -0.23181930, -0.54312563,  0.15678431,  0.11541642,
-4.02349234,  1.09067440, -0.94936466, -0.94698775, -0.64417297,
1.33173978,  1.19598091,  0.60152602, -1.71687996, -0.32354167],
[-1.37551093,  0.53867716,  1.14147651,  0.75911707,  0.60456169,
-0.39371085,  1.09126151,  0.40128496,  0.20201759,  1.76694763,
-1.90801167, -0.30953187, -0.14494842,  0.39712042, -0.91256326,
0.58106613,  0.35296690, -1.34865499, -1.05024314, -0.05232841]]
embeddings_cls[:, 2, :20]:
[[-1.01459241,  0.25639206, -0.52442497, -0.78820735,  0.02778245,
-1.77636433, -1.16884017, -0.69088089,  0.34502584,  0.60756481,
-3.80801296,  1.56891406, -0.84827286, -0.24529281, -0.97400403,
0.21175295, -0.40491876,  0.94469374,  0.15253644,  0.44595960],
[ 0.59894556, -1.16830647, -0.35044086, -1.02457511,  0.47519094,
-0.54397160, -0.82733262, -0.69585860,  0.50984484,  0.70106894,
-4.61783409,  0.89723241, -1.03709316, -1.36343861, -1.03756356,
0.82032251,  0.97853953,  0.10334171, -1.33009422, -0.10655852],
[-0.37394369, -0.39951679, -0.00646931, -0.11586973,  0.48873645,
-0.70540166, -0.06809238, -1.13572860,  1.18095231,  0.46109867,
-5.50498486,  0.67741048,  1.10808492,  0.81550378, -1.02222455,
-1.58559287,  0.10839003, -0.39578721, -0.48898828,  0.96963543]]

embedding_cls shape should be 31768. It should embeddings[:, 0, :20], or do I missed something?

That will be great.
And If you failed with paddle-gpu/paddlenlp, and you are using CUDA11.2, you could try the command:
python -m pip install paddlepaddle-gpu==2.3.2.post112 -f https://www.paddlepaddle.org.cn/whl/windows/mkl/avx/stable.html
pip install paddlenlp==2.4.0 -i https://pypi.tuna.tsinghua.edu.cn/simple

It should work.
If you could not find gpu in docker, you could refer to nvidia-docker.
Moreover, the output of test.py should be:

embeddings_cls[:, 0, :20]:
[[-1.77774417,  0.00598652, -0.40756163,  1.48542976,  1.00323224,
-1.12849081, -0.59234452,  0.59155643,  1.02687085,  0.59921616,
-5.34262943, -1.67266476, -0.62758511,  0.09924193, -0.14699396,
-1.84140718, -1.54178190, -0.25275639, -2.94911885,  0.62381172],
[ 0.44317573, -1.52981794, -0.44530281, -0.74898922,  1.12226999,
0.35646021, -1.07076824,  0.54384816, -0.32401377,  0.92292696,
-2.39912391, -0.89087659, -2.03453350,  0.09272160,  0.14290528,
0.66565150,  0.54146516, -0.15687519, -2.35277724, -0.30207837],
[-0.76079893,  0.00487283, -0.85944021,  1.80532372,  0.26026809,
-1.22708309, -0.37417799,  0.68968868,  0.73462498,  0.41082427,
-4.79745197, -1.21026504,  0.31113577,  1.00250030, -0.63518941,
-1.27956653, -2.22530723, -0.24357563, -0.52435875,  1.58814859]]
embeddings_cls[:, 1, :20]:
[[-0.73240942, -0.39978394, -0.76209849, -1.28554451,  0.77304256,
-0.70314878, -1.09901369, -0.90953970,  0.85369569,  0.82643604,
-5.99799013,  0.45907605, -0.34646827, -1.24245358, -0.15599670,
-0.76430464, -0.39999729, -0.03397463, -0.59768838,  0.25285265],
[ 0.75230992, -1.93841732, -0.29124042, -0.98008269,  0.95582294,
0.79164785, -0.23181930, -0.54312563,  0.15678431,  0.11541642,
-4.02349234,  1.09067440, -0.94936466, -0.94698775, -0.64417297,
1.33173978,  1.19598091,  0.60152602, -1.71687996, -0.32354167],
[-1.37551093,  0.53867716,  1.14147651,  0.75911707,  0.60456169,
-0.39371085,  1.09126151,  0.40128496,  0.20201759,  1.76694763,
-1.90801167, -0.30953187, -0.14494842,  0.39712042, -0.91256326,
0.58106613,  0.35296690, -1.34865499, -1.05024314, -0.05232841]]
embeddings_cls[:, 2, :20]:
[[-1.01459241,  0.25639206, -0.52442497, -0.78820735,  0.02778245,
-1.77636433, -1.16884017, -0.69088089,  0.34502584,  0.60756481,
-3.80801296,  1.56891406, -0.84827286, -0.24529281, -0.97400403,
0.21175295, -0.40491876,  0.94469374,  0.15253644,  0.44595960],
[ 0.59894556, -1.16830647, -0.35044086, -1.02457511,  0.47519094,
-0.54397160, -0.82733262, -0.69585860,  0.50984484,  0.70106894,
-4.61783409,  0.89723241, -1.03709316, -1.36343861, -1.03756356,
0.82032251,  0.97853953,  0.10334171, -1.33009422, -0.10655852],
[-0.37394369, -0.39951679, -0.00646931, -0.11586973,  0.48873645,
-0.70540166, -0.06809238, -1.13572860,  1.18095231,  0.46109867,
-5.50498486,  0.67741048,  1.10808492,  0.81550378, -1.02222455,
-1.58559287,  0.10839003, -0.39578721, -0.48898828,  0.96963543]]

embedding_cls shape should be (3,1,768). It should embeddings[:, 0, :20], or do I missed something?

Sorry, embeddings_cls is embeddings. Typo.

That will be great.
And If you failed with paddle-gpu/paddlenlp, and you are using CUDA11.2, you could try the command:
python -m pip install paddlepaddle-gpu==2.3.2.post112 -f https://www.paddlepaddle.org.cn/whl/windows/mkl/avx/stable.html
pip install paddlenlp==2.4.0 -i https://pypi.tuna.tsinghua.edu.cn/simple

It should work.
If you could not find gpu in docker, you could refer to nvidia-docker.
Moreover, the output of test.py should be:

embeddings_cls[:, 0, :20]:
[[-1.77774417,  0.00598652, -0.40756163,  1.48542976,  1.00323224,
-1.12849081, -0.59234452,  0.59155643,  1.02687085,  0.59921616,
-5.34262943, -1.67266476, -0.62758511,  0.09924193, -0.14699396,
-1.84140718, -1.54178190, -0.25275639, -2.94911885,  0.62381172],
[ 0.44317573, -1.52981794, -0.44530281, -0.74898922,  1.12226999,
0.35646021, -1.07076824,  0.54384816, -0.32401377,  0.92292696,
-2.39912391, -0.89087659, -2.03453350,  0.09272160,  0.14290528,
0.66565150,  0.54146516, -0.15687519, -2.35277724, -0.30207837],
[-0.76079893,  0.00487283, -0.85944021,  1.80532372,  0.26026809,
-1.22708309, -0.37417799,  0.68968868,  0.73462498,  0.41082427,
-4.79745197, -1.21026504,  0.31113577,  1.00250030, -0.63518941,
-1.27956653, -2.22530723, -0.24357563, -0.52435875,  1.58814859]]
embeddings_cls[:, 1, :20]:
[[-0.73240942, -0.39978394, -0.76209849, -1.28554451,  0.77304256,
-0.70314878, -1.09901369, -0.90953970,  0.85369569,  0.82643604,
-5.99799013,  0.45907605, -0.34646827, -1.24245358, -0.15599670,
-0.76430464, -0.39999729, -0.03397463, -0.59768838,  0.25285265],
[ 0.75230992, -1.93841732, -0.29124042, -0.98008269,  0.95582294,
0.79164785, -0.23181930, -0.54312563,  0.15678431,  0.11541642,
-4.02349234,  1.09067440, -0.94936466, -0.94698775, -0.64417297,
1.33173978,  1.19598091,  0.60152602, -1.71687996, -0.32354167],
[-1.37551093,  0.53867716,  1.14147651,  0.75911707,  0.60456169,
-0.39371085,  1.09126151,  0.40128496,  0.20201759,  1.76694763,
-1.90801167, -0.30953187, -0.14494842,  0.39712042, -0.91256326,
0.58106613,  0.35296690, -1.34865499, -1.05024314, -0.05232841]]
embeddings_cls[:, 2, :20]:
[[-1.01459241,  0.25639206, -0.52442497, -0.78820735,  0.02778245,
-1.77636433, -1.16884017, -0.69088089,  0.34502584,  0.60756481,
-3.80801296,  1.56891406, -0.84827286, -0.24529281, -0.97400403,
0.21175295, -0.40491876,  0.94469374,  0.15253644,  0.44595960],
[ 0.59894556, -1.16830647, -0.35044086, -1.02457511,  0.47519094,
-0.54397160, -0.82733262, -0.69585860,  0.50984484,  0.70106894,
-4.61783409,  0.89723241, -1.03709316, -1.36343861, -1.03756356,
0.82032251,  0.97853953,  0.10334171, -1.33009422, -0.10655852],
[-0.37394369, -0.39951679, -0.00646931, -0.11586973,  0.48873645,
-0.70540166, -0.06809238, -1.13572860,  1.18095231,  0.46109867,
-5.50498486,  0.67741048,  1.10808492,  0.81550378, -1.02222455,
-1.58559287,  0.10839003, -0.39578721, -0.48898828,  0.96963543]]

another question is that, it is a paddlepaddle whl for windows, not for linux.

Yes, of course. You could check paddle's official
https://www.paddlepaddle.org.cn/install/quick?docurl=/documentation/docs/zh/install/pip/windows-pip.html

in docker, the error information is below:
λ 354ec81262ed /home python test1.py
grep: warning: GREP_OPTIONS is deprecated; please use an alias or script
/usr/local/python3.7.0/lib/python3.7/site-packages/paddlenlp/transformers/image_utils.py:213: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
  resample=Image.BILINEAR,
/usr/local/python3.7.0/lib/python3.7/site-packages/paddlenlp/transformers/image_utils.py:379: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
  resample=Image.NEAREST,
/usr/local/python3.7.0/lib/python3.7/site-packages/paddlenlp/transformers/ernie_vil/feature_extraction.py:65: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
  resample=Image.BICUBIC,
/usr/local/python3.7.0/lib/python3.7/site-packages/paddlenlp/transformers/clip/feature_extraction.py:64: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
  resample=Image.BICUBIC,
W0625 06:48:42.434362   282 gpu_resources.cc:61] Please NOTE: device: 0, GPU Compute Capability: 8.0, Driver API Version: 12.2, Runtime API Version: 10.2
W0625 06:48:42.439152   282 gpu_resources.cc:91] device: 0, cuDNN Version: 7.6.
Traceback (most recent call last):
  File "test1.py", line 23, in <module>
    rna_ernie = ErnieModel.from_pretrained("output/BERT,ERNIE,MOTIF,PROMPT/checkpoint_final/", use_task_id=False)
  File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddlenlp/transformers/model_utils.py", line 340, in from_pretrained
    model = cls(*base_args, **base_kwargs)
  File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddlenlp/transformers/utils.py", line 159, in __impl__
    init_func(self, *args, **kwargs)
  File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddlenlp/transformers/ernie/modeling.py", line 846, in __init__
    task_id, use_task_id)
  File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddlenlp/transformers/ernie/modeling.py", line 62, in __init__
    weight_attr=weight_attr)
  File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle/nn/layer/common.py", line 1457, in __init__
    is_bias=False)
  File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 424, in create_parameter
    default_initializer)
  File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle/fluid/layer_helper_base.py", line 381, in create_parameter
    **attr._to_kwargs(with_initializer=True))
  File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle/fluid/framework.py", line 3572, in create_parameter
    initializer(param, self)
  File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle/fluid/initializer.py", line 474, in __call__
    'std', self._std_dev, 'seed', self._seed)
SystemError: (Fatal) Operator truncated_gaussian_random raises an thrust::system::system_error exception.
The exception content is
:parallel_for failed: cudaErrorNoKernelImageForDevice: no kernel image is available for execution on the device. (at /paddle/paddle/fluid/imperative/tracer.cc:307)

in docker, my error information is below:
λ 354ec81262ed /home python test1.py
grep: warning: GREP_OPTIONS is deprecated; please use an alias or script
/usr/local/python3.7.0/lib/python3.7/site-packages/paddlenlp/transformers/image_utils.py:213: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
  resample=Image.BILINEAR,
/usr/local/python3.7.0/lib/python3.7/site-packages/paddlenlp/transformers/image_utils.py:379: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
  resample=Image.NEAREST,
/usr/local/python3.7.0/lib/python3.7/site-packages/paddlenlp/transformers/ernie_vil/feature_extraction.py:65: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
  resample=Image.BICUBIC,
/usr/local/python3.7.0/lib/python3.7/site-packages/paddlenlp/transformers/clip/feature_extraction.py:64: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
  resample=Image.BICUBIC,
W0625 06:48:42.434362   282 gpu_resources.cc:61] Please NOTE: device: 0, GPU Compute Capability: 8.0, Driver API Version: 12.2, Runtime API Version: 10.2
W0625 06:48:42.439152   282 gpu_resources.cc:91] device: 0, cuDNN Version: 7.6.
Traceback (most recent call last):
  File "test1.py", line 23, in <module>
    rna_ernie = ErnieModel.from_pretrained("output/BERT,ERNIE,MOTIF,PROMPT/checkpoint_final/", use_task_id=False)
  File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddlenlp/transformers/model_utils.py", line 340, in from_pretrained
    model = cls(*base_args, **base_kwargs)
  File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddlenlp/transformers/utils.py", line 159, in __impl__
    init_func(self, *args, **kwargs)
  File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddlenlp/transformers/ernie/modeling.py", line 846, in __init__
    task_id, use_task_id)
  File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddlenlp/transformers/ernie/modeling.py", line 62, in __init__
    weight_attr=weight_attr)
  File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle/nn/layer/common.py", line 1457, in __init__
    is_bias=False)
  File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 424, in create_parameter
    default_initializer)
  File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle/fluid/layer_helper_base.py", line 381, in create_parameter
    **attr._to_kwargs(with_initializer=True))
  File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle/fluid/framework.py", line 3572, in create_parameter
    initializer(param, self)
  File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle/fluid/initializer.py", line 474, in __call__
    'std', self._std_dev, 'seed', self._seed)
SystemError: (Fatal) Operator truncated_gaussian_random raises an thrust::system::system_error exception.
The exception content is
:parallel_for failed: cudaErrorNoKernelImageForDevice: no kernel image is available for execution on the device. (at /paddle/paddle/fluid/imperative/tracer.cc:307)

the error seems to be correlated with incompatible between compiled cuda version in docker with my GPU architecture. just record here for other users.

Is the docker the original one?
This maybe caused by CUDA version:
https://aistudio.baidu.com/paddle/forum/topic/show/993141

That will be great.
And If you failed with paddle-gpu/paddlenlp, and you are using CUDA11.2, you could try the command:
python -m pip install paddlepaddle-gpu==2.3.2.post112 -f https://www.paddlepaddle.org.cn/whl/windows/mkl/avx/stable.html
pip install paddlenlp==2.4.0 -i https://pypi.tuna.tsinghua.edu.cn/simple

It should work.
If you could not find gpu in docker, you could refer to nvidia-docker.
Moreover, the output of test.py should be:

embeddings_cls[:, 0, :20]:
[[-1.77774417,  0.00598652, -0.40756163,  1.48542976,  1.00323224,
-1.12849081, -0.59234452,  0.59155643,  1.02687085,  0.59921616,
-5.34262943, -1.67266476, -0.62758511,  0.09924193, -0.14699396,
-1.84140718, -1.54178190, -0.25275639, -2.94911885,  0.62381172],
[ 0.44317573, -1.52981794, -0.44530281, -0.74898922,  1.12226999,
0.35646021, -1.07076824,  0.54384816, -0.32401377,  0.92292696,
-2.39912391, -0.89087659, -2.03453350,  0.09272160,  0.14290528,
0.66565150,  0.54146516, -0.15687519, -2.35277724, -0.30207837],
[-0.76079893,  0.00487283, -0.85944021,  1.80532372,  0.26026809,
-1.22708309, -0.37417799,  0.68968868,  0.73462498,  0.41082427,
-4.79745197, -1.21026504,  0.31113577,  1.00250030, -0.63518941,
-1.27956653, -2.22530723, -0.24357563, -0.52435875,  1.58814859]]
embeddings_cls[:, 1, :20]:
[[-0.73240942, -0.39978394, -0.76209849, -1.28554451,  0.77304256,
-0.70314878, -1.09901369, -0.90953970,  0.85369569,  0.82643604,
-5.99799013,  0.45907605, -0.34646827, -1.24245358, -0.15599670,
-0.76430464, -0.39999729, -0.03397463, -0.59768838,  0.25285265],
[ 0.75230992, -1.93841732, -0.29124042, -0.98008269,  0.95582294,
0.79164785, -0.23181930, -0.54312563,  0.15678431,  0.11541642,
-4.02349234,  1.09067440, -0.94936466, -0.94698775, -0.64417297,
1.33173978,  1.19598091,  0.60152602, -1.71687996, -0.32354167],
[-1.37551093,  0.53867716,  1.14147651,  0.75911707,  0.60456169,
-0.39371085,  1.09126151,  0.40128496,  0.20201759,  1.76694763,
-1.90801167, -0.30953187, -0.14494842,  0.39712042, -0.91256326,
0.58106613,  0.35296690, -1.34865499, -1.05024314, -0.05232841]]
embeddings_cls[:, 2, :20]:
[[-1.01459241,  0.25639206, -0.52442497, -0.78820735,  0.02778245,
-1.77636433, -1.16884017, -0.69088089,  0.34502584,  0.60756481,
-3.80801296,  1.56891406, -0.84827286, -0.24529281, -0.97400403,
0.21175295, -0.40491876,  0.94469374,  0.15253644,  0.44595960],
[ 0.59894556, -1.16830647, -0.35044086, -1.02457511,  0.47519094,
-0.54397160, -0.82733262, -0.69585860,  0.50984484,  0.70106894,
-4.61783409,  0.89723241, -1.03709316, -1.36343861, -1.03756356,
0.82032251,  0.97853953,  0.10334171, -1.33009422, -0.10655852],
[-0.37394369, -0.39951679, -0.00646931, -0.11586973,  0.48873645,
-0.70540166, -0.06809238, -1.13572860,  1.18095231,  0.46109867,
-5.50498486,  0.67741048,  1.10808492,  0.81550378, -1.02222455,
-1.58559287,  0.10839003, -0.39578721, -0.48898828,  0.96963543]]

Remarkable work. But I failed to reproduce the embeddings.
I used the Docker image of registry.baidubce.com/paddlepaddle/paddle:2.3.2-gpu-cuda11.2-cudnn8
then pip3 install biopython==1.78 paddlenlp==2.4.0 and run test.py, yet print(embeddings[:, 0, :20]) gives
Tensor(shape=[3, 20], dtype=float32, place=Place(gpu:0), stop_gradient=True,
       [[-1.67208195, -0.53518271, -0.49305263,  1.33484435,  0.93666548,
         -0.81713992, -0.98522162,  1.31676650,  0.53529423,  1.34836614,
         -2.33305836, -0.76671398, -0.47538409, -0.07811834, -0.01321072,
         -1.32791495, -1.22602844, -0.02752305, -2.00940180,  0.20836182],
        [ 0.51898336, -1.22038937, -0.85066754, -1.52177238,  0.74062485,
          0.03760993, -0.47369927,  0.26919988,  0.02379241,  0.58055133,
         -3.33222580, -0.97376209, -2.70073462, -0.17489678, -0.12052849,
          0.13721487,  0.93731660,  0.02093063, -1.90570378, -0.35856661],
        [-0.14025885, -0.42401713, -0.93420225,  1.22201741, -0.07398830,
         -0.92484510,  0.18494038,  0.61836243,  0.42724448,  1.00634503,
         -3.73101974, -1.25191975, -0.26088372,  0.91419125, -0.12234388,
         -1.31755149, -0.79623038, -0.37579566, -0.39976898,  1.63649654]])

Good! Now the embeddings seem much closer.

Considering most of the researchers will prefer to use transformers and pytorch as backend. So, I transfer my work to transformers and train a pytorch model from scratch. The new model is trained with more powerful settings: The max model length is up to 2048 now and the pretraining dataset is the newest version of rnacentral, which contains about 31 million RNA sequences after length filtering (<2048). This pytorch version model has been uploaded to huggingface at https://huggingface.co/WANGNingroci/RNAErnie and the training framework/tokenization is located at https://github.com/CatIIIIIIII/RNAErnie2. (NOTE: the tokenization is a little different from the original paddle implementation). Moreover, Multimolecule are implementing current most powerful RNA language model with transformers and pytorch. Our model also could be accessed at https://huggingface.co/multimolecule/rnaernie.


Investigate codegen performance for large apps with many packages

This refers to the performance of the vite plugin during development / optimized build.
How well does it scale when a lot of packages (potentially large) are referenced from the app?
Developer experience and build time should not suffer too much

We did not encounter any performance problems so far. Closing this for the time being.


Error on start

When trying to run a PE32, sandbox matching does not have a proper failure execution path.
main                 INFO     > Processing: _testing/5DA8C98136D98DFEC4716EDD79C7145F
main                 INFO     > Args: None
main                 INFO     > Timeout: 30
Report               INFO     > Report Dir: ./reports/1641057176-58189cbd4e6dc0c7d8e66b6a6f75652fc9f4afc7ce0eba7d67d8c3feb0d5381f
main                 INFO     > CPU: x64
main                 INFO     > FileType: PE32+ executable (GUI) x86-64, for MS Windows
Sandbox              INFO     > Starting...
Traceback (most recent call last):
  File "detux.py", line 65, in <module>
    result = sandbox.run(samplerun, report)
  File "./DetuxNG/core/sandbox.py", line 182, in run
    target_env.name,
AttributeError: 'NoneType' object has no attribute 'name'

Fixed in previous commit


perf: Improve Survey Results Memory Usage

Pull Request Key Information
Reason for change
During the memory profiling, the results of surveys were found to be taking up over 70% of the memory allocated during simulation.
What was changed
This change reduces the memory usage of survey results, resulting an approximately 97% decrease in memory allocated during simulation in the case being tested.
Intended Purpose
Lower memory usage during simulation.
Level of version change required
Patch
Testing Completed
Manual testing with memory profiling - See Sample results here:
memory_profiling_results.zip
All unit tests passing:
unit_test_results.txt
All End-to-End tests passing:
V4_simple_non_repairable_emissions_df7cdf012ba7c4aef1dbd54e02f1f18bd9b0bfa3.log
V4-simple_repairable_emissions_df7cdf012ba7c4aef1dbd54e02f1f18bd9b0bfa3.log
Target Issue
N/A
Additional Information
N/A

New unit test results:
unit_test_results.txt
New End-to-End test results:
V4_simple_non_repairable_emissions_c67e910bf3e029273da17ce6bfa5a0ae997f9839.log
V4-simple_repairable_emissions_c67e910bf3e029273da17ce6bfa5a0ae997f9839.log


Edit pass on grpc e-book content

Book content would benefit from an editorial review.
Here's the entry point for the book:
https://docs.microsoft.com/en-us/dotnet/architecture/grpc-for-wcf-developers/index

@mairaw Has the edit pass for the entire book been completed? Should this be closed?

closing as all related PRs have been merged.


[Preact X] Not sync with existing dom nodes

https://codesandbox.io/s/p5op59yp7m
When click the 'Increment' button many times, the old dom nodes still exists.

Thanks for the great bug report and the sandbox :tada: That looks indeed nasty! I can confirm the issue with provided sandbox and it's a great test case we should include in our test suite :+1:
Wrapping <DataViewer /> with a simple div works around that problem, so it seems like we seem to mess up child ordering in diffChildren().
Probably related #1343 , #1323 , #1326

I'm also seeing this for our app, using PreactRouter.

I think I ran into this as well.

Hello all! I'm not sure if this is actually related, but this issue is the closest to my case. Nothing in the changelog is related to this, am I missing something here?
Isn't this supposed to render <Footer/> after the <Router> component? Awkwardly this doesn't work in codesandbox.io (https://codesandbox.io/s/6z2zzw5r73) so there's a repo linked bellow.
import { h, Component } from 'preact';
import { Router } from 'preact-router';

import Footer from './footer';
import Header from './header';

// Code-splitting is automated for routes
import Home from '../routes/home';

export default class App extends Component {

  /** Gets fired when the route changes.
   *	@param {Object} event		"change" event from [preact-router](http://git.io/preact-router)
   *	@param {string} event.url	The newly routed URL
   */
  handleRoute = e => {
    this.currentUrl = e.url;
  };

  render() {
    return (
      <div id="app">
        <Header />
        <Router onChange={this.handleRoute}>
          <Home path="/" />
        </Router>
        <Footer />
      </div>
    );
  }
}

Versions:

preact: ^10.0.0-alpha.2
preact-router: ^3.0.0
node: v10.15.3
yarn: 1.15.2 / npm 6.4.1

Expected:
HEADER
Home
FOOTER

Result
HEADER
FOOTER
Home

Example: https://github.com/douggr/preact-10-dom-bug

Yes guys, I'm seeing this as well.
I have items rendered outside the root DOM node of the component where they are rendered.
Unable to reproduce it right-now, I'll try as soon as I can.

@marvinhagemeister @developit right after I wrote my last comment I spotted the cause of the bug afflicting my app: using the key prop outside a loop cause unpredictable behaviour (preact-x)!
I'll open a new issue as soon as I can make the codesandbox,.

@weijarz Good news: I can't reproduce this issue in master anymore :tada: :100:


Plugin must be installed on both sides to function.

Lets get this clear, I have no idea of how to code anything, but i have messed around with this plugin. And I've noticed that when I try to edit an item I'm holding on any of my friend's severs, it doesn't work.
Is there a way this could operate on only the client side? If not or if that would require remaking everything then I'm totally fine.
just curious.

it's a mod, not a plugin

The only feature that would work on a non-modded server is updating an item that is in your inventory if you're in creative mode. Doing anything else would require tinkering with commands / command blocks (which would also require you to be an operator on the server), and it is quite a lot of work to do, but it's something that I have in mind.


Z Wave LED component (Fibaro Controller) does not allow RGB control since .42

Make sure you are running the latest version of Home Assistant before reporting an issue.
You should only file an issue if you found a bug. Feature and enhancement requests should go in the Feature Requests section of our community forum:
Home Assistant release (hass --version):
.42+
Component/platform:
Fibaro Z Wave LED Controller
Description of problem:
So I think this issues started with .42, but Im not entirely sure.
I have a fibaro z wave LED controller that I have been turning on with the following below.
Since .42, if I add the RGB paramater, the light wont turn on. I just updated to .43 in the hopes the z wave fix would fix this issue but it doesnt seem to.
Looking at the entity page, I also dont see a lot of the attributes that used to be there (such as nothing talking about brightness or RGB). Here is what the entity now looks like.
When the controller is on the front end, it also no longer gives the color picker option.
Expected:
Color Picker to be present on front end. LED should turn on with designated RGB color when turned on by scene.
Problem-relevant configuration.yaml entries and steps to reproduce:
light.fibaro_system_fgrgbwm441_rgbw_controller_level_18:
      state: on
      brightness: 10
      rgb_color: [255, 0, 0]

Traceback (if applicable):
No errors.
Additional info:

It sounds similar to this problem: https://community.home-assistant.io/t/z-wave-rgb-broken-after-last-update/15777/9
Are you sure there's no errors in your log?

I can confirm this behavior.
For me it throws following error in the Logs:
2017-05-13 13:16:32 ERROR (MainThread) [homeassistant.core] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/homeassistant/core.py", line 1015, in _event_to_service_call
yield from service_handler.func(service_call)
File "/usr/local/lib/python3.6/site-packages/homeassistant/components/light/init.py", line 248, in async_handle_light_service
yield from light.async_turn_on(**params)
File "/usr/local/lib/python3.6/concurrent/futures/thread.py", line 55, in run
result = self.fn(*self.args, **self.kwargs)
File "/usr/local/lib/python3.6/site-packages/homeassistant/components/light/zwave.py", line 321, in turn_on
self._color_channels & COLOR_CHANNEL_WARM_WHITE or
TypeError: unsupported operand type(s) for &: 'NoneType' and 'int'

@armills
Didn't initally connect these errors but after seeing @pascalmattes, I do have similar errors in my log:
  File "/srv/homeassistant/lib/python3.4/site-packages/homeassistant/components/light/__init__.py", line 246, in async_handle_light_service
    yield from light.async_turn_on(**params)
  File "/usr/lib/python3.4/asyncio/futures.py", line 388, in __iter__
    yield self  # This tells Task to wait for completion.
  File "/usr/lib/python3.4/asyncio/tasks.py", line 286, in _wakeup
    value = future.result()
  File "/usr/lib/python3.4/asyncio/futures.py", line 277, in result
    raise self._exception
  File "/usr/lib/python3.4/concurrent/futures/thread.py", line 54, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/srv/homeassistant/lib/python3.4/site-packages/homeassistant/components/light/zwave.py", line 334, in turn_on
    self.values.color.data = rgbw
  File "/srv/homeassistant/lib/python3.4/site-packages/openzwave-0.3.2-py3.4.egg/openzwave/value.py", line 292, in data
    self._network.manager.setValue(self.value_id, value)
  File "src-lib/libopenzwave/libopenzwave.pyx", line 2827, in libopenzwave.PyManager.setValue (src-lib/libopenzwave/libopenzwave.cpp:21766)
TypeError: expected bytes, str found
17-05-11 21:35:37 WARNING (Thread-5) [pyecobee] Error connecting to Ecobee while attempting to get thermostat data.  Refreshing tokens and trying again.
17-05-11 22:00:00 ERROR (MainThread) [homeassistant.core] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/usr/lib/python3.4/asyncio/tasks.py", line 233, in _step
    result = coro.throw(exc)
  File "/srv/homeassistant/lib/python3.4/site-packages/homeassistant/core.py", line 1015, in _event_to_service_call
    yield from service_handler.func(service_call)
  File "/srv/homeassistant/lib/python3.4/site-packages/homeassistant/components/light/__init__.py", line 246, in async_handle_light_service
    yield from light.async_turn_on(**params)
  File "/usr/lib/python3.4/asyncio/futures.py", line 388, in __iter__
    yield self  # This tells Task to wait for completion.
  File "/usr/lib/python3.4/asyncio/tasks.py", line 286, in _wakeup
    value = future.result()
  File "/usr/lib/python3.4/asyncio/futures.py", line 277, in result
    raise self._exception
  File "/usr/lib/python3.4/concurrent/futures/thread.py", line 54, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/srv/homeassistant/lib/python3.4/site-packages/homeassistant/components/light/zwave.py", line 334, in turn_on
    self.values.color.data = rgbw
  File "/srv/homeassistant/lib/python3.4/site-packages/openzwave-0.3.2-py3.4.egg/openzwave/value.py", line 292, in data
    self._network.manager.setValue(self.value_id, value)
  File "src-lib/libopenzwave/libopenzwave.pyx", line 2827, in libopenzwave.PyManager.setValue (src-lib/libopenzwave/libopenzwave.cpp:21766)
TypeError: expected bytes, str found

@pizzapants To fix this, you can update your openzwave, but for me this isn't solving the problem.

@pascalmattes youve updated ozw and you are still having the same issue?

@pizzapants I have an other error as you can see in my log. It is not related to "expected bytes, str found".
Don't know what exactly is causing this problem. I am on hass.io 0.25

@pascalmattes Sorry, just to be clear are you still having the same issue with not being able to adjust color after updating or did that fix it?
What is the proper way to update OZW on a hassbian install? I was having trouble figuring that out the other day.

@pizzapants The error you're seeing is the same as the thread I linked to. The details are there, but the fix you'll need is to upgrade python-openzwave to version 0.3.3.
@pascalmattes This is an unrelated error, but it's a new problem we haven't seen before. Can you paste the section of your zwcfg_***.xml file for this node? It seems that the color channels values aren't being loaded.

@armills Sure, no Problem.
There you go: https://pastebin.com/pLt2QvAq
I also can't use the Colorpicker for this Node. The same Issue in Log appears then.
Btw.: I'm on a fresh new install of Hass.IO. Maybe it is related to that.

@armills Thanks, can you direct me to the proper steps to update ozw with hassbian? Is it just running these commands again?
(homeassistant)$ mkdir /srv/homeassistant/src
(homeassistant)$ cd /srv/homeassistant/src
(homeassistant)$ git clone https://github.com/OpenZWave/python-openzwave.git
(homeassistant)$ cd python-openzwave
(homeassistant)$ PYTHON_EXEC=`which python3` make build
(homeassistant)$ PYTHON_EXEC=`which python3` make install

@pascalmattes It's possible, but I would think that if your version of OZW is picking up the color channels value (Line 113), hass should have no problem reading it. Can you add some debugging outputs to your hass install? Specifically in this file, I'm guessing that one of these two returns is triggering, so the color is never read., although it could be that self.values.color_channels.data is returning None in the line below. If we can determine where it's failing we can start to trace down the root cause.
@pizzapants If you've already performed those steps once, you can probably just use the following commands. If you need any more help though, you'll be better off asking in the chatroom or forums. I don't use hassbian myself so I'm not 100% sure.
(homeassistant)$ cd /srv/homeassistant/src/python-openzwave
(homeassistant)$ git pull
(homeassistant)$ PYTHON_EXEC=`which python3` make build
(homeassistant)$ PYTHON_EXEC=`which python3` make install

@armills  ok, just did that. How do I verify the version installed?

Figured it out sort of, to this thread. Had to fully uninstall and reinstall with the following commands:
sudo su -s /bin/bash homeassistant
source /srv/homeassistant/bin/activate
cd /srv/homeassistant/src/python-openzwave
make uninstall
make clean
make update
make build
make install

Now Im seeing the color picker in the front end, but it doesnt actually work (the color doesnt change), and activating via scene still doesnt turn it on.
Although now that Im going through my error logs, Im finding this error again so maybe OZW isnt updated?
Traceback (most recent call last):
  File "/usr/lib/python3.4/asyncio/tasks.py", line 233, in _step
    result = coro.throw(exc)
  File "/srv/homeassistant/lib/python3.4/site-packages/homeassistant/core.py", line 1015, in _event_to_service_call
    yield from service_handler.func(service_call)
  File "/srv/homeassistant/lib/python3.4/site-packages/homeassistant/components/light/__init__.py", line 246, in async_handle_light_service
    yield from light.async_turn_on(**params)
  File "/usr/lib/python3.4/asyncio/futures.py", line 388, in __iter__
    yield self  # This tells Task to wait for completion.
  File "/usr/lib/python3.4/asyncio/tasks.py", line 286, in _wakeup
    value = future.result()
  File "/usr/lib/python3.4/asyncio/futures.py", line 277, in result
    raise self._exception
  File "/usr/lib/python3.4/concurrent/futures/thread.py", line 54, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/srv/homeassistant/lib/python3.4/site-packages/homeassistant/components/light/zwave.py", line 334, in turn_on
    self.values.color.data = rgbw
  File "/srv/homeassistant/lib/python3.4/site-packages/openzwave-0.3.2-py3.4.egg/openzwave/value.py", line 292, in data
    self._network.manager.setValue(self.value_id, value)
  File "src-lib/libopenzwave/libopenzwave.pyx", line 2827, in libopenzwave.PyManager.setValue (src-lib/libopenzwave/libopenzwave.cpp:21766)
TypeError: expected bytes, str found

Well that was a giant PITA. Had to completely uninstall OZW but now Im on the newest version and everything seems to be working properly.

@armills So I added the logger component, but I don't think it is helping us...
Error Log:
2017-05-14 11:21:38 DEBUG (MainThread) [homeassistant.components.websocket_api] WS<PHONE_NUMBER>: Received {'type': 'call_service', 'domain': 'light', 'service': 'turn_on', 'service_data': {'entity_id': 'light.fibaro_system_fgrgbwm441_rgbw_controller_level_8_0_2', 'rgb_color': [0, 0, 0], 'white_value': '255', 'brightness': '120'}, 'id': 10} 2017-05-14 11:21:38 INFO (MainThread) [homeassistant.core] Bus:Handling <Event call_service[L]: domain=light, service=turn_on, service_data=entity_id=light.fibaro_system_fgrgbwm441_rgbw_controller_level_8_0_2, rgb_color=[0, 0, 0], white_value=255, brightness=120, service_call_id=1962064848-3> 2017-05-14 11:21:38 ERROR (MainThread) [homeassistant.core] Error doing job: Task exception was never retrieved Traceback (most recent call last):   File "/usr/local/lib/python3.6/site-packages/homeassistant/core.py", line 1015, in _event_to_service_call     yield from service_handler.func(service_call)   File "/usr/local/lib/python3.6/site-packages/homeassistant/components/light/__init__.py", line 248, in async_handle_light_service     yield from light.async_turn_on(**params)   File "/usr/local/lib/python3.6/concurrent/futures/thread.py", line 55, in run     result = self.fn(*self.args, **self.kwargs)   File "/usr/local/lib/python3.6/site-packages/homeassistant/components/light/zwave.py", line 321, in turn_on     self._color_channels & COLOR_CHANNEL_WARM_WHITE or TypeError: unsupported operand type(s) for &: 'NoneType' and 'int'

I reinstalled Hass.IO but the bug is still there.

@pascalmattes
OK, Can you try downloading this file into your config directory, as <config_dir>/custom_components/zwave/__init__.py? You'll need to make sure logger is configured so homeassistant.components.zwave is set to debug level as described here. Next time you start hass it should produce a lot of output when zwave is starting up. Can you hastebin the output?

@armills
Sorry for the late answer.
I just did the steps you asked for.
Here is the output:
https://paste.ee/p/Il5eQ (was to big for pastebin)

OK, sorry for the confusion there. I rebased it on 0.44, and just to be safe adjusted the imports. Tested it on my machine so it should be working now. You'll have to update this file, and also download services.yaml into the custom_components/zwave folder because of how the code is written. It should work then.

After updating to 0.45 the error is gone. Don't know why but now it is working! Thanks anyway @armills

Glad to hear it! It was pretty strange problem.


How can we help?
I would like someone else to build the manifest.
Please read and ensure the following

[X] The installer meets the above requirements

Please provide the following information
Download Page Url: https://librepcb.org/download/
Publisher: LibrePCB
Package Name: LibrePCB
Description: LibrePCB is a free, cross-platform, easy-to-use electronic design automation suite to draw schematics and design printed circuit boards – for makers, students and professionals, from beginners to experts.
Package Version: 1.0.0
Installer URL: https://download.librepcb.org/releases/1.0.0/librepcb-installer-1.0.0-windows-x86.exe

Is this package support silent installation? wingetcreate.exe cannot inspect it from the installer wizard.

Available options
Usage: librepcb-installer-1.0.0-windows-x86.exe [options] Key=Value

Options:
  -?, -h, --help                           Displays this help.
  --version                                Displays version information.
  --framework-version                      Displays the version of the Qt
                                           Installer Framework.
  -v, --verbose                            Verbose mode. Prints out more
                                           information.
  --proxy                                  Use system proxy on Windows and
                                           Linux. This option has no effect on
                                           macOS.
  --no-proxy                               Do not use system proxy.
  --script <file>                          Execute the script given as
                                           argument.
  --checkupdates                           Check for updates and return an XML
                                           description.
  --updater                                Start application in updater mode.
  --manage-packages                        Start application in package manager
                                           mode.
  --no-force-installations                 Allow deselecting components that
                                           are marked as forced.
  --no-size-checking                       Disable checking of free space for
                                           installation target.
  --show-virtual-components                Show virtual components in installer
                                           and package manager.
  --logging-rules <rules>                  Enables logging according to passed
                                           rules. Comma separated logging rules
                                           have the following syntax:
                                           loggingCategory=true/false. Passing
                                           empty logging rules enables all
                                           logging categories. The following
                                           rules enable a single category:
                                           ifw.*=false,ifw.category=true The
                                           following logging categories are
                                           available:
                                           ifw.componentChecker
                                           ifw.resources
                                           ifw.translations
                                           ifw.network
  --create-local-repository                Create a local repository inside the
                                           installation directory. This option
                                           has no effect on online installers.
  --addRepository <URI,...>                Add a local or remote repository to
                                           the list of user defined
                                           repositories.
  --addTempRepository <URI,...>            Add a local or remote repository to
                                           the list of temporary available
                                           repositories.
  --setTempRepository <URI,...>            Set a local or remote repository as
                                           temporary repository, it is the only
                                           one used during fetch.
                                           Note: URI must be prefixed with the
                                           protocol, i.e. file:///, https://,
                                           http:// or ftp://.
  --startserver <mode,socketname,key>      Starts the application as headless
                                           process waiting for commands to
                                           execute. Mode can be DEBUG or
                                           PRODUCTION. In DEBUG mode, the option
                                           values can be omitted.Note: The
                                           server will not shutdown on his own,
                                           you need to quit the process by hand.
  --startclient <socketname,key>           Starts the application to debug the
                                           client-server communication. If a
                                           value is omitted, the client will use
                                           a default instead. Note: The server
                                           process is not started by the client
                                           application in that case, you need to
                                           start it on your own.
  --installCompressedRepository <URI,...>  Installs QBSP or 7z file. The QBSP
                                           (Board Support Package) file must be
                                           a .7z file which contains a valid
                                           repository.
  --silentUpdate                           Updates all packages silently.
  --platform <plugin>                      Use the specified platform plugin.
  --squish-port <port number>              Give a port where Squish can connect
                                           to. If no port is given, default port
                                           11233 is used. Note: To enable Squish
                                           support you first need to build IFW
                                           with SQUISH_PATH parameter where
                                           SQUISH_PATH is pointing to your
                                           Squish installation folder:
                                           <path_to_qt>/bin/qmake -r
                                           SQUISH_PATH=<pat_to_squish>

Arguments:
  Key=Value                                Key Value pair to be set.

--silentUpdate                           Updates all packages silently.

I think this issue is blocked because the installer wizard of LibrePCB is created by Qt Framework and does not support silent installation.
Maintainers of LibrePCB are looking for another installer wizard packaging technology (see LibrePCB/LibrePCB#1256). If they successfully switched the technology, then it might be able to support by winget.exe. I'll be glad to submit the manifest for it.

I would very much like to install LibrePCB using WinGet. Is it possible yet?


Duplicate updater.sh

Merging this project via any of the provided methods (emerge or pip) installs a duplicate updater.sh into the PATH. As far as I can tell, gentoo-update should be the only public facing script.
Please fix the build system if this is indeed a bug.

@antecrescent Thank you for noticing this (and improving the ebuild 🫶)
It was designed like that as a temporary workaround, I will fix it in the next version.

Great, thanks!

version 0.1.4 is in GURU repo now and is available for download


Index navigation page for reverse proxy

It would be nice if our reverse proxy setup would come with an index page out of the box for navigation purposes and to link to from deployments to have some kind of starting point to access services.

Fixed


Improve lifecycle

Hello,
I noticed that nodes, publishers, services and clients expose a void Dispose() method which is used for disposing resources while also having methods like bool RemovePublisher(IPublisherBase) which should be called on other objects to remove them and as implemented in #37 call Dispose().
While users should call this remove methods it is possible that they just call Dispose() (for example when using using ) which can lead to disposed objects being kept around until the object containing their remove method is disposed.
Furthermore, most disposable objects call Dispose() in their finalizers to account for users forgetting to call it.
Since accessing other managed objects in finalizers is tricky it should not be done, which means that for example accessing the collection of publishers associated with a Node  during finalization is risky (the collection and its managed internals having no finalizers is an implementation detail and should not be relied on as far as I know).
Possible Solution
The remove methods should be removed and the Dispose() method of the objects to be removed should handle all cleanup, which allows users to use using and prevents a node for example from being collected by the GC while a publisher exists.
Furthermore, the Ros2cs class should be replaced by an non-static class to prevent confusion which code is responsible for calling Ros2cs.Shutdown() and allows implementing IExtendedDisposable.
The problem of accessing managed objects during finalization can be solved by allowing some resources to leak if Dispose() was not called, which may be acceptable since node and context finalization typically happen on application shutdown.
Resources leaked include context and node handles, since we can not assure that associated resources are disposed (the context can however be shut down).
Handles of resources like publishers can be finalized during finalization.
Unity integration can be accomplished by calling Dispose() in ROS2UnityComponent but finalizing ROS2UnityCore and ROS2Node by GC would be difficult, which could be solved by removing them.
This is a class diagram containing the new methods used for lifecycle management (Publisher as an example of the other resources):

I could implement these changes if you agree with my ideas.

@Deric-W these seem to be beneficial changes. So far we were only interested in disposing of subscribers and publishers during runtime, not the nodes themselves. Could you submit a PR?