Misaligned stack access offset of a tuple (i32, i64)

bpftrace --info
System
  OS: Linux 5.7.0+ #2 SMP Mon Jun 1 12:50:01 JST 2020
  Arch: x86_64

Build
  version: v0.11.0-113-g3a46
  LLVM: 10
  foreach_sym: yes
  unsafe uprobe: no
  bfd: yes
  bpf_attach_kfunc: yes
  bcc_usdt_addsem: no
  libbpf: yes
  libbpf btf dump: yes
  libbpf btf dump type decl: yes

Kernel helpers
  probe_read: yes
  probe_read_str: yes
  probe_read_user: yes
  probe_read_user_str: yes
  probe_read_kernel: yes
  probe_read_kernel_str: yes
  get_current_cgroup_id: yes
  send_signal: yes
  override_return: yes
  get_boot_ns: no

Kernel features
  Instruction limit: 1000000
  Loop support: yes
  btf (depends on Build:libbpf): yes

Map types
  hash: yes
  percpu hash: yes
  array: yes
  percpu array: yes
  stack_trace: yes
  perf_event_array: yes

Probe types
  kprobe: yes
  tracepoint: yes
  perf_event: yes
  kfunc: yes

What reproduces the bug?
% sudo ./src/bpftrace -e 'BEGIN { @ = ((int32)1, 0);}' -v
BTF: using data from /sys/kernel/btf/vmlinux
Attaching 1 probe...

Error log:
0: (b7) r1 = 1
1: (63) *(u32 *)(r10 -24) = r1
2: (b7) r1 = 0
3: (7b) *(u64 *)(r10 -20) = r1 // <------------ should be *(u64 *)(r10 - 16)
misaligned stack access off (0x0; 0x0)+0+-20 size 8
processed 4 insns (limit 1000000) max_states_per_insn 0 total_states 0 peak_states 0 mark_read 0

ERROR: Error loading program: BEGIN

@ = (1, (int32)0) compiles, but has another problem: #1502

Both seem to be related to #1091.

This one is a bit tricky.  The u64 store is indeed wrong, but after fixing that it still breaks.
 bpftrace -e ' BEGIN { @ = ( (int8)40, (int32) -100, (int8) 10); exit() }' -v
BTF: using data from /sys/kernel/btf/vmlinux
Attaching 1 probe...

Error log:
0: (bf) r6 = r1
1: (b7) r1 = 10
2: (73) *(u8 *)(r10 -11) = r1
3: (18) r1 = 0xffffff9c
5: (63) *(u32 *)(r10 -15) = r1
misaligned stack access off (0x0; 0x0)+0+-15 size 4
processed 5 insns (limit 1000000) max_states_per_insn 0 total_states 0 peak_states 0 mark_read 0

As we use a packed struct aligned to 8 bytes it r10-15 is the right address. But the store will have to be split in this case. I can't figure out how to get LLVM to do this.
I think we're better of not packing our tuples, avoids a lot of alignment magic.


Slide tolarance

I tried almost all slider plugins this is the best (working great resizing, ssr and other features) just one piece missing, slide tolarence option.

Thanks! What would that prop do? I'm unfamiliar with it.

let's say user wants to add product to basket , when they click button or slider itself , drag:start will trigger after 5px movement . Because sometimes it will break @click event on slider itself .

What kind of markup are you using in those add to cart buttons?  I've tried to handle this use case by preventing dragging on interactive elements and images here:
https://github.com/BKWLD/vue-ssr-carousel/blob/25fe08b94e17d2a80ddc06b36968ca6e7cc85281/src/concerns/dragging.coffee#L233-L241
Maybe I need to add <button> to the list?


Loading Webface during training process.

Hello. I'm now training adaface models (ir50 and ir101) using Webface Dataset (4M, 12M, 42M).
During training process, I've found that the structure of sub-dirs in Webface Dataset is not proper for directly using Pytorch's ImageFolder class.
So I've made csv file which contain 'person_id_num' and 'image path', but I cannot train the model with webface 42M because of OOM error.
Would you give me some advice for loading such huge dataset for effiently setting-up batchsize during training process?

Hi khahn0213
One thing you could do is avoid loading the whole dataset. I would suggest using the technique used in https://github.com/deepinsight/insightface, that is using mxrecord file format for storing and reading the data. It is analogous to TFRecord file in tensorflow. One would need to bundle the dataset into the train.rec format as provided in InsightFace. Some useful discussion is in https://github.com/deepinsight/insightface/issues/1503.
After that, you can use the training procedure in this repository that uses the train.rec file. https://github.com/mk-minchul/AdaFace/blob/master/README_TRAIN.md

Thanks for your kind and helpful answer!
Hope you and I both do well in our respective places.

Do you still use your snu email? I have one or two questions about FR research :-)

Hi khahn0213 One thing you could do is avoid loading the whole dataset. I would suggest using the technique used in https://github.com/deepinsight/insightface, that is using mxrecord file format for storing and reading the data. It is analogous to TFRecord file in tensorflow. One would need to bundle the dataset into the train.rec format as provided in InsightFace. Some useful discussion is in deepinsight/insightface#1503.
After that, you can use the training procedure in this repository that uses the train.rec file. https://github.com/mk-minchul/AdaFace/blob/master/README_TRAIN.md

Hi @mk-minchul,
I did exactly what you mentioned here, but i still got an out of memory error when training adaface on webface42m, i used train.rec , and 8 * A100 GPUS with 40 GB each. i'm not sure did  I miss something ? did this happen with you?

@khahn0213 were you able to train adaface models (ir50 and ir101) using Webface Dataset 42M ?

@Gagan2002 Yes, I changed GPU.

@Gagan2002 I use four NVIDIA RTX A6000 D6 48GB.
For stable training, batch_size is as shown below.
batch_size=600
total_batch_size= 600 *4=2400


included new version of Jsonm to fix stack overflow

Jsonm module copied from github.com/hhugo/jsonm.git (branch: jsoo-friendly). This fixes the stack overflow error from parsing large objects in JSOO.

Pushed to staging:
https://tzcomet.io/#/explorer%3Fexplorer-input%3DKT1WwEZ98PyXu6BqGrDVBZJYbzCRUbPduuT8

https://tzcomet.io/staging/#/explorer%3Fexplorer-input%3DKT1WwEZ98PyXu6BqGrDVBZJYbzCRUbPduuT8

This fixes #60 (and #62).


Add function to display logo on config and menu pages

For code re-use and streamlining, build a function to dump the Base64 code for the logo rather than including the Base64 code in the index.php in multiple locations.

Implemented includeLogo().


Factorization of HierarchicalMatrices

The factorization phase of a H::HierarchichalMatrix with off-diagonal low-rank matrices/operators requires computation of H.A:
H.A = [I B; C I]

where B and C are dense and the identities are of conformable sizes. To simplify solution of later linear systems involving H.A, a factorization is stored in H.factorization. Should this be:

the QR factorization of H.A
the LU factorization of H.A
the block LDU factorization of H.A via

[I B; C I] = [I 0; C I]*[I 0; 0 I-C*B]*[I B; 0 I]

Advantage of QR is backward stability, while LU is simplest, and block LDU is fastest but both LU and block LDU may amplify ill-conditioning coming from, in particular, I-C*B.
Note that it's likely that C = B^T.

The size of H.A is r1+r2 x r1+r2, where these are the ranks of the off-diagonal matrices/operators at that level. So usually not very big, definitely less than 100 x 100.


get_data as_text does wrong thing

I was looking at Response.get_data. It does the wrong thing in many different ways when as_text=True is specified.
First off, the default charset is specified (RFC7231) or not ISO-8859-1 (RFC2616).  It should only work on text/* mime-types, and then lookup the default charset for the subtype, and never use utf-8 by default.
It also does not parse the charset from the content-type argument, which means if you use as_text=True w/ a non-utf-8 text/* document, you'll get the wrong results.
I have not written a test case for it, but it shouldn't be hard to write one.

The fact that it doesn't validate is the reason it's not part of the default Request. It is a contrib module from ancient times. If you'd like to improve that by somehow adding an approved list of encodings, or better documenting why it's not enabled or encouraged, I'd be happy to review a PR.

Please at least update the documentation for get_data as_text so that this is called out and document the work around.  I shouldn't have to file a bug to find out how to do something.  And that you don't call out the security issue in the DynamicCharsetREsponseMixin despite you knowing it's issues is a problem as well.

If you'd like to improve that by somehow adding an approved list of encodings, or better documenting why it's not enabled or encouraged, I'd be happy to review a PR.


Document considerations for streamMap(_ => Stream(...).evalMap) vs streamMap(_ => Stream(...)).evalMap

Fixed in #236


No cross-package analysis for site-packages

Azure-common package contains conditional aliases:
try:
    from msrestazure.azure_exceptions import CloudError
except ImportError:
    raise ImportError("You need to install 'msrestazure' to use this feature")

It seems to be a problem for PTVS

There is no problem if I import directly the class

This is actually because we don't do sophisticated enough cross-module analysis when analyzing installed packages. We do msrestazure and azure separately, and they don't know anything about each other, which is why the imports don't resolve.
We could probably scan the install metadata and selectively load other packages to handle this. If we try and load everything all the time then analysis will never complete.

Even if it's not solving the whole problem, if I add a __all__ statement this can help? Because in that case, you will have the explicit set of stuff I want to export.

No, the only way to help is to copy the entire type definition over (or put a stub definition under if False).


Set up Kanban board

We are two members in the group
This is the link to the project board

Hi Team,
Good job so far, indeed you have spent time on this project!
There are some issues that you still need to work on to go to the next project but you are almost there!

Highlights

Kanban usage ✔
Readme file ✔

Required Changes ♻
Check the comments under the review.

[ ] I noticed you missed the N+1 aspect, I will encourage you to add it since it is part of the project.

Optional suggestions
Every comment with the [OPTIONAL] prefix is not crucial enough to stop the approval of this PR. However, I strongly recommend you to take them into account as they can make your code better.
Cheers and Happy coding!👏👏👏
Feel free to leave any questions or comments in the PR thread if something is not 100% clear.
Please, remember to tag me in your question so I can receive the notification.
Please, do not open a new Pull Request for re-reviews. You should use the same Pull Request submitted for the first review, either valid or invalid unless it is requested otherwise.

As described in the Code reviews limits policy you have a limited number of reviews per project (check the exact number in your Dashboard). If you think that the code review was not fair, you can request a second opinion using this form.

STATUS: APPROVED 🟢
Hi @touredri, and @Kwaku28👋,
Your project is complete!  There is nothing else to say other than... it's time to merge it :shipit:
Congratulations! 🎉

Cheers and Happy coding!👏👏👏
Feel free to leave any questions or comments in the PR thread if something is not 100% clear.
Please, remember to tag me in your question so I can receive the notification.

As described in the Code reviews limits policy  you have a limited number of reviews per project (check the exact number in your Dashboard). If you think that the code review was not fair, you can request a second opinion using this form.

Thank you for the feedback


Create CONTRIBUTING.md

Fixes Issue
#10
Check List (Check all the applicable boxes)

[x] My code follows the code style of this project.
[x] My change requires changes to the documentation.
[x] I have updated the documentation accordingly.
[x] All new and existing tests passed.
[x] This PR does not contain plagiarized content.
[x] The title of my pull request is a short description of the requested changes.

Hey buddy, this is a contributing guide, not a code of conduct. The contributing guide will contain things like how to clone the repo, explanation of the project structure, how to add components, how to run linters etc.

Are you still working on this?


WebApp - Improve UI for Mission pools feature

Describe the task
Link design : https://www.figma.com/file/2nYhrm0rlJtfJoMwrT51Ep/SubWallet-Apps?type=design&node-id=11172-114612&mode=design&t=VPBPqgaoKQTKcbgQ-4

[x] Background is blurry

[x] Change logo of Twitter

[x] Check case when change responsive

Version test : https://1e38b491.subwallet-webapp.pages.dev/

[ ] Update logo and line arccording to design
Link : https://www.figma.com/file/2nYhrm0rlJtfJoMwrT51Ep/SubWallet-Apps?type=design&node-id=11172-114612&mode=design&t=zkBRr2x0cXeQ3dBY-4

[ ] Update padding

[ ] Update UI when change responsive arccording to extension design

Version test : https://96696565.subwallet-webapp.pages.dev/
Validation passed in case :

Update logo twitter

Change responsive

When filter and search


proposition for churn loading

not sure it is nicer - but this is what I tried to describe in the comment.

OK - thanks for explaining. What about this?
let churn = fs::File::open(churn_path) // type: io::Result<File>
    .map_or(BufReader::empty(), BufReader::new) // type: io::Result<BufReader>
    .map(parse_churn) // type: io::Result<anyhow::Result<Vec>>
    .context("Failed to parse churn info.")?; // type: Vec // handle parse_churn error

or
let churn = fs::File::open(churn_path) // type: io::Result<File>
    .map_or(Ok(vec![]), |f| parse_churn(BufReader::new(f))) // type: anyhow::Result<Vec>
    .context("Failed to parse churn info.")?; // type: Vec // handle parse_churn error

I misread the empty, it's indeed not in BufReader: https://doc.rust-lang.org/std/io/fn.empty.html
force-pushed your proposition - even though I prefer a single map_or over a map.map_or, but I consider it's your code ;)

As this commit will get erased when it's merge with #74 I moved it to https://github.com/c4dt/arti/commit/e25510044ccc33cf3bb65b52147809807a17a8f4


Multiple variations on cargo config

Prior to ch. 7 on Registers, each chapter by default uses the overall project's default src/.cargo/config.toml, which some chapters suggest modifying to use your platform's specific gdb.
This is overridden for the first time in Ch 7 by an unmentioned src/07-registers/.cargo/config with a default assumption that arm-none-eabi-gdb is to be used. In my case, no such gdb exists, and cargo run fails. Editing as follows allowed me to continue, but some may not understand the resulting error from Cargo well enough (or realize that you can have a multitude of .cargo/config or .cargo/config.toml files) to fix the issue:
diff --git a/src/07-registers/.cargo/config b/src/07-registers/.cargo/config
index f5a1d95..e11ff2f 100644
--- a/src/07-registers/.cargo/config
+++ b/src/07-registers/.cargo/config
@@ -1,5 +1,5 @@
 [target.thumbv7em-none-eabihf]
-runner = "arm-none-eabi-gdb -q -x openocd.gdb"
+runner = "gdb -q -x openocd.gdb"
 #rustflags = [
 #  "-C", "link-arg=-Tlink.x",
 #]

Maybe it's worth adding a helpful note in chapters where a custom config exists so users know exactly what to modify to get back to a working state?
[user@projects discovery]$ find . -type f -wholename "*/.cargo/config*" | sort
./src/.cargo/config.toml
./src/07-registers/.cargo/config
./src/08-leds-again/.cargo/config
./src/09-clocks-and-timers/.cargo/config
./src/11-usart/.cargo/config
./src/14-i2c/.cargo/config
./src/15-led-compass/.cargo/config
./src/16-punch-o-meter/.cargo/config
./src/WIP-async-io-the-future/.cargo/config

Hmm, I guess those are left overs not removed in #308.
@winksaville is there any reason why those other config files should stay?

I deleted the .cargo directory under f3discovery/src/07-registers, and now cargo run is successfully using the appropriate config from f3discovery/src/.cargo.


HAWQ-1061. Bugfix for specific eof is zeor and check greater than zero(thanks @wcl14).

cc @wcl14 @ictmalili

Merged into master.


iptsd-0.4

this is the result if I try iptsd:
ninja -v -j7 -l0 -C /var/tmp/portage/sys-firmware/iptsd-0.4/work/iptsd-0.4-10:41:03
ninja: Entering directory `/var/tmp/portage/sys-firmware/iptsd-0.4/work/iptsd-0.4-build'
ninja: no work to do.

Source compiled.

The complete build log is located at '/var/tmp/portage/sys-firmware/iptsd-0.4/temp/build.log'.
The ebuild environment file is located at '/var/tmp/portage/sys-firmware/iptsd-0.4/temp/environment'.

The complete build log is located at '/var/tmp/portage/sys-firmware/iptsd-0.4/temp/build.log'.
The ebuild environment file is located at '/var/tmp/portage/sys-firmware/iptsd-0.4/temp/environment'.
Working directory: '/var/tmp/portage/sys-firmware/iptsd-0.4/work/iptsd-master'
S: '/var/tmp/portage/sys-firmware/iptsd-0.4/work/iptsd-master'

Right I've changed the ninja function. Could you please try again, please also resync the repo.

No error, but nothing is installed
ninja -v -j7 -l0 -C /var/tmp/portage/sys-firmware/iptsd-0.4/work/iptsd-0.4-build
ninja: Entering directory `/var/tmp/portage/sys-firmware/iptsd-0.4/work/iptsd-0.4-build'
ninja: no work to do.

Source compiled.
Test phase [not enabled]: sys-firmware/iptsd-0.4

Install sys-firmware/iptsd-0.4 into /var/tmp/portage/sys-firmware/iptsd-0.4/image
ninja: Entering directory `/var/tmp/portage/sys-firmware/iptsd-0.4/work/iptsd-0.4-build'
ninja: error: unknown target 'DESTDIR=/var/tmp/portage/sys-firmware/iptsd-0.4/image'
Completed installing sys-firmware/iptsd-0.4 into /var/tmp/portage/s11:06:35are/iptsd-0.4/image

Final size of build directory: 920 KiB                              11:06:49

Final size of installed tree:    4 KiB                              11:07:01
                                                                11:07:02

Installing (1 of 1) sys-firmware/iptsd-0.4::linux-surface

checking 0 files for package collisions

Merging sys-firmware/iptsd-0.4 to /                                11:07:33
sys-firmware/iptsd-0.4 merged.
Regenerating /etc/ld.so.cache...
11:07:43
Recording sys-firmware/iptsd in "world" favorites file...
Auto-cleaning packages...

No outdated packages were found on your system.

Regenerating GNU info directory index...

Processed 118 info files.

IMPORTANT: 5 news items need reading for repository 'gentoo'.

Use eselect news read to view new items.

root@welle:~> equery f iptsd

Searching for iptsd ...
Contents of sys-firmware/iptsd-0.4:
root@welle:~>

Apologies for the quality, this should finally be fixed.

Jepp!!! Everytghing is fine now :)


Pinned grid copy value from unpinned grid issue

with first result set grid  pinned, run a query to generate a second grid (unpinned).
Select FIRST pinned grid and copy the first (row/column) value.
When you paste it, it will paste the same position value from the SECOND (unpinned) grid.
If you close one of those grids, it will copy the correct value.
Big issue!!

I can't reproduce this. Perhaps I didn't understand the use case.
Could you make a small gif video (e.g. with ScreenToGif  or some similar tool).


GLTF serializer: Add bones and morph target animation support

Follow-up of #4131

Does this issue mean that BabylonJS currently does not support loading a skeleton/bones from GLTF scenes?

Nope it means we do not export them from a babylon.js scene
From a loading standpoint we support EVERYTHING from a gltf file :)

Continuing from #6628, this PG:
https://playground.babylonjs.com/#T087A8#7
reproduces the issue.

I think I have a good plan of action here:
Looking at the current result of exporting the alien scene:
alien_babylon_glTF2_export.zip
We can see that our current joint structure is preserved (as we iterate through all nodes in the babylon scene):

well, for the most part anyways...due to #7152 we lose all non-mesh leaf nodes in the scene:

But this should be a trivial fix of removing this conditional we use to evaluate what nodes get exported when traversing the scene tree:
https://github.com/BabylonJS/Babylon.js/blob/v4.0.0/serializers/src/glTF/2.0/glTFExporter.ts#L1342
I'll check this change in as a part of this enhancement.
Now, back to implementation details, looks like we should be able to shim this functionality into createSceneAsync after we convert materials, and create our node map (mainly so that we already have a babylon scene node id to gltf node id to work with)
I should be able to add createSkinsAsync and createMorphTargetsAsync functions (Sticking with the asynchronous convention even though the export is performed synchronously?)
Only major concerns I have right now is in exporting the reverse bind matrix for my skin joints, I fear that LHC/RHC setting will cause some issues, but that a problem for Future Drigax™

I wish Future Drigax™ good luck :D

Hey @deltakosh , I'm noticing some divergance between Babylon and glTF's nomenclature for a bone in Babylon, is bone._restPose equivalent to the bind pose? (and therefore would need to be inverted to fit glTF's schema?)

I am using https://playground.babylonjs.com/#NSDE3A#1 as a test scene for the morph target and skeleton round-tripping.

Hey @Drigax , I know you are working very hard on this job, I just want to know when you fix this problem, I've been following it since last year, version 4.1. I have a project and really need GLTF serializer with animation.
thank you

Skinning is in via #8513, working on Morph targets now


docs(readme): fix invalid URL

Closes: https://github.com/convco/convco/issues/154

The scope in the commit message can be removed. You can also squash and only keep 1 commit. If I find time I will do it later this week

merged with https://github.com/convco/convco/commit/19dbc42b631551b97e4e56a26f2db6aa30d3e7a2


Provide call to close/abort native picker file explorer

Use case
When Using ImagePicker in Web on a MacOS or Windows, it will launch the Gallery in the form of a file explorer
Normally one would select the intended image and clicks the 'Open' button.
Alternatively one could drag the image onto the ImagePicker widget to drag/drop the image.
When this is done, the file explorer remains active. Not a problem, but would be nice to programmatically close the File Explorer window. Or to programmatically abort the ImagePicker action.
Proposal
Add a call to complete/cancel or abort the action, so the File Explorer that is shown natively is closed.

Alternatively one could drag the image onto the ImagePicker widget to drag/drop the image.

What widget are you referring to here?

Add a call to complete/cancel or abort the action, so the File Explorer that is shown natively is closed.

I'm not clear from the description above of the use case what the goal here is. You want to be able to close the UI out from under the user while they are still interacting with it? What's the use case where this would be desirable?

Code triggered onTap of widget:
                    final ImagePicker picker = ImagePicker();
                    final XFile? xFileImage = await picker.pickImage(source: ImageSource.gallery);

See attached video of use in Safari browser on MacOs
https://github.com/flutter/flutter/assets/9477833/2f2dca23-2303-4578-87c1-64caf150bb38

@jakusb I think this is out of the plugin's scope as the native picker belongs to the system side. Please refer to this SO's answer. If you see an official API on web that supports this, please share it here. Thanks!


Support development on mac

If I run nix develop I get
error: flake 'git+file:///Users/sjoerdvisscher/Documents/GitHub/atlas' does not provide attribute 'devShells.x86_64-darwin.default', 'devShell.x86_64-darwin', 'packages.x86_64-darwin.default' or 'defaultPackage.x86_64-darwin'

Is there a way around this?

Yes, being worked on in PR #120.

Works perfectly, thanks!


fix: pre-read requests prevent read/write index confusion

Closes #112
Signed-off-by: tzssangglass<EMAIL_ADDRESS>Please answer these questions before submitting a pull request

Why submit this pull request?

[x] Bugfix

[ ] New feature provided

[ ] Improve performance

Related issues

Bugfix

Description
refer to #112

Read to request headers, index error.

How to fix?

New feature or improvement

Describe the details and related test reports.

Source branch

Related commits and pull requests

Target branch

hi @leyuuu, can you verify that your problem is solved with this PR?

@tzssangglass Yes, it works. Thank you ❤


jmespath search unexpected behavior when expression contains equality operators

When using the #search method with equality operators there is some unexpected behavior. For example, if I have a hash:
irb(main):092:1* hash = {
irb(main):093:2*   "top_level_key" => {
irb(main):094:2*     "inner_key" => "Hello World!"
irb(main):095:1*   }
irb(main):096:0> }

and an expression that does not match the hash:
irb(main):102:0> expression = 'non_existent_key.inner_key'
=> "non_existent_key.inner_key"

I expect that if I call JMESPath.search(expression, hash) I should get nil, and that works as expected:
irb(main):103:0> JMESPath.search(expression, hash)
=> nil

However, if I include an equality operator like:
irb(main):104:0> expression = 'non_existent_key.inner_key == "Hello World!"'
=> "non_existent_key.inner_key == \"Hello World!\"

I would expect that JMESPath.search would return false, yet I get true:
irb(main):106:1* hash = {
irb(main):107:2*   "top_level_key" => {
irb(main):108:2*     "inner_key" => "Hello World!"
irb(main):109:1*   }
irb(main):110:0> }
=> {"top_level_key"=>{"inner_key"=>"Hello World!"}}
irb(main):111:0> expression = 'non_existent_key.inner_key == "Hello World!"'
=> "non_existent_key.inner_key == \"Hello World!\""
irb(main):112:0> JMESPath.search(expression, hash)
=> true

I believe the literal expression in your search should use single quotes (') instead of double (").  Eg, the following correctly returns false for me:
expression = "non_existent_key.inner_key == 'Hello World!'" # use single quotes for the literal
JMESPath.search(expression, hash) # now returns false

The double quotes are used for escaping an identifier.  So in this case you are searching for a key "Hello World", which in this hash will give you null and null == null is true, so that is being returned.

Ah, Thanks for the clarification!


No option to insert image from in memory image

For programmatically generated MD string it is very usefull to feed not only external images from URI or app resources, but from inmemory image - for example I want to add LiveCharts chart as image, but have no find options to feed these kind or images as source.
It is possible to save temp image and feed it to MD as a local image, but it is not a good solution.

In v0.10.12 and v11.0.0-a4, I add the UseResource property.
If this property is set to True, Markdown.Avalonia looks for images from Resources.
The value, which is set to Resources, should be Avalonia.Media.Imaging.Bitmap or Avalonia.Controls.Control
If there is no image in Resources, Markdown.Avalonia searches using the same rules as before.
example
<!-- xmlns:md="clr-namespace:Markdown.Avalonia;assembly=Markdown.Avalonia" -->

<Panel>
  <Panel.Resources>
    <Image x:Key="MyImage" Source="/Assets/ResourceImage.png"/>
  </Panel.Resources>

  <md:MarkdownScrollViewer UseResource="True" xml:space="preserve">
    <md:MarkdownScrollViewer.Resources>
      <Button x:Key="MyButton" Content="Hello Button"/>
    </md:MarkdownScrollViewer.Resources>

    ## Image ##

    ![](MyButton)

    ![](MyImage)

  </md:MarkdownScrollViewer>
</Panel>


necinnost: DP children born in CZ, nezletile, vady zadosti + some minor fixes

for necinnost:

one more residence type - dlouhodoby pobyt for children born in CZ
nezletile
vady zadosti

minor fixes and improvements

Looks good, thank you! I added the checkbox for vady zadosti as well

Deployed at production, thanks!


WIP: Feature cn 235 cv on demand

DO NOT MERGE UNTIL DEPENDENCY CHANGES ARE CONCLUDED
Changes are dependent on an RCL "datatable-translatable", so:

these have to be reviewed and merged into master
then published to NPM
then here, the package.json file needs to be updated with the new version of RCL on NPM
then this change needs to be committed and push in this feature branch

Then and only then may the PR be reviewed.

This change is

src/components/translatable/TranslatableTSV.js, line 23 at r3 (raw file):

import * as cv from 'uw-content-validation';
import * as csv from '../../core/csvMaker';

If future apps need this functionality, perhaps csvMaker could be moved to the uw-content-validation library?


Inline mantle wires from slice

Fixes https://github.com/rdaly525/coreir/issues/899
Depends on https://github.com/leonardt/verilogAST-cpp/pull/53

Adds a metadata field for enabling the inlining of a specific wire instance (by default wires are not inlined)
Adds verilog codegen support for mantle wires (there's a subtetly where mantle wires can have "in" be an output depending on the type used in the generator)

While this avoids intermediate mantle wire instances, the mantle wire modules are still generated.  I'll need to update the verilog code gen logic to figure this out, it's not necessarily easy since the module dependency graph is computed generically.  We'd want to check that all instances of a module are inlined, then skip it.  Not sure how blocking this is, so we can merge this with the extra unused modules code generated or wait for that improvement to be added.

Turns out the mantle wire codegen issue was fairly trivial given instanceGraphNode.getInstanceList(), latest commit adds that feature.


Dual support for ESM and CJS

[x] Update deps to fix issues with peer deps not satisfied
[x] Add tsup to build ESM & CJS
[x] Update scripts in package.json
[x] Add module resolution for ESM & CJS builds in package.json
[x] Setup tsup config
[x] Added formatter to keep code consistency (Can be removed if not needed)
[ ] Tested the lib

Closes #19

dist is now built and bundled into 3 main files.

index.js - CJS
index.mjs - ESM
index.d.ts - Type declarations

dist was originally just being built by tsc with no bundler, it is now built with tsup that bundles this into a single file with rollup. This saves space and cuts the amount of files needed to be pushed to prod walst adding full support for both ESM & CJS

@devnote-dev ready for review

Please only PR to the main branch

I've done some general testing and everything works from what I can see, however, typings don't show up for some methods/objects, for example Shard instances and Permissions objects. Is there a fix for this?

This looks like a config issue, will look into it and see


unit-test pkg/local/configmaps and secrets

unit-test

pkg/local/configmaps
pkg/local/secrets

Minor nitpick - the test methods need the GIVEN/WHEN/THEN comments that were discussed in a recent meeting


SW-163 Support processor decoration

I got the branch name wrong, closing this one in favor of #12


Support Kernels for SVM

Kernels in support vector machines are really usefull. I would like to test some object detection algorithms in tensorflow using svm and a radial basis kernel.
I am willing to help implementing kernels for support vector machines, though I am not familiar with the sdca optimization.
Is there any design doc, or plan how to add more features to the svm estimator api?

You might if anybody has done anything in this vein already by asking on stackoverflow. I don't know of any work internally to build out the svm functionality more or any design doc.
@martinwicke, do you have any knowledge of others interested in this or a good course of action? Marking as contributions welcome for now.

@petrosmol should have the best context for this. I believe there is work in progress for kernel-SVMs.

TL;DR: Yes, we are actively working on kernel methods in general. As of last week there is a kernel_methods package which is part of tensorflow.contrib. Kernel SVMs are supported indirectly (with some more work on user's behalf, see below). Note that this is all very recent so you need to build from source (and some changes might still need some time to propagate). Read on for details.

We don't have support for exact kernel methods and AFAIK, there is no plan to support these any time soon. However, we do support primal (approximate) kernel methods. We have implemented Random Fourier Features (RFF) that approximate the RBF kernel. This is the equivalent of sklearn's RBFSampler. See here. You can use RFF to transform your input features and then apply a linear model on top of it. These are typically much faster than exact kernels and perform pretty well in practice (quality-wise). Note that this works with dense input features only (so images should be fine)
If you want to use a pre-packaged estimator, we have implemented a Linear Classifier that uses RFF to preprocess the input features. This is not kernel SVMs, you can think of it as a "kernelization" of the LinearClassifier model from tensorflow.contrib.learn. The loss used is logistic (for binary) and cross entropy (for multiclass). See here for the estimator and here for sample usage.
If you really want to SVMs, you have the following 2 options:

If your problem is binary classification you can use the pre-packed SVM estimator (which uses SDCA) but you should first apply RFF to your features (as part of your input_fn). Make sure you update your feature columns accordingly.
For multiclass, you can implement your own 1-vs-all or 1-vs-1 using binary SVMs. There is no multiclass support for SVMs currently (but there are plans for it)

Thank you! I will try the different options. Binary SVM is sufficient for me right now.

Hello, @petrosmol !  Has multiclass SVMs api been supported in TF version 1.6 now?

There is no pre-packaged multiclass SVM Estimator yet (it is in the future plans though). But since this bug was opened, I have implemented multiclass SVM loss (see https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/kernel_methods/python/losses.py). If you want to turn this into an Estimator, possibly the easiest way is to create a class that inherits from _KernelEstimator and pass to its constructor a head that uses the multiclass SVM loss (see KernelLinearClassifier model for instance that uses a cross-entropy multiclass loss)

Thank you very much! @petrosmol  I will try your advice.  :-  )

Hello, @petrosmol ! I would like to know whether the implemented multiclass SVM loss  (see [https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/kernel_methods/python/losses.py]) in tensorflow could be optimized by Gradient Descent Algorithm.

@Lidaguo. Yes, the loss should work with any gradient based optimizer. I have tried it in the past in some experiments with MNIST with no issues AFAIR.

Hello, @petrosmol ! Thanks for your particular reply. But now, I have another problem. In my case, I'd like to use the implemented tf.contrib.kernel_methods.RandomFourierFeatureMapper to transform the features from network, and then use the transformed features to construct my loss function.  All the things go well at the beginning of my training procedure. However,  with the increasing of the training epochs, the occupation of GPU grow up. After tens of epochs, the OOM error of GPU occurs. If I don't use the transformation, everything works fine. my simple code as following:
kernel_mapper = tf.contrib.kernel_methods.RandomFourierFeatureMapper(input_dim = 100, output_dim = 1000,  stddev =5.0, name='rffm') model_train = My_model(..., kernel_mapper) for epoch_index in xrange(total_epochs):     run_epoch(..., model_train)
Here is the definition of the model,
class My_model(..., kernel_mapper):      ....      input_feature = ....      mapped_feature = kernel_mapper.map(input_feature)     ....
I guess that the  tf.contrib.kernel_methods.RandomFourierFeatureMapper class always stores values for each epoch(or each called situation) and don't free the Memory. Is there any solution for this?

Hello, @petrosmol ! Thanks for your particular reply. But now, I have another problem. In my case, I'd like to use the implemented tf.contrib.kernel_methods.RandomFourierFeatureMapper to transform the features from network, and then use the transformed features to construct my loss function.  All the things go well at the beginning of my training procedure. However,  with the increasing of the training epochs, the occupation of GPU grow up. After tens of epochs, the OOM error of GPU occurs. If I don't use the transformation, everything works fine. my simple code as following:
`
kernel_mapper = tf.contrib.kernel_methods.RandomFourierFeatureMapper(input_dim = 100, output_dim = 1000,  stddev =5.0, name='rffm')
model_train = My_model(..., kernel_mapper)
for epoch_index in xrange(total_epochs):
run_epoch(..., model_train)
`
Here is the definition of the model,
`
class My_model(..., kernel_mapper):
....
input_feature = ....
mapped_feature = kernel_mapper.map(input_feature)
....
`
I guess that the  tf.contrib.kernel_methods.RandomFourierFeatureMapper class always stores values for each epoch(or each called situation) and don't free the Memory. Is there any solution for this?

@Lidaguo.
Hi, RandomFourierFeatureMapper should be able to handle input/output dimensions of 100/1000. Related question: Do you see the same issue on CPU?
Is it possible that you add the map ops multiple times (one per epoch) in your training graph and that's why your are getting OOM errors after a few iterations?
RandomFourierFeatureMapper only adds map OPs (and does not store any values in a way that is different than what any TF graph would do it).
Can you make sure that, under the hood, your call does not call .map method multiple times?
For debugging, there are ways to print the size of the TF graph after each iteration/epoch. If this is growing with each iteration then you are likely adding ops to your graph during training (which is bad).
If the above suggestions don't help you fix the problem, send me the full code so that I can reproduce the issue.
Thanks
Petros

@petrosmol . Hello, I did add the map ops multiple times in one per epoch in my training graph. I have fixed this problem. Thank you very much. And I have a another problem about the implemented multiclass SVM loss (see )

@petrosmol . Hello, I did add the map ops multiple times in one per epoch in my training graph. I have fixed this problem. Thank you very much. And now, I have a problem about the implemented multiclass SVM loss (see https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/kernel_methods/python/losses.py), like the following picture:

In my case,
self._labels = tf.placeholder(tf.int32, [None,],name='data_labels')
Dose it mean that the labels or logits sent into the multiclass SVM loss class  must be a constant tensor ?

@Lidaguo
Hard to tell what is going on without seeing code. But both labels and logits should work with unspecified shape. For instance the following code should work:
import numpy as np
BATCH_SIZE = None
NUM_CLASSES = None
logits_np = np.array([[1.2, -1.4, -1.0], [1.4, 1.8, 4.0], [0.5, 1.3, -1.0]])
labels_np = np.array([0, 2, 1], dtype=np.int32)
logits = tf.placeholder(
tf.float32, shape=(BATCH_SIZE, NUM_CLASSES), name='logits')
labels = tf.placeholder(tf.int32, shape=[BATCH_SIZE,], name='labels')
loss = tf.contrib.kernel_methods.sparse_multiclass_hinge_loss(labels, logits)
with tf.Session() as sess:
loss_eval = sess.run(loss, feed_dict={logits: logits_np, labels: labels_np})
print(loss_eval)
Initial version of kernel_methods/python/losses.py was not working with unknown shape though. This was fixed end of 2017 (see https://github.com/tensorflow/tensorflow/commit/2dce1ee2e37e314772467bfa150b57b8d4311598#diff-c7afce59582d7278b51608c8024bf615)
So, you might need to upgrade to a more recent version of tf or patch these changes locally.

@petrosmol
Yes! I have ungraded the version of tf. Now, my model works. Thank you very much. : - )


Readme ?

Maybe provide a Readme with dependencies and a short example ?
Thanks. Really enjoyed the movie.

And requirements.txt

Alright, just added a README.md which talks about dependencies and getting the code up and running.  Let me know if it doesn't makes sense!

Basic README and requirements.txt are now up.


Twisted issue #9446 causes ProtocolError with WAMP RPC and large messages/busy server on Windows

Edit: The root cause is with Twisted #9446. See end of thread.
One of our applications is calling WAMP procedures with large arguments (GraphQL queries) returning large query results. We are getting regular errors autobahn.wamp.exception.ProtocolError: INVOCATION received for request ID <id> already invoked. We suspect there is an issue when calls/results are split over frames concurrently.
Tried to reproduce this with an isolated test case but the code below produces autobahn.wamp.exception.ProtocolError: RESULT received for non-pending request ID <id> for more than half of the runs (which id actually fails is varying) which is at leat originating from the very same function as the exception above. Code works fine when ARG_SIZE is choosen << 1000.
We are running Python 3.6.0 64bit on Windows 7. Any ideas?
from autobahn.asyncio.component import Component
from autobahn.wamp.types import RegisterOptions
from asyncio import Future, gather, get_event_loop, sleep

component = Component(
    transports=[
        {
            "type": "websocket",
            "url": "ws://<IP_ADDRESS>:5000/ws",
            "endpoint": {
                "type": "tcp",
                "host": "<IP_ADDRESS>",
                "port": 5000
            }
        }
    ],
    authentication={
      "wampcra": {
        'authid': 'backend',
        'secret': 'super secret'
      }
    },
    realm='default'
)

_last = Future()

@component.register('org.test.echo_next_call', options=RegisterOptions(details_arg='details'))
async def echo_next_call(arg, details):
    """last call will be ready when `echo_next_call` gets called again, interlacing calls and results"""
    global _last

    if not _last.done():
        _last.set_result(arg)
    else:
        # only get here when things went wrong
        pass

    future = Future()
    _last = future

    return await future

ARG_SIZE = 10000

@component.on_ready
async def on_ready(session):
    global _last

    future = gather(*[session.call('org.test.echo_next_call', 'bla' * ARG_SIZE) for _ in range(8)])

    # wait long enough such that we are sure to .set_result on the last future, otherwise future will never finish
    await sleep(1.)

    if not _last.done():
        _last.set_result('done')
    else:
        # we only get here when an exception was raised in gather(...)
        pass

    await future

    session.leave()

if __name__ == '__main__':
    loop = get_event_loop()
    future = component.start(loop)
    loop.run_until_complete(future)

Raises the following exception in about half of all runs:
Traceback (most recent call last):
  File "C:\virtualenvs\weboptions\lib\site-packages\autobahn\wamp\websocket.py", line 95, in onMessage
    self._session.onMessage(msg)
  File "C:\virtualenvs\weboptions\lib\site-packages\autobahn\wamp\protocol.py", line 828, in onMessage
    raise ProtocolError("RESULT received for non-pending request ID {0}".format(msg.request))
autobahn.wamp.exception.ProtocolError: RESULT received for non-pending request ID 2

Not sure if this is relevant to this at all, but there was a past instance of a 32-bit number being used for IDs -- if any request-ids got that small again, this could be a symptom (i.e. with enough outstanding requests, you start getting occasional collisions).

to track this down, more info that allows to bisect the root cause would be good. here are some things that could shed new light:

description of setup (what windows, both crossbar and client windows? what network? any firewalls, proxies, ..)
try it using rawsocket as transport, instead of websocket
try your exact same test program running on Linux, but crossbar on windows. other wise round. and both linux
try https://s3.eu-central-1.amazonaws.com/download.crossbario.com/crossbar.exe
run the autobahn testsuite on windows

the latter is a statically built crossbar (a very new version) for windows I created recently: use it for testing only (it is fine, I promise) ..
using the single-file exe allows us reduce the number of variables in this game, as we can easily run both the exact same bits of crossbar.

@meejah I think above shows that - regardless what the specific issue in this case (if any) is - we should really expand our CI, and integrate (so it automatically runs) what we already have

AB websocket testsuite
AB Py examples (autobahn/examples)
AB JS tests https://github.com/crossbario/autobahn-js/tree/master/test
CTS

Thanks for the detailed bug-report @asodeur !
I have been playing with your example code on Debian -- if I turn up the "concurrent outstanding calls" to 1M I start to see some failures (e.g. low-level send()-s failing), and some of these do result in suspicious-looking failures at the Python level (although not in the same way as your traces). I see different failures if I use two different components (i.e. one registers the echo RPC, the other calls it).
Have you tried with two components? (Looking at the code, we shouldn't overlap IDs until 2^53 outstanding calls -- but that doesn't necessarily rules that possibility out.)

@oberstet yes, getting more automated testing "actually fully automated" would be really good!
I'm not very familiar with the websocket testsuite, perhaps making it run when github wants it to would be a good excuse :)

(Interestingly, I can only get the 'weird things' to happen on asyncio, not twisted). Also I have to turn off the ping-timeouts (or make the timeout bigger) using the example-router config.

I have auto-ping turned off as well. So it looks like Linux and MacOS  cope pretty well but Windows is pretty fragile.
The self._transport.send(reply) issue shows up with my test after the first double invocation kills the transport. A quick and dirty fix might be skipping the reply if the transport went away before the success callback gets called (patch attached). Probably a better strategy would be to cancel() the InvocationRequests.
Another tiny py3 glitch I noticed running the autobahn-testsuite is an integer division in WebSocketProtocol.
0009-Skipping-Yield-message-if-transport-gets-closed-befo.txt
0064-integer-division-for-py3.txt
With that the autobahn-testsuite seems to pass for testeeclient vs fuzzingserver and fuzzingclient vs testeeserver. (I am saying seems because I had to do a quick port to py3 since our py2.7.3 does no longer work with the new pypi, and did update to autobahn 18.4.2.dev. Not 100% I got all that right.)
Using the static crossbar.exe from above did not change results.
If I find some time on Wed I'll try if I some of the 9.x.x tests from autobahn-testsuite break down on Windows under sufficient load.

if you have the testsuite (the fuzzing client part), the you can run that directly against crossbar, as it has a built in websocket testee endpoint: https://github.com/crossbario/crossbar-examples/tree/master/benchmark/websocket
this is a streaming websocket capable endpoint, so it will be "apples and oranges" comparing to WAMP, but nevertheless, if you can pump through big chunks during the websocket tests using endpoint A (testee type) in crossbar, while at the same time endpoint B (running in the very same crossbar) fails for largish WAMP payloads, that would further reduce the code lines of possible source of issue that are candidates.
anyways: yeah, crossbar on windows hasn't been taken under load and fire as on *nix ..

I found a working configuration and tracked down the main issue to Twisted's IOCPReactor by now.
Crossbar's websocket testee endpoint passed the testsuite after some wrestling with the new config file format. There is a typo in the Autobahn's asyncio raw socket implementation that causes trouble with messages split over TCP frames (0002-fixed-issue-with-split-messages.txt). After this fix the failures became pretty deterministic in rawsocket mode.
The issue with Twisted's IOCPReactor seems to be that if you write new data to a saturated TCP connection all in-flight data gets send again (iocpreactor.abstract.FileHandle.doWrite can get called again before  ._handleWrite got the chance to update the bytes already written). If the router worker is configured to run on the select reactor everything works fine (I did overlook crossbar --reactor=select start is not enough to change the router worker's reactor).
I hope I'll get the time to write-up a bug report to the Twisted folks next week. Would it make sense to make select the default reactor on Windows in the meantime?
PS:  Would you be interested in the py3/autobajn 18.4.2 port of autobahn-testsuite? Shall I prepare a pull request?

@asodeur woah! you've been digging deep;) ok, if the issue is with backpressure on the IOCP Twisted reactor, then yeah, best is to first file an issue on Twisted. But it will require (obviously) digging even deeper. The IOCP reactor is based on the Windows completion port model ("signal when done") which is quite different from the Unix model ("signal when actionable"), so the handling of backpressure etc .. could be tricky. anyways, thanks for all the research work!
btw, I fixed the bug you found in rawsocket/asyncio as part of this https://github.com/crossbario/autobahn-python/compare/59f119272e61^...acdb623e3d03#diff-bdbe67d81608026a166c457814e89a2f
rgd using the select reactor on windows: so you patched the code? because it should work with:
CROSSBAR_REACTOR=select crossbar start

on windows in principle. with master. because I restructured the whole reactor loading, in part because of windows work we recently did, and other reasons. if it doesn't work as above on windows, that is a bug we should fix. that being said, select reactor has at least the limits on number of file descriptors (255 on windows pseudo-unix if I remember), and .. the signal handling again .. mmh. no clue how reliable that is. and the worker to node controller communication (runs over pipes). all these are dependent on deep down reactor features in the end.

Would you be interested in the py3/autobajn 18.4.2 port of autobahn-testsuite

yeah, please submit it! cool. I really cannot promise when I will time to look deep into .. because a lot has happened in the code;) and there is still the idea to pretty much version freeze the testsuite (we also have a docker image) to have a stable reference testsuite (even though an imperfect one of course). but this can be looked at from another angle too ... so actually, have an open PR would allow to gather feedback on those two views already. so yeah, would be great to have that work at least saved in a PR!

Closing this as the rot cause is with Twisted #9446. Also renaming the issue to refer to the Twisted issue and mark it as Windows only.
Using the select reactor is a workaround if the open connection limit is not an issue.

@asodeur thanks for coming back and filing the twisted issue. I added an issue on our side (even though its upstream business) to make sure we eventually follow up on this.

one practical idea: did you try to increase TCP window buffers size manually in windows? eg HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\TcpWindowSize => 16777216. normally, this is auto-tuned at run-time and pre connection these days, but I wonder if you could work around this as it is backpressure related ..

The current iocp reactor implementation will run into trouble as soon as the Windows system call is not writing all data immediately and returning synchronously. I have no clue about Windows internals but would be surprised if writing to sockets is synchronous until TCP window size is exceeded.The good news is I am fairly confident the issue with the iocp reactor has a very easy fix. Already opened a pull request.A-------- Original Message --------Subject: Re: [crossbario/autobahn-python] Twisted issue #9446 causes ProtocolError with WAMP RPC and large messages/busy server on Windows (#993)From: Tobias Oberstein To: crossbario/autobahn-python CC: asodeur ,Mention @asodeur thanks for coming back and filing the twisted issue. I added an issue on our side (even though its upstream business) to make sure we eventually follow up on this.
one practical idea: did you try to increase TCP window buffers size manually in windows? eg HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\TcpWindowSize => 16777216. normally, this is auto-tuned at run-time and pre connection these days, but I wonder if you could work around this as it is backpressure related ..
—You are receiving this because you were mentioned.Reply to this email directly, view it on GitHub, or mute the thread.
{"@context":"http://schema.org","@type":"EmailMessage","potentialAction":{"@type":"ViewAction","target":"https://github.com/crossbario/autobahn-python/issues/993#issuecomment-394974682","url":"https://github.com/crossbario/autobahn-python/issues/993#issuecomment-394974682","name":"View Issue"},"description":"View this Issue on GitHub","publisher":{"@type":"Organization","name":"GitHub","url":"https://github.com"}}
{"api_version":"1.0","publisher":{"api_key":"05dde50f1d1a384dd78767c55493e4bb","name":"GitHub"},"entity":{"external_key":"github/crossbario/autobahn-python","title":"crossbario/autobahn-python","subtitle":"GitHub repository","main_image_url":"https://assets-cdn.github.com/images/email/message_cards/header.png","avatar_image_url":"https://assets-cdn.github.com/images/email/message_cards/avatar.png","action":{"name":"Open in GitHub","url":"https://github.com/crossbario/autobahn-python"}},"updates":{"snippets":[{"icon":"PERSON","message":"@oberstet in #993: @asodeur thanks for coming back and filing the twisted issue. I added an issue on our side (even though its upstream business) to make sure we eventually follow up on this.\r\n\r\n\u003e one practical idea: did you try to increase TCP window buffers size manually in windows? eg HKEY_LOCAL_MACHINE\\SYSTEM\\CurrentControlSet\\Services\\Tcpip\\Parameters\\TcpWindowSize =\u003e 16777216. normally, this is auto-tuned at run-time and pre connection these days, but I wonder if you could work around this as it is backpressure related .."}],"action":{"name":"View Issue","url":"https://github.com/crossbario/autobahn-python/issues/993#issuecomment-394974682"}}}
{
"@type": "MessageCard",
"@context": "http://schema.org/extensions",
"hideOriginalBody": "false",
"originator": "37567f93-e2a7-4e2a-ad37-a9160fc62647",
"title": "Re: [crossbario/autobahn-python] Twisted issue #9446 causes ProtocolError with WAMP RPC and large messages/busy server on Windows (#993)",
"sections": [
{
"text": "",
"activityTitle": "Tobias Oberstein",
"activityImage": "https://assets-cdn.github.com/images/email/message_cards/avatar.png",
"activitySubtitle": "@oberstet",
"facts": [
]
}
],
"potentialAction": [
{
"name": "Add a comment",
"@type": "ActionCard",
"inputs": [
{
"isMultiLine": true,
"@type": "TextInput",
"id": "IssueComment",
"isRequired": false
}
],
"actions": [
{
"name": "Comment",
"@type": "HttpPOST",
"target": "https://api.github.com",
"body": "{\n"commandName": "IssueComment",\n"repositoryFullName": "crossbario/autobahn-python",\n"issueId": 993,\n"IssueComment": "{{IssueComment.value}}"\n}"
}
]
},
{
"targets": [
{
"os": "default",
"uri": "https://github.com/crossbario/autobahn-python/issues/993#issuecomment-394974682"
}
],
"@type": "OpenUri",
"name": "View on GitHub"
},
{
"name": "Unsubscribe",
"@type": "HttpPOST",
"target": "https://api.github.com",
"body": "{\n"commandName": "MuteNotification",\n"threadId": 325595789\n}"
}
],
"themeColor": "26292E"
}


Studio: Reload performance insufficient

Describe the bug
When I reload a page in Fusion/Studio it took about 60 seconds (!!!) to reload the page. That is a totally inacceptable bad performance. Even 10 seconds would be bad already! That needs to fixed for usability of working with Fusion.
What part of Nexus is affected

[x] Fusion
[ ] Delta
[ ] CLI
[ ] Documentation

To Reproduce
Steps to reproduce the behavior, e.g. for Fusion:

Go to 'Studio'
Select a studio (in the example "test 62227")
3.reload the page

Expected behavior
Page should reload in like 5 seconds.
Screenshots

Additional context
env: stage Nexus Fusion v.1.8.0-M3-5-g7c346d1e
user: adietz
speedtest: 47.38 Mbps download, 31.92 Mbps upload

Agreed, some queries when run on a SPARQL engine can run a long time. The reason for that is that this engine might not be suited for some kinds of queries (e.g. listing of numerous items). A work around (that is at the discretion of the user building the Studio) is to use Elasticsearch for such kinds of queries. In the future, we would like to leverage Search results (which run in Elasticsearch) to build a Studio Dashboard, both improving performance and making the creation of a Studio dashboard much easier (no need to write a SPARQL or Elasticsearch query).
I am relabelling this an improvement rather than a bug.


added my name

added my name Kwinsi Thakkar and link of github account

added my name Kwinsi Thakkar and link of github account


Implement AnyOp and AllOp operators

Is your feature request related to a problem or challenge?
Follow on to https://github.com/apache/arrow-datafusion/issues/6119
It would be really nice to implement SQL Any and All operators in arrow-datafusion. These two operators have a wide scope.
Describe the solution you'd like
sqlparser-rs supports AnyOp and AllOp operators (See https://docs.rs/sqlparser/latest/sqlparser/ast/enum.Expr.html).
Describe alternatives you've considered
No response
Additional context
https://docs.rs/sqlparser/latest/sqlparser/ast/enum.Expr.html#variant.AnyOp
https://docs.rs/sqlparser/latest/sqlparser/ast/enum.Expr.html#variant.AllOp

I just want to confirm the functionality, I can do something like foo > all(1); to check if all values are greater than 1. Or foo != any(6); to ensure no values in the array are equal to 6. Does this seem accurate?


Getting ERRO[0000] [default] can't find status mapper for Alertmanager 0.16.0

While running Karma, I am getting the above mentioned error. Not able to find anything about this. Any idea what this is about?

You need to upgrade alertmanager.
It's documented on top of https://github.com/prymitive/karma/blob/master/README.md
Starting with karma v0.56 only Alertmanager V2 API is supported, which requires Alertmanager >=0.17.0.

But I have downloaded Karma v0.55.

0.16.0 doesn't have required API, see v0.55 README
Alertmanager 0.16.x is NOT supported by karma due to changes in the API, see this issue for details. A new API endpoint was added for 0.17.0 and alertmanager >=0.17.0 is supported.

Okay, thanks. So, does any version of Karma support Alertmanager 0.16.x?

No, 0.16 dropped the v1 API karma uses, v2 API for that was only added in 0.17.x


[terra-worklist-data-grid] Changed focus styling for cells

Summary

What was changed:
Updated the css stylings for cell focus to have a dashed black outline.
Why it was changed:
To match the other stylings in terra, such as terra-table.
Testing

This change was tested using:

[x] WDIO
[x] Jest
[x] Visual testing (please attach a screenshot or recording)
[ ] Other (please describe below)
[ ] No tests are needed

Reviews
In addition to engineering reviews, this PR needs:

[x] UX review
[x] Accessibility review
[ ] Functional review

Additional Details

This PR resolves:
UXPLATFORM-9141

Thank you for contributing to Terra.
@cerner/terra

Tested this with Microsoft Edge and it looked good to me.  Also did a quick test with JAWS to confirm nothing changed.  +1.  @eawww  once you've had a chance to test with your Mac and if it looks good please update the Accessibility label as well as the UX one. Thanks!

@artpark - It does not appear that you are covering the use case where the user clicks or presses space while on a cell.  In that scenario, my understanding is that we will preserve the styling that is used today.  The keyboard focus will move with the navigation.  However, the active selection state does not change.  There do not appear to be screen shots to show this behavior.

@artpark - With this update, we would never have our blue background when clicking a cell, correct?  Would we want to merge this state?

@artpark - With this update, we would never have our blue background when clicking a cell, correct? Would we want to merge this state?

I think it's alright since https://github.com/cerner/terra-framework/pull/1676 should be adding the blue background, as well as the cell selected functionality.

The focus styling looks good. I tested with Chrome, Edge and Safari. I also did some screen reader testing to see how it looked with the screen reader. JAWS worked as expected. With VoiceOver, the dashed border displayed when a cell had focus. However, VoiceOver is not announcing row and column headers as I navigate the table cells. Not a show stopper, but something we need to address in the future.

Looks great in the body.
I'm noticing with the header row cells the focus indicator is getting chopped a bit. ☹


Add ARM64 support

Fixes #20

We should probably only release linux/arm64.
People ask this for RPIs mostly. So maybe we can list linux/arm64 as an exception to the matrix.

I've explicitely ignored the windows and darwin builds for arm64, but it seems to be the case by default. Here is what is printed in debug output of goreleaser release:
      • BUILDING BINARIES
         • skipped invalid build     target=darwin_arm64
         • skipped invalid build     target=windows_arm64

Ah if that's the default we don't need to ignore I suppose. But verbose is ok.


investigate deno regression causing examples to fail

#890

https://github.com/denoland/deno/issues/18740


fixing operation name

@SeanFeldman : Thanks for your contribution! The author, @cgillum, has been notified to review your proposed change.

@cgillum bot says you're to be pinged 🙂

#sign-off


Hovering themes/tags causes wrap behavior

On themes and tags page, hovering on themes and tags causes the divs to wrap, making them unclickable

fixed by adding an empty div to the end of the row of tags that disappears upon hovering on any of the tabs... however this causes some weird behavior when you hover on the empty div area


Reconfirm RoadRunner tuning of new bots

GLaDOS and Wheatley may need retuning of RoadRunner as the weight/center of gravity has been changed

Wheatley bot does not need a retune, this issue has been moved for GLaDOS who has had a major design change


fix python api && dockerfile && imdb benchmark

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.MRXLT seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


[Networking] Configuration of Cisco devices using Telnet and SSH

Proposed title of article
Configuration of Cisco devices using Telnet and SSH
Proposed article introduction
Most network administrators are used to configuring devices physically either using the console management port. With the need for connectivity, there was a need for a much easier way to configure Cisco devices without having to be physically present. Telnet and SSH provided a solution to this problem by establishing a session with the Cisco devices remotely. In this article, we will take a look at both Telnet and SSH and determine which is the better option when configuring Cisco devices. The reader will be able to configure the Telnet and SSH server on Cisco IOS router or switch and use a client to establish a session for configuration.
Key takeaways
At the end of the article, the reader will know:

What is telnet
What is SSH
The advantages and disadvantages of telnet and SSH
How to enable telnet
How to configure devices using SSH
Troubleshooting

Article quality
This article will help the reader determine which is the best option for him/her when it comes to configuring Cisco devices. It will provide a step-by-step procedure of how to configure cisco devices using telnet and SSH with a simple network topology. The reader will also be able to troubleshoot and know the important things to take into consideration when configuring Telnet and SSH.
References
N/A

I think you received a feedback here as regards to this topic suggestion https://github.com/section-engineering-education/engineering-education/issues/4649


Upgrade versions-maven-plugin 2.17.1 -> 2.18.0

This PR contains the following updates:

Package
Type
Update
Change

versions-maven-plugin (source)
build
minor
2.17.1 -> 2.18.0

Release Notes

mojohaus/versions-maven-plugin (versions-maven-plugin)
v2.18.0
Compare Source
❗ NOTICE
Due to Doxia 2.x stack maven-site-plugin 3.20+ is required.
https://cwiki.apache.org/confluence/display/MAVEN/Towards+Doxia+2.0.0+Stack
Changes
🚀 New features and improvements

Upgrade to Doxia 2.x stack (#1180) @slawekjaranowski
Remove implementation of dynamic version goal (#1179) @slawekjaranowski
fix: Removed non-fully functional dynamic version goal (#1172) @jimisola
Resolves #1131: Suppress logging when allowMajorUpdates, allowMinorUp… (#1132) @gmshake
Move expression: {blah} no value System.out to logs (#1119) @JackPGreen
Improve speed of versions:resolve-ranges in case parents contain many version properties (#1122) @m-schutte-ohra-nl

🐛 Bug Fixes

Resolves #1060: Bugfix in UpdateParentMojo + add tests (#1181) @andrzejj0
Resolves#1066: Fixed a possible NPE in interpolateVersion (#1166) @andrzejj0
Resolves #1067: Bug fix: incorrect arguments of PomHelper::createProjectBuilderRequest (#1164) @andrzejj0
Resolves #1137: using PomHelper.getGroupId if groupId of the current project root is empty (#1162) @andrzejj0
bugix/ protect internal data in multi thread env. (#1133) @jgraglia
Fixed #1123 - Missing link in examples to recoding changes and use dy… (#1124) @khmarbaise

📦 Dependency updates

Upgrade to Doxia 2.x stack (#1180) @slawekjaranowski
Bump byteBuddyVersion from 1.15.7 to 1.15.10 (#1173) @dependabot
Bump com.fasterxml.woodstox:woodstox-core from 7.0.0 to 7.1.0 (#1171) @dependabot
Bump org.codehaus.groovy:groovy-all from 2.0.0 to 2.4.21 in /versions-maven-plugin/src/it/it-property-updates-report-002-slow (#1161) @dependabot
Bump byteBuddyVersion from 1.15.5 to 1.15.7 (#1163) @dependabot
Bump byteBuddyVersion from 1.15.4 to 1.15.5 (#1160) @dependabot
Bump byteBuddyVersion from 1.15.3 to 1.15.4 (#1156) @dependabot
Bump org.codehaus.mojo:mojo-parent from 85 to 86 (#1139) @dependabot
Bump byteBuddyVersion from 1.15.1 to 1.15.3 (#1151) @dependabot
Bump org.codehaus.plexus:plexus-utils from 4.0.1 to 4.0.2 (#1148) @dependabot
Bump commons-io:commons-io from 2.16.1 to 2.17.0 (#1143) @dependabot
Bump byteBuddyVersion from 1.15.0 to 1.15.1 (#1135) @dependabot
Bump org.apache.commons:commons-lang3 from 3.16.0 to 3.17.0 (#1136) @dependabot
Bump byteBuddyVersion from 1.14.19 to 1.15.0 (#1134) @dependabot
Bump byteBuddyVersion from 1.14.18 to 1.14.19 (#1129) @dependabot
Bump org.codehaus.mojo:mojo-parent from 84 to 85 (#1128) @dependabot
Bump org.apache.commons:commons-lang3 from 3.15.0 to 3.16.0 (#1126) @dependabot
Bump org.hamcrest:hamcrest from 2.2 to 3.0 (#1125) @dependabot
Bump commons-codec:commons-codec from 1.17.0 to 1.17.1 (#1117) @dependabot
Bump org.apache.commons:commons-lang3 from 3.14.0 to 3.15.0 (#1118) @dependabot

📝 Documentation updates

Fix #1144 / appends .+ to .+-(alpha|beta) (#1145) @onacit

👻 Maintenance

Drop Maven Wrapper from project (#1178) @slawekjaranowski
Bump project version to 2.18.0-SNAPSHOT (#1177) @slawekjaranowski
Move to modello-stax (#1176) @andrzejj0
Using Stax2 (Woodstox) instead of plexus-xml (Xpp) to process XML (#1174) @andrzejj0
Resolves#1169: Clean-up, remove deprecated methods, remove Plexus StringUtils in favour of Apache Commons (#1170) @andrzejj0
Update site descriptor to 2.x (#1167) @slawekjaranowski
Disable ITs with not set CI Friendly with newer Maven version (#1152) @slawekjaranowski

🔧 Build

Use shared release drafter GH Action (#1127) @slawekjaranowski

[ ] If you want to rebase/retry this PR, check this box

Suggested commit message:
Upgrade versions-maven-plugin 2.17.1 -> 2.18.0

See:
- https://github.com/mojohaus/versions/releases/tag/2.18.0
- https://github.com/mojohaus/versions-maven-plugin/compare/2.17.1...2.18.0


Demote some Docker e2e tests to flaky as per #7840

Demote some Docker e2e tests to flaky as per #7840
@yujuhong @ixdy for review and merge

Shippable and Travis are flakes.  I think this PR is good to go.

Needs a rebase. Also conflicts with #10810

Replaced by #10904


unbound request

more of a feature request than an issue, but could we get unbound integration into pihole for recursive dns. I'm still dabbling in to docker so didn't really delve into integrating them with the internet monitor docker-compose but i think it would be really a comprehensive solution for running the full suite of pihole with unbound and getting that just tiny bit more speed wrt dns query resolution.

It is simple. The unbound guide you are probably following expects that both pi-hole and unbound will be running on the same host and makes you use <IP_ADDRESS> as a DNS server in Pi-Hole. Wont work since hole is in a container and unbound is not. Just configure unbound to listen to your host's IP (for example) and then tell pi-hole to use the same IP as a DNS server.

Looking into using https://github.com/chriscrowe/docker-pihole-unbound/tree/master/one-container for this


We2_iic_bootloader_recover doesnot compile

Describe the bug
I need to do bootloader recover as the vision v2 doesnot seem to be flashing from sensecraft AI
But the bootloader recover file doesnot compile throws all sorts of errors
To Reproduce
Steps to reproduce the behavior:

Go to 'Arduino IDE-> SEEED STUDIO SSCMA-> We2_iic_bootloader_recover '
Click on 'verify'
Scroll down to 'bottom on output console'
See error

Expected behavior
i get this error
Compilation error: call of overloaded 'requestFrom(uint8_t&, size_t, bool)' is ambiguous
Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

OS: Windows
Browser Chrome
Version (device version) 2023.12.14

Smartphone (please complete the following information):

Device: [e.g. iPhone6]
OS: [e.g. iOS8.1]
Browser [e.g. stock browser, safari]
Version [e.g. 22]

Additional context
Add any other context about the problem here.

Works only with ESP32 boards not with Arduino ATMEGA 328p boards


Improved regex handling.

Parse Haxe regex literals correctly.
Take into account regex literals can follow an object literal property.
Take into account regexes in FRSearch.

It could cause some (minor) completion issues but more importantly, it's responsible for this:
https://twitter.com/_bigp/status/594314905878134786
EDIT: Surely regex parsing should take into account the possible flags as well.

Modified FRSearch to get the source file name and use it to check for regex literals.

Forgot to add that the latest commit also checks for regex flags.

Fair enough


convert main function to immediately invoked function expression

because why not?

Thanks for the PR but this code change is not an improvement.


build: Replace GCC ARM Embedded with GNU Arm Embedded

The old GCC ARM Embedded website on launchpad
(https://launchpad.net/gcc-arm-embedded) has been superseeded by the new
GNU Arm Embedded one
(https://developer.arm.com/open-source/gnu-toolchain/gnu-rm).
This also means a change of name from "GCC" to "GNU". Reflect this in
the enviroment variables so that the proper term is used henceforth.
Signed-off-by: Carles Cufi<EMAIL_ADDRESS>

recheck

@dbkinder @nashif CI fails due to the amount of boards I changed, times out since it would take a long time to run all the tests on them.

@SebastianBoe mind taking a look at this?

Merging with CI failing due to the amount of boards this change affects, which makes sanitycheck timeout

@MaureenHelm will do thanks!


Use controller manager sync rather than re-queue

@richardcase suggested using the controller manager sync capability rather than re-queuing custom resources.
This will mean the interval field in the spec is not required and the sync period will be set at controller runtime.
This change will mean every instance of the AddonsLayer custom resources will be processed by the controller every 'sync period'

https://github.com/fidelity/kraan/pull/67


Simplify information fragment layout

Using ConstraintLayout for the "App information" list makes the layout unnecessarily complicated because each row needs to reference the previous one. To avoid that, I recommend using a vertical LinearLayout like in this PR. It probably also increases performance by a minor bit.
The appearance of the layout should be the same:

Can you target branch release/1.9.x with your PR?

I don't think I can do that without opening a new PR, can I?

Oh, I can, I see. I rebased my branch onto release/1.9.x as well.

Your PR currently contains unrelated changes.

Sorry, I didn't notice. I'm not sure how that happened. I removed the unrelated commit that I accidentally picked up somewhere.

Except for the new padding looks as intended, the padding isn't too bad either, up to you @harambasicluka


Slider selects text

as shown here: https://github.com/kolibril13/jupyter-splitview/pull/24

@kolibril13, this seems to be an issue with Jupyter Lab only. This, #19 and #20 seem to only exist in Jupyter Lab. Do you have any explanation for this?

I've tried it again, but still same issue.
But should the submodule version not be updated in the pull request?
Currently it is

but there are newer versions available:

@kolibril13, this seems to be an issue with Jupyter Lab only. This, https://github.com/Octoframes/compare_view/issues/19 and https://github.com/Octoframes/compare_view/issues/20 seem to only exist in Jupyter Lab. Do you have any explanation for this?

I've never seen this before, and I don't have an idea what might cause it.

Sorry, my bad.

Just tried again, still same error.

Odd, I don't have this issue...
I only tried this on Firefox—what browsers did you test?
EDIT: This is also an issue on Edge.

I tested it now in the following browsers:
Google Chrome: Bug present
Safari: No bug
Firefox: No bug
Additional note: VS Code Notebook: Widget is not showing at all.

I think I solved it; could you try again, please?

...

Thanks!


aligning code samples and including cpp sample in build-a-test-app.md

Look ok, @andreimatei and @knz? Andrei, we should put comments in basic-sample.cpp as well. Can you add that to this PR as well?

This change is

Jesse, what am I supposed to add?

Review status: 0 of 6 files reviewed at latest revision, all discussions resolved.

Comments from the review on Reviewable.io

Andrei, I just added comments to basic-sample.cpp. Can you check those? Do we need a comment explaining pqxx::nontransaction w(c);? And where's the right place to put the "Close the database connection" comment?

Comments from the review on Reviewable.io

Reviewed 4 of 6 files at r1, 2 of 2 files at r2.
Review status: all files reviewed at latest revision, 1 unresolved discussion.

_includes/app/basic-sample.cpp, line 10 [r2] (raw file):
No that comment is not accurate. There's no "importing of the driver" to speak of in C++.

Comments from the review on Reviewable.io

Beware that this is likely to merge conflict with #162. Which one do you want to get in first?

comments in basic-sample.cpp look good, minus the one on the import.
I don't think we need a comment on nontransaction.
And there's no good place for the "close connection" comment. The connection gets closed automatically when the pqxx::connection object goes out of scope. It's what you'd expect in C++. Oh how I miss this sweet language.
I thing these examples generally need some experience with the specific drivers. If the user doesn't have it, she'll have to read a bit on the driver first. I think that's unavoidable.

Review status: all files reviewed at latest revision, 1 unresolved discussion, some commit checks failed.

_includes/app/basic-sample.cpp, line 10 [r2] (raw file):
yeah, this line doesn't need a comment.

Comments from the review on Reviewable.io

Going to update from master, resolve conflicts, and the merge.

Review status: 5 of 6 files reviewed at latest revision, 1 unresolved discussion, some commit checks failed.

_includes/app/basic-sample.cpp, line 10 [r2] (raw file):
Sorry about that. Remove the comment.

Comments from the review on Reviewable.io


Is it possible to hide endpoints in Swagger based on IP Address?

I have a setup that requires that certain endpoints of the generated Swagger.json are hidden based on the requester's IP address.
The best place for that logic would be in my opinion a DocumentFilter that is applied during AddSwaggerGen() processing and called each time the swagger.json is created.
My problem is that it is not possible to add DocumentFilters that require the injection of external dependencies via constructor. As I am aware I can add a new DocumentFilter like
options.DocumentFilter<CustomSwaggerDocFilter>();
to the process and I could also pass statically parameters to that filter that will be forwarded to the constructor.
Does it therefore make sense to create a feature request for this or is there a better way to achieve my objective?
Thanks in advance!

From the readme:

NOTE: Filter pipelines are DI-aware. That is, you can create filters with constructor parameters and if the parameter types are registered with the DI framework, they'll be automatically injected when the filters are instantiated.

So, you can register the filter as you've done above, and if it has a constructor with parameters, they will automatically be retreived from DI at the time of instantiation. With that said, I'm not sure if you can inject something within the request scope - I'd be interested to know if you try this out.
Regardless, if you want to modify the doc with request context, you can do it with a PreSerializeFilter as described in the readme:
https://github.com/domaindrivendev/Swashbuckle.AspNetCore#modify-swagger-with-request-context


Saving for QC and results plots.

Some functions (e.g., cell2location.utils.filtering.filter_genes or RegressionModel.plot_QC) generate plots, but have no way (that I could think of) for saving the plots from a script. Workarounds don't always work, for example the code below only saves one of the two plots generated when interactively running the code.
fig, ax = plt.subplots()
mod.plot_QC()
# only saves first plot unfortunately
fig.savefig(QC_dir / 'QC_plots_signature_model.png', dpi=300)

Since sometimes running cell2loc interactively is not practical, it would be great to have improved plot saving capabilities! Thank you for your great work!

Thanks for using cell2location @bednarsky! We indeed also often use cell2location non-interactively and in general, save a range of plots. Is mod.plot_QC() and cell2location.utils.filtering.filter_genes the only methods that have this issue?
@yozhikoff what do you think is the best interface for saving plots? File path argument to the function or just making sure that mod.plot_QC() creates only one plot?

Cell2location plot_QC() should be save-able:
if True:
    mod.plot_QC()
    plt.savefig(f"{scvi_run_name}/reconstruction_accuracy_histogram.png",
                bbox_inches='tight')
    plt.close()

We also often look at these additional QC metrics:
if True:
    fig_dir = f"{scvi_run_name}/spatial/"
    import os
    if not os.path.exists(fig_dir):
        os.mkdir(fig_dir)
    adata_vis.obs['total_cell_abundance'] = adata_vis.uns['mod']['post_sample_means']['w_sf'].sum(1).flatten()
    fig = plot_spatial_per_cell_type(adata_vis, cell_type='total_cell_abundance', prefix='');
    fig.savefig(f"{fig_dir}total_cell_abundance.png", bbox_inches='tight')
    fig.clear()
    plt.close(fig)

    adata_vis.obs['detection_y_s'] = adata_vis.uns['mod']['post_sample_q05']['detection_y_s']
    fig = plot_spatial_per_cell_type(adata_vis, cell_type='detection_y_s', prefix='');
    fig.savefig(f"{fig_dir}detection_y_s.png", bbox_inches='tight')
    fig.clear()
    plt.close(fig)

    fig = plot_spatial_per_cell_type(adata_vis, cell_type='total_counts', prefix='');
    fig.savefig(f"{fig_dir}total_RNA_counts.png", bbox_inches='tight')
    fig.clear()
    plt.close(fig)

which requires a custom plotter that visualizes the same obs column for all visium sections:
def plot_spatial_per_cell_type(adata,
                                   cell_type='total_counts',
                                   samples=adata.obs['sample'].cat.categories,
                                  ncol=3, prefix='', figsize=(24, 6), vmax_quantile=0.992):
    n_samples = len(samples)
    nrow = int(np.ceil(n_samples / ncol))
    fig, axs = plt.subplots(nrow, ncol, figsize=figsize)
    if nrow == 1:
        axs = axs.reshape((1, ncol))

    col_name = f'{prefix}{cell_type}'
    vmax = np.quantile(adata.obs[col_name].values, vmax_quantile)
    adata.obs[cell_type] = adata.obs[col_name].copy()

    from itertools import chain
    ind = list(chain(*[[(i, j) for i in range(nrow)] for j in range(ncol)]))

    for i, s in enumerate(samples):
        sp_data_s = select_slide(adata, s, s_col='sample')
        sc.pl.spatial(sp_data_s, cmap='magma',
                      color=cell_type,
                      size=1.3, img_key='hires', alpha_img=1,
                      vmin=0, vmax=vmax, ax=axs[ind[i][0],ind[i][1]], show=False
                                            )
        axs[ind[i][0],ind[i][1]].title.set_text(cell_type+'\n'+s)

    if len(samples) < len(axs.flatten()):
        for i in range(len(samples), len(axs.flatten())):
            axs[ind[i][0],ind[i][1]].axis('off')

    fig.tight_layout(pad=0.5)

    return fig

plot_spatial_per_cell_type(adata, cell_type='total_counts');

Also total cell abundance histograms per sample:
if True:
    # look at the total cell abundance for each experiment
    with mpl.rc_context({'axes.facecolor': 'white', 'font.size': 10,
                         'figure.figsize': [5, 15]}):
        for i, n in enumerate(adata_vis.obs['sample'].unique()):
            plt.subplot(len(adata_vis.obs['sample'].unique()), 1, i+1)
            total_cell_abundance = adata_vis.uns['mod']['post_sample_means']['w_sf'].sum(1).flatten()
            total_cell_abundance_ = total_cell_abundance[adata_vis.obs['sample'] == n]
            plt.hist(total_cell_abundance_, bins=100);
            plt.title(n)
            plt.xlabel(r'Total cell abundance ($sum_f w_sf$) for each experiment')
            plt.xlim(0, total_cell_abundance.max())
        plt.savefig(f"{scvi_run_name}/total_cell_abundance.png",
                    bbox_inches='tight')
        plt.close()

Finally, this is how you can save plots for A) one cell type in all samples and B) all cell types in one sample:
if True:
    # add 5% quantile, representing confident cell abundance, 'at least this amount is present',
    # to adata.obs with nice names for plotting
    adata_vis.obs[adata_vis.uns['mod']['factor_names']] = adata_vis.obsm['q05_cell_abundance_w_sf']

    from re import sub
    import os
    for c in adata_vis.uns['mod']['factor_names']:
        fig = plot_spatial_per_cell_type(adata_vis, cell_type=c)
        fig_dir = f"{scvi_run_name}/spatial/"
        if not os.path.exists(fig_dir):
            os.mkdir(fig_dir)
        fig_dir = f"{scvi_run_name}/spatial/per_cell_type/"
        if not os.path.exists(fig_dir):
            os.mkdir(fig_dir)
        fig.savefig(f"{fig_dir}W_cell_abundance_q05_{sub('/', '_', c)}.png",
                    bbox_inches='tight')
        fig.clear()
        plt.close(fig)

    f"{scvi_run_name}/spatial/per_cell_type/"

    with mpl.rc_context({"axes.facecolor": "black"}):
        clust_names = adata_vis.uns['mod']['factor_names']

        for s in adata_vis.obs['sample'].unique():

            s_ind = adata_vis.obs['sample'] == s
            s_keys = list(adata_vis.uns['spatial'].keys())
            s_spatial = np.array(s_keys)[[s in i for i in s_keys]][0]

            fig = sc.pl.spatial(adata_vis[s_ind, :], cmap='magma',
                                color=clust_names, ncols=5, library_id=s_spatial,
                                size=1.3, img_key='hires', alpha_img=1,
                                vmin=0, vmax='p99.2',
                                return_fig=True, show=False)

            fig_dir = f"{scvi_run_name}/spatial/"
            if not os.path.exists(fig_dir):
                os.mkdir(fig_dir)
            fig_dir = f"{scvi_run_name}/spatial/per_sample/"
            if not os.path.exists(fig_dir):
                os.mkdir(fig_dir)

            plt.savefig(f"{fig_dir}W_cell_abundance_q05_{s}.png",
                        bbox_inches='tight')
            plt.close()

Also adding the history plots for completeness:
# plot ELBO loss history during training, removing first 100 epochs from the plot
mod.plot_history(5000)
plt.savefig(f"{scvi_run_name}/training_ELBO_history_minus5k.png",
                   bbox_inches='tight')
plt.close()
mod.plot_history(0)
plt.savefig(f"{scvi_run_name}/training_ELBO_history_all.png",
                   bbox_inches='tight')
plt.close()

@yozhikoff what do you think is the best interface for saving plots? File path argument to the function or just making sure that mod.plot_QC() creates only one plot?

@vitkl
I would just return a list of Figure objects, so that the user could do whatever is needed - save, modify, add something else.

hi ! up :)

@yozhikoff what do you think is the best interface for saving plots? File path argument to the function or just making sure that mod.plot_QC() creates only one plot?

@vitkl I would just return a list of Figure objects, so that the user could do whatever is needed - save, modify, add something else.

I am using the mod.plot_QC(), which produces two graphs simultaneously. I wish I can save those two graphs to the files.

Cell2location plot_QC() should be save-able:
if True:
    mod.plot_QC()
    plt.savefig(f"{scvi_run_name}/reconstruction_accuracy_histogram.png",
                bbox_inches='tight')
    plt.close()

When I run this, I end up with a blank png, which I believe is because the function mod.plot_QC() has plt.show(), resulting in a blank figure being saved (running plt.savefig after plt.show). Is there a way to save figures with the current functions?

Could you make a PR suggesting specific changes?
On Thu, 22 Aug 2024 at 19:06, Science Peak @.***> wrote:

Cell2location plot_QC() should be save-able:
if True:
mod.plot_QC()
plt.savefig(f"{scvi_run_name}/reconstruction_accuracy_histogram.png",
bbox_inches='tight')
plt.close()
When I run this, I end up with a blank png, which I believe is because the
function mod.plot_QC() has plt.show(), resulting in a blank figure being
saved (running plt.savefig after plt.show). Is there a way to save figures
with the current functions?
Hi,
Inspired by your comments. I found that when I specify show=False in the
sc.pl.spatial() function, I can draw the graph to the hard drive. Without
the show=False, I just got a blank file on the hard drive, which means I
have to manually interactively export or save the graph to the hard drive.
Showing the figures by default is a problem for programming style coding
that handles many samples running in the back end of the computer. I still
can not figure out the mod.plot_QC(), which lacks the parameter of
show=False and draws two figures at the same time.
—
Reply to this email directly, view it on GitHub
https://github.com/BayraktarLab/cell2location/issues/191#issuecomment-2305351712,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AFMFTV4FCYZVIJCZG24XJG3ZSYSDNAVCNFSM6AAAAABMSYIRB2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMBVGM2TCNZRGI
.
You are receiving this because you were mentioned.Message ID:
@.***>

Thanks for using cell2location @bednarsky! We indeed also often use cell2location non-interactively and in general, save a range of plots. Is mod.plot_QC() and cell2location.utils.filtering.filter_genes the only methods that have this issue?
@yozhikoff what do you think is the best interface for saving plots? File path argument to the function or just making sure that mod.plot_QC() creates only one plot?

One function creates only one plot would be a better solution. If two plots are needed, perhaps it is good to split the function into two functions with similar names.
Adding an parameter show=False to the function would be a better solution. If you could point out where the source code and documentation of mod.plot_QC() is, I would like to take a look, though my Python skill on cell2location might not be good enough to submit a PR.
If the two "better" solution is realized at the same time, it would be a perfect solution for needs of programmers, who can not use interactive notebook to handle many samples.

Cell2location plot_QC() should be save-able:
if True:
    mod.plot_QC()
    plt.savefig(f"{scvi_run_name}/reconstruction_accuracy_histogram.png",
                bbox_inches='tight')
    plt.close()

Hello! I still do not know how to save the 2 plots that are generated by mod.plot_QC(). It only works if the output is one plot (e.g. for the spatial model). How can this be adressed? It would be very helpful to include show and save options. Even the workaround with matplotlib doesn't work like expected for this function since 2 figures are created.


Refactor README.md to improve project overview and add detailed table…

@Durgesh4993 merge my code.

@Durgesh4993 merge code..


Incorrect namespace in RPC documentation for get_activated_protocol_features

The namespace of get_activated_protocol_features RPC is chain, but its documentation on Developer Portal is introduced under producer namespace.
https://developers.eos.io/eosio-nodeos/reference#get_activated_protocol_features

@dskvr

in the latest documentation the get_activated_protocol_features is under chain namespace:
see here:
https://developers.eos.io/manuals/eos/latest/nodeos/plugins/chain_api_plugin/api-reference/index#operation/get_activated_protocol_features

if it that is alright with you we shall close this one. @conr2d


https://nim-lang.org/choosenim/init.sh is outdated

https://nim-lang.org/choosenim/init.sh needs to be updated to the latest version in the repo. It is missing the latest changes in commit ac77dc5 "supports arm64 on macOS" .
This means that on macOS arm64 it still downloads the amd64 choosenim binary.
See also: https://github.com/dom96/choosenim/issues/347

Well, it has been linked to https://github.com/nim-lang/choosenim/blob/master/scripts/choosenim-unix-init.sh in the server

When I download the file (with curl -O) , I still get the version, which has this on line 127ff.:
      if [ "$HOSTTYPE" = "arm64" ] ; then
        # TODO: We don't have arm binaries for choosenim so far, rely on Rosetta.
        local ucpu="amd64"
      fi

which is the previous version.

@SpotlightKid Sorry for the inconvenience. We have had fixed it on the server. Does it work for you now?

Yes, an up-to-date init.sh is downloaded now. I don't have macOS so I can't test whether it downloads the right binary for macOS arm64 now, but it should.

https://github.com/nim-lang/choosenim/pull/55 will be added to ensure it works on macosx_arm


Adds extra closing tags when closing tag is inside go block

with
<span>{{ define whatever }}
</span>
{{ end }}

as input<EMAIL_ADDRESS>outputs:
<span><span>{{ define whatever }}
</span>
{{ end }}
 </span>

adding additional <span> tags
It doesn't error and return the original text like the invalid test case does, I think it should probably do so, since I don't see a way to reasonably format it

This might be a duplicate of #59?

Yes, exact duplicate of #59.


Refactoring title generation

Fixes #330
Removing PageTitleSeparator from settings
Using " - " by default
Possibility to define in views

Yes, the sitename is in my blogs branch, will port it.


We should use EL to evaluate variables within SCXML models

https://docs.oracle.com/javaee/7/api/javax/el/package-summary.html
EL is a Java standard for expressions. We already use EL to evaluate conditions within the SCXML Engine. Can we expand this support so that some of these features are enabled via EL instead of custom syntax?

Variable values (can variable values be assigned via expressions?)
Macros (can macros become functions which the user can call in an expression?)
Complex data types

EL would come with its own parsing and syntax as well.

Some additional thoughts:

This would be a significant (breaking) change for existing code. Is there any chance for backward compatibility?
What is the performance overhead of using EL?
There is still value in having Java as the native/basic language for extension.
Can we opt-in to pre-processing?

@mpeter28 thinking about how to express constraints. Add that to the additional thoughts at the top. We will decide later on the final direction.

Being replaced with 3.0 direction to use spark. @bryantrobbins ok with that ?


fix draft_curation_request bug

During the go script work, I realized users could still publish after requesting draft curation. By setting draft_curation_requested to false in the ensure block, it could never update to true & buttons for publish & request curation were always rendered. If I remember correctly, the initial idea for having it there was for error handling, but I think rescuing the CurationError does an adequate job of handling that scenario.

It might be a good idea to test that the boolean gets set properly in here: https://github.com/psu-libraries/scholarsphere/blob/main/spec/features/dashboard/work_version_form_spec.rb#L1148-L1158 . There's a lot of logic in this controller, so we should try to cover as much of it as possible somewhere.

+1


Solidifier Item Pushing Bug

When solidifiers push items to a chest, they're ghost items with incorrect amounts (not really there) and they seem to bug out and not stack?

Recent fix should resolve this.


fixed different memory issues in code resolves# 10

this pr only works in tandem with the async_op_loading pr that is currently open in teraslice

When merged, this PR closes #10

The changes in this PR require the following teraslice changes to also be accepted:
https://github.com/terascope/teraslice/pull/500