Change in Waze's Data

It appears as though Waze has, at some point, changed their formatting. The trip data is now kept in the second file instead of the third one. Even so, when I attempt to use it, the GPX data thinks I was in Antarctica for some reason.

@TheDravenator thanks for the report. Indeed, compared to data i exported in August - format has changed slightly. Looks like an easy fix though.
However, in my case all data is still in account_activity_3.csv file.
-a----        20.09.2024     01:22              0 account_activity_1.csv
-a----        20.09.2024     01:22              0 account_activity_2.csv
-a----        20.09.2024     01:22        1102108 account_activity_3.csv
-a----        20.09.2024     01:22            690 account_info.csv
-a----        20.09.2024     01:22        2212800 image_2_1.png

I'd appreciate if you share your data export - my email can be found on GitHub profile

@TheDravenator  Added support for new coordinate format in ba14d0fcde62ab08ac0b12c2088c722babc6e9d3
As for data in files other than account_activity_3.csv - i cannot reproduce with my Waze account. I need someone to share an archive i can investigate.

Solved the problem of figuring out which of the account_activity_* files contains data by implementing multiple file upload. Just upload all CSV files and the app will extract trips from all of them.
7a72fbb3c03e297cf89fcc77a739e15151008bea


The sound is not reproduced through the jack, but through the HDMI

The sound is not reproduced through the jack, but through the HDMI
https://pastebin.com/Exnt3WHZ
`
/etc/default/raspotify -- Arguments/configuration for librespot
Device name on Spotify Connect
#DEVICE_NAME="EspejoDAD"
Bitrate, one of 96 (low quality), 160 (default quality), or 320 (high quality)
#BITRATE="160"
Additional command line arguments for librespot can be set below.
See librespot -h for more info. Make sure whatever arguments you specify
aren't already covered by other variables in this file. (See the daemon's
config at /lib/systemd/system/raspotify.service for more technical details.)

To make your device visible on Spotify Connect across the Internet add your
username and password which can be set via "Set device password", on your
account settings, use --username and --password.

To choose a different output device (ie a USB audio dongle or HDMI audio out),
use --device with something like --device hw:0,1. Your mileage may vary.
OPTIONS="--device hw:0"
Uncomment to use a cache for downloaded audio files. Cache is disabled by
default. It's best to leave this as-is if you want to use it, since
permissions are properly set on the directory `/var/cache/raspotify'.
#CACHE_ARGS="--cache /var/cache/raspotify"
By default, the volume normalization is enabled, add alternative volume
arguments here if you'd like, but these should be fine.
#VOLUME_ARGS="--enable-volume-normalisation --linear-volume --initial-volume=10$
Backend could be set to pipe here, but it's for very advanced use cases of
librespot, so you shouldn't need to change this under normal circumstances.
#BACKEND_ARGS="--backend alsa"`
Thanksss :D

Please see the troubleshooting section of the wiki:
https://github.com/dtcooper/raspotify/wiki/Troubleshooting


Extension in-app page width

New extension page within the app is way too narrow. This extremely tunneling view brings headache and nausea. It needs to be full width and without large fixed elements.

https://github.com/microsoft/vscode/commit/d2ed4c8213f26306b5c7b7dbe693e2ca1b854d55 seems to have added this:
https://github.com/microsoft/vscode/blob/cdd975671e18fe63b9747b2a506baffd624637dc/src/vs/workbench/contrib/extensions/browser/media/extensionEditor.css#L12
On my 2560x1440 125%-scaled display using current Insiders:

/confirm
After using Devtools to disable that, then swapping to Changelog tab and back to Details (to redraw the webview):

This reinstates the 882px max-width that the body of the webview has apparently had since 1.44 (https://github.com/microsoft/vscode/commit/10a794185481c6f4fa0e16145bc5048c6ef652c7)
Between 1.41 and 1.44 it was 780px (https://github.com/microsoft/vscode/commit/b5959b356c5a0fa7d410b70ed26a49fc4619121f)

Removing the 882px constraint I get this:

This change was intentional to align the contents in the center just like other editors like settings editor, markdown preview.,
@misolori any suggestions for max-width for extension editor?

We use 1200px on the main welcome/getting started page but I think I'd be open to increase that for larger screens.

Changed max-width it to 80% of the editor width, so bigger the screen bigger the width will be.

the more, the better


Run data analysis on multiple servers / cluster

See "Auto-scaling scikit-learn with Apache Spark": https://databricks.com/blog/2016/02/08/auto-scaling-scikit-learn-with-apache-spark.html

Also see "Parallel and Large Scale Machine Learning with scikit-learn": https://speakerdeck.com/ogrisel/parallel-and-large-scale-machine-learning-with-scikit-learn


Exception when scrolling text editor with node.js code

Got a random exception while scrolling the text editor with node.js code. I really can't think of any special circumstances that caused this nor can I reproduce. Perhaps the only non-standard condition is that I'm working in a dual monitor setup where my text editor tabs are in a separate window than the IDE main frame.
Here is my AcitivityLog.xml
Running latest build version v1.1.Dev-5.13.2015 (44bd6d6962ab7acb49d1bbe538af1f350edc928f)
VS 2013 Ultimate, win7 x64

The error is coming from the Node.js classifier, which is running in the background... Does it reproduce consistently?

Nope. I thought the log might be enough to find the source and fix. This
happened just once and I use NTVS consistently.
On May 28, 2015 5:17 AM, "Sara Itani"<EMAIL_ADDRESS>wrote:

The error is coming from the Node.js classifier, which is running in the
background... Does it reproduce consistently?
—
Reply to this email directly or view it on GitHub
https://github.com/Microsoft/nodejstools/issues/130#issuecomment-106148640
.

No worries. It should hopefully be enough to find a fix, but it's always good to understand the reason we ended up in that state to begin with.

Thx @syagev for reporting, and thx @jaredpar for fixing it :smiley:. Available in the latest dev build


Can we run a test with a user details for authentication

Hi,
Is there a way to test for the logged in user details.
I have some functionality that requires testing for username and email of the logged in user.
I was using ADAL earlier in which it was fairly simple to test it.

Can you please share the link for the documentation, where I can access the logged in user.

https://github.com/Mimetis/msal-react-tester/blob/1b13d366bb5c9fa509159c7eb35f88449e257fe6/create-react-app-sample/src/pages/HomePage.test.tsx#L43-L61

Thanks @Mimetis


Multiple Accounts

Hi There
Thanks for the great integration
I don't actually have two meters as such but have one account for electricity and one account for my solar FIT that has been added since Octopus took over the bulb
I would like to be able to see the details from the second account also in HA as with my electricity reading
This a completely second account on their system under my login
I would like to be able to see it as a separate set of entities and not an actual meter, I assume as its FIT it will only have meter readings for previous, current, and an amount paid/earned etc
As I have just been moved from bulb to octopus for my fit I don't have anything in the account yet to even see but could try and pull the data if needed
Does that make sense?

I'm afraid this is not supported, and quite low on my list as the integration would require quite the significant re-write. There is already an open issue for this.
I'll therefore close this issue and if you could upvote the linked issue it will help me gauge how many people require the feature.


Implement Waku Filter Protocol

Problem
The Waku Filter protocol allow light clients to save bandwidth but only receiving messages of the target content topic, and not relaying/forwarding messages. See https://rfc.vac.dev/spec/12/ for details.
DoD

[ ] Filter protocol implemented with tests using nim-waku
[ ] Doc published with new example that demonstrate filter used with light push

Effort already made to make it work with go-waku.
Need to test/merge:

https://github.com/status-im/go-waku/pull/241
https://github.com/status-im/js-waku/pull/711

Initial implementation done: https://github.com/status-im/js-waku/pull/762 but only works in NodeJS with a listening node.
Blocked by:

https://github.com/status-im/go-waku/issues/245
https://github.com/status-im/nwaku/issues/948

Initial implementation done: #762 but only works in NodeJS with a listening node.
Blocked by:
* [Filter does not push messages to js-waku in the Browser go-waku#245](https://github.com/waku-org/go-waku/issues/245)

* [Waku Filter In The Browser nwaku#948](https://github.com/waku-org/nwaku/issues/948)

I see both the PRs have been merged. Is the status updated?

Yep done :)


How to cancel future?

Say I have a long-running background calculation, that the user should be able to cancel. The current implementation does not include some communication mechanism for signalling this event. How could such a feature be implemented? I think the following concerns should be addressed:

Signal the background calculation of the cancellation; it would be up to the future to check for this flag and stop further processing.
Signal the promise of some "cancel" or "failure" state; so that any subsequent callbacks also recognize the cancellation of the promise/future.

Since our Future is primarily modeled after Scala's future, my initial reaction is: "What would Scala do?" I don't know, but a quick internet search revealed that Scala's futures are not cancellable (yet) and people miss it (here and here).
I agree that it is practical to be able to cancel a background calculation / future / 'task', I use this with BFTasks. I'll have to think about whether it should be part of Future or if it should be part of a subtype that is more like a 'task', e.g. see ReactKit/SwiftTask.
One related question: what if you cancel a future that is created from an andThen, map and the like. Would it also cancel the original future that it originates from?
I will create an issue here soon that deals with the pure vs practical future (internal) debate that I alluded to in the second paragraph and will certainly consider adding a way to cancel a future.

To answer my own question related to te propagation of cancel: probably not a good idea, see https://github.com/ReactKit/SwiftTask/pull/18.

Having the cancel event propagating up the chain might not be expected indeed. However the cancel event should propagate down the chain; maybe cancelling them as well or put them in the error state.
For example:
A -> B -> C

So when cancelling B; C should be cancelled as well, A remains untouched.

I've had the time to think about this for a while now and I feel like cancelling (and pausing) a task should not be part of a Future. I've started working on a wrapper around NSProgress that provides a way to cancel and pause a task: https://github.com/Thomvis/GoodProgress. There could eventually very well be a third library that combines BrightFutures and GoodProgress and thereby achieve the functionality that you are looking for.


Make the validation of the MID signature certificate optional (or remove it)

There is no more revocation option in the Mobile ID itself (beside if CA will do it).
Therefore we should make it optional and by default "off" or even remove it completely.

Already implmented in v1.1. Setting the parameter DisableSignatureCertValidation to True disable the verification of certificate chain. However, same basic checks of the MID user certificate are still be done, e.g. time validity.


This commit is a test to see the protection state of the branch

You better not accept this PR since it apport nothing to the original project.

I'll answer to the issue and revert my previous commit.


Binance Futures API

I implemented binance futures api according to https://binance-docs.github.io/apidocs/futures/en

Rest API
WebSocket Connection

With a simple examples .

Great!! Many thanks to all of you.
This is a very important work.

I completely exploded code  !
Note : Please merge previous my commit before this .

assets api removed (Not working anymore)
Implemented new BinanceApiConfig (Now support: Spot, Futures, Testnet)
I wrote new example codes
Abstract factory design pattern implemented for choosing true engine (Spot, Futures, Testnet)
Previous example codes updated
and some more improvements ....

Note : Please see my new example codes , you can understand what happened on new version of this library .

after 48h develop and programming :
I refactor whole project (FORCED)
1. new implementation of BinanceApiConfig with ApiConfig annotation
2. implemented BinanceEngineType with types (SPOT, MARGIN, SWAP, FUTURES, TESTNET)
3. BinanceAbstractFactory replaced by old your factory pattern
4. improvement JavaDoc
5. new examples
6. cleanup codes
7. and other improvements ....
Your Welcome .
Thank you :+1:

Binance UserDataStream improved and bug fixed .

I am not able to use this maven library with this dependency its not finding it
<dependency>
  <groupId>com.binance.api</groupId>
  <artifactId>binance-api-client</artifactId>
  <version>1.0.0</version>
</dependency>

version changed to 1.0.2 (not pushed to maven central yet)

I am not able to use this maven library with this dependency its not finding it, and i cant download it
<dependency>
  <groupId>com.binance.api</groupId>
  <artifactId>binance-api-client</artifactId>
  <version>1.0.0</version>
</dependency>

my implementation not merged yet .

Hi, do you have plans to merge this PR?
We need to use the futures API so our current solution would be to fork the repo and merge it on our own, but we'd prefer to not stray too far from the main trunk


Fixes issue #70

Fixes the check in the Position constructor, also includes a simple test.

Nice (particularly for the unit test), thanks.


design change for structured search panels

Changes in this PR:
Update design of structured search panel so that it..

was more easy on the eye, (better visibility of search options).
reduced depth by taking out the form label margin,
retained a light touch of yellow sign post each panel.

Trello card / Rollbar error (etc)
https://trello.com/c/f26ashyA/758-🔍-pui-browse-design-an-alternative-look-and-feel-for-the-advanced-search-form-that-doesnt-use-the-bright-yellow-background
Screenshots of UI changes:
Before

After

[ ] Requires env variable(s) to be updated

Codecov Report

Merging #620 (b89f830) into main (5f03514) will not change coverage.
The diff coverage is n/a.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
@@           Coverage Diff           @@
##             main     #620   +/-   ##
=======================================
  Coverage   84.13%   84.13%
=======================================
  Files          17       17
  Lines         750      750
  Branches       48       48
=======================================
  Hits          631      631
  Misses        104      104
  Partials       15       15

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Handle busy exception on fast/snap WS Download process

PR description
This PR is an attempt to stop the worldstate download pipeline from breaking when RocksDB returns a Status.Busy Exception.
We currently don't handle that scenario and forest does seem to trigger that often.
I suspect Bonsai rarely hits this scenario but it could also leverage from the exception handler.
Fixed Issue(s)

Fixes #5300

Closing PR after getting 100% of instances eventually triggering world state issues


Write a GO program to convert string to pascalcase

Description
Write a GO program to convert string to pascalcase
Pascalcase is a style of writing in which the first letter of each word is capitalized.
Input  : "hello world"
Output : "HelloWorld"

How to contribute

Save the solution in program/convert_string_to_pascalcase.go file
Add convert_string_to_pascalcase.go file in convert-string-to-pascalcase folder

!assign

Hey @MadhuS-1605,
This issue has been assigned to you for more than 15 days.
We are unassigning you from this issue.
If you are still interested in contributing to this issue, please get auto-assigned on the issue by commenting !assign again.
Thanks for your contributions :)


refs #37 SMTP support

Play mailer library is used as MailerService as another implementation of generic MailService interface.
https://github.com/playframework/play-mailer

Thank you. I added configuration switch. In configuration file mail.service.implementation can be mailer or mailgun.


Automatically reload index after running Workspace Sync

Please Select if your Request is Either something new or an Enhancement

[ ] Enhancement of an existing Feature.
[ ] Request of a new feature.

Please select the area your request applys to. (Multiple selections are Possible)

[ ] Onboard. Initial vault setup and import/export pods
[ ] Create. Note creation, lookup, snippets and templates
[ ] Retrieve. Backlinks, references, graph view
[ ] Structure. Refactoring, multi-vault and schemas
[ ] Publish. Sharing your repo with the world
[x] Misc (Choose this if your not sure)

Is your feature request related to a problem? Please describe
After running the Workspace Sync command:

lookup can't find any new notes added by the sync
if a new daily note for today was synced in, running the Create Daily Journal Note command will replace it with a blank daily note
V1 tree view does not update (not sure about V2 tree)

Describe the solution you'd like
Dendron automatically reloads its index after the sync is complete.
Describe alternatives you've considered
Use the ryuta46.multi-command extension to sequentially run dendron.sync and dendron.reloadIndex commands:
// settings
"multiCommand.commands": [
  {
      "command": "multiCommand.dendronSyncAndReload",
      "sequence": ["dendron.sync", "dendron.reloadIndex"]
  }
]

// keybindings.json
  {
      "key": "ctrl+k s",
      "command": "extension.multiCommand.execute",
      "args": { "command": "multiCommand.dendronSyncAndReload" }
  }

Thanks for the suggestion @aleksey-rowan.  In the end state, ideally we can remove the reloadIndex command altogether and have the engine do what it needs to do to properly reflect the workspace state at the right times (but with a better implementation than us just running the reloadIndex command behind the scenes all the time, since this can be slow when the workspace is large :) .
This is particularly painful around workspace sync though, since it's a really common operation.
@Harshita-mindfire - can you take a look at this one later?


Pivot transpose and mode changes

Renames transpose() to pivot_transpose() implements, transpose() and T for table transposition.
Fixes mode value calculation in string types.
Couldn't get the datetime test to work even in the original commits, suspect it has something to do with locales.

Codecov Report

Merging #42 (289696d) into master (596923d) will increase coverage by 0.09%.
The diff coverage is 98.03%.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
@@            Coverage Diff             @@
##           master      #42      +/-   ##
==========================================
+ Coverage   83.06%   83.16%   +0.09%
==========================================
  Files          38       38
  Lines        6851     6897      +46
==========================================
+ Hits         5691     5736      +45
- Misses       1160     1161       +1

Flag
Coverage Δ

pytest
83.16% <98.03%> (+0.09%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

tablite/core.py
73.69% <94.73%> (+0.17%)
:arrow_up:

tablite/utils.py
79.51% <100.00%> (+0.30%)
:arrow_up:

tablite/version.py
100.00% <100.00%> (ø)

tests/test_transpose.py
100.00% <100.00%> (ø)

tests/test_utils.py
100.00% <100.00%> (ø)

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


STORM-1719 Introduce REST API: Topology metric stats for stream

Path: /api/v1/topology/:id/metrics
This API provides detailed metrics for topology

shows metrics per component, which are aggregated by stream

add documentation about new REST API

please refer docs/STORM-UI-REST-API.md for details

This should be reviewed and merged since 1.x version (#1347) of STORM-1719 is already merged.
@harshach
This is a pull request for STORM-1719 against master branch.
I have confirmed that it runs well with 2.0.0 SNAPSHOT.
Please take a look. Thanks!

+1. looks good.


MRG: Refactor coreg transformations and fits

This PR builds on #4966 so that must be merged first.
This PR:

Refactors the transforms to only ever transform the MRI points. be done in the _coreg_gui module instead of the _viewer.py module / visuals.
Speeds plotting / interaction substantially by making transform parameters in a list that can be updated at once (previously, updating 3 rot + 3 trans params caused 6 plot updates, now it causes 1)
Refactors the transforms to respect Neuromag head coordinates (rather than being naison-centric)
Adds weighting factors for nasion/LPA/RPA/EEG/HPI. For example, setting the nasion factor to 100 weights the nasion 100x more than any dig point (i.e., pretty much fixes it in place)
Changes the fitting method from fit_point_cloud-based (which uses nearest calculations on every objective function evaluation) to iterative-closest-point (ICP; computes nearest neighbors and does the fit using those matched pairs). This should replicate what people are presumably more accustomed to from mne_analyze and in the MEG literature. It is fairly quick, so pressing it multiple times to see/watch fit convergence isn't bad and also has some pedagogical value I think.
Adds coordinate frame axes to the display
Make display more compact
Save and restore scene scale factor and size
Adds status bar to monitor changes / convergence
Sets head opaque when editing MRI fiducials, sets back to transparent afterward
Sets background color black by default
Fixes popup parenting (closing window will close options dialogs)
Changes units to most readable / natural (mm, deg, percent)
Allows ICP fitting only if >= 10 points are available
Switches to doing nearest-neighbor calculations using high res surface by using a nearest_calc class-based scheme that is sufficiently fast

Also:

[x] fixes (by disabling) scaling of sphere surfaces
[x] removes Save trans button that was errantly added
[x] does not require MRI fiducials for transforms to take effect
[x] allows weights for EEG electrodes and HPI coils (default to zero)
[x] adds "reset" buttons for scaling and trans
[x] changes "Save subject as..." to "Save as..." for more generic term
[x] allow setting view coordinate frame to head- or MRI-based
[x] fix bug with point orientation updates
[x] fix bug where excluding outliers (exclude 10mm, then exclude 5 mm) causes an error
[x] fix OSX HiDPI size saving
[x] make SCENE_SCALE persist
[x] save scene size
[x] fix bug with saving old source spaces (negative distance)
[x] fix layout on OSX (make left panel scrollable)
[x] fix switching terrain->arcball
[x] add ICP fitting options (n_iterations, convergence criteria)
[x] use high-res mesh for ICP
[x] improve correspondence between mne_analyze and mne coreg ICP
[x] fix scaling single param source space distances
[x] MRI fiducials should be in mm not m
[x] add nearest-fid-mode for ICP
[x] fix double window popup?
[x] see if latest libs avoid warnings
[x] add tests for _DistanceQuery
[x] query and set window size rather than scene size
[x] ensure KIT guis work

Looks really good, I much enjoy the didactic twist of moving the MRI into the head point cloud, completely agree on moving away from nasion-centered transforms. Thank you for finally figuring out the EEG electrode situation (my failed attempt a few years back is still nagging :)
I guess the "Save trans"-button is WIP?

I errantly added that one, actually. The "Save as..." button saves only the trans if you did no scaling, and saves the surrogate + trans if you did scaling. I didn't realize it did the former until I had added the button, then forgot to remove it...

Ah! Suggest going with "Save as..." then, instead of the current "Save Subject As..."; the latter implied to me that a scaled subject would be saved to subjects_dir... Otherwise +1 to have a single button for saving that works in either scenario.
BTW: Any idea how the HPI coil orientation is determined? I happen to get this one funky one on the rhs (looks like it's 90 deg rotated, pretty sure it wasn't during recording, I think I would have felt it!)

It is determined by the orientation of the nearest surface point, in this
case must be on the ear.

Okay @cjayb @christianbrodbeck I think this fixes everything we talked about from the previous PR and adds a bunch of functionality (see top comment). Feel free to give it a try.
@agramfort this might even fix things on your system! A good starting command is:
mne coreg --subjects-dir=~/mne_data/MNE-sample-data/subjects/ \
    --subject sample --fif ~/mne_data/MNE-sample-data/MEG/sample/sample_audvis_raw.fif \
    --trans ~/mne_data/MNE-sample-data/MEG/sample/sample_audvis_raw-trans.fif

I refactored a bunch of display params on Linux and now OSX can be unhappy. I'll set back to WIP and let people know when it's fixed. In the meantime you can still test it.

Some initial thoughts on GUI elements:

I'm not crazy about the drop-down list for fiducials: adds some clicking; IMHO the previous radio buttons were nicer
the Omit-button works, yay!
how's the ICP supposed to work? Is the intention to let it run once per click, and leave it to the user to decide when to stop? It /is/ fast, but... But would you consider running a few iterations based on some stopping criterion (delta-trans or delta-rot)?

Indeed my immediate trouble is with displaying (qt5, pyqt5, vtk8, os x 10.13.3)

initially started the gui (mne coreg ... sample ...) while connected to a large external screen: resizing the window works ok
unplug: window no longer fits on the 13" screen (retina display in the default 1280x800 configuration)

resizing from top corners doesn't work, bottom corners outside of view
going fullscreen doesn't help (hidpi-related?)

close gui, restart it (only main 13" screen attached): window still larger than screen, and additionally the below exception on startup (is there an old config file I need to delete somewhere?)

cjb@hathor $ mne coreg --subject sample --subjects-dir=~/data/templates/MNE-sample-data/subjects --fif ~/data/templates/MNE-sample-data/MEG/sample/sample_audvis_raw.fif --trans ~/data/templates/MNE-sample-data/MEG/sample/sample_audvis_raw-trans.fif
/Users/cjb/miniconda_envs/mne3/lib/python3.6/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  from ._conv import register_converters as _register_converters
    Triangle neighbors and vertex normals...
    Triangle neighbors and vertex normals...
Exception occurred in traits notification handler.
Please check the log file for details.
Exception occurred in traits notification handler for object: <mne.gui._viewer.HeadViewController object at 0x1c2921a570>, trait: interaction, old value: trackball, new value: terrain
Traceback (most recent call last):
  File "/Users/cjb/miniconda_envs/mne3/lib/python3.6/site-packages/traits/trait_notifiers.py", line 538, in _dispatch_change_event
    self.dispatch( handler, *args )
  File "/Users/cjb/miniconda_envs/mne3/lib/python3.6/site-packages/traits/trait_notifiers.py", line 501, in dispatch
    handler( *args )
  File "/Users/cjb/src/git/mne-python/mne/gui/_viewer.py", line 114, in on_set_interaction
    self.scene.interactor.interactor_style
AttributeError: 'NoneType' object has no attribute 'interactor_style'

I'm not crazy about the drop-down list for fiducials: adds some clicking; IMHO the previous radio buttons were nicer

I agree, but it was necessary to save space (shaved over 100 pixels from the left and right columns). The other problem I had before was that LPA/Nasion/RPA lined up with the X/Y/Z, which makes it visually confusing (they seem vertically related but aren't). These two things made it seem worth the switch even for extra clicks.

window no longer fits on the 13" screen

Yes this is a HiDPI resolution problem and on the todo list above. I need to refactor the there panels on the left into a single panel so that it, too, can scroll like the right one.

close gui, restart it

I ended up getting rid of the scene-size-saving step because of some TraitsUI limitations, so this bug is already "fixed". (You'll need to manually set your mne config values to fix it, but users of the final PR won't have that problem.)

AttributeError: 'NoneType' object has no attribute 'interactor_style'

This is a weird one, I don't get this on OSX or Linux. But it should be fixable by delaying the interactor-setting step.

how's the ICP supposed to work? Is the intention to let it run once per click, and leave it to the user to decide when to stop?

Yes, that's more or less how it's done in mne_analyze for example. We could add a number-of-iterations parameter but usually don't see people use it much in mne_analyze. We could try to automate the procedure, too, but I worry about degenerate cases somehow breaking things (e.g., if the dig is bad or points aren't omitted properly and parameters start to explode, especially during rescaling steps), so I was going to omit that for simplicity for now.

I'm not crazy about the drop-down list for fiducials

I agree that the drop-down menu for this seems clumsy

it was necessary to save space

Maybe some less frequently used items like "interaction" and "Display options" could go into a menu bar to save space? I think it should be possible with traitsui: http://docs.enthought.com/traitsui/traitsui_user_manual/handler.html#menus-and-menu-bars

You'll need to manually set your mne config values to fix it

Sorry, which parameter is it exactly?

I worry about degenerate cases somehow breaking things

Agree, the risk of automating the ICP iteration-process is risky, fine to let it be user-controlled

Maybe even better would be to replace the mayavi toolbar that sits on top of the scene? I don't know if it's possible to modify it so that the x/y/z buttons line up with our coordinate system?

Maybe even better would be to replace the mayavi toolbar that sits on top of the scene?

I thought about removing this but that toolbar is pretty useful actually. You can tweak viz and save very easily which is nice.

Sorry, which parameter is it exactly?

MNE_COREG_SCENE_WIDTH / HEIGHT

Maybe some less frequently used items like "interaction" and "Display options" could go into a menu bar

I can move the scale and interaction mode into "Display options", and move that button somewhere that it doesn't add space (maybe to the right of top/right/front/left). I need to refactor that panel anyway.

I agree that the drop-down menu for this seems clumsy

To me the bigger concern than space (though I listed this second) was the false impression that the L/N/R were related to X/Y/Z due to visual vertical alignment. I'll try to find some way to avoid this while putting buttons back.

(toolbar) I thought about removing this but that toolbar is pretty useful actually

Yeah that's why I was wondering whether it could be modified so that x/y/z work properly, or just by adding additional buttons...

MNE_COREG_SCENE_WIDTH / HEIGHT

I basically get random behaviour setting these in mne-python.json :( No combination (even halving the value to accommodate hidpi) seems to make the window fit onto my (blasted) screen.

Yes there is a height limitation due to the concatenation of three panels on the left. I'm going to fix it by:

trimming vertical space on the elements
embedding them in a single panel (what is done on the right side) to allow using a scrollbar when necessary, but this will require a lot of refactoring

Okay, layout works on HiDPI OSX now by tightening up the layout and relying a bit more on tooltips and implicit labels rather than explicit labels. Here is master on OSX:

This PR:

And when in MRI fiducial editing mode master:

And this PR:

I think it's ready for another look. @cjayb @christianbrodbeck you might need to set MNE_COREG_SCENE_WIDTH/HEIGHT in ~/.mne/mne-python.json. In particular height should be sufficiently small (700 or less) on 13" HiDPI OSX.

[x] previous AssertionError Re: 'interactor_style' fixed
[ ] something not working with my config, no effect on scene height

cjb@hathor $ grep SCENE_HEIGHT ~/.mne/mne-python.json
"MNE_COREG_SCENE_HEIGHT": "400",

Here's a shot of my whole screen. If I Grab the window, the parts not show on-screen are visible on the screenshot :-/ Any suggestions on which config to flush...?

@cjayb argh okay. I'll need to actually do the scrolling refactoring I mentioned (I hoped to avoid it by saving space). It's probably worthwhile anyway.
BTW command-shift-4 then "space" on OSX will allow you to capture a single window.

I'll need to actually do the scrolling refactoring

What if you put the view panel back below the scene? You could make it less tall by moving the view options to the side (or into a menu bar)?

Or change the coordinate system to be consistent with Mayavi's x/y/z? Then the view buttons would not even be needed because the the Mayavi toolbar x/y/z buttons could be used instead?

What if you put the view panel back below the scene?

This will have the sad side effect of reducing the scene size in most cases on widescreen systems, no?

This will have the sad side effect of reducing the scene size in most cases on widescreen systems, no?

I've never felt this was an issue but I also rarely work on a laptop; it would really be ideal if we could append our buttons to the mayavi toolbar that is already there and can be hidden, but I assume it's not exposed?

Then the view buttons would not even be needed because the the Mayavi toolbar x/y/z buttons could be used instead?

We are already in the right coordinate system for this. The problem is that their "+Y" (anterior) viewpoint does not maintain "+Z"-up, instead sets "+X"-up (90 degree roll). They had to make a choice one way or the other, and it wasn't the right one for us. This turns the head sideways. A similar thing happens for the "top" view -- the head faces away rather than toward (180 degree roll).

I've never felt this was an issue but I also rarely work on a laptop; it would really be ideal if we could append our buttons to the mayavi toolbar that is already there and can be hidden, but I assume it's not exposed?

I don't know if it's exposed, but I doubt it's supposed to be changed. This would make it potentially fragile so I'm wary of this.

We are already in the right coordinate system for this. The problem is that their "+Y" (anterior) viewpoint does not maintain "+Z"-up, instead sets "+X"-up (90 degree roll). They had to make a choice one way or the other, and it wasn't the right one for us. This turns the head sideways. A similar thing happens for the "top" view -- the head faces away rather than toward (180 degree roll).

Right – then the alternative would probably be replacing the toolbar, with view actions according to our need? Any useful actions should not be too hard to re-add? Might still be easier than refactoring the panels?

Any useful actions should not be too hard to re-add? Might still be easier than refactoring the panels?

Some of those actions are in HeadViewController (which gets passed to FiducialsPanel so that it can set views), and others are not. So this would still require a lot of refactoring, just of a different type.

The "∆" don't come out right for me, probably because of the old VTK...

When switching the subject from sample to fsaverage I get this in the terminal:
Exception occurred in traits notification handler for object: <mne.gui._viewer.PointObject object at 0x132df7470>, trait: project_to_points, old value: [[-0.001028   -0.0008233   0.1151672 ]
 [ 0.0722266  -0.0007632   0.0678615 ]
 [ 0.0216248   0.0835264   0.0665323 ]
 ...,
 [ 0.015821    0.0205052  -0.12551451]
 [ 0.0012782   0.0246383  -0.1238603 ]
 [ 0.0050989   0.0119841  -0.12529761]], new value: [[-0.08270434 -0.01890772  0.0182912 ]
 [-0.08267697 -0.02638755  0.01428479]
 [-0.08323214 -0.02036837  0.00828501]
 ...,
 [ 0.08477898 -0.02493742 -0.01049101]
 [ 0.08478923 -0.0271198  -0.0228901 ]
 [ 0.08583546 -0.02239441  0.01172523]]
Traceback (most recent call last):
  File "//anaconda/envs/mne-dev/lib/python2.7/site-packages/traits/trait_notifiers.py", line 519, in _dispatch_change_event
    self.dispatch( handler, *args )
  File "//anaconda/envs/mne-dev/lib/python2.7/site-packages/traits/trait_notifiers.py", line 482, in dispatch
    handler( *args )
  File "/Users/christian/Code/mne-python/mne/gui/_viewer.py", line 343, in _update_projections
    method=method)[2:4]
  File "/Users/christian/Code/mne-python/mne/surface.py", line 260, in _project_onto_surface
    surf_geom = _get_tri_supp_geom(surf)
  File "/Users/christian/Code/mne-python/mne/surface.py", line 985, in _get_tri_supp_geom
    r1 = surf['rr'][surf['tris'][:, 0], :]
IndexError: index 2033 is out of bounds for axis 0 with size 2033

When switching the subject from sample to fsaverage I get this in the terminal:

I confirm, here's a screenshot right after which may help debugging? Any rendering update fixes the wicked-looking orientations.

Okay layout on OSX works with scrolling now, as does size saving and loading (on Qt backend at least).

Yes! I confirm works on my 13" hidpi screen.
I'm a little confused at to how the 'Project' tick box under Display options is intended to work? Over at #4966 @christianbrodbeck commented on it too but I'm not sure it got resolved.
Seems to me that EEG points inside the surface are projected outward, but points outside are not project "in"?
How does this projection-business interact with the ICP? The projected locations are used for the next iteration?

I'm a little confused at to how the 'Project' tick box under Display options is intended to work?

It's supposed to project the EEG electrodes onto the scalp, but I can confirm that it does not seem to be working any more, when toggling it some of the EEG points seem to change inside/outside status but they don't seem to move.... With sample's own MRI as well as fsaverage. There's no traceback in the terminal.

The "∆" don't come out right for me

I forgot some u prefixes, hopefully it's fixed now.

When switching the subject from sample to fsaverage I get this in the terminal:

This should now be fixed. While in there I sped things up a bit by:

Letting VTK compute the surface normals.
Not doing .clear() and .plot() on changes but rather updating the rr and tris directly.

This required a bit more refactoring but seemed worth it.

It's supposed to project the EEG electrodes onto the scalp, but I can confirm that it does not seem to be working any more

I fixed this once in this PR and then broke it again :) It should be working now, though.

ow does this projection-business interact with the ICP? The projected locations are used for the next iteration?

It doesn't. It's for visualization purposes only. Would it be good to add this as a note somewhere (some tooltip?)

@cjayb sizes and tips about EEG projections should now be fixed (tested on my OSX machine).

Gone! (horizontal bar) I'm going to fade out now, check in again tomorrow morning CET for any residuals.
Until then: a great big, fat, +1 from this reviewer! Can't wait to tell folks about this :)

Great. In the meantime I've changed the default background to black (maybe this should be save-loadable?) and also made it so that the head opacity automatically switches to 1. when doing the MRI fiducials (and back to whatever it was when switching out of that mode).

"∆" and other issues are fixed. Sorry for having a new list of issues :)

I've changed the default background to black (maybe this should be save-loadable?)

It would be nice if visualization preferences were persistent... Maybe there could be a more efficient way of saving them all by going to/from a dictionary?
I don't like the black background because it makes it harder to see the transparent fiducials:

It seems strange that the hair intervenes between the two items concerning point exclusion – did you mean to put the status text above the hair?

When saving a scaled brain I get an error, after hitting save:
Exception occurred in traits notification handler.
Please check the log file for details.
Exception occurred in traits notification handler for object: <mne.gui._coreg_gui.CoregPanel object at 0x12e69b830>, trait: queue_len_str, old value: Queue length: 1, new value:
Traceback (most recent call last):
  File "//anaconda/envs/mne-dev/lib/python2.7/site-packages/traits/trait_notifiers.py", line 519, in _dispatch_change_event
    self.dispatch( handler, *args )
  File "//anaconda/envs/mne-dev/lib/python2.7/site-packages/traits/trait_notifiers.py", line 482, in dispatch
    handler( *args )
  File "//anaconda/envs/mne-dev/lib/python2.7/site-packages/traits/has_traits.py", line 2735, in _sync_trait_modified
    del locked[ name ]
KeyError: 'queue_len_str'
    1 BEM surfaces found
    Reading a surface...
[done]
    1 BEM surfaces read
    1 BEM surfaces found
    Reading a surface...
[done]
    1 BEM surfaces read

I also get horizontal scroll bars now which don't go away by giving the panel more space:

I think the radio buttons of edit/lock are reversed.
When I click edit it locks the fiducials and vice versa

also when I click on LPA if shows me the nose rather than the left ear.

why do I need to click many times "fit ICP" in mne_analyze it can do 10 or
more steps in one click.

I think the radio buttons of edit/lock are reversed.
When I click edit it locks the fiducials and vice versa
also when I click on LPA if shows me the nose rather than the left ear.

None of these happen for me on OSX or Linux, or presumably for @cjayb or @christianbrodbeck. This suggests that you have some problem, likely with Traits/Traitsui/pyface combination. Are you running the environment.yml or something else?

why do I need to click many times "fit ICP" in mne_analyze it can do 10 or more steps in one click.

I never found this particularly helpful, because even after those 10 iterations, you still need to click one more time to see if it converged fully. If it had really converged in 5 iterations, you spent 20 iterations worth of wait time (downside) but you did only have to press twice instead of those five times (benefit). Since it sounds like this is something you want and think others will want, I can add a number-of-iterations (defaulting to 1) parameter.
What we could do is add an ICP options dialog with a parameter Iterations that specifies the maximum number of iterations to do, stopping early if all of the specified criteria parameters for changes in position (mm), angle (deg), and scale (%) are met. That way someone can set it to 10, and if convergence is met sooner, it stops. And if it for some reason goes crazy, no more than 10 iterations will be done anyway. Sound good?

@dengemann see #4754, you need to update traitsui

@larsoner arff. So I need a dev version of traitsui? ...

Yes. You can use the versions pinned in our environment.yml or just use latest master

@larsoner
Traceback (most recent call last):
  File "/Users/dengeman/github/mne-python/mne/commands/mne_coreg.py", line 112, in <module>
    run()
  File "/Users/dengeman/github/mne-python/mne/commands/mne_coreg.py", line 105, in run
    scale=options.scale, verbose=options.verbose)
  File "<string>", line 2, in coregistration
  File "/Users/dengeman/github/mne-python/mne/utils.py", line 728, in verbose
    return function(*args, **kwargs)
  File "/Users/dengeman/github/mne-python/mne/gui/__init__.py", line 177, in coregistration
    from ._coreg_gui import CoregFrame, _make_view
  File "/Users/dengeman/github/mne-python/mne/gui/_coreg_gui.py", line 51, in <module>
    from mayavi.core.ui.mayavi_scene import MayaviScene
  File "/Users/dengeman/anaconda3/lib/python3.5/site-packages/mayavi/core/ui/mayavi_scene.py", line 12, in <module>
    from tvtk.tools.ivtk import IVTK
  File "/Users/dengeman/anaconda3/lib/python3.5/site-packages/tvtk/tools/ivtk.py", line 32, in <module>
    from pyface.api import FileDialog, GUI, OK, PythonShell
  File "/Users/dengeman/anaconda3/lib/python3.5/site-packages/pyface/api.py", line 18, in <module>
    from .application_window import ApplicationWindow
  File "/Users/dengeman/anaconda3/lib/python3.5/site-packages/pyface/application_window.py", line 20, in <module>
    ApplicationWindow = toolkit_object('application_window:ApplicationWindow')
  File "/Users/dengeman/anaconda3/lib/python3.5/site-packages/pyface/toolkit.py", line 129, in toolkit_object
    __import__(be_mname)
  File "/Users/dengeman/anaconda3/lib/python3.5/site-packages/pyface/ui/qt4/application_window.py", line 17, in <module>
    from pyface.action.api import MenuBarManager, StatusBarManager
  File "/Users/dengeman/anaconda3/lib/python3.5/site-packages/pyface/action/api.py", line 17, in <module>
    from .action import Action
  File "/Users/dengeman/anaconda3/lib/python3.5/site-packages/pyface/action/action.py", line 20, in <module>
    from traitsui.ui_traits import Image
  File "/Users/dengeman/github/traitsui/build/lib/traitsui/ui_traits.py", line 27, in <module>
    from pyface.ui_traits import (Alignment, Border, HasBorder, HasMargin, Image,
ImportError: No module named 'pyface.ui_traits'

Now pyface complains ... have you seen this before?

If you update one of traits/traitsui/pyface you should update all of them (hey have interdependencies).

If you update one of traits/traitsui/pyface you should update all of them (hey have interdependencies).

I hope it won't break Mayavi ... for Python 3.5 you need specific versions.

Alright. Now it works.

@larsoner looks good so far. I just have the feeling that the ICP here is somewhat different form mne_analyze, as if it would get stuck earlier after fewer iterations. I'd also follow Alex in wishing a steps option.

I just have the feeling that the ICP here is somewhat different form mne_analyze, as if it would get stuck earlier after fewer iterations.

I'm not sure what you mean, you'll have to elaborate.

I'm not sure what you mean, you'll have to elaborate.

When applying ICP I have the impression that it converges earlier in some local optimum after fewer steps. I was just wondering how far the ICP implementation here is from the mne-c one.

It should be very similar, assuming you start each from the same location

... and the same weighting to dig points (not sure if mne_analyze includes HPI, EEG, and LPA/RPA/Nasion or not), and the same point-omitting.
One difference is that mne_analyze does not use MRI fiducials, so those are closest-point matched, whereas in mne coreg there are MRI fiducials so these are always matched to each other. But this will be a small difference, and given the number of other dig points setting these values to zero should make the behavior even more similar. One other small difference is that we use distances to the downsampled surface, but again this should be a small difference.

When saving a scaled brain I get an error, after hitting save:

@christianbrodbeck I can't replicate this. Are you on environment.yml? If not, can you try it? I suspect this is actually a traits / traitsui problem.
I reverted the background to gray, will wait on saving all viz prefs for another PR.
I have added ICP fitting with convergence criteria (default uses up to 10 steps), feel free to try it.

what error do you get with sample? I get 3.1mm +/- 1mm.
what do you envision for the weights? do you expect to set them to something
different from 0 or 1?
ICP with 10 iters works great.

I would use 1 in most cases, yes. Also 1 for EEG depending on how closely
those electrodes were digitized to the scalp. At UW we are careful to get
close to the scalp for these points, too, for example. For people confident
in their fiducial digitization (and MRI fiducials) they should set those
higher, as they are always matched.
Can't check error right now but can later. I think 10 steps with auto stop
is nice. Do you think the stop criteria are reasonable?
The next most interesting thing will be dealing with points inside the
head. IMO these should be avoided if possible via a heavier penalty or a
(COBYLA) fitting constraint. But even for the sample data it's impossible
to keep them all out with a reasonable non-scaling transform. I don't think
this problem is that common, though.

ok thanks a lot
can someone, independant from me or eric try the new GUI vs. mne_analyze
and report
differents in estimated trans and distances reported between the 2 GUIs?
I'd like to be quite confident that we can advertise the coreg GUI for the
majority.
I have to admit I am still using mne_analyze for my coregs...

 Can someone
suggest an appropriate metric for scoring similarity between 2 transforms?
On Sun 11 Mar 2018 at 16:51, Alexandre Gramfort<EMAIL_ADDRESS>wrote:

ok thanks a lot
can someone, independant from me or eric try the new GUI vs. mne_analyze
and report
differents in estimated trans and distances reported between the 2 GUIs?
I'd like to be quite confident that we can advertise the coreg GUI for the
majority.
I have to admit I am still using mne_analyze for my coregs...
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/mne-tools/mne-python/pull/4971#issuecomment-372125484,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AB0firCqcOYN8zx9ji-LmGgW7CniM2w5ks5tdUgUgaJpZM4ST8pM
.

For me the horizontal scroll bars still appear when the vertical ones appear, i.e. there is enough space in the area for the panel itself but not for the vertical scroll-bars... In newer version (environment.yml) the vertical scroll bar fades out when it is not used, that's why you probably don't see it.
Also, since the last update the GUI recomputes distances upon uniform scaling which takes forever and is not necessary, and did not happen before. Did something change about fsaverage handling?

When saving a scaled brain I get an error, after hitting save:

@christianbrodbeck I can't replicate this. Are you on environment.yml? If not, can you try it? I suspect this is actually a traits / traitsui problem.

I'm on the environment I uploaded earlier which is conda-forge based, so I think a good number of people use it. These are my settings:

Terminal:
2018-03-11 12:23:16.104 python[33566:4059470] modalSession has been exited prematurely - check for a reentrant call to endModalSession:
Exception occurred in traits notification handler.
Please check the log file for details.
Exception occurred in traits notification handler for object: <mne.gui._coreg_gui.CoregPanel object at 0x12c9d95f0>, trait: queue_len_str, old value: Queue length: 1, new value:
Traceback (most recent call last):
  File "//anaconda/envs/mne-dev/lib/python2.7/site-packages/traits/trait_notifiers.py", line 519, in _dispatch_change_event
    self.dispatch( handler, *args )
  File "//anaconda/envs/mne-dev/lib/python2.7/site-packages/traits/trait_notifiers.py", line 482, in dispatch
    handler( *args )
  File "//anaconda/envs/mne-dev/lib/python2.7/site-packages/traits/has_traits.py", line 2735, in _sync_trait_modified
    del locked[ name ]
KeyError: 'queue_len_str'
    1 BEM surfaces found
    Reading a surface...
[done]
    1 BEM surfaces read
    1 BEM surfaces found
    Reading a surface...
[done]
    1 BEM surfaces read
    Reading a source space...
    Computing patch statistics...
    Patch information added...
    Distance information added...
    [done]
    Reading a source space...
    Computing patch statistics...
    Patch information added...
    Distance information added...
    [done]
    2 source spaces read
scaling source space None:  fsaverage -> sample-fsa2
Scale factor: [ 0.97054443  0.97054443  0.97054443]
Recomputing distances, this might take a while
Calculating source space distances (limit=inf mm)...

With environment.yml I don't get it but I get other weird stuff like

and
QWidgetWindow(0x7fe94a26cc50, name="QMainWindowClassWindow") ( QScreen(0x7fe94638c320, name="SMS27A850") ): Attempt to set a screen on a child window.
QWidgetWindow(0x7fe94a26cc50, name="QMainWindowClassWindow") ( QScreen(0x7fe94638c320, name="SMS27A850") ): Attempt to set a screen on a child window.
QWidgetWindow(0x7fe94a26cc50, name="QMainWindowClassWindow") ( QScreen(0x7fe94638c320, name="SMS27A850") ): Attempt to set a screen on a child window.
QWidgetWindow(0x7fe947dbe450, name="_ToolBarClassWindow") ( QScreen(0x7fe94638c320, name="SMS27A850") ): Attempt to set a screen on a child window.
QWidgetWindow(0x7fe94a26cc50, name="QMainWindowClassWindow") ( QScreen(0x7fe94638c320, name="SMS27A850") ): Attempt to set a screen on a child window.
QWidgetWindow(0x7fe947dbe450, name="_ToolBarClassWindow") ( QScreen(0x7fe94638c320, name="SMS27A850") ): Attempt to set a screen on a child window.
QWidgetWindow(0x7fe94a2569b0, name="_GroupSplitterClassWindow") ( QScreen(0x7fe94638c320, name="SMS27A850") ): Attempt to set a screen on a child window.
QWidgetWindow(0x7fe94a26cc50, name="QMainWindowClassWindow") ( QScreen(0x7fe94638c320, name="SMS27A850") ): Attempt to set a screen on a child window.
QWidgetWindow(0x7fe94a26cc50, name="QMainWindowClassWindow") ( QScreen(0x7fe94638c320, name="SMS27A850") ): Attempt to set a screen on a child window.
QWidgetWindow(0x7fe94a26cc50, name="QMainWindowClassWindow") ( QScreen(0x7fe94638c320, name="SMS27A850") ): Attempt to set a screen on a child window.
QWidgetWindow(0x7fe947dbe450, name="_ToolBarClassWindow") ( QScreen(0x7fe94638c320, name="SMS27A850") ): Attempt to set a screen on a child window.
QWidgetWindow(0x7fe947dbe450, name="_ToolBarClassWindow") ( QScreen(0x7fe94638c320, name="SMS27A850") ): Attempt to set a screen on a child window.

and a second floating toolbar

and crash when I try to move it

In newer version (environment.yml) the vertical scroll bar fades out when it is not used, that's why you probably don't see it.

I just checked and they are not there. I wouldn't be surprised if it has to do with OSX version. I pushed a commit to tighten up the left panel a little bit more. But really the horizontal scrollbar isn't so terrible if it isn't needed anyway, so we're probably hitting diminishing returns here.

Also, since the last update the GUI recomputes distances upon uniform scaling which takes forever and is not necessary, and did not happen before. Did something change about fsaverage handling?

I don't think so, but I can look into it. It really should be a fast operation, though. Maybe something else is taking up the time. Did you use e.g. time.time() insertion or line profiling to ensure that it's really due to the distance calculation? (I assume you have sklearn so that BallTree is available?)
Regarding the version problems, I'll try updating to latest master of some of the libraries. Perhaps we can pin a different / newer version to fix some of these issues on our recommented environment.yml.

I think the recomputing distances issue is related to always submitting scale as 3 parameters. This PR logs Scale factor: [ 0.98211367  0.98211367  0.98211367] whereas the divergence point with master logs Scale factor: 0.966808630935. When there is one parameter the scaling functions just scales the distances, but when there are 3 parameters it recomputes. I do have sklearn installed and it takes >20 minutes, it recomputes not just once but for every source space.

I think the recomputing distances issue is related to always submitting scale as 3 parameters.

Ahh yes somehow I missed where you said it was about saving. Setting all 3 each time in unified mode simplified some of the code paths and unified some behaviors, so I'd like to keep that change. It should be simple enough to fix in the scaling function (if all are equal, treat as one) to avoid the slower code path, though.

@agramfort I did this with sample:

Reset params
Fid fiducials
Omit 20 mm (to remove the 1 clearly errant point)
Fit ICP

Converges on iteration 10 (3.1 +/ 1.2mm for HSP).
In mne_analyze I did roughly the same thing (using omit=22mm to get the same 1 point removed, probably due to variability in my initial-fiducial alignment) with 10 iterations. After saving both transforms and using this code:
import numpy as np
import mne
from mne.transforms import _angle_between_quats, rot_to_quat, invert_transform
coreg_fname = 'sample-mne-coreg-trans.fif'
c_fname = 'sample-mne_analyze-trans.fif'
sample_fname = (mne.datasets.sample.data_path() +
                '/MEG/sample/sample_audvis_raw-trans.fif')
coreg_trans = invert_transform(mne.read_trans(coreg_fname))['trans']
c_trans = mne.read_trans(c_fname)['trans']
sample_trans = mne.read_trans(sample_fname)['trans']

def delta(a, b):
    print(np.rad2deg(_angle_between_quats(rot_to_quat(a[:3, :3]),
                                          rot_to_quat(b[:3, :3]))))
    print(1000 * np.linalg.norm(a[:3, 3] - b[:3, 3]))

delta(coreg_trans, c_trans)
delta(c_trans, sample_trans)

I get 4.78° and 5.25mm differences between the mne coreg and mne_analyze results, which is bigger than I expected. For comparison, the sample_audvis_raw-trans.fif and the mne_analyze one I just made are only off by 1.07° and 1.94mm.
One possible difference is that I use the low-res mesh for the ICP steps. Maybe it's worth using the high-res mesh to improve the accuracy. It's at least worth a shot to see if that improves the correspondence.
Also we should probably save the invert_transform version of the transform for consistency.

I tried using the high res surface and it didn't matter (though it was much slower).
What did matter (and brought the numbers for mne coreg vs mne_analyze to 1.20° and 1.46mm, comparable to mne_analyze vs sample_audvis_raw-trans.fif) was putting my MRI fiducials way lower than I had been:

I guess maybe this is where they were originally digitized? If not, then the trans we get in mne coreg should actually be more accurate than the one produced by MNE-C / mne_analyze.
In any case, I think that given a proper and consistent definition of the fiducials, mne coreg and mne_analyze should agree. But it would still be good for others to cross check this.

It should be simple enough to fix in the scaling function (if all are equal, treat as one) to avoid the slower code path, though.

Yes I think that would actually be preferable, makes the case more efficient where they are entered as three

Couple of UI issues have appeared on my py3/vtk8/macos setup as well as on py27/ubuntu (based on environment2.yml, though with current master of enthought mods):

The ICP fit button is greyed out even when scaling mode is None (can't do ICP on sample), even after Lock'ing

ICP works in Scaling mode

I consistently get 2 windows popping up when I click on 'Display options...' (though on occasion, only one, can't figure out the rule!)
(New comment) the FID coordinates (left panel) are given in meters and with many decimals -> not readable

suggest using mm (should be space to the right of the RPA-box)
and rounding to e.g. 2 decimals

The ICP fit button is greyed out even when scaling mode is None (can't do ICP on sample)

There are two "Fit ICP" buttons. They are both grayed out? The top one should be and the bottom one shouldn't be.

mne_analyze is less strict with fiducial repositioning using dig points?

@agramfort AFAIK mne_analyze does not use the MRI fiducial locations. As such the fiducial digitizations in mne_analyze are just like any other dig points during ICP (they can be matched to any surface point in the distance calculations). In mne coreg we always match the (head) digitized fiducials to the MRI fiducial points during ICP fitting, rather than to the closest surface point. This should give more accurate results assuming you did your fiducials accurately during acquisition. (If you didn't, you can lower the weight assigned to these points.) We could add a fiducials-to-nearest mode (rather than the current fiducials-to-MRI points), do you think it's necessary?

There are two "Fit ICP" buttons. They are both grayed out? The top one should be and the bottom one shouldn't be.

The bottom one stays grayed whatever I do, to top one reacts as expected (only used when scaling).

The bottom one stays grayed whatever I do, to top one reacts as expected (only used when scaling).

Is this with sample data, or data of your own? If it's yours, do you only have EEG dig points? There is probably a bug where the logic needs to be changed regarding enabling.

I think that relying on 100 dig points is better than forcing some
constraints on 3 fiducials that will be inherently more variable.
So I am sceptical that it's a good default

I think that relying on 100 dig points is better than forcing some constraints on 3 fiducials that will be inherently more variable. So I am sceptical that it's a good default

To be clear, both mne_analyze and mne coreg rely on the 100 dig points, i.e. use the same number of points in fitting. The question is whether the 3 fiducials should be matched to whatever the nearest point on the head surface happens to be (mne_analyze behavior) or specifically to the MRI fiducials that the user marks (mne coreg behavior). Assuming you actually know where you mark the fiducials, the latter should be more accurate. Do you think this is unlikely / uncommon?

@cjayb can you try changing the digitization source (you can use sample_audvis_raw.fif or any -ave.fif in that directory) to see if that fixes it? If so, there is some problem with traits updates that we can probably fix.
@christianbrodbeck I've fixed the one-parameter scaling problem with the latest commit.
I also changed the user-facing units in all dialogs to be easier to enter/tweak (including MRI fiducials). They are now mm instead of m, degrees instead of radians, and percent instead of proportion for scale.

@cjayb I tweaked the logic for button availability, maybe it will fix your issue. If not, please also try what I suggested above about loading a new digitization source to see if it works.

I tweaked the logic for button availability, maybe it will fix your issue. If not, please also try what I suggested above about loading a new digitization source to see if it works.

Both buttons now 'activate' once I choose a (any) source, but only after I do anything to force a redraw of the right panel (scroll or resize the window).
Another oddity: going back-and-forth with Lock-Edit-Lock-Edit sometimes greys/ungreys the right panel, other times not! I can not [sic ;)] find the rule! Regardless, in Edit mode, the buttons, scrolling or any other parts of the right panel don't work. As expected, but really weird for the end-user... Speaking of which, I still get two Display options-windows for one click on the button :-/
No doubt Traits-wraiths at play, going to be a real ***** to debug across versions & platforms! To that extent: a py2.7 env with vtk and pyqt=5 from the default channel doesn't suffer from the grey-weirdness, but hidpi is screwed (Enthought-bits from git master). My point: corner-cases will remain on the UI-side, I'm afraid, which might just have to be dealt with on the mailing list. Oh Python...[-sigh-]

Do you experience these on the recommended environmement.yml? You shouldn't.
For HiDPI VTK8 installed via pip followed by our pinned versions of traits, traitsui, pyface, and mayavi works (other than the weirdness with the toolbar @christianbrodbeck mentioned, still need to see if that's fixed in a newer version).
I tried to update environment.yml to use VTK8 at some point, but it broke some tests IIRC. Might be time to try again.

I think that relying on 100 dig points is better than forcing some
constraints on 3 fiducials that will be inherently more variable.

I tend to agree, though just choosing the closest surface point might be massively varying due to the complex geometry of the ear. Not that it'll matter much in a 3-to-100-situation, but I don't think that xfm deserves gold standard status.

Assuming you actually know where you mark the fiducials, the latter should be more accurate. Do you think this is unlikely / uncommon?

Unfortunately: I think it's quite common, particularly when the person measuring and analysing isn't the same. Just look at the points we need to choose on sample... Also, many people like to choose the tragus as L/RPA: ostensibly easy-to-find protrusion, but also severely displaced in most MRIs due to cushioning used for hearing protection and immobilisation.
With the new GUI I might just use the fiducials for an initial fit, omit outliers, then set their weight to zero and ICP the daylights out of the head shape (extra) points.

To be clear, both mne_analyze and mne coreg rely on the 100 dig points,
i.e. use the same number of points in fitting. The question is whether the
3 fiducials should be matched to whatever the nearest point on the head
surface happens to be (mne_analyze behavior) or specifically to the MRI
fiducials that the user marks (mne coreg behavior). Assuming you actually
know where you mark the fiducials, the latter should be more accurate. Do
you think this is unlikely / uncommon?

you're maybe right but maybe I biased here as I mostly analyse data that I
did not collect myself so
I just have good guesses for the fiducials. That's why I think that the
mne_analze approach may be
a better default, especially in a time of large analysis of public data
that we did not collect ourselves.
Makes sense?

looks like some large distance brain synchrony just happened here :)

Do you experience these on the recommended environmement.yml? You shouldn't.

No grey weirdness, no, but 1/4 hidpi-mess of course

For HiDPI VTK8 installed via pip followed by our pinned versions of traits, traitsui, pyface, and mayavi works (other than the weirdness with the toolbar @christianbrodbeck mentioned, still need to see if that's fixed in a newer version).

I couldn't get VTK8 working with the pinned versions, had to go with master. I can give it another shot tomorrow, just to make sure I didn't do anything daft. I think the latest relevant commits there were like 6 days old, so maybe hold on and see if they release something stable? Might save you a lot of work.

looks like some large distance brain synchrony just happened here :)

hehe, high-five :)

With the new GUI I might just use the fiducials for an initial fit, omit outliers, then set their weight to zero and ICP the daylights out of the head shape (extra) points.

I'll add a mode for using matching or nearest for fids during ICP.

I'll add a mode for using matching or nearest for fids during ICP.

This is now done. Using this as the default and setting a few others to match mne_analyze-like behavior (nasion weight=10, hpi weight=1, eeg weight=1, max icp iterations=20) I can replicate the existing trans:

Run:mne coreg --subjects-dir=~/mne_data/MNE-sample-data/subjects/ --subject sample \
     --fif ~/mne_data/MNE-sample-data/MEG/sample/sample_audvis_raw.fif`

Fit fid
Omit using 20mm (drops 1 point)
Fit (ICP)
Save ...

I get differences of 1.5°/1.3mm, which is on the same scale as the differences between sample_audvis_raw-trans.fif and the one I redid in mne_analyze (1.07°/1.94mm).
Whether or not these are the best defaults is probably site-specific, so at some point soon we should allow saving/loading the prefs. But in the meantime, perhaps this should replicate what mne_analyze does by default?

I get differences of 1.5°/1.3mm, which is on the same scale as the differences between sample_audvis_raw-trans.fif and the one I redid in mne_analyze (1.07°/1.94mm).

Could you confirm that the 1.5°/1.3mm is between sample_audvis_raw-trans.fif and the mne coreg-calculated trans, using the code you copied in above? (minus the invert_transform) I get slightly higher differences (1.9/2.8), doubtful if FID-placement can explain it. NB default Nasion weight is 100 (not 10).

Ahh yes, I was looking at the differences between the one I did in C and the mne coreg one. I'm actually okay with this because I have tried to replicate the same procedure in both places (but I don't know if my procedure is identical to what was done originally for sample).
FWIW I get 2.27 and 3.41 between the sample one and mne coreg.

doubtful if FID-placement can explain it.

I wonder if it could be actually. If where each of us put the nasion it differs by a couple mm, we tell it to stay put during fitting so it will affect the transformation.
Can you upload your sample-fiducials.fif file? Then I can replicate and ensure it's not a system issue.

Here are my fiducials, the screenshot of parameters used, and the resulting xfm. NB This is with only omitting the clear outlier. The previous delta-values I referred to were after additional tweaking, this is 4deg/3mm from sample-trans.fif.

cjayb-sample-fids-xfm.zip

@cjayb using your MRI fiducials and doing this on my OSX and my Linux machine:

Set nasion weight 10. (pushing a commit for this now so if you update you won't need to)
Fit Fid.
Omit 20 (one point)
Fit ICP

On iteration 7 it converges, giving 2.9° 3.4mm. (I've updated the code snippet above to remove the unnecessary invert_transform and clean up the output). Is this the same procedure you used?

Is this the same procedure you used?

Yes, same procedure, exact same errors (phew!).
Apart from minor UI-oddities (that I'll try to dig into elsewhere), I'm very pleased with this tool! Both within-subject and fsaverage-like coregs are working nicely, I have a good feeling about the quality of the fit based on the new visuals; @agramfort it might be time to "turn from your wicked ways" and embrace mne coreg ;)

Apart from minor UI-oddities (that I'll try to dig into elsewhere), I'm very pleased with this tool!

Excellent. The bulk of the credit belongs to @christianbrodbeck, I am just pushing it in a mne_analyze direction plus a few features I always wished I'd had in mne_analyze to try to increase adoption a bit. If you have other ideas for improvements as you use it, feel free to comment in #3934

shall we merge?

I should add some tests for _DistanceQuery (can do today) then good to go from my end.
@christianbrodbeck okay for you? I don't know if we can do much about the GUI warnings (and OSX toolbar segfault) except bother the upstream libraries about them.

I've tested it by using and that was great, didn't see any issues, if you want to wait I could have another look over the code tonight. About the OS X issues I agree, I think as long as it works in the stable (2.7) env, which it does, that's enough.

I've added / refactored the last bit for better test coverage (though it might still go down a little bite).
@christianbrodbeck feel free to look at the code again if you have time. Otherwise we can merge in the morning.

Ugh, just noticed that this PR breaks the kit2fiff GUI which depends on the same visualization objects. I get a lot of errors in the terminal, the HPI coil positions are not plotted, and the layout has become unwieldy because it displays all the new point display options (which probably don't apply, since we don't have head based normals)...
I can try to look into it in the next few days if you're not familiar enough with it. Objects to test it are at mne/io/kit/tests/data
The Coreg GUI on the other hand looks great! Just two more thougts:

Based on trying to coregister sample to fsaverage I wonder whether the nasion weight should be 1 instead of 10 by default.
I liked R/A/S labels instead of X/Y/Z because they are less ambiguous to me, but I am not strongly attached if there are other reasons to prefer X/Y/Z.

Ugh, just noticed that this PR breaks the kit2fiff GUI which depends on the same visualization objects.

I have never used this, so it's probably worth waiting until you have time to check. Also it would be good to fix these tests to make them more robust to regressions at the same time (if possible / within reason) since right now mne/gui tests pass :(

I am not strongly attached if there are other reasons to prefer X/Y/Z.

I think elsewhere in MNE we tend to call / think of the coordinate frames in terms of X/Y/Z. Thinking for example of where a the origin of an expansion is in the head frame, or MRI vertex coords, etc. these are referred to IIRC as X/Y/Z and not as R/A/S (although our conventions make these equivalent). I think X/Y/Z is also what mne_analyze uses, though I can't check at the moment because I need to roll back an XQuartz update that broke it on my OSX laptop...

Based on trying to coregister sample to fsaverage I wonder whether the nasion weight should be 1 instead of 10 by default.

Hmm... I'm surprised by this because I thought that increased nasion confidence was the reason you originally had the coordinate frames snasion-centric (and wanted a Nasion weight). But in practice maybe it causes more problems than it solves (good to have on sometimes but not by default)? This would be consistent with @agramfort not even wanted the points matched to the MRI markings, too, I guess.
Although I think "try to keep nasion in place" is the default in mne_analyze (need to check), I'm okay with changing it to 1. for now.

Also it would be good to fix these tests to make them more robust to regressions

Yeah I'll check whether these things can be tested

I'm surprised by this because I thought that increased nasion confidence was the reason you originally had the coordinate frames snasion-centric (and wanted a Nasion weight)

I know :) I can see arguments both ways, for sample the result looks wrong. The reason for trusting the nasion more applies in particular to optical head scans which include hair, so I guess it just needs to be adjusted based on the data.

WIP fixing kit2fiff GUI at https://github.com/larsoner/mne-python/pull/30

@christianbrodbeck I just realized that currently we scale the sphere and sphere.reg surfaces. I think that these should just be copied, not be modified by scaling. Agreed?

Yes that’s right.
You could also look over and merge the current PR for now – it should just change the tests so that errors in trait handlers are raised rather than logged. It looks like Travis did not run the last commit though. I am traveling and won’t be able to continue until at least the weekend, sorry for the delay.

On Mar 21, 2018, at 8:11 AM, Eric Larson<EMAIL_ADDRESS>wrote:
@christianbrodbeck I just realized that currently we scale the sphere and sphere.reg surfaces. I think that these should just be copied, not be modified by scaling. Agreed?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

@christianbrodbeck I made it so that there are no longer any errors in mne/gui tests locally.
Can you see if the kit2fiff gui actually works for you?

I’ll check when I get back – the other issue that could be solved without knowing about the KIT system was that there were display options on the GUI that did no apply for the points (making the GUI very wide), so we’d probably need a separate view for points depending on which GUI they’re in.

On Mar 21, 2018, at 9:48 AM, Eric Larson<EMAIL_ADDRESS>wrote:
@christianbrodbeck I made it so that there are no longer any errors in mne/gui tests locally.
Can you see if the kit2fiff gui actually works for you?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

Great. It's okay if it takes a week to sort this out, we are not close to making a release for 0.16 yet.

The "phantom bar" on OSX is due to layout='split' (https://github.com/enthought/mayavi/issues/623). I'm going to removing this, since it's probably no longer necessary because the panels are "the right size" now.

Argh of course setting split=False makes it so something else breaks -- the scene no longer resizes to be larger when increasing the window size.

A new minor issue, I noticed if I start the GUI multiple times it gets a little smaller every time :)

Hmm... this seems to happen to me, too. I might see if it's possible to query the window size on exit since that's what we actually need, rather than trying to compute it based on the scene size (and other dimensionality).

if I start the GUI multiple times it gets a little smaller every time :)

Okay this should be fixed now. I've only tested it on OSX but it should hold for Linux, too. It now queries the window size on exit and sets it on startup.
The only thing left on my todo list is ensure that Kit GUIs still work -- @christianbrodbeck do you agree, and you're planning on investigating that bit when you get a chance right?

The only thing left on my todo list is ensure that Kit GUIs still work -- @christianbrodbeck do you agree, and you're planning on investigating that bit when you get a chance right?

yes and yes

Okay everything should be working and I think everyone is happy with this. I'll go ahead and merge if CIs come back happy!

ok !

Nothing like a +2,086 −1,572 PR to send us into the weekend :)


Bump to controller-runtime 0.3.0 and Kubernetes 1.15

Description of the change:
Bump dependencies:

controller-runtime to v0.3.0
kubernetes to v1.15.4
controller-tools to v0.2.2

Motivation for the change:

To support new features from controller-runtime, controller-tools, and kubernetes

/hold
Waiting for #1949 to merge.
@camilamacedo86 good catch on the CHANGELOG. I'll update it.

/hold cancel

/test e2e-aws-ansible
/test e2e-aws-go
/test e2e-aws-helm

/test marker


Verbose Errors

Seperated Error Logging and Expected, for easy custom errors when needed.
Expected functions now log out what they found instead aswell.
Solves issues #1123

Can't seem to figure out why this test is failing:
test("a\n#let \nb", 7..7, "i", true);

Why should source.len_bytes()  and range.len() not be equal, I dont see how changing the error message affected this.

I will try to find a solution for #1123 that integrates better with the current error system.


Wjh/simplify hooks

This PR is a draft of the changes to simplify the SCAPI hooks. The two primary goals are to fix the function signature (current develop has duplicate arguments and unnecessary complexity) and to move as much logic as possible from the endpoint hooks (which are generated) into the helper useQuery/useMutation hooks.
Very much still a WIP, but early feedback is appreciated! Currently only the query hooks are done, I haven't touched mutation hooks yet.
To avoid confusing myself during development, I've generated the ShopperBaskets hooks in a new Test directory. Once the design is more finalized, I'll remove Test and generate the real hooks.

Description

Types of Changes

[ ] Bug fix (non-breaking change that fixes an issue)
[ ] New feature (non-breaking change that adds functionality)
[ ] Documentation update
[ ] Breaking change (could cause existing functionality to not work as expected)
[ ] Other changes (non-breaking changes that does not fit any of the above)

Breaking changes include:

Removing a public function or component or prop
Adding a required argument to a function
Changing the data type of a function parameter or return value
Adding a new peer dependency to package.json

Changes

(change1)

How to Test-Drive This PR

(step1)

Checklists

General

[ ] Changes are covered by test cases
[ ] CHANGELOG.md updated with a short description of changes (not required for documentation updates)

Accessibility Compliance
You must check off all items in one of the follow two lists:

[ ] There are no changes to UI

or...

[ ] Changes were tested with a Screen Reader (iOS VoiceOver or Android Talkback) and had no issues
[ ] Changes comply with WCAG 2.0 guidelines levels A and AA
[ ] Changes to common UI patterns and interactions comply with WAI-ARIA best practices

Localization

[ ] Changes include a UI text update in the Retail React App (which requires translation)

This refactor should branch off the develop instead of the feature branch, the goal of the feature branch is to integration the hooks into the Retail App. Technically, it should not involve too much of a change from the commerce-sdk library. Besides, the feature branch is a long-run branch for this release, so I don't this we gonna merge it any time soon, and this refactor is likely to merge before the feature branch.
Making refactor from develop is also better because users can actually get to test out new code change when we do another release instead of having to wait until the feature branch is fully completed.
We can always merge this back into feature branch when it is done, it is easier than another way around


Optimise BasicMode::clear RAM usage and optimize size of Ssd1306::char_to_bitmap

Hi! Thank you for helping out with SSD1306 development! Please:

[ ] Check that you've added documentation to any new methods
[X] Rebase from master if you're not already up to date
[ ] Add or modify an example if there are changes to the public API
[X] Add a CHANGELOG.md entry in the Unreleased section under the appropriate heading (Added, Fixed, Changed, etc)
[X] Run rustfmt on the project with cargo fmt --all - CI will not pass without this step
[X] Check that your branch is up to date with master and that CI is passing once the PR is opened

PR description
This PR fixes issues with small AVRs:

BasicMode doesn't work on MCUs with less than ~1.5Kb of RAM because there's a stack array of 1024 bytes allocated to zero-out pages. Instead, we can zero-out in small pieces.
Refactor glyph search to use lookup table instead of one big match. Saves around 3.5Kb of compiled code on AVR ATMega-s. Didn't test on other architectures, but pretty sure it will save some size too.

AVR size difference
Before:
text    data     bss     dec     hex filename
   0    8446       0    8446    20fe object.hex

After:
text    data     bss     dec     hex filename
   0    4988       0    4988    137c object.hex

STM32 (thumbv7em) size difference
Before:
verified 7212 bytes

After:
verified 6392 bytes

@jamwaffles No problemo!
Although I saved some space, I still can't squeeze my program into atmega168 - bme680 driver is a resource hog. So I'll try to switch to a STM32 MCU, I can test this change there too later this week. I also have ESP8266 at hand, but I've never used it with Rust, so not sure if I can quickly check it.

LOL, I've just noticed that all the first and last bytes of each glyph in the font are zeros, so I just made the data array smaller by another 500 bytes. It is a bit clunky, but the font is hardcoded (and quite nice), so I don't think it is a problem. -3.5Kb total.

Tested on NUCLEO-F446ZE and ATMega168 with 128x64 and 128x32, all work fine.

Got rid of extra-sized arrays in Command::send, cuts 80 bytes on Cortex-M4 and 260 bytes on ATMega168.

Thanks for the review. I accepted all your suggestions, a newbie in Rust, so make simple mistakes. Also, the merge conflict should be resolved now.

@jamwaffles checked and committed your changes. Thanks for spending so much time on this one!

This is released in v0.8.3. cargo update should pull in the changes.

FYI I'm firmly opposed to https://github.com/jamwaffles/ssd1306/pull/195/commits/e309b41027e4219a17ba28200034eddcff25da77 as it means I'll have to duplicate the whole command table for the async interface.


[improve][broker] defer the ownership checks if the owner is inactive (ExtensibleLoadManager)

Motivation
When a broker shutdown or gets killed, the leader broker will get notified and soon reassign the orphan bundle ownerships. Meanwhile, the ownership lookup to these orphan bundles should be deferred til the new brokers own them. This will further reduce the lookup request retries when brokers are shutdown or get killed.
Modifications

In ServiceUnitStateChannelImpl.getOwnerAsync, additionally, check if the current owner broker is active or not in the broker registry. If active, return the current owner. Otherwise, defer the ownership lookup requests til the new owner owns it.
During the orphan ownership cleanup, when there is no selected owner, the leader overrides the orphan states to Free.
Do not throw exception from writeLoadReportOnZookeeper writeResourceQuotasToZooKeeper

Verifying this change

[x] Make sure that the change passes the CI checks.

Added a test case testActiveGetOwner to cover this.
Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

[ ] Dependencies (add or upgrade a dependency)
[ ] The public API
[ ] The schema
[ ] The default values of configurations
[ ] The threading model
[ ] The binary protocol
[ ] The REST endpoints
[ ] The admin CLI options
[ ] The metrics
[ ] Anything that affects deployment

Documentation

[ ] doc
[ ] doc-required
[x] doc-not-needed
[ ] doc-complete

Matching PR in forked repository
PR in forked repository: https://github.com/heesung-sn/pulsar/pull/58

Codecov Report
Attention: 5 lines in your changes are missing coverage. Please review.

Comparison is base (99d06b9) 73.58% compared to head (3535156) 73.54%.
Report is 11 commits behind head on master.

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #21811      +/-   ##
============================================
- Coverage     73.58%   73.54%   -0.04%
- Complexity    32254    32278      +24
============================================
  Files          1858     1858
  Lines        138021   138110      +89
  Branches      15111    15137      +26
============================================
+ Hits         101558   101579      +21
- Misses        28606    28670      +64
- Partials       7857     7861       +4

Flag
Coverage Δ

inttests
24.16% <16.36%> (-0.02%)
:arrow_down:

systests
23.68% <21.81%> (-0.11%)
:arrow_down:

unittests
72.84% <90.90%> (-0.04%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Files
Coverage Δ

...lance/extensions/ExtensibleLoadManagerWrapper.java
58.62% <ø> (ø)

...ache/pulsar/client/impl/MemoryLimitController.java
93.75% <89.47%> (-1.81%)
:arrow_down:

...xtensions/channel/ServiceUnitStateChannelImpl.java
84.46% <91.66%> (+0.30%)
:arrow_up:

... and 65 files with indirect coverage changes


Cannot use custom components in Nuxt

Actual Behavior
I cannot use custom components in the ~/emails folder, I get the following error
[Vue warn]: Failed to resolve component: EmailCustomLink
If this is a native custom element, make sure to exclude it from component resolution via compilerOptions.isCustomElement.

Expected Behavior
It should work as described in https://github.com/vue-email/vue-email/issues/64
Reproduction
https://stackblitz.com/edit/nuxt-starter-a69obu?file=emails%2FCustomLink.vue

Hi, you just need to use the name of the component, without the Email prefix

Thanks, that works 👍🏻🚀


Documentation: Add NpgsqlTypeMap, MySqlDbTypeMap and SqlDbTypeMap into the API Docs.

Updated the ReadTheDocs documentation just now.


measurement Seq Num  needs punctuation - cosmetic but confusing

When displaying overview of all sensors there is a space but no punction
between thousands and hundredths.

Expected 45,019 (or 45.019)
Actual   45 019

The web app currently follows European (Finnish) standards and we do not use commas or periods here. We are planning to add support for browser (or operating system) locale settings on these things later on. Closing this issue.


Can I use parts of this in my project?

https://github.com/alichtman/shallow-backup
I'm planning on adding automatic restic backups to my back up utility. I'm asking for permission because I noticed there was no license in the repo. Is that okay?

Yep, of course. I added a MIT license - hope that works for you.  I never considered a use case outside of a personal utility.
p.s. It's only ever been used/tested on my environment as far as I know - so I will welcome any improvements you make.
Cam

MIT is perfect. I'll open PRs for any necessary changes. Thanks!


Branch stable is failing

Trying to build stable at commit 51c4cc780986340f611c3c0e365081836a4fcff5 failed.
More info at: https://travis-ci.com/Qiskit/qiskit-terra/jobs/201799042

The jobs was rerun and the reason for the fail got lost :S


When you support tls version 1.3？

Hi @husttb . Can you ask this question on the new Mbed TLS list?

By the way, there was an older ticket which has some information (near the end) that you might find useful too: https://github.com/ARMmbed/mbedtls/issues/508#issuecomment-601258457

I'm closing this since the question is answered and we're trying not to use the issue tracker for questions anymore.


Wikilinks failing to be created

So, in my markdown I have
[[Power of Attorney]]
[[Power of Attorney#Conclusion]]

But, it gets rendered as
https://alphacreat.atlassian.net//wiki/spaces/FINANCE/pages/65575
https://alphacreat.atlassian.net//wiki/spaces/FINANCE/pages/65575#Conclusion

Is this related to the latest updates?

Ahh :( Sorry a bug I need to fix. You have an extra / on the end of your confluence site domain. If you remove that it will all start working.
I thought I fixed that sorry

No worries.
Hope that is the only issue :)

Yup it is :)
Unrelated question. Is it possible to prevent the folder Confluence Page to be published as a page?

Nevermind, i was using folder notes wrongly!


Restdocs tests to generate stubs for renderer

Sagan will be soon retired. A note has been added to the README.
We don't accept contributions anymore as a result.
Thanks for your help!


SpriteFrameCache::removeUnusedSpriteFrames doesn't work

Can't remove frames from cache, relative PR: #459
auto spriteFrameCache = SpriteFrameCache::getInstance();
spriteFrameCache->addSpriteFramesWithFile("ghosts.plist");
spriteFrameCache->removeUnusedSpriteFrames();

@rh101 Could you take a look about this issue?

Will do

No worries, I'll take a look at it now

Ok, so the issue has nothing to do with commit https://github.com/axis-project/axis/commit/4ad4bf93b0657d7bdeed9d69701e2614b621b5e8. That commit reduced the reference count from 3 to 2, which is great, but the issue doesn't come from there.
The issue is that the frames are auto-release objects, so when they are created, they have a reference of 1, like here:
SpriteFrame* SpriteFrame::createWithTexture(Texture2D* texture,
                                            const Rect& rect,
                                            bool rotated,
                                            const Vec2& offset,
                                            const Vec2& originalSize)
{
    SpriteFrame* spriteFrame = new SpriteFrame();
    if (spriteFrame->initWithTexture(texture, rect, rotated, offset, originalSize))
    {
        spriteFrame->autorelease();
        return spriteFrame;
    }

    delete spriteFrame;
    return nullptr;
}

When they're added to the cache, the reference count increases to 2.
On the next update loop, the auto-release code will decrease all frame references by 1, so they will all be equal to 1.
This works to clear the frames, since it calls it on the next loop, after the auto-release code has run:
auto spriteFrameCache = SpriteFrameCache::getInstance();
spriteFrameCache->addSpriteFramesWithFile("ghosts.plist");
scheduleOnce(
    [spriteFrameCache](float) {
        spriteFrameCache->removeUnusedSpriteFrames();
    }, 0.0f, "CLEAR_FRAMES");

This issue is in the cocos2d engine as well, and it seems that it's always been this way.
Loading them removing the frames within the same loop isn't a typical use-case, but it be good to fix it, since it will mean less code to write in order to clear the frames, and no need to wait till the next loop.
Now, a problem.  On the current loop, ref = 2, but on the next loop, ref = 1.  We can't check for ref = 2 in the SpriteFrameCache::removeUnusedSpriteFrames() method, since we don't know if it's in auto-release or not.  We can probably do this:
if (spriteFrame->getReferenceCount() == 1 || (spriteFrame->getReferenceCount() == 2 && poolManager->isObjectInPools(this)))
{
    toRemoveFrames.push_back(iter.first);
    spriteFrame->getTexture()->removeSpriteFrameCapInset(spriteFrame);
    CCLOG("cocos2d: SpriteFrameCache: removing unused frame: %s", iter.first.c_str());
    removed = true;
}

But, I'm not sure what kind of performance impact this may have, since it's checking for the object in the pool for every single frame.  The more frames, and the more objects in the pool, the slower it will be.
The alternative is removing the object from the auto-release pool, but no method exists to remove it in the AutoreleasePool class.  It only has addObject() and contains(), but what do you think if we add removeObject?
We can then call AutoreleasePool::removeObject(spriteFrame) before adding it to the cache, so the reference will be 1, like this:
void PlistSpriteSheetLoader::addSpriteFramesWithDictionary(ValueMap& dictionary,
                                                           Texture2D* texture,
                                                           std::string_view plist,
                                                           SpriteFrameCache& cache)
{
...
            spriteFrame = SpriteFrame::createWithTexture(texture, Rect(x, y, w, h), false, Vec2(ox, oy),
                                                         Vec2((float)ow, (float)oh));

...
        // add sprite frame
        PoolManager::getInstance()->getCurrentPool()->removeObject(spriteFrame);
        cache.insertFrame(spriteSheet, spriteFrameName, spriteFrame);

...
}

The other option is adding new SpriteFrame::createXXXXXX methods that do not add frames to the auto-release pool, or maybe just add a parameter to the existing methods named autorelease, with default value of true, like this:
SpriteFrame* SpriteFrame::create(std::string_view filename,
                                 const Rect& rect,
                                 bool rotated,
                                 const Vec2& offset,
                                 const Vec2& originalSize,
                                 bool autorelease = true)
{
    SpriteFrame* spriteFrame = new SpriteFrame();
    if (spriteFrame->initWithTextureFilename(filename, rect, rotated, offset, originalSize))
    {
        if (autorelease) {
            spriteFrame->autorelease();
        }
        return spriteFrame;
    }
    delete spriteFrame;
    return nullptr;
}

Which do you prefer, or do you have any ideas on how else we could deal with this?

I think it's ok current behavior same with cocos2dx


Fixed typos

Fixed typos and some small errors in math I noticed while reading through the Sets, Logic, Computation textbook build.

Thank you!


Feat rework txn struct

Reworked txn struct. Made a bunch of changes to the code using txns, to use new struct.
Also included a proposal for NativeToken so that we don't use different units in some place by misatke

Typo on the description


Reduce the Vulkan interop buffer kernel workgroup size to 256

Using local_size_x = 512 in the shader used in the Vulkan interop tests fails on older hardware because this exceeds the maximum barrier size limit.
Changed to 256, which is still a multiple of 32 and 64, and is below the hardware limitations. The tests are not performance tests anyways, so the performance penalty, if any, is acceptable.
This fixes issue #1818

Merging as discussed on Memory subgroup call of November 14, 2023.


Add artifact validation to run_rc_validation

This will ensure that the nexus repo we're about to publish is up to date and has all files we need.
2 decisions to call out here:

I decided to manually keep a list of artifacts and fail if its out of date. I did this because there's not a good way to determine which gradle artifacts should be published without walking the file system and parsing contents AFAIK. This shouldn't be too noisy and should keep itself up to date. The consequences of a bad signal are ~5 minutes of toil.
I ran this during rc_validation instead of during build_rc_candidate. This is because there's not a good way to access open staged nexus repos programatically, so you basically have to close it first.

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

Run Portable_Python PreCommit

Run GoPortable PreCommit

Run Python_Integration PreCommit

R: @chamikaramj

R: @jrmccluskey


set useQuerystring to true in request_arg

The slack Web API will give an invalid_array_arg error if you pass arrays with the default serialization to the endpoint. This pops up when trying to post, for example, messages with an array of attachements. The request library supports an alternate serialization of the form foo=bar&foo=baz by setting useQuerystring=true in the options.
See https://github.com/request/request#requestoptions-callback and https://api.slack.com/methods/chat.postMessage

Thanks!


mediumroast_js: Update installation instruction with the URL for the formal GitHub App

Introduction
The installation instructions should report to the user the URL of the GitHub application such that they are able to have it installed in their organization.

The installation instructions have been updated with key information related to the location of the GitHub app and a shell URL for the application on https://www.mediumroast.io

Verified in setup closing


Make an example request for all Forms with AJAX.

Can you make an example request for all Forms with AJAX.
Note:

The code must be maintainable
Diberikan komentar akan lebih bagus

Thank you!

I will work on this