How to make it work with Microsoft 365 and modern authentication?

Hi,
Basic authentication is no longer supported with Microsoft 365 services....
https://learn.microsoft.com/en-us/exchange/clients-and-mobile-in-exchange-online/deprecation-of-basic-authentication-exchange-online#pop-imap-and-smtp-auth
therefore, when trying to configure the plugin to gather the emails from a Microsoft 365 IMAP account, doesn't work ... We've tried to create an App Password for basic authentication but it doesn't work either.
As per thte article I've posted, the way to proceed when trying to access a mailbox from a Microsoft 365 (doesn't matter which protocol, POP3, IMAP)  is using the modern authentication mecanism, is it implemented in this camunda add-on? is there any plan to support it?
Many thanks and best regards,

Hi,
You can try using the officially supported Microsoft 365 connector from Camunda:
https://docs.camunda.io/docs/components/connectors/out-of-the-box-connectors/microsoft-o365-mail/
Which uses modern authentication methods. These could potentially be added to this community connector


istio rewrite http probe adds duplicate header

Bug description
Istio will by default enable the http probe rewrite when enable the mTLS.. However, when the pod probe definition includes the extra httpHeaders section, such as
        readinessProbe:
          failureThreshold: 3
          httpGet:
            httpHeaders:
            - name: bbbb
              value: "00000000"
            path: /http
            port: 5000
            scheme: HTTP
          periodSeconds: 15
          successThreshold: 1
          timeoutSeconds: 5

The underlying service container will receive multi header of "bbbb", not just single one.  Such as, this is one of the log about the detail request header which service container received:
User-Agent: kube-probe/1.18
Accept-Encoding: gzip
Bbbb: 00000000
Bbbb: 00000000
Connection: close

[ ] Docs
[ ] Installation
[X] Networking
[ ] Performance and Scalability
[ ] Extensions and Telemetry
[ ] Security
[ ] Test and Release
[X] User Experience
[ ] Developer Infrastructure
[ ] Upgrade
Expected behavior
There should be one single header received, not duplicate header.
Steps to reproduce the bug
Deploy any deployment on an mTLS enabled istio environment, add an extra http readiness probe with httpHeaders.
Version (include the output of istioctl version --remote and kubectl version --short and helm version --short if you used Helm)
> istioctl version --remote
client version: 1.8.2
control plane version: 1.8.2
data plane version: 1.8.2 (4 proxies)

> kubectl version --short
Client Version: v1.18.6
Server Version: v1.18.6

How was Istio installed?
Using IstioOperator
Environment where the bug was observed (cloud vendor, OS, etc)
Azure AKS, Linux ubuntu
Additionally, please consider running istioctl bug-report and attach the generated cluster-state tarball to this issue.
Refer cluster state archive for more details.

Dupe of https://github.com/istio/istio/issues/28466. Fix will be shipped with 1.8.4.

Will close since this is already fixed. Please reopen if 1.8.4 does not fixed the issue. Thanks!


⚠ Bofrost.de has degraded performance

In bf809ae, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 1567 ms

Resolved: Bofrost.de performance has improved in 1af667f after 7 minutes.


flake-info: fix compilation warnings

Removes some dead code, including the unimplemented --elastic-user, --elastic-pw and --gc flags. Builds and tests with no warnings.

--gc is used though
You can (could?) run the import on a fresh store which the gc flag would clean up after completion.
I dont know if that is still useful to anyone though and could probably be implemented by just passing extra nix arguments

for the elastic-pw/user one I wonder if it would be better to implement these rather than remove them and use those flags rather than the url encoded authentication

--gc is used though You can (could?) run the import on a fresh store which the gc flag would clean up after completion. I dont know if that is still useful to anyone though and could probably be implemented by just passing extra nix arguments

It's not used. The run_gc function is never called. So I think it's safe to assume it's not useful to anyone :)

for the elastic-pw/user one I wonder if it would be better to implement these rather than remove them and use those flags rather than the url encoded authentication

Sure, you can do that in a follow-up PR, I'm just commenting them out for now.

I agree in both points

@ysndr could you approve please?

Thanks


cli: update-tree: mutating command line args, rich exception handling, etc.

Codecov Report

Merging #94 (b9130c1) into main (19b31de) will decrease coverage by 1.98%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##             main      #94      +/-   ##
==========================================
- Coverage   29.60%   27.62%   -1.99%
==========================================
  Files          36       36
  Lines        1949     1991      +42
  Branches      359      376      +17
==========================================
- Hits          577      550      -27
- Misses       1358     1435      +77
+ Partials       14        6       -8

Impacted Files
Coverage Δ

src/dvc_data/__main__.py
0.00% <0.00%> (ø)

src/dvc_data/cli.py
0.00% <0.00%> (ø)

tests/conftest.py
50.68% <0.00%> (-36.99%)
:arrow_down:

src/dvc_data/build.py
0.00% <0.00%> (ø)

src/dvc_data/db/index.py
0.00% <0.00%> (ø)

src/dvc_data/hashfile/state.py
31.13% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 19b31de...b9130c1. Read the comment docs.


[WIP] Add ability to create and execute Stripe fixtures

Scenario:
User wants to create and execute Stripe fixtures. Maybe from an existing event? And to be able to customize the data emitted.
Create fixtures

In the event tree view, we could add new context menu items so a context menu would show:
Trigger this event again
Create fixture from event
User clicks on  Trigger this event again  => stripe trigger <eventname gets run
User clicks on Create fixture from event
VS Code grabs the matching fixture from https://github.com/stripe/stripe-cli/tree/master/triggers
Stores the content in a new file inside the workspace: ./.stripe/fixtures/<event name.json
Opens a new editor with ./.stripe/fixtures/<event name.json

How to execute fixtures?

Option 1:  New command to execute current open file as fixture?
Option 2: New list of fixtures in a view or something and right click option to execute?

Related (but not perfectly related) feedback from user feedback:
"it would be super cool to have something like "send new subscription created with a card sucessfully via connect" or "new subscription payment is about to be charged" or "new payment was charged successfully" - so that it delivers all the events as they would normally come. even cooler would be to be able to specify some basic params for these event-chains, e.g. stripe customer id for the  "new subscription payment is about to be charged" or stripe subscription id for the "new subscription payment is about to be charged"."

feature released in v2.0.8
https://jira.corp.stripe.com/browse/DX-6881


Server not working

I am trying to set alarms up with the server. I ran
sudo python main.py -s
The unicornhat package needs to be installed !
The LightUpHardware folder contains a README file with more info.
Mock unicornhat module imported.
Running Python version 2.7.9

======= Parsing Command line arguments =======
Server selected

=========== Launching LightUpPi Alarm ==========
 * Running on http://<IP_ADDRESS>:8000/
<IP_ADDRESS> - - [20/Dec/2016 18:14:18] "GET /LightUpPi/ HTTP/1.1" 304 -
<IP_ADDRESS> - - [20/Dec/2016 18:15:07] "GET /LightUpPi/ HTTP/1.1" 304 -

and the webpage loads but it looks like:

when I click okay and the page loads it just says.

Note: I manually changed the ip address and port so it would not confilct with another application of mine.

It's been a year an a half since the last time I worked on this, so I honestly cannot remember if the web interface was working correctly if hosting from the pi and accessing from another device on the network. The mobile app should have been working correctly as I used that mostly, accessing the web interface from the raspberry pi as well should be working too, as I was doing a lot of development that way.
There is an issue already opened similar to this one. At the moment I am a bit busy to work on this right now, but I am planning to finish it up around February. I'll see if I can give it a quick go during this Christmas break first to at least get this server component working or clearly stating what still needs to be implemented.
Having a very very brief look I see a lot of hardcoded localhost on https://github.com/carlosperate/LightUpPi-Alarm/blob/master/LightUpWeb/js/lightuppi_controller.js so you could try to edit that with the IP of your Pi and see if it works?


WE-676 implement message comparison

Fixes
This pull request fixes WE-676
Description

Implement message comparison functionality
Rework displayMissingLogModal to accept any ObjectType.

Type of change

New feature (non-breaking change which adds functionality)

Impacted Areas in Application

Frontend

Checklist:
Communication

[x] PR is related to an issue
[ ] I have made corresponding changes to the documentation

Code quality

[x] Code follows the style guidelines
[x] I have self-reviewed my code
[x] No new warnings are generated

Test coverage

[x] Existing tests pass
[ ] New code is covered by passing tests

Greenlit by @marmid74 and Thomas Halland to merge


Achele - Single Listing Communty Page details

how is this going? any blockers

@ibimina I’m waiting for the Single listing community page to be merged. Need to work off that branch

okay

ADOPTED! 🎉  @Achele leave the Focus Areas implementation out of this work, we do not have time to build that feature the way it should be.

Okay


LCFS - Show Organization Details on BCeID dashboard

Description:
Show the organization details on BCeID dashboard:

Show operating name of organization
Show address for service
Show organization telephone and email
"Users" link to Organization page
"Create new BCeID account" links to https://www.bceid.ca/

Wireframe: Private Zenhub Image
Purpose and benefit to user:
User is displayed their organization information
Acceptance Criteria:

[ ] Given I am a , When I am , then
[ ] Given I am a , When I am , then

Development Checklist:

[ ] A
[ ] B
[ ] C

Notes:

@Kailee-Douglas @airinggov the wireframe I included is based on this mockup from UX pin (https://preview.uxpin.com/59145260c7d8cc48969ac79401cc03d84843c3d9#/pages/164349076/simulate/sitemap). Can we revise the mockup of the Organization Details portion of the BCeID dashboard to include the organization telephone number and email (these are new info requirements from the new Act). See org details page for example info: https://preview.uxpin.com/59145260c7d8cc48969ac79401cc03d84843c3d9#/pages/164329808/simulate/sitemap

@justin-lepitzki  here is an updated mock-up for your review: https://preview.uxpin.com/59145260c7d8cc48969ac79401cc03d84843c3d9#/pages/164908212

Hey team! Please add your planning poker estimate with Zenhub @hamed-valiollahi @kevin-hashimoto @jig-patel @prv-proton @AlexZorkin

@Kailee-Douglas can we add the grey / white contrast into the wireframe? The current screenshot in the card description has it over the whole page, but I think we should have it around the white box of the Organization Details widget/box.
Example: https://api.zenhub.com/attachedFiles/eyJfcmFpbHMiOnsibWVzc2FnZSI6IkJBaHBBMzdwQVE9PSIsImV4cCI6bnVsbCwicHVyIjoiYmxvYl9pZCJ9fQ==--b0cabb69d229e140fe90a85ab728d9f1262f2444/Dashboard-BCeID-Org_Details.png

Hello @justin-lepitzki  is this what you are envisioning? https://preview.uxpin.com/59145260c7d8cc48969ac79401cc03d84843c3d9#/pages/164908212/simulate/no-panels
As we gradually add more content, we can update the size of the grey contrast in new associated mock-ups.

@Kailee-Douglas looks great! I have updated the wireframe link. Thank you :)

Requires access to BCeID user account to test.


Verify whether Validator metrics port on Lighthouse and Lodestar should be 5056 (vs. 5054?)

In Run a Validator section, for Lighthouse and Lodestar, there are two metrics ports defined that are different

Consensus section: uses 5054 as metrics port, maps 5054:5054/tcp to localhost port 5054
Validators section: uses 5056 as metrics port

Question

Do Consensus and Validators use different metrics ports?
If they do, do we need to map 5056:5056/tcp to also expose that to the user?

@CVJoint - FYI (cc @alebanzas)

Lighthouse Docs seem to suggest 5064
https://lighthouse-book.sigmaprime.io/advanced_metrics.html#validator-client-metrics
Lodestar Docs seem to suggest 8008
https://chainsafe.github.io/lodestar/usage/prometheus-grafana/

Will change it to the default ports

Beacon node and validator have different metric ports (Lighthouse and Lodestar):

Lighthouse
Lodestar
Teku

Beacon
5054
8008
8008

Validator
5064
5064
8008

For Teku, will use 5064 as validator metrics port


[Bug] Connection fails with: "Error: self signed certificate in certificate chain"

What did I do
Using MQTTX exactly as before with CA cert, client cert and key.
What happened
"Error: self signed certificate in certificate chain"
Broker logs just show:
1701288672: New connection from x.y.z.w:64544 on port 8883.
1701288672: OpenSSL Error[0]: error:0A000126:SSL routines::unexpected eof while reading
1701288672: Client  disconnected: Protocol error.
Expected
Connection should proceed as normal and not get terminated due to usage of self signed certs.
Environment
Win10 Pro
MQTTX 1.9.6, also tried some order versions and same thing happens. Tried going back t 1.8.0 but the GUI does not start with that or older versions at all anymore even though the processes are running.

MQTTX logs show:
2023-11-29 22:10:52] [INFO] APP init
[2023-11-29 22:11:11] [INFO] MQTTX client with ID e9ac200d-19d0-45e9-ac52-bc4fdd064d41 assigned
[2023-11-29 22:11:11] [INFO] Connect client Demo, MQTT/SSL connection: mqtts://x.y.z.w:8883
[2023-11-29 22:11:11] [ERROR] Demo connect fail, MQTT.js onError trigger, Error: self signed certificate in certificate chain
at TLSSocket.onConnectSecure (_tls_wrap.js:1497:34)
at TLSSocket.emit (events.js:315:20)
at TLSSocket._finishInit (_tls_wrap.js:932:8)
at TLSWrap.ssl.onhandshakedone (_tls_wrap.js:706:12)

Also a strange thing is I cannot find this error text in the MQTTX code to check why this happens.

Ok so the error is coming from Node itself. Online sources say that env variable NODE_TLS_REJECT_UNAUTHORIZED has to be set to 0 for this to work.
I did that on the command line (Windows), then started mqttx.exe manually from same shell but same error comes. Node acknowledges that certificate verification is disabled:
[2023-11-29T22:44:45.457] [INFO] default - APP init
[2023-11-29T22:44:47.522] [INFO] default - MQTTX client with ID 2c194093-99ac-4fcd-aad6-86adaa85c7f9 assigned
[2023-11-29T22:44:47.524] [INFO] default - Connect client Demo, MQTT/SSL connection: mqtts://x.y.z.w:8883
(node:5828) Warning: Setting the NODE_TLS_REJECT_UNAUTHORIZED environment variable to '0' makes TLS connections and HTTPS requests insecure by disabling certificate verification.
(Use MQTTX --trace-warnings ... to show where the warning was created)
[2023-11-29T22:44:47.697] [ERROR] default - Demo connect fail, MQTT.js onError trigger, Error: self signed certificate in certificate chain
at TLSSocket.onConnectSecure (_tls_wrap.js:1497:34)
at TLSSocket.emit (events.js:315:20)
at TLSSocket._finishInit (_tls_wrap.js:932:8)
at TLSWrap.ssl.onhandshakedone (_tls_wrap.js:706:12)
[2023-11-29T22:44:47.709] [INFO] default - Demo connect close, MQTT.js onClose trigger

So, no progress really, not sure if this is the root cause. But see
https://github.com/nodejs/node/blob/8787acb1e62a48cf77c361b571b7aa641e59fb99/lib/internal/options.js#L54C1-L54C1

The way I set the environment variable for the user is
[Environment]::SetEnvironmentVariable("NODE_TLS_REJECT_UNAUTHORIZED", "0", "User")
I checked that the variable is set in a separate shell:
gci env:
....
NODE_TLS_REJECT_UNAUTHORIZED   0
...
So this is not effective as a fix but let me know if you have any ideas. One suggestion would be to add
process.env['NODE_TLS_REJECT_UNAUTHORIZED'] = 0;
to MQTTX code.

Seems like it might have been an error on my side after all. I recreated the certificates and then MQTTX connection works.

The issue can be closed.


Integrate Fermionic QAOA(FQAOA) Module and Associated Files into OpenQAOA

Description
This pull request introduces the FQAOA (Fermionic Quantum Approximate Optimization Algorithm) module along with its associated files into the OpenQAOA framework. The integration includes the following components:

fqaoa_workflow.py: Implements the FQAOA workflow, enabling the use of fermionic methods within the OpenQAOA framework.
fqaoa_utils.py: Provides utility functions to support the FQAOA implementation, including essential operations and helper functions.
test_fqaoa.py: Contains unit tests for the fqaoa_utils.py, ensuring the correctness and reliability of the new functionality.
test_workflows.py: Tests the overall workflow integration, confirming that FQAOA integrates smoothly with existing components.
test_analytical_simulator.py: Includes tests for the analytical simulator, verifying the exception handling in the FQAOA approach.
16_FQAOA_examples.ipynb: A Jupyter notebook demonstrating an example of solving the portfolio optimization problem using FQAOA. This notebook showcases the practical application of the FQAOA method and provides a step-by-step guide for users.

All tests have been successfully executed, demonstrating the stability and effectiveness of the integrated features. This addition enhances the OpenQAOA framework by introducing the capability to handle constrained combinatorial optimization problems using FQAOA.
Checklist

[x] I have performed a self-review of my code.
[x] I have commented my code and used numpy-style docstrings
[x] I have made corresponding updates to the documentation.
[ ] My changes generate no new warnings
[x] I have added/updated tests to make sure bugfix/feature works.
[x] New and existing unit tests pass locally with my changes.

Type of change
Please delete options that are not relevant.

[x] New feature (non-breaking change which adds functionality)

How Has This Been Tested?
The following unit tests have been added to ensure the robustness and accuracy of the new features:
test_fqaoa.py: Tests the core functionality of the fqaoa_utils.py.

test_workflows.py: Validates the integration of fqaoa_workflow.py within the overall workflow.

test_analytical_simulator.py: Confirms the exception handling in the FQAOA approach.

I've just pushed an additional notebook 17_FQAOA_advanced_parameterization.ipynb to this branch that provides a tutorial on advanced circuit parametrization, including Fourier parametrization for FQAOA. This notebook is intended to demonstrate more complex usage scenarios of FQAOA and complement the existing documentation.

Thank you for your review and the kind words!
I appreciate your feedback and will go through the comments to make the suggested changes.
I’ll also review the typos and other minor issues you pointed out.
I will pay particular attention to the functions, such as givens rotation, to make sure that they are correct.
Thanks again for your time and valuable feedback!

Thank you for the detailed review! I've pushed a new commit that addresses the typos and minor corrections as per your comments. Here's a summary of the changes:

Fixed typos and made small corrections based on the review feedback.
Updated variable names and improved documentation for clarity and consistency.

Please note that the error mitigation SPAM twiling is still pending and will be tackled in a separate commit.
I've also performed a final spell check using codespell and aspell to ensure there are no lingering issues. I apologize for any inconvenience caused by the previous commits. Let me know if you have any further suggestions or if anything needs additional adjustments!

I want to express my sincere gratitude for your incredibly detailed review.
Your comments have been invaluable in helping me improve the quality of this pull request.
I'm truly grateful for your time and expertise.
If you have any further concerns or suggestions, please let me know.

I've added a risk factor to the objective function and updated the related tests.
This risk factor was included in the argument but not reflected in the objective function.
Integrate risk factor into objective function and update related tests

Modified the objective function to include risk factor in calculations.
Updated tests to account for changes in the objective function.
Adjusted Jupyter notebooks to reflect the new implementation of the risk factor.


Implement browser metrics in a new page

Visualize browser metrics in UI, as following

cc: @Jtrust

👌

Maybe could we try to replace the bar chart with a line chart？ Because the line chart can better show the trend of the data over time. @Fine0830

@Jtrust I saw the prototype pages, only comparison charts are using the bar chart. Which one are you asking?

@wu-sheng App PV and Error sum in the duration

Maybe could we try to replace the bar chart with a line chart？ Because the line chart can better show the trend of the data over time. @Fine0830

Sure. Line and bar are linear charts,  so they can do both.

Oh, if that is for one app, or one page, then yes, it should be a linear graph

The first version will be provided through #5686

One last thing is the log page seems to have some incorrect names.


add happi support

Description

Adding happi support for REManagerAPI objects.
Motivation and Context

End users can find it cumbersome to manually create REManagerAPI objects due to esoteric (to them) arguments. Happi allows them to grab such objects quickly. Happi is familiar to users who are already using it for device management.
Summary of Changes for Release Notes

Added

REManagerAPI objects can now be managed by happi.

How Has This Been Tested?

No tests.

@dmgav any thoughts on this PR?

I was thinking about this idea. I am not against merging the PR, but is there a use case for this? I always thought happi was intended primarily for Ophyd objects.

I'm weakly in favor of this, but I think that the happi dependency should be put in an extra dependency flag rather than being a hard dependency.
Should probably also be in a public module?

This looks convenient to me.
First-class support of happi - from RE manager objects to using happi items to instantiate devices for usage - in the queue server would be great.
cc @ZLLentz @tangkong

Just FYI when I actually glanced at this last night I noticed that I totally forgot to add the actual entry points. So please don't merge until I do that. I should probably also add tests so that we know the idea actually works.

I always thought happi was intended primarily for Ophyd objects.

I think about Happi as a "configuration file" for any objects that can be instantiated with serializable parameters. We have other mechanisms for recognizing which of those are devices.

Should probably also be in a public module?

I'll think about this. The way I'm personally used to using Happi I don't ever dig down and make items by hand using Python. But I haven't really played with this workflow.
I'm hearing "we're okay adding this" so please give me a moment to actually implement something that works practically.

First-class support of happi - from RE manager objects to using happi items to instantiate devices for usage - in the queue server would be great.

This is my dream, to use Happi as the one true reference for trucking configuration to the queueserver and back.


Update YARN images

Depends on https://github.com/kbase/cdm-prototype-yarn/pull/7

Sigh...
I don't understand why it would core dump here but not in the yarn repo.
The line it's core dumping on is the very last line:
"$@"

$@ should be yarn [resource|node]manager

Might be a docker version / MacOS version / Rosetta issue: https://github.com/docker/for-mac/issues/7172


user-home-test branch

As mentioned in neurostars, the last few versions of the fmriprep docker images fail to run when specifying a user (docker run -u myuser ...).
@oesteban created a new docker image (poldracklab/fmriprep:user-home-test).
This new image fixes the permissions problem with $TEMPLATEFLOW_HOME.  However, now I get an error similar to the one in a different Neurostats post when processing the (GRE) fieldmap:
Node: fmriprep_wf.single_subject_Pilot005_wf.func_preproc_ses_day1_task_TASK_acq_normal_run_01_echo_1_wf.sdc_wf.phdiff_wf.meta
Working directory: /tmp/work/fmriprep_wf/single_subject_Pilot005_wf/func_preproc_ses_day1_task_TASK_acq_normal_run_01_echo_1_wf/sdc_wf/phdiff_wf/meta

Node inputs:

bids_dir = None
bids_validate = False
fields = <undefined>
in_file = /data/phelpslab/Linda/BIDSdata/sub-Pilot005/ses-day1/fmap/sub-Pilot005_ses-day1_acq-GRE_run-01_phasediff.nii.gz
undef_fields = False

Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/plugins/multiproc.py", line 69, in run_node
    result['result'] = node.run(updatehash=updatehash)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 473, in run
    result = self._run_interface(execute=True)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 557, in _run_interface
    return self._run_command(execute)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 637, in _run_command
    result = self._interface.run(cwd=outdir)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/base/core.py", line 371, in run
    outputs = self.aggregate_outputs(runtime)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/base/core.py", line 472, in aggregate_outputs
    raise error
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/base/core.py", line 465, in aggregate_outputs
    setattr(outputs, key, val)
  File "/usr/local/miniconda/lib/python3.7/site-packages/traits/trait_handlers.py", line 172, in error
    value )
traits.trait_errors.TraitError: The 'run' trait of a ReadSidecarJSONOutputSpec instance must be a unicode string, but a value of 1 <class 'int'> was specified.

This error was supposed to be fixed after v.1.3.0.post2, so I'm not sure from which version the user-home-test branch was created...
Thanks.

Hi @pvelasco, we've just released 1.3.0.post3 that should take care of both issues.
I'm going to close this one in favor of the neurostars thread (https://neurostars.org/t/singularity-fmriprep-permissionerror-errno-13-permission-denied-cache/3693). Please feel free to reopen if this is still an issue.

Hi @oesteban,
I tested 1.3.0.post3 (specifying a user) and I got a different error (also related to permissions inside the docker image):
Process Process-2:
Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/usr/local/miniconda/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/cli/run.py", line 755, in build_workflow
    err_on_aroma_warn=opts.error_on_aroma_warnings,
  File "/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/workflows/base.py", line 218, in init_fmriprep_wf
    err_on_aroma_warn=err_on_aroma_warn,
  File "/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/workflows/base.py", line 516, in init_single_subject_wf
    num_bold=len(subject_data['bold']))
  File "/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/workflows/bold/base.py", line 399, in init_func_preproc_wf
    bold_reference_wf = init_bold_reference_wf(omp_nthreads=omp_nthreads)
  File "/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/workflows/bold/util.py", line 121, in init_bold_reference_wf
    omp_nthreads=omp_nthreads, pre_mask=pre_mask)
  File "/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/workflows/bold/util.py", line 302, in init_enhance_and_skullstrip_bold_wf
    'epi_atlasbased_brainmask.json')),
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/ants/registration.py", line 935, in __init__
    super(Registration, self).__init__(**inputs)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/ants/base.py", line 76, in __init__
    super(ANTSCommand, self).__init__(**inputs)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/base/core.py", line 645, in __init__
    super(CommandLine, self).__init__(**inputs)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/base/core.py", line 182, in __init__
    self.load_inputs_from_json(from_file, overwrite=True)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/base/core.py", line 495, in load_inputs_from_json
    with open(json_file) as fhandle:
PermissionError: [Errno 13] Permission denied: '/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/data/epi_atlasbased_brainmask.json'

The problem is that the package_data in fmriprep gets installed with the same permissions as in /src/fmriprep, which are -rw-rw---- (only root and its group have rw access).
I have a fix for it, and will be submitting a PR shortly.
(Note: I can only submit PRs to branches, not tags, so which branch do you want me to submit the PR to?  To master, since the problem is still there?)

This is surprising, why tests would then even work?. Yes, send the PR to master, please.

Sorry, I got it wrong: in the master branch, the permissions are correct.
I think the problem is the --no-cache-dir in the pip install .[all]
I tried building the Docker image for 1.3.0.post3 with
pip install .[all]

(omitting --no-cache-dir) and it runs for a regular user.
Bottom line: tag 1.3.0.post3 is fine except for the --no-cache-dir.
So I'm closing the PR.  Thanks a lot for your help!

Hi, the latest release 1.3.1 is out. Please let us know if that version resolves this problem!

Hi @oesteban,
Yes, it does.  It works fine.
Thanks a lot!


Add float precision math support

As we all know, 0.1+0.2 does not produce 0.3 in Javascript, as one would expect. Instead, it results in 0.30000000000000004. The issue is explained in greater detail in this SO question.
Numeral has add, subtract, multiply, divide and difference methods, which currently naively do math operations with floats and give the same kind of non-correct results.
This PR fixes the issue with math operations with floats. The implementation is borrowed from sinful.js, with some modifications to make it a bit lighter.
So the below now works:
numeral(0.2).add(0.1).value(); // 0.3
numeral(0.3).subtract(0.1).value(); // 0.2
numeral(0.1).multiply(0.2).value(); // 0.0.2
numeral(5.3).divide(0.1).value(); // 53

@heme you ever find a resolution for this? sorry to bring this back from so long ago

@tsjoberg nope. I think we ended up using 2 libraries.
decimal.js for the maths
and numeral for the formatting.
example: http://jsfiddle.net/AYZ63/6/


Provide a link to the image release in logs

Description
In scope of this PR we are adding a link to the image release in logs.
Related issue: #1860
Check list

[x] Related issue / work item is attached
[ ] Tests are written (if applicable)
[ ] Documentation is updated (if applicable)
[ ] Changes are tested and related VM images are successfully generated

Should this work? (log from the last couple of minutes)
Current runner version: '2.277.1'
Operating System
Virtual Environment
  Environment: ubuntu-20.04
  Version: 20210309.1
  Included Software: https://github.com/actions/virtual-environments/blob/ubuntu20/20210309.1/images/linux/Ubuntu2004-README.md
  Image Release: https://github.com/actions/virtual-environments/releases/tag/ubuntu20%2F

@jsoref thanks for bringing this up! It looks like the variable name is wrong — should be image_version instead of imageVersion


TS 1.5.4, VS2015, compile on save is not working

I've tested this on two different machines with vs2015 pro and enterprise versions. 1.5.4 broke the compile on save.  ( I didn't touch default settings, compilation type is AMD).  When I reverted to 1.5.3 it worked fine again.

Hm, we shipped 1.5.4 in part to fix a compile on save issue in 1.5.3 with --out https://github.com/Microsoft/TypeScript/issues/3939, haven't seen other reports like this yet. Are you using any other tools/extensions in your build process that could be affecting it?

is it like this #3949?

Here are all the VS extensions I have. I have neither tsconfig.json nor Redirect JavaScript output to directory set (Project type is Telerik Appbuilder project and it doesn't have such an option). What I notieced is when I saved .ts file, I got a visual studio message in the left bottom corner which said "output generation cancelled".

What kind of project is this, can you share a small sample project this happens in?

Here is the sample template project telerik provide
https://github.com/Lanayx/TelerikAppbuilderTypescriptSample
To run it you need to create a temp account here: https://platform.telerik.com/. Also, Telerik Appbuilder extension is needed. What I expect is when saving index.ts file, index.js should also be changed. It has been working for me more than a year before 1.5.4 typescript version.

Is this the only project type where Compile on Save doesn't work for you?

I've found out the actual issue, that happens for any type of project.
It depends on the TypeScriptCompile tag for the file. If it exists, compile succeeds, if not it fails. All my existing files doesn't have such a tag, that's why none of them compiles.
Currently, if you add a .ts file through the Add new item, this tag is added to the project file, but if you copy a .ts file from existing one, this tag is not added and compile on save fails.

I'm running 1.5.4 and I still have the problem when using "${ProjectDir}/output" as the place to compile the files. If I specify the absolute path, compile on save works as expected.

@cannehag Yes, the aliases don't work for 1.5.4, that's something we're looking at for 1.6.

@Lanayx where do you copy the file from, a project in the same solution or a file from the file explorer?

A project in the same solution. Haven't you reproduced that? What I want is files to be compiled even without TypeScriptCompile tag like in previous versions.

Hey,
Both myself and a colleague have hit this issue today.
We have a number of projects that use TypeScript, each .TS file has been added through the Add New Item menu and so each one has a TypeScriptCompile element in the project file. The projects in question are either library or console projects.
We are compiling our typescript as AMD modules, the JavaScript files are not combined or minified or have any other processing done to them as they are simply embedded resources in the assemblies.

@ProfessionalNihilist did that work before? Which version of VS are you using? And to confirm you're using a "Windows Console App".

@Lanayx I couldn't reproduce that with any of the standard project in Visual Studio, it seems to only happen in the Telerik Appbuild project, and in there Compile on Save works as expected when you set the content type to TypeScriptCompile.
So closing this as External.
@ProfessionalNihilist since you're issue doesn't seem related I copied it over to #4350

@ProfessionalNihilist  @Lanayx  do you see "output generation cancelled" when you save ts files?

@Serguzest - yes, I do

I get the "Output generation canceled" too.

Me either. Had a big project developed in VS 2015 RC without any problem, and now after the uprade it's happing even if i open a new templated ts project.

Verify your tsconfig.json file.  I was getting the Output generation canceled message and when I added my .ts file to the config, it worked.

Already tried without success... Thanks

@dorad007 can you try the latest 1.6-beta release

Thanks for your reply, I didn't understand how to install it into vs. Anyway, I downgraded to vs2015rc until it's expirtaion, hope the issue to be resolved.

why not use VS 2015 RTM if i may ask. you can always install community edition, with no expiration periods.

Because of this issue the community edition is useless for me, ts is not compiling on save...

@dorad007 this issue is fixed in latest release: https://github.com/Microsoft/TypeScript/releases/tag/v1.6.0-beta, can you give this a try?

I have checked, 1.6.0-beta does fixes the issue

OK, Reinstalling.

@Lanayx there are multiple issues referenced in this post. which issue are we talking about here? is it copying files do not set their content type to "TypeScriptCompile"? or something else?

@mhegazy It's about the header issue - compile on save was broken in 1.5.4 and fixed in 1.6.0, I guess all the issues in this thread originate from the same root.

@Lanayx there are a few different issues. if 1.6 beta is not working for you, then your issue is one we are not aware of. i would appreciate it if you can point us to which issue you are still running into so we can fix it.

I can confirm that 1.5.4 also broke compile on save for ts file without a project (or with virtual project). Steps to replicate: File - New - File - TypeScript File then add anything valid TS - save to desktop (or where ever), then edit and save again. The js is not updated with new compilation. I am running VS 2015 Community.

Just tried installing 1.6.10 (beta) and it fixes the compile on save for non-project ts file.

I reinstalled VS2015 Community, opened a new project and put "npm install -g typescript" in command line

What else do i have to do, because it's not working yet.

It's working. thanks!


{AKS} Refactor a thirteenth part of the create sub-command in acs module

[DO NOT MERGE]
Description

refactor identity & identity profile & auto upgrade profile

Testing Guide

History Notes

[Component Name 1] BREAKING CHANGE: az command a: Make some customer-facing breaking change.
[Component Name 2] az command b: Add some customer-facing feature.

This checklist is used to make sure that common guidelines for a pull request are followed.

[ ] The PR title and description has followed the guideline in Submitting Pull Requests.

[ ] I adhere to the Command Guidelines.

[ ] I adhere to the Error Handling Guidelines.

AKS


Default Environment Backup and Restore

How do I backup and restore the Default Environment?
We are building a Project for the Web solution in here and had a situation that would have been resolved by restoring yesterday's backup; however, there are no backups for the Default Environment.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: d9c58fb1-e6a2-5210-3774-4ea4bbff4572
Version Independent ID: 57c1780c-b097-af8c-82fa-47e38a63995c
Content: Back up and restore environments - Power Platform
Content Source: power-platform/admin/backup-restore-environments.md
Service: power-platform
GitHub Login: @jimholtz
Microsoft Alias: jimholtz

@donalmc
Thanks for the feedback. We've updated the following topics:
https://docs.microsoft.com/en-us/power-platform/admin/backup-restore-environments#manual-backups
https://docs.microsoft.com/en-us/power-platform/admin/environments-overview#the-default-environment


[WIP at best] Quick and dirty tcp support

Featuring:

Improper error management
Impropper shutdown/reload support
Bad code
Breaking code
Ineffective code
A lot of "funny" prints/noise.

Config like:
[entryPoints]
  [entryPoints.tcp]
  mode = "tcp"
  address = ":2222"

[file]
[backends]
  [backends.backend1]
    [backends.backend1.servers.server1]
    url = "<IP_ADDRESS>:22"

[frontends]
  [frontends.frontend1]
  backend = "backend1"
  entrypoints = ["tcp"]
    [frontends.frontend2.routes.test_1]
    rule = "Host:backend2.com"

This sort of features has been requested by multiple users.
We have been discussing cmux as a possible solution. Have you looked at this?

We have been discussing cmux as a possible solution. Have you looked at this?

Nope, I haven't, but it could definitely make it simpler. I will take a closer look, if I can find some time.
How do you think the configuration should look like? A tcp/mode switch for both frontend/backend?

@dtomcej Are your discussions in a public forum? I have a need to use TCP with Kubernetes and would like to get involved if I could.

@andrewrynhard you can chat with us on slack!
http://traefik.slack.com/

I'm closing this as we have switched to Kubernetes and therefor switched to the Nginx ingress controller. Also https://github.com/containous/traefik/issues/1174 is a very serious issue, which is now fixed.


Send emailList as string when removing from workspace

Details
We're currently sending an array of strings to the API when removing a user from a workspace which isn't working on iOS (the API receives nothing). In the Web repo for the same command we send a comma-separated list of emails, and the API expects a comma-separated list as well, so this PR updates the usage here to be in line with our other usages.
Fixed Issues

$ https://github.com/Expensify/App/issues/4299
Tests/QA

Launch the app
Click on FAB menu
New Workspace
Click on Get Started!
Select People
Invite 3 users to your workspace
Select one of the users/ user and Click Remove
Click Remove button on the Remove Members popup, confirm user is removed successfully

Tested On

[x] Web
[x] Mobile Web
[x] Desktop
[x] iOS
[x] Android

Screenshots
Mobile

Web

:hand: This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

🚀 Deployed to staging         in version: 1.0.81-5🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
failure ❌

🕸 web 🕸
success ✅

🚀 Deployed to production         in version: 1.0.82-7🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅


Record timings for Shoot creation and deletion operations

How to categorize this PR?

/area monitoring
/kind enhancement
What this PR does / why we need it:
Add prometheus Histogram metrics for Shoot Create and Delete operations and expose via gardenlet metrics.
Glimpse into possible monitoring dashboard for the metrics added::

Which issue(s) this PR fixes:
Result of 2024-12 Gardener Hackathon.
Special notes for your reviewer:
/cc @timebertt
Release note:

Add `gardenlet` metrics for `Shoot` creation and deletion timings.

/close
In favor of #10971.
/cc @simcod


RabbitMq connection retry-task should swap connection under lock

And only if existing connection is bad.

Updated csproj file to generate nuget-package
Updated to NLog 4.5.0 RTM

Resolves #1 , Resolves #4 , Resolves #5
To generate nuget-package:
dotnet pack src\Nlog.RabbitMQ.Target --configuration Release --include-symbols

Thanks! I hope this gets merged into v.soon!

Discovered that I changed the behavior when buffer was full, then it didn't attempt to restart the connection. Created #11 to resolve this.


Better opacity scaling for SZ map segments

@austinjpaul proposed using a linear scale on slow time instead of % slowdown. I think this does a much better job at emphasizing the problem areas. Thoughts?
Before:

After:

I wanted to play with a square root curve as well to give more variation in the low end, while retaining visible differences in the high end.
 Thoughts?


Npm install problems

webpack built 6e3e56702d892b1a769e in 1565ms
Version: webpack 1.15.0
Time: 1565ms
            Asset     Size  Chunks       Chunk Names
static/js/main.js  2.22 MB       0       main

ERROR in ./~/node-xmpp-client/lib/Client.js
Module not found: Error: Cannot resolve module 'child_process' in C:\Dev\clinical.messaging\node_modules\node-xmpp-client\lib
 @ ./~/node-xmpp-client/lib/Client.js 15:11-35

ERROR in ./~/node-xmpp-client/lib/session.js
Module not found: Error: Cannot resolve module 'tls' in C:\Dev\clinical.messaging\node_modules\node-xmpp-client\lib
 @ ./~/node-xmpp-client/lib/session.js 3:10-24

ERROR in ./~/node-xmpp-core/lib/Connection.js
Module not found: Error: Cannot resolve module 'net' in C:\Dev\clinical.messaging\node_modules\node-xmpp-core\lib
 @ ./~/node-xmpp-core/lib/Connection.js 3:10-24

ERROR in ./~/node-xmpp-core/lib/SRV.js
Module not found: Error: Cannot resolve module 'dns' in C:\Dev\clinical.messaging\node_modules\node-xmpp-core\lib
 @ ./~/node-xmpp-core/lib/SRV.js 3:10-24
webpack: Failed to compile.

To install through npm all I used was
npm install node-xmpp

try node-xmpp-client

The above error is shown in cmd when I use npm start. I did try using npm install node-xmpp-client' and I get the Cannot GET /` error. I'm just confused on how to implement node-xmpp into my node.js project.

It appears that some of the dependencies are not included maybe?

@jonator plugins are for the rewrite which is not released yet.
I don't understand what you're trying to do, why do you use webpack or npm start?

I have a project with an elm front-end and node/express back-end. I need a good library for using xmpp to communicate with ports to the front-end
And I don't know why we're using webpack, I forked this project as a template to work with elm

My pasted console result above is from using npm start. It's giving errors from the node-xmpp-core module, stating that there are unresolved sub-modules. By trying to require('node-xmpp-client') I get the Cannot GET / on my webpage, sorry for the horrible wording above.

Are you trying to use node-xmpp-client client or server side?

client side right now

@jonator I've never really used webpack so I don't know how it works but with some googling I found this https://github.com/webpack-contrib/css-loader/issues/447#issuecomment-285598881
An other solution is to use the browser bundle.
<script src="node_modules/node-xmpp-client/browserify.js"></script>

window.XMPP

In any case if your project can wait a little bit, we're gonna release the rewrite soon (before Jully 1) and it should be much easier to use with webpack.

Ok, thanks for the support it is appreciated!

This is a really old issue but here's a solution that worked for my project. Add this to your webpack config
node: {
    net: 'empty',
},


Introduce utility method to look up drive props by drive name.

Introduce get_properties_by_drive_name() utility method. This looks up the drive props against the name of the drive. It also checks for duplication in the database.
Use it in web.py to reduce code duplication, and to remove client side parameters in drive.html (somewhat more secure)

This change shaved almost 8k off the size of drives.html ;)

@rdmark Please can you check the tests are passing before merging. I've approved as I think the fixes might be in another one of your PRs already.


Inform collection exercise when a sample is validated

What and why?
Adding collection exercise client so sample can call back to collection exercise to inform it that a sample has been validated and distribute. These calls allow collection exercise to transition it's state.
How to test?
Trello

/deploy baileyw

/deploy baileyw


ci: Increase timeout for coverage job

Description
This is in hopes of reducing tarpaulin time below 15 mins,
so it fits in GHA timeout.
Right now it times out after 15 mins:
https://github.com/kubewarden/kwctl/actions/runs/11442780601
Relates to https://github.com/kubewarden/kwctl/issues/852.

Test
Tested locally, but cannot know about GHA timeout.

Additional Information
Tradeoff

Potential improvement

I'm fine with that, but I think we also realized we were running tarpaulin with some flags that made it slower than needed.
@fabriziosestito do you remember which project was fixed?
I see policy-server has some different options: https://github.com/kubewarden/policy-server/blob/9737ba8cfa17f578f6e6451f938611c98be8cbd8/Makefile#L46-L49

Good point. I have force-pushed, removing the timeout increase and setting the same args as we use for policy-server instead.

It still got killed, maybe we have to raise the limit too, but to something higher than 30 minutes

Note that the CI run takes the code from main, not the PR though.


Error in README 'load the plugin'

Current README says:
function preload(){
game.plugins.add(PhaserNineSlice.NineSlice);
}

Which outputs an error to the console:
Uncaught TypeError: game.cache.getNineSlice is not a function
The correct way of adding the plugin can be seen in the example:
game.plugins.add(PhaserNineSlice.Plugin);

yes, well spotted. Thank you!


Iframe source is in a different origin

In my specific scenario, we need to include an iframe in our React application of which its source cannot utilize the iframe-resizer.
Can I still use this library?  Can I still utilize at the parent level the messaging, if the same contract is utilized by the iframe source?

Have you seen
https://github.com/davidjbradshaw/iframe-resizer-react


feat: CSS

Just went over the CSS files, did some cleaning up, merging of some files, moving around, renaming, and organizing.

Looks like just one conflict


Fixes a slight typo in the Knowledge Base/Databases page

Just a tiny typo fix!

Thank you for the PR!


Restart option crashes game

When you select "Restart" in the pause menu, the game crashes. Will need significant debugging.

Fixed in #26


feat: Add 'sni_mode' for CDN resource/DS

What does this fix or implement?

Checklist

[x] PR name added as appropriate (e.g. feat:/fix:/doc:/test:/refactor:)
[x] Tests added or updated
[x] Documentation updated
[x] Changelog updated and version incremented (label: upcoming release)
[ ] Github Issue linked if any
[ ] Jira task updated

The API isn't released, same for the SDK, so after the review we will place this on-hold and we will merge this as soon as the API becomes available for the clients. This is why I didn't add any files related to the SDK (I worked with a SDK that was generated locally).


Supporting range argument on rollup method

Using the range argument will be useful for calculating and recalculating rollups
of large data sets that cant be counted all at once

My bad, it looks like the test was failing. Fixed in the commit above.
Thanks for the PR! Will push a new release shortly.


proposal: move slave syncer out of tlogserver

In the current slave sync design, the slave syncer is a component part of the TlogServer, syncing a vdisk's (meta)data into a configured slave cluster using the played back transaction logs (using the TLogPlayer). This makes the design much more complex than it needs to be, gives tasks to the tlogserver it shouldn't be doing, and turns the 3-layer design into an actual 2-layer design. Normally the 3 layers of 0-Disk data storage are:

Primary Cluster: the beating heart of a vdisk, at all times directly controlled by the nbdserver, reading/writing data from/to this cluster;
Slave Cluster (to copy data from or to control directly in case one or multiple primary servers no longer function);
TLog: the last layer to fallback to. If configured, it stores all (write) transactions that happen to a 0-Disk from oldest to latest. If both a primary and slave server fail, the tlog layer can be used to restore that data and respread it over the primary and slave servers which are still online;

With the design as it currently is however, layer 2 will fail as soon as there is a problem in layer 3. Which makes it that layer 2 and layer 3 are in fact one and the same in the current design.
On top of that there are complexities that arise in this design, in terms of communication. When a primary server fails, it will initially just be marked as offline, and the goal is that the nbdserver will use the linked slave server in the meanwhile directly, as if it was the primary server itself. This has to be communicated to the slave syncer to ensure that it is from now on no longer touching this slave server. Right now this would have to be communicated between processes, due to the fact that the tlogserver and nbdserver live in different processes. On top of that it would have to be info stored in a permanent storage due to the fact that any tlogserver could suddenly shut down and be replaced by another tlogserver which would otherwise be unaware of the fact that it might not be allowed to certain slave servers, as they are currently in control by the nbdserver. Again, none of this is impossible to solve, but it makes the entire design very complex, while the idea is very simple.
The slave cluster idea
To understand how we got ourselves into this complex design, we need to understand first of all what the slave cluster is. We'll also have to understand why we moved the logic which was initially just part of the nbdserver into the tlogserver as an entirely separate component.
The idea of the slave cluster is very simple:

When a slave cluster is supported and configured it should have the exact same (meta)data as the primary cluster for a given vdisk;

For simplicity sake we'll also assume and require that the primary and slave cluster have the same amount of servers, this makes the design very simple as it allows to map each primary server directly to a slave server (a pair is identified by the shared index server);
In normal scenarios this slave cluster will be synced by something other than the normal vdisk storage code (whether that's just a different goroutine, or an entirely different process is not important for the design);
There should be no noticable performance cost for having the data being stored on 2 clusters instead of one;
When a vdisk or AYS notices a connection issue with a primary server it will be marked temporarly as offline, and the nbdserver's vdisk storage logic will take direct control over that slave server;

When this happens there should be nothing else writing data to this slave server, until the nbdserver's vdisk storage logic is no longer using this slave server directly (which will happen as soon as the primary server's state is updated from offline to either respread or repair);

That's it for the core idea. There are a couple of more details and nuances (most of which is described in more detail in https://github.com/zero-os/0-Disk/blob/master/specs/ardbfailurescenarios.md), but none of which has anything to do with the scope of this issue.
Original concept implementation
Here things become vague, as this feature request has been lingering around forever, and I can't fully remember all details from the initial discussions. On top of that I can't seem to find the initial commits of an original implementation that I thought existed. In fact I now start to think that no original implementation existed in fact.
However I think we all agreed since the beginning that syncing (meta)data to the slave cluster shouldn't be the responsibility of the tlogserver. That is also something highlighted by @robvanmieghem's comment from back in May: https://github.com/zero-os/0-Disk/issues/38#issuecomment-303419968. There was however not a bad idea back than, and so we continued with the idea of implementing this slave syncer component within the tlogserver, using the tlogplayer.
Proposed solution
I would want to propose to keep the slave syncer within the nbdserver process, and simply have it be a thing that runs for a given vdisk on one or multiple separate goroutines. This way the slave syncer would still async a vdisk's (meta)data to the slave cluster, but it would be done so within the same process. This solution would have the following advantages:

The tlogserver can now once again focus on its responsibilities, rather than also having to do extra things;
There is no more extra communication (start/stop slave sync) required between nbdserver and tlogserver;
The slave syncer becomes a lot cheaper as there is no longer an entire tlogreplay process (which has to decode, decrypt, decompress, etc...), and instead data is just written as is done with the regular primary cluster;
The entire slave sync implementation becomes a lot simpeler;
The slave cluster (layer 2) can now be used, whether or not a tlog cluster (layer 3) is supported and configured;
A critical failure in the tlogserver (cluster) (layer 3), will no longer affect the correctness and availability of the slave cluster (layer 2);

This solution has also some potential disadvantages:

It might hurt the performance of the nbdserver, as it now has to write data to 2 clusters per vdisk (which supports the slave cluster), instead of just 1;
It makes it harder to guarantee that the slave cluster is in sync with the primary clusters;
If an nbdserver crashes it would potentially make both the primary- and slave cluster corrupt, with no real way to prevent this as the slave syncer is now part of the nbdserver (not sure however if this is a new disadvantage, as I'm fairly sure that a crashing nbdserver is also a problem for the current slave syncer as part of the tlogserver);

So this proposition still has quite some big disadvantages. And while it does make the entire slave sync implementation a lot simpler, those disadvantages might make it less attractive for us to want to go for it.
An alternative solution that i had in mind would be to have some kind of message queue service (similar to or RabbitMQ itself). This would be used to the nbdserver for all data write calls. Than it would write all that data to the primary cluster, slave cluster and tlogserver. Making it from the nbdserver as simple as one external call, instead of 3. It does however mean that we would have yet another loose piece in the 0-Disk puzzle that might suddenly go up in fire. Therefore I'm not sure if we would ever want to go with something like this but it has been in my mind the past couple of days, so I wanted to get it out there, as I was at it anyhow.

Very interesting proposal, some quick into will be very helpful especially about the solution.
As slave syncer is indeed not normally a task for a tlog server.
If we can see some raw solution proposal soon, we might make this issue and #587 to block each other.

Sure thing. Will add it tomorrow as soon as possible. Today I was too busy with coding work. Sorry for that.

@iwanbk I added the description. It's quite a lot to take in though, and it's a bit chaotic as it's written directly from my mind. However I hope it helps us to think and decide about the future of the slave sync feature.
Feel free to discuss it tomorrow with others such as @zaibon as well.

However I think we all agreed since the beginning that syncing (meta)data to the slave cluster shouldn't be the responsibility of the tlogserver. That is also something highlighted by @robvanmieghem's comment from back in May: #38 (comment). There was however not a bad idea back than, and so we continued with the idea of implementing this slave syncer component within the tlogserver, using the tlogplayer.

Yep, it is correct.
Please also note that slave syncer also make the tlogserver slower

There is no more extra communication (start/stop slave sync) required between nbdserver and tlogserver

For now, the extra communication from nbdserver is only wait for syncing process to be finished

The slave syncer becomes a lot cheaper as there is no longer an entire tlogreplay process (which has to decode, decrypt, decompress, etc...)

No
The entire tlogreplay process only happens when the data source is from 0-stor.
if the data comes from tlogclient directly, tlogreplay don't need to decrypt & decompress.

An alternative solution that i had in mind would be to have some kind of message queue service (similar to or RabbitMQ itself).

Instead of rabbitmq, we might want something more lightweight like nsq

This would be used to the nbdserver for all data write calls. Than it would write all that data to the primary cluster, slave cluster and tlogserver.

Sounds reasonable for slave and tlog, but not sure about writing to primary cluster from an MQ.

Please also note that slave syncer also make the tlogserver slower

Interesting to note indeed. It makes the argument of going for the tlogserver-backed solution because of performance a lot weaker. As it could be argued that the performance cost has just been moved from the nbdserver to the tlogserver. While the total cost is still at least as much, if not more.

For now, the extra communication from nbdserver is only wait for syncing process to be finished

Sure but that doesn't matter. In the final design (if we should continue with the slave syncer as a component within the tlogserver) we require much more complex communication, as we need to be able to disable slave syncing to specific servers of a cluster when the nbdserver takes control over them, and only enable those servers for slave syncing again when the nbdserver no longer desires such direct control.

Instead of rabbitmq, we might want something more lightweight like nsq

Sure, or even a very simplistic custom solution. I was just saying RabbitMQ because of the concept of message queueing and how it executes that concept. Didn't really want to point to any implementation in specific, it was more about the idea.

Sounds reasonable for slave and tlog, but not sure about writing to primary cluster from an MQ.

Yeah it's going to cause problems. As there will quite a big window when the data is not yet in the actual ARDB, while it has already left the storage. But those same issues are also problematic for the slave servers, as we might switch at any point to those servers so that window is just as bad for those. And I would argue that the same also goes for the tlog data.
I wouldn't focus to much on the alternative MQ solution if I were you @iwanbk. I think it's more of a random side thought, that can probably be ignored.

The entire tlogreplay process only happens when the data source is from 0-stor.

That's interesting. Even though it doesn't make me any more of a fan of the current slave sync implementation. Still for the sake of completeness, could you point me to the current master code where you are writing "the data [which] comes from tlogclient directly", as you say. As I seem to recall that the only slave cluster writing happens using the tlogplayer.

Interesting to note indeed. It makes the argument of going for the tlogserver-backed solution because of performance a lot weake

Yeah, because more task for the CPU

"the data [which] comes from tlogclient directly"

just to make sure, it means from tlogclient-> tlogserver -> slave syncer

As I seem to recall that the only slave cluster writing happens using the tlogplayer.

Well, those decompress etc is done by 0-stor client, only needed when tlogplayer got data from 0-stor.
tlogplayer has API to replay data directly from an aggregation https://godoc.org/github.com/zero-os/0-Disk/tlog/tlogclient/player#Player.ReplayAggregationWithCallback
data is being sent to slave syncer here
https://github.com/zero-os/0-Disk/blob/master/tlog/tlogserver/server/vdisk.go#L470
received here
https://github.com/zero-os/0-Disk/blob/master/tlog/tlogserver/slavesync/slave_sync.go#L254
and then replayed with the func i listed above

Yeah I know more or less how that code looks, I've seen it plenty of times. But I have yet to see slave-sync code that writes data directly to the slave cluster as it is received from the tlogclient in the tlogserver.

Also we're getting too much in detail here. And it all doesn't matter too much as I think we all agree that going for the current solution (having the slave syncer as a component of the tlogserver) isn't a solution we want to roll with.
So instead of going back and forward about details like this, I would rather focus on finding and agreeing on a new slave syncer implementation, which is completely decoupled from the tlog module. In the spirit of this I would encourage you to think about this and propose solutions as you come up with them. Also like I said, it might be useful to discus them with others of the team while I'm offline for the night.

i edited the issue description, crossing the point about "need entire tlogreplay" and add point about "slavesync make tlogserver to be slower"

@iwanbk what do you in fact think about my actual proposed solution (where the slave-syncer is just an async part of the normal nbdserver vdisk storage code). Because I don't think you've shared your opinion on that one yet?
Also did you have any (alternation of) ideas of your own about the future of the slave syncer?

@iwanbk what do you in fact think about my actual proposed solution (where the slave-syncer is just an async part of the normal nbdserver vdisk storage code)

Basically i'm agree with this

Because I don't think you've shared your opinion on that one yet?

Yeah, because this proposal still very vague, hard to comment at this stage.

Also did you have any (alternation of) ideas of your own about the future of the slave syncer?

going to think about it today.

With the design as it currently is however, layer 2 will fail as soon as there is a problem in layer 3. Which makes it that layer 2 and layer 3 are in fact one and the same in the current design.

Actually, write to layer 1 will also failed if layer 3 failed.

After some thinking today, i think we could implement my (improved) first idea, proposed at https://github.com/zero-os/0-Disk/issues/38#issuecomment-302987755.
In which tlogserver don't fill the slave, but only assist the slave
So, we can setup slave cluster using redis/ardb replication mechanism, and let the master do the syncing to the slave.
In case of master become failed:

check value of nbd's last flushed sequence(lfs) in slave, the value was added as part of #230
execute tlogreplay to write data from sequence lfs+1 to the end.

The good part of this idea are:

should be faster than addding slave syncer to tlog or nbd
simpler of course
could be implemented as zeroctl command that could be invoked by the orchestrator

@robvanmieghem criticized the original version of this idea at https://github.com/zero-os/0-Disk/issues/38#issuecomment-303419968

@iwanbk @GlenDC Was thinking about that too originally. Thing is you need to know more or less to what point the slave caught up, monitor the master slave replication, detect a failing master, trigger this process, put a new config next to the nbdserver and SIGHUP it, all without causing io errors in the vm using the nbdserver.

Except the first two point, all other points remain true (well, except the SIGHUP part, but etcd part is basically the same).
For the first two point :

thing is you need to know more or less to what point the slave caught up : with nbd's last flushed sequence metadata, we know the point
monitor the master slave replication : we don't need to monitor it

For the other points:
we have to do it anyway, even with current slave syncer

Actually, write to layer 1 will also failed if layer 3 failed.

Yeah currently. But that shouldn't be happening imho. Hence what I was talking about in https://github.com/zero-os/0-Disk/pull/521#issuecomment-343496064 (I was by the way not proposing to do that work as part of that PR, it simply seemed the most natural place to bring up those thoughts).

I think the problem with using ardb/redis master/slave replication is that we have no control over it at all. Can we at any point know how far the slave has caught up with master? I suppose starting/stopping master/slave replication is possible, but as nbdserver doesn't know anything about the ARDB server processes, it cannot do so on its own and will have to do that via AYS, which is an issue as that is a delayed process.
In short, I really don't see that working with our current infrastructure.

I think the problem with using ardb/redis master/slave replication is that we have no control over it at all.

what is we here? AYS or nbdserver?
If ays, i guess they have

Can we at any point know how far the slave has caught up with master?

Yeah, it is on my proposal
thing is you need to know more or less to what point the slave caught up : with nbd's last flushed sequence metadata, we know the point

I suppose starting/stopping master/slave replication is possible, but as nbdserver doesn't know anything about the ARDB server processes, it cannot do so on its own and will have to do that via AYS, which is an issue as that is a delayed process.

How long is the delay?

what is we here? AYS or nbdserver? If ays, i guess they have

AYS does, but it doesn't just instantly react on stuff that propagate from nbdserver down you know. And neither is there currently a way for us to wait on some kind of answer from AYS. In fact there is not even a way for AYS to communicate back to us, other than to update our configs or shut is down.
So even if AYS has control over ardb/redis master/slave replication (which indeed it probably does have), there is no way that the nbdserver could suddenly turn on/off replication for a specific server. We could request it via an STDERR log to AYS, but than there would be a certain window where both the NBDServer could be writing to a slave server and the slave server itself is still being replicated from the primary server.
Well thinking about this, I guess this is perhaps not an issue, as we only switch to writing to slave server directly in case the primary server is down. And if that's the case than I guess the replication process can't be writing to the slave server anymore either, as there is no data to be replicated, as the primary server is down.

Yeah, it is on my proposal

In your proposal we would still have a highly coupled layer 2 (slave cluster) and layer 3 (tlog), which is entirely what we're trying to prevent with this issue... I think...

How long is the delay?

Who knows, as if you can put a number on that.

In your proposal we would still have a highly coupled layer 2 (slave cluster) and layer 3 (tlog), which is entirely what we're trying to prevent with this issue... I think...

Hmm.. not sure with your response.
What i mean is https://github.com/zero-os/0-Disk/blob/master/nbd/ardb/storage/tlog.go#L37
If it is also what you mean, well, while it is only used by tlogStorage this is not really specific to tlog, because it is last flushed to ardb not to tlog.
If what you mean is we still need to use tlogreplay, then i was thinking yesterday to use a kind of buffer in nbdserver. The buffer entry will be erased once it knows that it already synced to slave.

How long is the delay?
Who knows, as if you can put a number on that.

I mean, we also have delay with any async solution

My thought about this topic:
Regarding the solutoin propsed by @GlenDC to move slave syncer to nbdserver:
I mostly agree with this idea cause it cut out all the hard communication that needs to happens between tlogserver  and nbdserver. Which I think is can be a source of a lot problem and edge cases.
Now I have few question:

It might hurt the performance of the nbdserver, as it now has to write data to 2 clusters per vdisk (which supports the slave cluster), instead of just 1;

How would it have to impact performance, the write to the slave cluster has to be asynchronous.

It makes it harder to guarantee that the slave cluster is in sync with the primary clusters;

Why is that ? we still know what has been flushed or not with this design no ? To me this design also simplify that since the information stays local to the nbdserver

If an nbdserver crashes it would potentially make both the primary- and slave cluster corrupt,

This is not something you can prevent I think. If nbdserver crash, then the last valid state is the one from the last transaction that is on the tlog. This is the design

Introducion new queue is indeed something I wouldn't do. this bring complexity and that's what we're trying to avoid by keeping syncer in the nbdserver

Now some question regarding some comment from @iwanbk :

Please also note that slave syncer also make the tlogserver slower

Why is that? This should be done asynchronous, so no delay for the tlog server.
nbdserver shouldn't not wait for the tlog to have replicated the data on the slave cluster. But it should be able to know what it needs to still replay in case nbdserver needs to swtich to slave cluster. so tlog server can tell nbdserver, ok now you can start using slave cluster, it has the same state as your master before the crash.

Please also note that slave syncer also make the tlogserver slower
Why is that? this should be done asynchronous, so no delay for the tlog server

nbdserver shouldn't not wait for the tlog to have replicated the data on the slave cluster. But it should be able to know what it needs to still replay in case nbdserver needs to swtich to slave cluster. so tlog server can tell nbdserver, "ok now you can start using slave cluster, it has the same state as your master before the crash" But like I said this kind of communication seems pretty complex and error prone.

How would it have to impact performance, the write to the slave cluster has to be asynchronous.

Why is that? This should be done asynchronous, so no delay for the tlog server.

Async doesn't mean zero performance hit. It still takes your CPU and RAM.
But yeah, less hit than the sync one.

Why is that ? we still know what has been flushed or not with this design no ?

Yeah I'm talking more about in case we restart an nbdserver. Or mount an existing vdisk. There would currently be no way to know that the slave cluster is up to date with the primary cluster.
I do like the idea to implement the slave cluster syncing as a subscriber to the tlog stream, but at the same time that would mean that we would only provide the slave cluster support for vdisks that make use of the tlog repo. Than again I suppose that doesn't have to be a problem, as it all falls under the umbrella of data-redudancy, it might be fine if this is like this. Even so, we would still need to somehow have a way to validate (at startup/failover) that a slave server/cluster is up to date with the primary counterpart.


Still picking a disabled date

I have disabled some dates in future so user can't be able to pick them. Date picker is by default set to current date. Now the current date is the one that i've disabled previously so its set to the disabled date by default. User can now select the current date (i.e the disabled one). I want, if current date is the one that is being disabled then date picker should be set to next enabled date by default. Any quick way to do this? Thanks.

I am facing the exact same issue. I've disabled some dates including the current date. But the DatePicker's default selection is today's date. I need it to move on to the next nearest select-able date.
Have you figured it out? If yes, please let me know.

You'll have to provide an example that shows how this is broken. The original author of this issue indicated that it was an error on their end and there is a test covering exactly the scenario you're describing.

I came up with a heuristic logic of my own to overcome the problem described before.
Problem is:

Set current date to selected.
Disable some dates, including the current date.
Now, the selection should automatically move to the next selectable date but it doesn't.

Heuristic:

Make a list of selectables.
Subtract disables from selectables.
Make DatePickerDialog object with current date selectables.get(0)

This way it'll always be on the first selectable defined by the user. I hope I made it clear.

I tried your scenario and it works as expected.
Keep in mind that you are supposed to completely configure the picker before calling the show() method. The picker takes care of adjusting the default selection in onCreateView.


feat: support multiple lsp clients

Should now be able to fetch symbols from multiple servers.
Previously, if you had multiple servers, multiple buffers for each lsp result would be created. Results are now merged together, sorted and placed in the single buffer.
This also fixes issue #25 and #12

Thanks for the PR!


Incorrect type checking

Godot version
3.5
System information
Windows 10
Issue description
According to my understanding, the point of specifying variable types is to allow compiler-time checks even though Godot does not have static typing at run-time. However, when dealing with class hierarchies, Godot does not handle typed variables as expected. If we declare a variable to be an instance of class C, Godot compiler will allow the access to properties and functions that are not defined in given class C, but in its subclasses, which is incorrect.
It has to be fixed so that this code does not compile:
class_name Parent
func parentFunction():
    print("parent")
...
class_name Child
extends Parent
func childFunction():
    print("child")
...

var a: Parent = Child.new()
a.childFunction()            # this should not compile, because class Parent does not have childFunction

Steps to reproduce

define a class Parent
define a class Child, extending Parent
define a function m in class Child, such that it does not exist in Parent
define variable a of type Parent
initialize variable a with an instance of class Child
try to call function m, which exists in Child but not in Parent
this code should not compile, because it tries to call a function which is not defined in the type of variable a

Minimal reproduction project
project.zip

You can enable warnings:

Additionally, you can treat all warnings as errors:

In 4.0-dev you can set this up on a per-type basis:

Thank you for the clarification! I still a bit wonder what would be the use case for a developer to declare the type for a variable and then not want the compiler to enforce it. It results in confusing code.

GDScript is not compiled. The compiler thus can not enforce anything.
This does seem like a case though where in 3.5 you would have gotten an error in the IDE telling you about this. Since tracing back the fact the function does not exist should work just fine with static typing. Not just a warning.

By "compiler" I meant the thing which obviously parses the code and checks the syntax and semantics. It easly can be just IDE, I am not familiar with how gdscript gets executed. Anyway the point is: the language allows programmer to specify variable's type and then does not enforce it, which can be even more confusing than not specifying type at all.

Either way, this is completely fine behavior. Though I personally think it should raise an error instead of a warning.
Your variable obviously contains a Child, not a Parent, and so accessing Child functions isn't causing an error despite the typing being for Parent.
If the IDE did not know there was obviously a Child, then it would error as expected with a method not found. But... the method IS found.

No it's not fine. It must be treated as semantically incorrect code. My variable obviously contains a Child only in that minimal example. In reality the assignment can be anywhere and very far from sight. Consider:
func weirdFunction(a: Parent):
    a.childFunction()   # !!!!!!!

How on earth it can be fine that weirdFunction calls a method which cannot be found in Parent class? The function does not even need to know that class Child exists. If I expect that parameter a is of type Child it should be declared as Child. That's the whole point of declaring types.

func weirdFunction(a: Parent):
a.childFunction()   # !!!!!!! NO, this is not fine, class Parent does not have childFunction

I am pretty sure this example does raise an error though. And if it does not. Then that is a definitive bug.
It is:

var a: Parent = Child.new()
a.chilFunction()

Which does not raise an error. Because... well, it wouldn't error now would it?

var a: Parent = Child.new()
a.chilFunction()
Which does not raise an error. Because... well, it wouldn't error now would it?

It would not be an error if I did not restrict the type of a to be Parent.

But that's my point.
In any case. It may be worth investigating this further and establish a list of examples in which an error versus a warning would be expected.

var a: Parent = Child.new()
a.chilFunction()

This is valid, functional, executable, code. Despite the type miss match. If you run this, it will function just fine. (Type hints don't really exist at runtime.)
It is also a constant expression. There is no ambiguity about the fact that A holds a child by the time you try to call childFunction. So the IDE seemingly accepts this as correct.
I agree that it probably should not be allowed. But this needs more testing to determine if this is:

A bug that needs fixing.
An intentional decision to allow valid code but raise a warning due to type miss match.
Completely unaccounted for.

This is valid, functional, executable, code. Despite the type miss match. If you run this, it will function just fine. (Type hints don't really exist at runtime.)

This is not valid functional, executable code in any programming language with static typing. I know that types don't exist at run time in GDScript, but because at development time GDScript gives the possibility to specify (restrict) type of a variable, it should behave as expected. If I give a type to a variable, it means that I want to assume that variable will have exactly that type and I don't want to be allowed to call methods and access properties that don't belong to that type. And I want that Liskov Substitution Principle holds, so that the code works regardles of what object is used as value for a variable, as long as it is inherited from that variable's type.

specify (restrict) type of a variable

That is not what it does.

I don't want to be allowed to call methods and access properties that don't belong to that type

GDScript is not statically typed. It is interpreted. The type hints exist to allow the Editor to provide you with useful information. There is no way to prevent you from writing code that ignores the hints. You can save a script with a type error and run it just fine so long as you don't run into an actual execution error.
If you think of GDScript as being in the same class of language as JavaScript, your expectations may be more in line with how things work. Or TypeScript since you are using the optional type hints.
Point is. It will execute without error. So the editor, is not giving you an error as it stands right now. I am plainly explaining why things work how they do, and what frame of reference allows for this to be acceptable.
It probably should error. I will reiterate.
And I will reiterate. That we (you) should investigate under which circumstances it is not providing an error, even though we believe that it should.

Static typing is optional in GDScript. In particular, the following code is allowed by default, although it is not type-safe:
var my_var: MyClass = dict.key

If you want strict checks, I wrote above how to enable them. See also this comment.

Fair enough, thanks for replying.

Upon further investigation.
extends Node2D

func some_func_a:
	var a: FirstClass = SecondClass.new()
	a.second_class_func() # No code completion shown. No error. No warning.

func some_func_b(a: FirstClass):
	a.second_class_func() # No code completion shown. No error. No warning.

class FirstClass:
	func first_class_func():
		pass

class SecondClass extends FirstClass:
	func second_class_func():
		pass

Neither of these create an Error in the editor. Both will create a warning if the warning is enabled. Neither of them show up in code completion.
Everything is working correctly here. The warning is simply disabled by default.
@isokissa There is no bug and no false behavior present here. BUT, you can make a proposal about enabling these warnings by default if you wish to start a discussion on that. I do think it makes sense. And we might want to clarify the description of the warnings to not include "compile time" when that's not really a thing in the typically expected way.

BUT, you can make a proposal about enabling these warnings by default if you wish to start a discussion on that. I do think it makes sense.

Personally, I'm against it. Static typing is optional, type hints are disabled by default in the editor, static types are not used in official examples. This would be too unfriendly to beginners and those who prefer dynamic typing.
There would be a huge number (much more than even the infamous RETURN_VALUE_DISCARDED) of warnings that are useless to most, thus devaluing other, more critical warnings.
I find type safe lines to be a good tool and a compromise between code type safety and convenience. Type safe lines do not distract the developer, do not force them to constantly please the type system (which sometimes, on the contrary, leads to problems if you mask an error, as is the case with the as keyword). But type safe lines gently draw the developer's attention to potential problems, think about the health of the code if there are too many gray lines. If I see a gray line, then I check more carefully if everything is correct. But this does not mean that there is an error in this line.

If you were to hold a poll about safe lines. I am willing to bet only experienced users could tell you: They exist, what they do, and what they look like. Meanwhile typing is something everyone recommends and which actively prevents errors and makes the code easier to use.
In any case. This is veering off into discussion. There is no bug present in this issue, so it can probably be closed?

Personally, I'm against it. Static typing is optional, type hints are disabled by default in the editor, static types are not used in official examples. This would be too unfriendly to beginners and those who prefer dynamic typing.

Yes, static typing is optional, and those who do not like it should not use it. And for those who want it, it should work as expected. When I specify the type of a variable, this is because I care that the variable contains exactly that type and no other.

I can choose that option and enjoy the static typing.

You can do it now, without changing the defaults. And since you have more programming experience than beginners, it's easier for you to switch the settings to what you want than for them.
I don't consider myself a beginner, I use static typing wherever possible, but I deliberately don't activate these warnings. Because I think that personally these warnings will bring more inconvenience to me and force me to appease the type system without any benefit.

IMHO this is a bug.

It's definitely not a bug, it's a feature of the language's type system. Different languages have different ways how types work.

Closing per the above comments. Please continue the discussion in a proposal (but see https://github.com/godotengine/godot-proposals/issues/173 first).

Obviosly the wrong options are disabled/enabled by default. Nobody should care if you don't use function's return value. But it's pretty big thing if your "static typing" allows calling the method which does not exist.


Error Reading Compressed Data?

Interesting issue when using the flozano/kafka image's latest version:
  Failed to decompress message at events:49@3141 RangeError: Trying to access beyond buffer length
  RangeError: Trying to access beyond buffer length
      at _Reader.Reader.demand (/app/node_modules/module-workercore/node_modules/avrolanche/node_modules/bin-protocol/lib/reader.js:79:15)
      at _Reader.define.read (/app/node_modules/module-workercore/node_modules/avrolanche/node_modules/bin-protocol/lib/index.js:118:14)
      at _Reader._read (/app/node_modules/module-workercore/node_modules/avrolanche/node_modules/bin-protocol/lib/reader.js:42:12)
      at apply (/app/node_modules/module-workercore/node_modules/avrolanche/node_modules/lodash/lodash.js:411:27)
      at _Reader.wrapper [as raw] (/app/node_modules/gaming-module-workercore/node_modules/avrolanche/node_modules/lodash/lodash.js:4837:16)

Any ideas? It seems that it's failing when reading a Gzipped message from the stream because it's going 'over the end' of the buffer - potentially an issue when handling compressed messages? I've not dug too deeply into it yet - but I'm wondering if there's a semi-obvious reason that might be causing this.
Will be rebuilding the message corpus without compression when producing to see if that abates it. Was using Codec 1 (GZIP)

Does it happen with any gzipped message or some specific?

Here's where it got a little wierder: if I set the codec to 0 (NONE) on the publisher, blow away kafka and reingest the data - I get different errors. Effectively what I'm seeing under very high load (100,000's of messages) - is that occasionally we get messages pulled out that are incomplete in the buffer.
I suspect that the issue is that under some circumstance, the data comes back either too long/short in the buffer relative to it's declared size. Is there some way for me to dump out the specific message? The data is an avro-serialzied byte buffer - and we get failures reading it back via the AVSC library. I suspect the real problem is that we get partial messages back at the tail of a fully packed buffer - and that this causes the decompression (when GZIP is on) and the deserialization (when it's just a raw buffer) to fail, because the data is incomplete.

What is no-kafka version being used?

Tried a few: we used it with latest and also a hacked up version we made without snappy. I suspect the issue is potentially because when we start a new consumer at -2/start of log, perhaps the bulk of data overflows something. I've a theory on how to reproduce it I'll put to practice tomorrow at work.
(New Kafka, create topic, write 1m messages, then read back from offset -2, I suspect messages overflowing size of buffer is issue)
Sent from my iPhone

On 4 Aug 2016, at 20:14, Oleksiy Krivoshey<EMAIL_ADDRESS>wrote:
What is no-kafka version being used?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

Is that the only stack trace available?

Yes. I've had to leave things stable during the day for demonstrating some other part of the platform. I'll be working on a small-scale repro shortly.

@oleksiyk - Narrowed it down to it being the gzip stuff - it seems that we'd actually been ignoring the codec flag and manually setting it to 1, so everything was gzipped synchronously.

Closing: I suspect there's something here, but we've switched off compression for the time being and I'm not in a position to reproduce.


Oxygen Vent massive lag when placing many simultaneous blocks

Version of Advanced Rocketry
1.5.0.-125
Have you verified this is an issue in the latest unstable build
Y
Version of LibVulpes
0.4.0.-49
Version of Minecraft
1.12.2
Crash report
This report is of when the server crashed due to not having any ticks in 60 seconds (from placing 500 blocks simultaneously)
https://pastebin.com/SEy55idw
Description of the problem
When large numbers of blocks are placed simultaneously in an oxygen vent, it causes massive lag, scaling proportionately to the number of blocks being placed. The easiest way to reproduce this is with a large setting for the oxygen vent radius (I used a radius of 80 and a calculation method of 3) and an RFTools Shield Projector using a shape card. Building a shield of 25 blocks resulted in about 5000ms between ticks, and 500 blocks was over 60 seconds, which caused the server to shut itself down.
My guess is that it recalculates the fill area once for every new block that is placed, so when many blocks are placed simultaneously it recalculates the area hundreds of times.

Is this still a issue? Please comment if it is and i will reopen if if you are unable? We are closing old issues to see what is still relevant and needs to be fixed.


Datasets for 3D pose estimation

Hello! I notice that you provide two scripts for 3d pose estimation networks, human3.6m only or multiple datasets (coco, mpii...), respectively. Would you mind sharing the performance difference between these two configurations (I didn't find any related ablation results in the paper) and which configuration is used in the final checkpoint in the paper?
Thanks very much for your reply!

And what's the difference between the provided checkpoints~(i.e., mesh_mpii3d, mesh_3dpw, mesh_h36m), because you only provide two training scripts for mesh prediction. Which datasets are used for mesh_mpii3d.pth.tar training?

I think I might understand the first question. But I still have two problems:

what are the training datasets for mesh_mpii3d.pth.tar, is it the same as mesh_hm36m which used only one dataset for training, testing, and evaluation.
To train the model for 3dpw, you mentioned you use 5 datasets for training, but for human3.6m you only used one dataset for training and testing. In some views, human3.6m is an in-lab dataset in which the scenes, backgrounds, and postures are limited compared to in-wild datasets. I wonder if training only on human3.6m will show overfitting on human3.6m datasets, archiving good performance but showing poor results in other datasets (e.g., 3dpw)?

Thanks very much for your reply!

And what are the validation datasets for human3.6m and mpii3d training ...

The 2 configurations are because the different joint sets detected by different 2d pose detectors (CPN for human3.6m and ViTPose for 3DPW and MPII3D), where the 3dpw joint set has 19 joints and the h36m joint set has 17 joints. You can refer to Pose2Mesh we followed for the illustration.
The training datasets for mpii3d are mixed datasets, the same as 3dpw. And it is tested on mpii3d testing dataset. It can directly use the checkpoints of 3dpw to test, thus we did not provide its training script. In other words, you can use the training script of 3dpw to train a mpii3d model.
Human3.6m is trained and tested on its dataset, following the same setting of pose-based methods. But as you mentioned, it may cause overfitting. It's interesting to add more datasets to train, it may be useful for both accuracy and generalization ability.
For human3.6m, following previous methods, such as VIBE, TCMR, our model is trained on 5 subjects (S1, S5, S6, S7, S8) and tested on the other 2 subjects (S9, S11). There is no validation set. For mpii3d, it is only tested by using the checkpoint of 3dpw.

The 2 configurations are because the different joint sets detected by different 2d pose detectors (CPN for human3.6m and ViTPose for 3DPW and MPII3D), where the 3dpw joint set has 19 joints and the h36m joint set has 17 joints. You can refer to Pose2Mesh we followed for the illustration.
The training datasets for mpii3d are mixed datasets, the same as 3dpw. And it is tested on mpii3d testing dataset. It can directly use the checkpoints of 3dpw to test, thus we did not provide its training script. In other words, you can use the training script of 3dpw to train a mpii3d model.
Human3.6m is trained and tested on its dataset, following the same setting of pose-based methods. But as you mentioned, it may cause overfitting. It's interesting to add more datasets to train, it may be useful for both accuracy and generalization ability.
For human3.6m, following previous methods, such as VIBE, TCMR, our model is trained on 5 subjects (S1, S5, S6, S7, S8) and tested on the other 2 subjects (S9, S11). There is no validation set. For mpii3d, it is only tested by using the checkpoint of 3dpw.
Hope this reply helps you!

Thanks very much for your reply!


Removed unused imports and fixed some indentation errors.

I signed it!
On Mon, Aug 15, 2016 at 4:35 PM, googlebot<EMAIL_ADDRESS>wrote:

Thanks for your pull request. It looks like this may be your first
contribution to a Google open source project. Before we can look at your
pull request, you'll need to sign a Contributor License Agreement (CLA).
📝 Please visit https://cla.developers.google.com/
https://cla.developers.google.com/ to sign.
Once you've signed, please reply here (e.g. I signed it!) and we'll
verify. Thanks.

If you've already signed a CLA, it's possible we don't have your
GitHub username or you're using a different email address. Check your
existing CLA data https://cla.developers.google.com/clas and verify
that your email is set on your git commits
https://help.github.com/articles/setting-your-email-in-git/.
If you signed the CLA as a corporation, please let us know the
company's name.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/GoogleCloudPlatform/appengine-search-python-java/pull/4#issuecomment-239817899,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAOFlb__CB6IdjbGGNKW6jdhXZw3L0Lsks5qgHkigaJpZM4Jkbs-
.

--
\Chris

Thanks!


check code coverage

...before publishing 0.10.0.

this may be useful in the quest for ES6 Istanbul coverage: https://www.npmjs.com/package/remap-istanbul