Content Security Policy is overwritten

The fix for issue #143 created an issue where a pre-existing content-policy is overwritten by this module which can cause stuff to break when testing.  My particular use-case that breaks this is that I have inline data images.  The CSP that this module uses to overwrite my own CSP blocks the images from loading.  I don't think completely overwriting an existing CSP is the proper method.  I'm not sure adding or updating the CSP is proper either, as it could give a developer a false sense of their app working through the PhoneGap app, but when they actually build the APK and deploy, it would fail.  The developer should be responsible for ensuring the proper CSP is in place that supports their app and allows for debugging with the PhoneGap App.

Hi @mattzuba,
Thanks for your research and insights on this issue. We will need to discuss this before we can implement anything.

Ping @mwbrooks & @timkim

Thanks for the issue @mattzuba!
I agree with @mattzuba that the developer should be responsible for ensuring the proper CSP is in place, but we should also ensure that getting started is as easy as possible.
I think the solution should be:
1. When no CSP exists (or app was created from a PhoneGap template)

Inject the CSP

Update the current CSP is be as wide open as possible, including inline data images

Display a warning message in the console

Link to the appropriate documentation
Display the injected CSP

2. When CSP exists

Do not inject the CSP
Display a warning message

Auto-reload may not work under serve

Optionally, we can inspect the current CSP and make suggestions to support serve functionality

I think making it an option in the phonegap-cli could be another way to go rather than detecting if a csp exists or not. Something like:
phonegap serve --inject-csp // default true and replaces or injects our csp
phonegap serve --no-inject-csp // don't touch the csp

But then you can still run into issues like @mattzuba where we inject CSP and then a user does a full build only to find out that some features are not working like they were in the dev app.

That does not resolve the issue @timkim
The injected csp needs to be the combination of both the serve-csp and the developers csp because something will break if it is one or the other.
In @mattzuba 's case, if you inject the serve-csp, his images won't load, and if you just use his csp, then reload won't work.
It shouldn't be too hard to just add the 3 or so serve-csp requirements to an existing csp if one is present.

It shouldn't be too hard to just add the 3 or so serve-csp requirements to an existing csp if one is present.

I agree that this is a good outcome, if feasible. I also think it's important for the CLI to let developers know that their CSP is being modified and to what (as per Michael's suggestions above)...

Yes, no secret magic.
This is what is injected, so we need to add
var cspTag = "<meta http-equiv="Content-Security-Policy" content="default-src *; style-src 'self' 'unsafe-inline'; script-src * 'unsafe-inline' 'unsafe-eval'">";
csp leads to fugly quotes

Do we know when there will be a fix for this?  It is causing me great pain at the moment.  Or, is there a patch for this issue?  My meta tag is getting overwritten.

Hi @mattzuba and @scottburch
I'm currently looking at this issue and I want to make sure your scenarios are working with the update. However, I noticed that when I tried to use inline images, it already worked for me on the current version of connect-phonegap.
eg:
<img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUA
AAAFCAYAAACNbyblAAAAHElEQVQI12P4//8/w38GIAXDIBKE0DHxgljNBAAO
9TXL0Y4OHwAAAABJRU5ErkJggg==" alt="Red dot" />

Could you provide a particular html snippet to test with?

Thanks for the reply.  Let me know if I can be of any help.  I looked at the code and have temporarily just commented out the replace.  But, it seems like it would not be hard to either send a warning, if your permission is not there, or, while probably harder, add the one to the existing string.
I'm using the style-loader with webpack.  So, I needed to add 'blob:' This is what is getting overwritten.  It is generally a bad idea to overwrite someones security settings.

I don't have any snippets to share at the moment, the project I was working
on I've put to the side for now.  However the inline images I was using
were in CSS, not the html.
On Fri, Aug 26, 2016, 4:25 PM Scott Burch<EMAIL_ADDRESS>wrote:

Thanks for the reply. Let me know if I can be of any help. I looked at the
code and have temporarily just commented out the replace. But, it seems
like it would not be hard to either send a warning, if your permission is
not there, or, while probably harder, add the one to the existing string.
I'm using the style-loader with webpack. So, I needed to add 'blob:' This
is what is getting overwritten. It is generally a bad idea to overwrite
someones security settings.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/phonegap/connect-phonegap/issues/163#issuecomment-242875256,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABbOuX0obw4eQ4N368QHWNTx7gkxBBqDks5qj3XWgaJpZM4JGNCv
.

--
Thanks,
Matt Zuba
Sent from my mobile telecommunication device, please excuse any tpyos.

Ok, I'm nearly done fixing - just need to fix some failing tests and add some new ones.
The new changes will modify an existing CSP to allow normal dev app + connect-phonegap communication, but also leave existing values if they exist (except for 'none' - this value will be cleared out). It should modify the CSP so that it will allow at least these policies:
default-src *
style-src 'self' 'unsafe-inline' data: blob:
script-src *  'unsafe-inline' 'unsafe-eval' data: blob:

Users will be warned from a console.log that their CSP has changed.

Ok, the changes should be in the latest<EMAIL_ADDRESS>Closing now.

Thank you.

phonegap-cli 6.3.3
[phonegap] [console.warn] Content Security Policy has been modified to be:
[phonegap] [console.warn] Content Security Policy has been added:
[phonegap] [console.warn] Content Security Policy has been added:
[phonegap] 200 /api/appzip
is injected every time even thou i explicit set  tag in index.html

please help, it works in the browser fine, but not in phonegap developer app on android.. do i need an update to something ?
//BO thanks

When using the PhoneGap Developer App, it's own CSP will be enforced, overwriting yours. You will need to compile your application in the standard manner if you wish to test your app with your CSP in place.

Hi @additionbo
Like @chrisgriffith mentioned, connect-phonegap now modifies your CSP to ensure certain rules are set so communication between server and your device is possible. The reason why you don't see the browser logging the CSP warning is that the CSP modifcation only happens when we zip up your project on the server side.

... What is the point of the phone app if my images aren't working? I don't mind if you enable everything, but don't restrict things I've already enabled.

The modifications to CSP should only make it less restrictive, so that the features required by phonegap serve are enabled.  Are you seeing an issue @Aeolun ? are your images suddenly not showing?  If so, please post your CSP fragment (minus anything private) and we'll look into it.

So, What is the point of having a phonegap desktop application ?, by doing this I cant preview properly my apps before deploying it in a real device.

@SimonRuaGitHub,
We need to overwrite the CSP so that the desktop app will work properly along with the mobile developer app. If you need to test your app using your own defined CSP than you should create a full build using the phonegap-cli or PhoneGap Build service so that you can install it onto your device.

We need to make the existing CSP less strict so it will allow dev-app specific requests.
Currently it is discarding any existing CSP where it should be simply adding the needed values for dev-app to function.

Currently it is discarding any existing CSP where it should be simply adding the needed values for dev-app to function.

@purplecabbage no that's incorrect. It is not discarding the existing csp - it adds the needed values plus whatever the user has defined.

Great!  So that is why this is closed then ... (duh)
@SimonRuaGitHub are you experiencing this issue?
If so, please post your CSP fragment (minus anything private) and we'll look into it.

I have the CSP:
<meta http-equiv="Content-Security-Policy" content="default-src 'self' data: gap: 'unsafe-inline' 'unsafe-eval'; script-src 'self' www.google-analytics.com maps.googleapis.com apis.google.com cdnjs.cloudflare.com assets.pinterest.com log.pinterest.com www.youtube.com connect.facebook.net data: gap: 'unsafe-inline' 'unsafe-eval'; img-src *; media-src *; style-src 'self' fonts.googleapis.com maxcdn.bootstrapcdn.com data: gap: 'unsafe-inline'; font-src fonts.gstatic.com maxcdn.bootstrapcdn.com data: gap: 'unsafe-inline'; frame-src accounts.google.com staticxx.facebook.com www.facebook.com data: gap: 'unsafe-inline';" />
But When using the Developer app for iOS, It keeps throwing the following error:
undefined Content Security Policy has been modified to be: <meta http-equiv="Content-Security-Policy" content="default-src * 'unsafe-inline' gap: ws: https://ssl.gstatic.com;style-src 'self' 'unsafe-inline' data: blob:;media-src *;img-src * 'self' data: content:;script-src * 'unsafe-inline' 'unsafe-eval' data: blob:;">
The developer app doesn't seem to recognize the existence of the CSP, even though it is working on my browser when I visit the subnet IP address.

Hi @AugustBrenner,
The developer app needs a specific set of CSP rules in order to work properly. So if we detect that some of the necessary rules are not there we will modify appropriately. In order to test your app with your defined CSP you will need to compile a full build and then install it to a device.

so how can I enable eval function ?


don't merge - rails 5 support

With Rails 5's support for connection_specification_name, we can drop a crap-ton of our monkey patches and simplify the hell out of stuff.
Note that this PR totally drops support for rails < 5.  we should perhaps merge this to a branch or something.
@bquorning

Superseded by #70.


Revisit __stop() method

Description:
Abstract listener's __stop() method is removed and 2 new method was introduced.

__gracefulStop()
__immediateStop()

Revisit stdlib __stop() method implementation to align expected behavior with these two methods. PR https://github.com/ballerina-platform/ballerina-lang/pull/17558 migrates __stop() implementation to __immediateStop()

Need to check if API docs need to be updated WRT this.

This change is already completed. Hense closing


Translations update from Hosted Weblate

Translations update from Hosted Weblate for raspiblitz-web/translations.
Current translation status:

Should wait till Albanian has some content maybe?

Doesn't matter, since it's not included in the i18n.ts file yet :)


Cranelift: support vector instructions without optional ISA extensions

In #3809, a new extension to our fuzzing found that some of our Wasm-SIMD support requires nominally optional ISA features, at least on x86-64.
I think we should try to have fallbacks such that we can support all relevant features on any machine that supports the base ISA. On x86-64, this means just SSE2. Or if this is very difficult for some reason, we should document why and then possibly consider raising our minimum required ISA version.

Reopened; GH mis-parsed "once we fix #3810" as "fix #3810" in #4231's commit.

I believe that this is now done, so closing.


ResourceExhaustedError after several iterations in a grid search

First off, make sure to check your support options.
The preferred way to resolve usage related matters is through the docs which are maintained up-to-date with the latest version of Talos.
If you do end up asking for support in a new issue, make sure to follow the below steps carefully.
1) Confirm the below

[x] I have looked for an answer in the Docs
[x] My Python version is 3.5 or higher
[x] I have searched through the issues Issues for a duplicate
[x] I've tested that my Keras model works as a stand-alone

2) Include the output of:
talos.__version__ == 0.6.7
3) Explain clearly what you are trying to achieve
I am running a grid search that gives 36 rounds.
After about 4 or 5 rounds, during a model.fit I suddenly get hit by a ResourceExhaustedError. I think this is very odd given that I am able to complete at least 3 rounds of fitting on the GPU (with a model and batch size that takes up pretty much all the gpu memory), so it seems that there is a small but significant memory leak somewhere. Any ideas what it could be?

My parameter dictionary is:
p = {
    "sigma_noise": [0, 0.01],
    "nb_filters_0": [16, 32, 64],
    "loss_func": ["cat_CE", "tversky_loss", "cat_FL"],
    "arch": ["U-Net"],
    "act": [Swish, ReLU],
}

And I'm running a U-net with 34 million trainable parameters (for nb_filters_0 == 64) and input dimensions of (208, 208, 3) with a batch size of 12 and 400 epochs.

UPDATE: I did a "quick" test, where I ran each model for only 50 epochs and I got a ResourceExhaustedError again in round 4  during the 5th epoch, and I think that was actually the exact same spot as before when each of the 3 previous model had run for +100 epochs. This tells me, that the models are not properly cleaned out of the GPU memory and on top of that, I might have a memory leak in my generator. @mikkokotila, what do you think?

Very interesting. Can you post your full trace.

Of course! See below. I also added the output leading up, because I think it gives some idea of how the exception occurs.

---------------------------------------------------------------------------
ResourceExhaustedError                    Traceback (most recent call last)
<ipython-input-10-d1427f7c3b24> in <module>
    104     params=p,
    105     experiment_name="talos/" + date_string,
--> 106     reduction_method='gamify',
    107 )
    108

~/.pyenv/versions/miniconda3-4.3.30/envs/tf_gpu/lib/python3.6/site-packages/talos/scan/Scan.py in __init__(self, x, y, params, model, experiment_name, x_val, y_val, val_split, random_method, seed, performance_target, fraction_limit, round_limit, time_limit, boolean_limit, reduction_method, reduction_interval, reduction_window, reduction_threshold, reduction_metric, minimize_loss, disable_progress_bar, print_params, clear_session, save_weights)
    194         # start runtime
    195         from .scan_run import scan_run
--> 196         scan_run(self)

~/.pyenv/versions/miniconda3-4.3.30/envs/tf_gpu/lib/python3.6/site-packages/talos/scan/scan_run.py in scan_run(self)
     24         # otherwise proceed with next permutation
     25         from .scan_round import scan_round
---> 26         self = scan_round(self)
     27         self.pbar.update(1)
     28

~/.pyenv/versions/miniconda3-4.3.30/envs/tf_gpu/lib/python3.6/site-packages/talos/scan/scan_round.py in scan_round(self)
     17     # fit the model
     18     from ..model.ingest_model import ingest_model
---> 19     self.model_history, self.round_model = ingest_model(self)
     20     self.round_history.append(self.model_history.history)
     21

~/.pyenv/versions/miniconda3-4.3.30/envs/tf_gpu/lib/python3.6/site-packages/talos/model/ingest_model.py in ingest_model(self)
      8                       self.x_val,
      9                       self.y_val,
---> 10                       self.round_params)

~/myapps/mmciad/src/mmciad/utils/hyper.py in talos_model(x, y, val_x, val_y, talos_params)
    301                 class_weight=class_weights,
    302                 verbose=internal_params["verbose"],
--> 303                 callbacks=model_callbacks + opti_callbacks,
    304             )
    305         return history, model

~/.pyenv/versions/miniconda3-4.3.30/envs/tf_gpu/lib/python3.6/site-packages/keras/legacy/interfaces.py in wrapper(*args, **kwargs)
     89                 warnings.warn('Update your `' + object_name + '` call to the ' +
     90                               'Keras 2 API: ' + signature, stacklevel=2)
---> 91             return func(*args, **kwargs)
     92         wrapper._original_function = func
     93         return wrapper

~/.pyenv/versions/miniconda3-4.3.30/envs/tf_gpu/lib/python3.6/site-packages/keras/engine/training.py in fit_generator(self, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, validation_freq, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch)
   1730             use_multiprocessing=use_multiprocessing,
   1731             shuffle=shuffle,
-> 1732             initial_epoch=initial_epoch)
   1733
   1734     @interfaces.legacy_generator_methods_support

~/.pyenv/versions/miniconda3-4.3.30/envs/tf_gpu/lib/python3.6/site-packages/keras/engine/training_generator.py in fit_generator(model, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, validation_freq, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch)
    218                                             sample_weight=sample_weight,
    219                                             class_weight=class_weight,
--> 220                                             reset_metrics=False)
    221
    222                 outs = to_list(outs)

~/.pyenv/versions/miniconda3-4.3.30/envs/tf_gpu/lib/python3.6/site-packages/keras/engine/training.py in train_on_batch(self, x, y, sample_weight, class_weight, reset_metrics)
   1512             ins = x + y + sample_weights
   1513         self._make_train_function()
-> 1514         outputs = self.train_function(ins)
   1515
   1516         if reset_metrics:

~/.pyenv/versions/miniconda3-4.3.30/envs/tf_gpu/lib/python3.6/site-packages/tensorflow/python/keras/backend.py in __call__(self, inputs)
   3290
   3291     fetched = self._callable_fn(*array_vals,
-> 3292                                 run_metadata=self.run_metadata)
   3293     self._call_fetch_callbacks(fetched[-len(self._fetches):])
   3294     output_structure = nest.pack_sequence_as(

~/.pyenv/versions/miniconda3-4.3.30/envs/tf_gpu/lib/python3.6/site-packages/tensorflow/python/client/session.py in __call__(self, *args, **kwargs)
   1456         ret = tf_session.TF_SessionRunCallable(self._session._session,
   1457                                                self._handle, args,
-> 1458                                                run_metadata_ptr)
   1459         if run_metadata:
   1460           proto_data = tf_session.TF_GetBuffer(run_metadata_ptr)

ResourceExhaustedError: OOM when allocating tensor with shape[16,192,208,208] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
	 [[{{node training/Adam/gradients/block1_u_conv1/convolution_grad/Conv2DBackpropInput}}]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

Have you looked at this SO post.

How much memory your GPU has?

I just check out your link and it does not appear to describe the issue I am having, though at first it did look similar. I am running with an Nvidia GeForce GTX 1080 TI with 11 GB ram.

Can you do this:
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.
...and share the output you get.

I would love to, but that option crashes my python kernel, so it's not really possible. This is a long-standing Keras bug, I believe.

Yes, it most certainly is an upstream bug in Keras or TensorFlow.
To avoid doubt, can you share your Scan() command.
Also, how about giving Talos 1.0 a shot. It will use different backend, so you might have better luck.

Sure! I use custom keras.utils.Sequence data generators, so I have two dummy variables for my scan command as shown below:
  dummy_x = np.empty((1, BATCH_SIZE, 208, 208))
   dummy_y = np.empty((1, BATCH_SIZE))

   scan_object = ta.Scan(
       x=dummy_x,
       y=dummy_y,
       disable_progress_bar=False,
       print_params=True,
       model=talos_model,
       params=p,
       experiment_name="talos/" + date_string,
       reduction_method='gamify',
   )

I will take a look at talos 1.0 right away!

So running talos 1.0 had the same outcome, but with a slightly different error message at the end:
 14% |█▌        | 5/36 [1:52:10<11:43:52, 1362.35s/it]
{'act': <class 'keras_contrib.layers.advanced_activations.swish.Swish'>, 'arch': 'U-Net', 'loss_func': 'cat_CE', 'nb_filters_0': 64, 'sigma_noise': 0.01}
tracking <tf.Variable 'block1_d_Swish1/scaling_factor:0' shape=() dtype=float32> scaling_factor
tracking <tf.Variable 'block1_d_Swish2/scaling_factor:0' shape=() dtype=float32> scaling_factor
tracking <tf.Variable 'block2_d_Swish1/scaling_factor:0' shape=() dtype=float32> scaling_factor
tracking <tf.Variable 'block2_d_Swish2/scaling_factor:0' shape=() dtype=float32> scaling_factor
tracking <tf.Variable 'block3_d_Swish1/scaling_factor:0' shape=() dtype=float32> scaling_factor
tracking <tf.Variable 'block3_d_Swish2/scaling_factor:0' shape=() dtype=float32> scaling_factor
tracking <tf.Variable 'block4_d_Swish1/scaling_factor:0' shape=() dtype=float32> scaling_factor
tracking <tf.Variable 'block4_d_Swish2/scaling_factor:0' shape=() dtype=float32> scaling_factor
tracking <tf.Variable 'block5_bottom_Swish1/scaling_factor:0' shape=() dtype=float32> scaling_factor
tracking <tf.Variable 'block5_bottom_Swish2/scaling_factor:0' shape=() dtype=float32> scaling_factor
tracking <tf.Variable 'block4_u_Swish1/scaling_factor:0' shape=() dtype=float32> scaling_factor
tracking <tf.Variable 'block4_u_Swish2/scaling_factor:0' shape=() dtype=float32> scaling_factor
tracking <tf.Variable 'block3_u_Swish1/scaling_factor:0' shape=() dtype=float32> scaling_factor
tracking <tf.Variable 'block3_u_Swish2/scaling_factor:0' shape=() dtype=float32> scaling_factor
tracking <tf.Variable 'block2_u_Swish1/scaling_factor:0' shape=() dtype=float32> scaling_factor
tracking <tf.Variable 'block2_u_Swish2/scaling_factor:0' shape=() dtype=float32> scaling_factor
tracking <tf.Variable 'block1_u_Swish1/scaling_factor:0' shape=() dtype=float32> scaling_factor
tracking <tf.Variable 'block1_u_Swish2/scaling_factor:0' shape=() dtype=float32> scaling_factor
Training |          | 0% 0/5 [00:00<?, ?it/s]
Epoch 0  |██▌        | [loss: 2.2988, acc: 0.1848, jaccard1_coef: 0.0575] : 25% 113/451 [01:19<03:12, 1.76it/s]
---------------------------------------------------------------------------
ResourceExhaustedError                    Traceback (most recent call last)
<ipython-input-10-050e8f7c8199> in <module>
    104         params=p,
    105         experiment_name="talos/" + date_string,
--> 106         reduction_method='gamify',
    107     )
    108

~/.pyenv/versions/miniconda3-4.3.30/envs/tf_gpu/lib/python3.6/site-packages/talos/scan/Scan.py in __init__(self, x, y, params, model, experiment_name, x_val, y_val, val_split, random_method, seed, performance_target, fraction_limit, round_limit, time_limit, boolean_limit, reduction_method, reduction_interval, reduction_window, reduction_threshold, reduction_metric, minimize_loss, disable_progress_bar, print_params, clear_session, save_weights)
    194         # start runtime
    195         from .scan_run import scan_run
--> 196         scan_run(self)

~/.pyenv/versions/miniconda3-4.3.30/envs/tf_gpu/lib/python3.6/site-packages/talos/scan/scan_run.py in scan_run(self)
     24         # otherwise proceed with next permutation
     25         from .scan_round import scan_round
---> 26         self = scan_round(self)
     27         self.pbar.update(1)
     28

~/.pyenv/versions/miniconda3-4.3.30/envs/tf_gpu/lib/python3.6/site-packages/talos/scan/scan_round.py in scan_round(self)
     17     # fit the model
     18     from ..model.ingest_model import ingest_model
---> 19     self.model_history, self.round_model = ingest_model(self)
     20     self.round_history.append(self.model_history.history)
     21

~/.pyenv/versions/miniconda3-4.3.30/envs/tf_gpu/lib/python3.6/site-packages/talos/model/ingest_model.py in ingest_model(self)
      8                       self.x_val,
      9                       self.y_val,
---> 10                       self.round_params)

~/myapps/mmciad/src/mmciad/utils/hyper.py in talos_model(x, y, val_x, val_y, talos_params)
    303                 class_weight=class_weights,
    304                 verbose=internal_params["verbose"],
--> 305                 callbacks=model_callbacks + opti_callbacks,
    306             )
    307         return history, model

~/.pyenv/versions/miniconda3-4.3.30/envs/tf_gpu/lib/python3.6/site-packages/keras/legacy/interfaces.py in wrapper(*args, **kwargs)
     89                 warnings.warn('Update your `' + object_name + '` call to the ' +
     90                               'Keras 2 API: ' + signature, stacklevel=2)
---> 91             return func(*args, **kwargs)
     92         wrapper._original_function = func
     93         return wrapper

~/.pyenv/versions/miniconda3-4.3.30/envs/tf_gpu/lib/python3.6/site-packages/keras/engine/training.py in fit_generator(self, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, validation_freq, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch)
   1730             use_multiprocessing=use_multiprocessing,
   1731             shuffle=shuffle,
-> 1732             initial_epoch=initial_epoch)
   1733
   1734     @interfaces.legacy_generator_methods_support

~/.pyenv/versions/miniconda3-4.3.30/envs/tf_gpu/lib/python3.6/site-packages/keras/engine/training_generator.py in fit_generator(model, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, validation_freq, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch)
    218                                             sample_weight=sample_weight,
    219                                             class_weight=class_weight,
--> 220                                             reset_metrics=False)
    221
    222                 outs = to_list(outs)

~/.pyenv/versions/miniconda3-4.3.30/envs/tf_gpu/lib/python3.6/site-packages/keras/engine/training.py in train_on_batch(self, x, y, sample_weight, class_weight, reset_metrics)
   1512             ins = x + y + sample_weights
   1513         self._make_train_function()
-> 1514         outputs = self.train_function(ins)
   1515
   1516         if reset_metrics:

~/.pyenv/versions/miniconda3-4.3.30/envs/tf_gpu/lib/python3.6/site-packages/tensorflow/python/keras/backend.py in __call__(self, inputs)
   3290
   3291     fetched = self._callable_fn(*array_vals,
-> 3292                                 run_metadata=self.run_metadata)
   3293     self._call_fetch_callbacks(fetched[-len(self._fetches):])
   3294     output_structure = nest.pack_sequence_as(

~/.pyenv/versions/miniconda3-4.3.30/envs/tf_gpu/lib/python3.6/site-packages/tensorflow/python/client/session.py in __call__(self, *args, **kwargs)
   1456         ret = tf_session.TF_SessionRunCallable(self._session._session,
   1457                                                self._handle, args,
-> 1458                                                run_metadata_ptr)
   1459         if run_metadata:
   1460           proto_data = tf_session.TF_GetBuffer(run_metadata_ptr)

ResourceExhaustedError: 2 root error(s) found.
  (0) Resource exhausted: OOM when allocating tensor with shape[16,64,208,208] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
	 [[{{node GaussianNoise_preout/cond/add-1-TransposeNHWCToNCHW-LayoutOptimizer}}]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

	 [[metrics/acc/Identity/_1095]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

  (1) Resource exhausted: OOM when allocating tensor with shape[16,64,208,208] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
	 [[{{node GaussianNoise_preout/cond/add-1-TransposeNHWCToNCHW-LayoutOptimizer}}]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

0 successful operations.
0 derived errors ignored.

Can you run the input model in a loop a few times and see if you can get the same result. If yes, I suggest posting this directly to TensorFlow.

Do you mean a simple loop like this:
model = create_model(*args, **kwargs)

for _ in range(6):
    model.compile(**kwargs)
    model.fit(train_generator, epochs=10)

Or should I try to add some sort of garbage collection to this?

No, just the simplest possible loop.

Is the above code example simple enough or can it be even simpler?

So far, when running a loop like the one I wrote earlier, I am not getting any ResourceExhaustedError. I have almost completed 5 iterations of the loop with 50 epochs pr iteration. With talos it crashed in the beginning of the fifth iteration.

Okay, so the loop was set to do ten training sessions of 50 epochs since I knew that 50 epochs was enough to get the ResourceExhaustedError  after 5 iterations in the talos Scan(). Now it has completed all 10 passes of the loop without any errors whatsoever. I assume this rules out it being a TensorFlow bug?

For good measure, I redid the Scan() just to confirm that updating some of the packages did not alter the outcome. Rather than getting the ResourceExhaustedError, my kernel crashed completely (though it may still be due to a ResourceExhaustedError). Any ideas on how to proceed?

I have now tested it on a different machine with a larger GPU (the NVIDIA Quatro RTX6000 with 24 GB ram) and the same thing happens.

To summarize
The bug(?) appears on two systems with the below configuration(s):

Nvidia GTX 1080 TI or Nvidia Quattro RTX 6000
Nvidia 418.87.00
CUDA 10.1
CuDNN 7.6.5
Python >=3.7.6
Both TensorFlow 1.13, 2.1, and >=2.2.0rc2
Talos >= 0.6.0

It does not appear to happen if a model is compiled and fitted several times in a simple loop, which seem to eliminate the possibility of this being a TensorFlow problem.

Is it possible for you to share a Jupyter notebook or colab which is self-contained, so I can just run and repeat.

Yes, create_model is identical. I will try and see if I can make a self-contained notebook. Currently, my solution has been to modify talos to accept a new boolean parameter allow_resume, which if True savew ParamSpace, list of keys/metrics and the various stores to files on disk and in the event of a crash (or interrupt) it will read these files and restore the important parts of the Scan object before executing scan_run(). It might sacrifice some efficiency, but it sure beats never getting to the finish line ;)
BTW, are there any special considerations behind doing method level imports rather than module/top level imports?

Sorry, I totally missed this.

BTW, are there any special considerations behind doing method level imports rather than module/top level imports?

Yes. Chunks of code are self-contained, readability improves, import only if need etc.
How about we implement the above-said feature into v1.1?

We could do that, but I'm not sure my hack is the best way to go at this point. It makes sense for me, but it could be a lot cleaner, I think. Perhaps storing everything in one file rather than having about three different files to read from :) I will be happy to show you the changes I made, though, and you can decide for yourself what you think of it.

You can look at the changes and additions here: https://github.com/bjtho08/talos/tree/1.0.1-dev

Thanks. Do I understand correctly, the feature is simply to:

allow storing a "restore point" as an option of Scan()
being able to refer to a file the "restore point" is  stored

Is there anything I'm missing?

Yep, that pretty much sums it up.
In the project directory (where the logging csv file is stored), three additional files are created: a pickle that contains the various stores from each run; a yaml that lists the remaining instances in the paramspace (dumped from self.param_object) and and a yaml file containing the self._all_keys, self._metric_keys, and  self._val_keys.
As I said, it can most likely be done in a cleaner fashion. I just hacked this together in a few days to work around my issue with constant crashing after a few iterations :)

Hello,
have you already found a working solution or a workaround for your problem? Because I am currently facing the same issue.

I will try to work on this next week.

Disabling eager execution solved the problem for me:
tf.compat.v1.disable_eager_execution()

@MolineraNegra wonderful.
@bjtho08 can you confirm if this works for you?

@mikkokotila
I'm confused, I thought keras disabled eager execution by default even for tf2.x?


Android app?

Any chance? I need this on my phone for absolutely no reason.

Every single phone on the planet has nocode installed on it, but many users aren't aware of it being there. It just doesn't show an icon on the app drawer or anything... Apparently it's running as a background process that can't be terminated even if you switch your phone off.
I believe there's a PR in the works to fix it.

I've always suspected it, but I couldn't quite confirm. Thanks for the info!

@drawcard This PR can fix it #1903


Add getChildPages optimization

👋 @kkga
This getChildPages optimization speeds up the 10k markdown file benchmark by 20-30sec on my M1 Pro.
I traced the benchmark and saw we were spending 60sec in this function:

We call getChildPages twice per page and end up doing duplicate work (looping the other pages).
This PR roughly halves the amount of time we spend in this function.
There are no performance benefits for small builds (e.g. docs).
As always, no pressure to merge this – just sharing for interest :)
Benchmark used
git clone https://github.com/Zettelkasten-Method/10000-markdown-files
time deno run -A --unstable main.ts --input 10000-markdown-files

Looks great! As always, thanks for your contribution :D


Add jules-verne-campus.de

@ArianMontgomery First stage of verification passed, official domain for Jules Verne Campus is jules-verne-campus.de as far as I verified. This is just an informational note. Pull request is still in the review. I appreciate your patience while waiting for the final decision.

@ArianMontgomery Thank you for your patience while waiting for a review and processing your request. Pull request merged.


Here is a negation overflow in lua

CVE-2020-24370 is a security vulnerability in lua. Although the CVE decription in CVE-2020-24370 said that this CVE only affected lua 5.4.0, according to lua this CVE actually existed since lua 5.2. The root cause of this CVE is the negation overflow that occurs when you try to take the negative of 0x80000000. Thus, this CVE also exists in MLN in the file MLN-Android/mlncore/src/main/jni/lua/ldebug.c.
Maybe you should try to backport the fix to the lua in MLN since the original fix is for 5.4 and several functions need to be changed. I'd like to open a PR for that if necessary.

thanks.
It would be great if you open a PR for that.


HIV Management - Case Management [10h]

Issue
Creating a followup

In a HIV Management encounter, the CHW has the option for follow-up in one day, one week, or one month. Any option should create a follow-up in a new "TB Management" section of the Case Management screen.

In any place where there is an HIV Lab test (e.g. ANC Labs) that gives a positive result a followup should be scheduled for one week from the positive test.

Logic for display on the case management "pills" should follow AI case management, that is

This week: follow-up due within 7 days
This month: follow-up due in 7 days or more
Due today: follow-up due today.
Past due: followup was due in the past

Resolution of a case management follow-up should occur if the patient has an HIV Management encounter when that follow-up is active (whether or not they entered through the case management screen). That is, any case management encounter should clear previous follow-ups, and a patient can only have one HIV Management follow-up at a time.
Tasks

Create the follow-up infrastructure
Create the logic for the creation of a follow-up and its resolution in both HIV management and in Lab Results
Render the page for HIV Management on the Case Management screen

@adamhstewart
Deployed to Nutrition env. Ready for QA.

https://nutrition-ihangane.pantheonsite.io/app
QA Review
Found issues:

None

Checked and approved:

Status of follow up displays correctly:

Past due
Due today

This week
This month


Top bar/click targets misaligned

Looks like for me, there's whitespace above the top bar, but the click targets for the buttons on the bar did not move. I have to click ABOVE the links on the top bar to hit them.

What's your browser? Also, do you observer this behavior on http://dillinger.io/ ?

@cem I am the resident "I will break your app because I use Edge" expert.
Dillinger.io is aligned correctly.

IE bug due to iframes :/
I'll add a workaround.

Should be fixed as of 68dd138e056124595481f1d8f643f045c839736d


build process failing on linkchecker

When opening a PR the PR is failing with link breaks on the 404 page
URL        `/reference/cli/'
Name       `Command Line Tool'
Parent URL file:///github/workspace/public/404.html, line 56, col 13901
Real URL   file:///reference/cli/
Check time 0.000 seconds
Result     Error: URLError: <urlopen error [Errno 2] No such file or directory: '/reference/cli/'>

Here is an instance https://github.com/cloud-native-toolkit/developer-guide/pull/470/checks?check_run_id=2822722794
cc @bwoolf1

This is due to a change to the mkdocs-materials container, which we use as the base image for the build action.  They changed the links in the 404.html file from relative to absolute, which breaks the linkcheckerrc, as the file links now start with /, so the root of the filesystem
Fix:
Modified the linkchecker config to set the site root : in PR #470
Updated the Dockerfiles to use a specified tag, so we can decide when to take new releases : in PR #472


Build failed when used as dependency with default-features = false

I added nfc1 as a dependency with:
$ cargo add nfc1 --no-default-features

Then ran the script.
$ cargo run
    Blocking waiting for file lock on build directory
   Compiling nfc1-sys v0.3.4
   Compiling nfc1 v0.5.1
error[E0432]: unresolved imports `nfc1_sys::pn53x_transceive`, `nfc1_sys::pn53x_read_register`, `nfc1_sys::pn53x_write_register`
  --> /Users/username/.cargo/registry/src/github.com-1ecc6299db9ec823/nfc1-0.5.1/src/device.rs:23:2
   |
23 |     pn53x_transceive,
   |     ^^^^^^^^^^^^^^^^ no `pn53x_transceive` in the root
24 |     pn53x_read_register,
   |     ^^^^^^^^^^^^^^^^^^^ no `pn53x_read_register` in the root
25 |     pn53x_write_register,
   |     ^^^^^^^^^^^^^^^^^^^^ no `pn53x_write_register` in the root

For more information about this error, try `rustc --explain E0432`.
error: could not compile `nfc1` due to previous error

It's referencing these lines:
https://github.com/alexrsagen/rs-nfc1/blob/258ae6a32890cee72858786e021589df0bbe34b1/src/device.rs#L23-L25
Probable cause
This is likely because when the default features are disabled, the default drivers in nfc1-sys are being disabled too:
https://github.com/alexrsagen/rs-nfc1-sys/blob/a8e2c8a999ca71e93c7af9e098218bc85717b219/Cargo.toml#L26-L32
Thereforebuild.rs does not genereates the bindings for the drivers.
Possible fix
When the --no-default-features flag is used the pn53x_* functions should not be included.
I suggest excluding these imports when the --no-default-features flag is enabled:
https://github.com/alexrsagen/rs-nfc1-sys/blob/a8e2c8a999ca71e93c7af9e098218bc85717b219/Cargo.toml#L26-L32

If you're not using the default features, you need to add every feature you need explicitly, as explained here: https://doc.rust-lang.org/cargo/reference/features.html
You need the "drivers" feature to generate bindings to driver-specific functions.
Your dependency line in Cargo.toml might look like:
nfc1 = { version = "0.5", default-features = false, features = ["drivers"] }`

Does this solve the issue?

Adding the drivers features "solves" the problem, in the sense that it links the drivers and generates bindings for the functions mentioned in the issue.
But I do not need any drivers related for the pn53x because I do not need to use the pn53x_* functions.
I think that the user shoul be allowed to link against the library without linking any vendor specific drivers, i.e. by importing the package like this: nfc1 = { version = "0.5", default-features = false }.
The documentation page that you linked provides an example in which the dependency jpeg-decoder is added having all the default features disabled, with no other dependency specified.

Thanks for clarifying.
I agree this should work. I'll look into what code depends on the driver-specific functions and add #[cfg(feature = "drivers")] where applicable.
I'll let you know when building without drivers feature works fine.


grafana output

Hi
Thank you Stephan for this work. I have a question about grafana. I try to access it at "<IP_ADDRESS>:3000" but it gives me this page

I'm not able to access dashboards, or any other thing from grafana? I'm not sure what I did wrong?
One other question please, what should I do to access collected metric values programmatically in python? should I use a specific library? or should I forward collected metrics to a database, then access it from python?
Regards

I'm seeing the same issue

For Prom 1.8 this has been fixed, if you want to stick with 1.8 download the release from here https://github.com/stefanprodan/swarmprom/releases

Thank you Stephan
The last swarmprom (with prometheus 2.2.0-rc.0) works. Happy I didn't have to go to 1.8
Regards


New version: ImageMagick.ImageMagick version <IP_ADDRESS>

- [X] Have you signed the Contributor License Agreement?

[ ] Have you checked that there aren't other open pull requests for the same manifest update/change?

[X] Does your manifest conform to the 1.4 schema?

Resolves microsoft/winget-pkgs#96508
Microsoft Reviewers: Open in CodeFlow

Still an issue:
ImageMagick-7.1.1-4-Q16-HDRI-x64-dll.exe
An unexpected error occurred while executing the command:
Download request status is not success.
0x80190194 : Not found (404).


The project has no license

Hi @domluna @baumanab,
I notice that our project doesn't have license, that's a little unprofessional for a open source project, especially there has much extends adoption and development in an online education project. "No license terms" means users don't know it can be used freely. Another reference is Can I call my program "Open Source" even if I don't use an approved license?.
So here I suggest the project add license, recommend we choose one of the below license, firstly because they're all Permissive Licenses, but they have different redistribution disclaimer:

MIT
Apache-2.0
BSD-3-Clause

Regards,
Brandon

Added!


Resolved Bug 667: Storybook > Component > Small and Large Mabile view save and cancel button is alignment issue all component.

Description

Resolve the issues on Component button alignment issue:
Make the changes to responsive.scss file.
Resolve alignment styles and CSS issues on all components mentioned above.

Screenshots :

Checklist:

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[x] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes

@Ketan-Khalate  Please close this pr as it has commits issue.
please refer PR: https://github.com/Wai-Technologies/raaghu-react/pull/721

@poojagurav21 ok as I this not from base branch I am closing this PR


Choose flags to include

Give an option to allow users to specify which flags to include in the output CSS.
Specify using a variable list in SCSS like so:
$include: 'as', 'bl', 'si';
If this variable is omitted then all flags are imported as normal.
Also moved the "more" flags into the main list as it seems pointless separating them. Maybe this could also be expanded to have an $exclude variable too so it includes them all apart from the ones specified. It then would allow full control in three different ways!

All flags
All but some (exclude)
Some but not all (include)

Thanks,
Neil

Can we please get this merged into develop? We really need this feature :)

This is actually quite interesting.. :)

and there is no breaking change here.. right?


Fix issue 24

This resolves issue 24. The order of initialization is changed, so that the edge error is calculated after the borders have been defined.  This error will not be obvious if a compiler initializes the border array to zero. In my experience, the issues are very obvious with the Clang compiler.
You can validate the issue and the resolution by downloading the face.zip mesh:
import pyfqmr
import trimesh as tr
face = tr.load_mesh('~/face.obj')
mesh_simplifier = pyfqmr.Simplify()
mesh_simplifier.setMesh(face.vertices, face.faces)
mesh_simplifier.simplify_mesh(target_count = 164600, aggressiveness=7, preserve_border=True, verbose=10)
vtx, fc, normals = mesh_simplifier.getMesh()
mesh = tr.Trimesh(vertices=vtx,faces=fc)
mesh.export("~/simplified.obj")

Ughh, my bad again. I pushed to my fork rather than this one.  Can you rebase again please?

Yay, CI happened.

Thanks @neurolabusc! Apologies for the polava.


fix ShallowWrapper check for IE

Fixes https://github.com/blainekasten/enzyme-matchers/issues/89.

Love this! Smart move. Thanks!


Fixes Issue No #46 and Issue No #54 | Adds Speaker Profile Info | Adds Social Media Details | Increases Sizes

Solves & Fixes Issue #46 and #54 .

Adds Speaker Profile Info
Adds Social Media Details
Increases Social Media Size Buttons
Photo added in detail view of speakers
Links and Social Channels given now
Session of speakers added - In Speaker Matthias Spielkamp's detail view
Organisation Link also Added
Speakers Overview Doesn't start with a comma
Other details required in the respective issues are also added

@mariobehling Please have a look at this PR.
Please suggest in case if any other changes are required to improve the code.
Solves & Fixes Issue #46 and #54 .
Thanks! :smile:

@mariobehling @creativepsyco Is this fine? Is this PR ready to be merged?

@jig08 Could you review this PR as well? Thanks! :smile:

@SaptakS @joerx @daminisatya @mariobehling @jig08 @AmayAgrawal @championswimmer @creativepsyco  Could someone review the PR, please! Thanks! :smiley:

Can the issues #46 and #54 be closed, and this PR can be merged?

@championswimmer Hi. So, I have made the required changes. So, can this be merged now?
@joerx @mariobehling @jig08 @AmayAgrawal @creativepsyco @SaptakS @daminisatya
Thanks! :smile:


space before \left( and after \right)

LaTeX example:
\begin{array}{c}
a\left(b\right)c
\\
a(b)c
\end{array}

there is some extra space before \left( and after \right) (See this post)
In Temml, the two lines are the same:

I believe that Temml is emitting the correct MathML code for these parentheses. They are an <mo> element, with the attributes for fence and form set correctly.
You are correct in that Firefox does not rendering these parens with any extra spacing. That could be regarded as an improvement. The post to which you linked is a discussion about how to get rid of that spacing. I tend to agree with them.
I do not intend to take any action. If you want that spacing, you'll need to take this up with the browsers.


Add String.split_by that uses a string delimitator. (fix #1399)

The signature is almost the same as String.split, we don't have any default value here. I'm not sure what'd make sense, \n, ' '?
There are some users of String.split in the stdlib and this can't be a drop-in replacement for those. If it's acceptable we can replace the current String.split uses with a Regex or move the current logic under another name (tokenize?).

We discussed this on the sync call.
The favored course of action is to accept this PR as-is, then later refactor to use type parameter inference (#1184) and specialisation (#1410) once those two features are implemented, so that the user can pass either String (for this new behaviour) or a ReadSeq[U32] (for the old behaviour of splitting by any of the given runes in the string).  We might also consider allowing ReadSeq[String] as well.  The combination of type parameter inference and specialisation means that we can have ergonomic usage of passing different types to the same method name as a zero-cost abstraction (instead of using a type union and unwrapping it - an abstraction with a cost).


Make sf::Ftp::SendCommand public

This is very useful for non-standard FTP commands (SITE CHMOD).
(I hope this is the right way to do a pull request. :P)

I'll probably do it later. But not for SFML 2.0.


Add syslog writer

Description
Linked issue:
#156
Linked website pull request, if the documention of https://tinylog.org/v2/ has to be updated:
tinylog-org/website#000
Definition of Done

[x] I read contributing.md
[x] There are no TODOs left in the code
[x] Code style follows the tinylog standard
[x] All classes and methods have Javadoc
[x] Changes are covered by JUnit tests including edge cases, errors, and exception handling
[x] Maven build works including compiling, tests, and checks (mvn verify)
[x] Changes are committed by a verified email address that is assigned to the GitHub account (https://github.com/settings/emails)
[x] Documentation on the tinylog website is up-to-date or updated by a pull request to the repository tinylog-org/website

Agreements

[x] I agree that my changes will be published under the terms of the Apache License 2.0 (mandatory)
[x] I agree that my GitHub user name will be published in the release notes (optional)

Thank you for your pull request! I will review it soon. However, your changes looks very good at first sight. :)

I'm done with the review. I could find only a few minors. You have written very good and maintainable code :)

By the way: Would you be able to add some documentation for the tinylog website?
Link:
https://github.com/tinylog-org/website/edit/v2/content/documentation/configuration.md

Thank you very much for your great pull request! It's really awesome to have a syslog writer in tinylog.


Trying to fix build-failure in docker hub

Success! https://hub.docker.com/r/cwspear/docker-local-persist-volume-plugin/builds/


[Service Bus] Incorrect logs related to actionAfterWaitTimeout - batching receiver

actionAfterWaitTimeout

Timers are set at different times

totalWaitTimer = setTimeout(actionAfterWaitTimeout, args.maxWaitTimeInMs);
totalWaitTimer = setTimeout(actionAfterWaitTimeout, getRemainingWaitTimeInMs());

Consider the scenario - with the workflow

timer-1 is set - maxWaitTimeInMs = 10000
1st message received at 1000 ms
timer-1 is cleared
timer-2 is added - getRemainingWaitTimeInMs() = 1000 ms
No messages received
timer-2 goes off
The log message would say Batching, max wait time in milliseconds 10000 over which is wrong, it should instead say 2000 ms

PS: Observed this while discussing the logs for https://github.com/Azure/azure-sdk-for-js/issues/8875, https://github.com/Azure/azure-sdk-for-js/issues/11633 with @anqyan (from the service-bus team)
TODO
Probably, pass the time as an argument for the actionAfterWaitTimeout just for the log statement and update the log statement accordingly.

Done with #12322


Add back getOptionsKey test that was reverted

Add back connection profile test that was previously revert to address https://github.com/microsoft/azuredatastudio/issues/23273.

hmm, it looks like this tests when restored.  I'll take a look.

@kburtram This is likely because you didn't bring over the other fix that I did with passing in the serverCapabilities as well. So following up with doing that as well should complete this.
From the original PR:
But while fixing this it seemed to me that all this logic was broken in the first place - we would never generate the same options key even with the original code even though the objects should be for all intents identical. So I'm fixing it to also pass in the serverCapabilities object when creating the IConnectionProfile - which will then fix this secondary issue as well.


Order form links

Do we want our order form links to go to the printable order form PDF?
OR
Do we want our order form links to go to the online order form?

Add online film Process + Scan link. Per conversation.

working on this now

completed. see what you all think. @afaiman @digijim

@afaiman and I went over all these changes and made slight mods here and there (some unrelated to the addition of online order links) All of our changes were in Pull Request #181


Websocket Boilerplate - Nextjs

Create a websocket client in the front-end application for sending requests to the collector.
Requirements:

Successfully connect to the collector (websocket server)
Send and receive messages from the websocket server

Websockets work and UI is receiving data


General: Route to not found page

Is your feature request related to a problem?
When a route cannot be found there is currently no clear error message but instead just re-routing to the base Url, which requires the user to notice the change of the Url to understand that the route cannot be found.
If the Url change is not noticed it seems like Artemis would not work properly.
Current redirect to base url:

Describe the solution you'd like
A not found page which

contains the requested route
a redirect button to the course overview

Duplicate of #6689


chore: remove Prettier from pre-commit config

Pre-commit prettier is not maintained anymore and if we want to run it
we should consider an alternative way
https://github.com/pre-commit/mirrors-prettier

RELEASE NOTES BEGIN
Remove Prettier from pre-commit
RELEASE NOTES END

thanks for noticing and opening the PR! We are using it across multiple other packit repositories, so we should consider using some other fork or some alternative

@lbarcziova could be solved by dprint #419. They don't have pre-commit but they have a GH action

Okay Biome is more popular and has a bit better maintainability for now. I'd vote in favor of that. In #447 currently I run into an issue again with prettier wanting to format a file and we can't fix it easily using pre-commit. I'd rather go to using husky or just a GitHub action for handling formatting using something like Biome
Where else is prettier used atm in other projects? It should only be a JS formatter @lbarcziova

@Venefilyn we use prettier in other projects to format Markdown, yaml and JSON (e.g. test response files).
Biome looks cool as a JS formatted, if I checked correctly it doesn't support yaml and Markdown, right?

@Venefilyn we use prettier in other projects to format Markdown, yaml and JSON (e.g. test response files).
Biome looks cool as a JS formatter, if I checked correctly it doesn't support yaml and Markdown, right?

@lbarcziova you're right, currently it doesn't support it https://biomejs.dev/internals/language-support/


[Transforms] Fix for a number of source map emit issues.

Fixes a large number of sourcemap related test failures.
Tests fixed:

Too many to list (over 150).

:+1:


apm install platformio-ide-terminal

[ ] I have reset Atom to defaults prior to submitting report.
[ ] I have not reset Atom to defaults prior to submitting report.

Description

Steps to reproduce

Expected behavior:
Actual behavior:
Reproduces how often:
Versions

> atom --version

> apm --version

OS name and version:
Platformio-ide-terminal version:
Additional Information

Fixed in https://github.com/platformio/platformio-atom-ide-terminal/releases/tag/v2.10.1
Please update platformio-ide-terminal to the latest version.


Heatmap is showing only one row.

Hi,
Im using version 0.4.6 on grafana v5.4.2 and using the zabbix datasource
Im trying to create a heatmap like the one on your screenshots.

As you can see it's displaying only one row.
It's one host in zabbix with +200 items(metrics) from the nodes it has connected.  each green square is one metric. It's possible to make them fit on multiple lines? or it's possible to disable the scroll bar? .

This is by design.
In my example, I was referring four different metrics for 200+ servers.  Obviously i use 1server*500 metrcis or 500 servers * 1 metic, I will end up in horizontal or vertical scroll  bar.
BTW, there is a chance to achieve this, if you group your data with some limited number of groups using alias.
BTW, What is the usecase to show 200 metrics from one host..??

Thanks for the quick response, i don't know how to group it by alias but im going to investigate it.
i want to have a grid(heatmap) like this to display the status of the 200+ nodes , we need to see if they're connected or disconnected and this is "easy" way to check them all at once.

the 200+ metrics are 200+ nodes(servers)

i'd love to see some multi row logic as well. Just like the panel repeat configuration perhaps, a "max per row" setting

I'm clearly missing something obvious, but how do you config the panel to display the heatmap results horizontally? I have a Prometheus query that returns CPU usage. It renders fine as a table. But, no matter what I try, it still displays the results in a table, vertically, one row per server.
I found this issue with screenshots, but it still didn't help. Must be missing something really obvious here :-(
I'm using v1.0.0 of the plugin.

I'm clearly missing something obvious, but how do you config the panel to display the heatmap results horizontally? I have a Prometheus query that returns CPU usage. It renders fine as a table. But, no matter what I try, it still displays the results in a table, vertically, one row per server.
I found this issue with screenshots, but it still didn't help. Must be missing something really obvious here :-(
I'm using v1.0.0 of the plugin.

Invert the col/row , as you can see on my screenshot 0 and 1 if i configure 1 on the row and the other on the col i will display it vertically.

Yeah, thanks for the prompt reply. Just this morning I got it working.  Digging into the row / column headers in debug mode pointed me in the right direction.
Thanks.


[Question] Does tsdx can publish many components ?

what is the purpose ?
Three days ago I searching to pass to 40 Mo with tsx, .d.ts and js files to uniq UMD file with many js file (the goal is achieve with ~3Mo i think)
in a nutshell
Actual 👎  :

My module have 100~130 components so make one component by repo is making for me no sense .
so right now my process is very silly but no regrets !
I give a try with https://www.npmjs.com/package/npm-dts but with critical failure (because of many files  and some "export * from "../someFolder")
const path = require('path');
const MiniCssExtractPlugin = require('mini-css-extract-plugin');
const CssMinimizerPlugin = require('css-minimizer-webpack-plugin');
const OptimizeCssAssetsPlugin = require('optimize-css-assets-webpack-plugin');
const TsconfigPathsPlugin = require('tsconfig-paths-webpack-plugin');
const NpmDtsPlugin = require('npm-dts-webpack-plugin')
module.exports = {
    plugins: [
        new NpmDtsPlugin({
            output: path.resolve(__dirname, 'dist')
        }),
        new OptimizeCssAssetsPlugin({
            cssProcessorOptions: {
                safe: true,
                discardComments: {
                    removeAll: true,
                },
            },
        }),
        new MiniCssExtractPlugin({
            filename: 'styles.css',
            chunkFilename: '[id].css',
        }),
    ],
    entry: {
        index: { import: './src/index.ts', dependOn: 'shared', },
        shared: 'lodash',
    },
    module: {
        rules: [
            {
                test: /\.tsx?$/,
                use: 'ts-loader',
                exclude: /(node_modules|\.css$)/,
                include: path.join(__dirname, "src"),

            },
            { test: /\.css$/, use: [MiniCssExtractPlugin.loader, 'css-loader'] },
            {
                test: /\.(png|jpg|gif|svg)$/i,
                use: [
                    {
                        loader: 'url-loader',//options: {limit: 8192,},
                    },
                ],
            },
        ],
    },
    resolve: {
        extensions: ['.tsx', '.ts', '.js'],
        plugins: [new TsconfigPathsPlugin({ configFile: "tsconfig.json" })]
    },
    output: {
        filename: '[name].js',
        chunkFilename: "[name].chunk.js",
        path: path.resolve(__dirname, 'dist'),
    },
    optimization: {
        runtimeChunk: 'single',
        splitChunks: { chunks: 'all', },
        minimize: true,
        minimizer: [new CssMinimizerPlugin(),],
    },
};

I'm not friendly with configurate configaration of config tools ... However I spend 3 days on webpack and no solution yet.
Webpack is powerfull but i want something so "basic things" ...
that is what I want 👍 :

troubles with webpack
so i want to acheive with webpack (no comment) .
I search into the deep web https://www.npmjs.com/package/npm-dts this.
in my opinion it is a "wow awesome" package :D
What I want to achieve ?

Really ... I send into npm every build: 800 files ! Hell things I agree but no have choice beetween
ultra complex config to achieve and ultra easy silly way.
Do you think this npm package can acheve it  ? or Can you please tell me some tips ?
I think this package do that :

Do you have an idea if this awesome coolest library can provide me for this usage ? (if cannot, can you tell me some tips ? at least it can help other people to deal with that )

up ! can you tell me if the package will be maintain ? (yes or not just wondering for lead my technical choice).


Fix agg result when column value could be null

reopen this one since it didn't  consider all null case.

have fixed all null case, remain this issue to impl group by key = null case


Characters shifting while autocomplete in fish

Hello,
I'm having an issue while using powerline-shell in fish:
When I use the autocompletion, the line is shifted to the right (the demo is in termite but it's the same on the TTYs)

When I'm using a "classic" prompt it works well, I have this issue only when I use powerline-shell ..
Here is the code I added to my config file for using powerline-shell in fish :
function fish_prompt
  powerline-shell --shell bare $status
end

I reinstalled everything on my pc, and I still have this problem but only on terminal emulators (it works correctly on TTYs)
And now, when I launch my graphical environment (I tried with Gnome X11, Gnome wayland and sway) from a TTY it works but when I launch it from a display manager (GDM or LightDM) I still have this bug ...


Feature: Interfaces

Is your feature request related to a problem? Please describe.
Large deployments can span sets of resources deployed to all number of different scopes. Given a tenant, a management group and a subscription, I may deploy resources at a global scope (those intended to be created only once) and deploy others on a regional basis (either one per region or sets of common individual regional deployments).
As the deployment is intended to span all the resources at a given scope, it introduces an enormous pile of parameters to keep track of, leading to a "root" file that dishes out a subset of these parameters to child modules that deploy the actual and dependent resources. There's an excellent example of this in @brwilkinson 's AzureDeploymentFramework repository, where you have a directory containing all the resources for any scope of deployment. The author's stated intent is that one can deploy everything as a greenfield deployment or incrementally push to only a single "spoke" in a given region.
Parameters are then supplied on a scoping basis (not only ARM scopes, but also including values like tenant name and deployment ID (keeping track of how many deployments exist in a region) and passed into the pipeline at that root, where they trickle down to each resource based on the configuration.
Unfortunately, in an effort to create a simplified version of this for my own purposes, I've found that this approach leaves room for some complexity - all the types and names of values are hidden away in these scoped parameter files and Bicep simply trusts that it'll find them in the blobbed 'object' types specified in the markup.
In an effort to ensure that each resource can be independently deployed, it becomes necessary in this example to pass whole untyped objects down from the root to the children in an identical fashion so that sub-properties can be accessed and consumed as necessary.
The shapes of modules are fairly clearly described. You have a number of input parameters which are generally clearly typed and that informs the developer about how to treat it. If I indicate that I have a string param, I can prescribe it allowed values and treat it as an enum in the rest of the module:
@allowed([
  'apple'
  'banana'
  'carrot'
])
param myOptions string

This helps me as the developer to understand what values it may or may not contain as I build out the module. Similarly, I can indicate a numerical input and put constraints on the range or a boolean and describes its default value. These all help the developer.
However, if I simply pass an untyped object or an untyped array, I can infer that there are subproperties of the former and that there are zero or more elements of the latter, but that's as far as I can see into the shape of the argument. Today, Bicep apparently takes a standoff-ish approach to handling these object properties as it just assumes they're accurate and doesn't share any feedback in VS Code when specifying them, because it has no source of truth from which to discern what the properties should or shouldn't be.
Describe the solution you'd like
I would like to propose the notion of an interface as used in TypeScript (for whatever reason, TypeScript's actual docs have deprecated their own interface page) to describe the shape of parameter objects strictly at design-time and without any impact on the build output.
Put simply, in any given module, I would propose a new language construct, the interface to be defined and provide a parameter-like approach for defining (exclusively for Bicep) the shapes of objects and the objects passed in arrays into other modules. For example, I'm picturing something like this:
//Module A
interface globalType = {
  prefix: 'az'
  environment: 'g'
  deploymentId: 2
}

Today, as long as the definition stays inside of a single module, I could just define this as a variable object and Bicep can infer both the names and types of the object for same-module definitions. But as soon as I pass this into another module, that's entirely lost because I can only pass it in as another untyped object parameter. I'd rather be able to do something like:
//Module B
param globalOptions globalType
param listOfOptions array<globalType>

Today, at best I can do:
//Module B
param globalOptions object
param listOfOptions array

And I lose all the clarity of what exactly is in globalOptions or listOfOptions, trusting both that I passed the appropriate object into whatever is calling this module and that the properties I'm calling on it are actually defined somewhere (and not just a runtime error waiting to happen).
To repeat, like TypeScript, this should not at all change what's actually produced by the compiler. At build time, Bicep would still just treat these parameters in Module B as object and an array as it does today, but it would allow the developer some peace of mind to catch typos, utilize Intellisense to know the properties of objects without leaving sets of other windows open and enable optional benefits of type safety without any impact on the dynamic nature of Bicep for anyone that wants to opt out.
Thanks for taking the time to read this proposal.

There are currently some investigations for how to achieve some of these asks.
Also the ability to define a custom object/types has been part of the same conversations.
I will add links here as I find them for reference.
https://github.com/Azure/bicep/issues/4158

Re-opening this to identify another variation that's more relevant to my proposal here than to #4158
I'm working on building out the infrastructure to deploy Azure Service Fabric applications via ARM and am starting with a generic module for the deployment.
Skipping over most of the project structure as it's just not relevant here, there are two kinds of services one can deploy to an SF cluster: stateless and stateful services. In ARM, they are both represented by a single type "Microsoft.ServiceFabric/clusters/applications/services" but each uses quite different properties of that type.
Ideally, with the proposal I outlined above, I could indicate something like (a number of different ideas illustrated in the first one):
// This one could be illustrated using #4158 since it's contained as a separate type altogether in the ARM markup
var serviceLoadMetrics interface = {
  defaultLoad: int = 10 //Default value, absence of it indicates it's a required value not unlike params in Bicep files, ideally infer type from it too without me explicitly indicating as much as in primaryDefaultLoad below
  name: string
  primaryDefaultLoad = 10
  secondaryDefaultLoad: int? //I don't specify a default value, but the ? indicates it's optional nonetheless
  weight: ['High', 'Low', 'Medium', 'Zero'] //Similar to using @allowed, but inline in the interface
}

// And from here on, we're in a world that I think #4158 doesn't suffice for
var serviceFabricService interface = {
  name: string
  partitionScheme: partitionScheme //Into which I can pass a namedPartitionScheme, a singletonPartitionScheme or a uniformIntRangePartitionScheme
}

var sfStatelessService interface as serviceFabricService = {
  instanceCount: int
  serviceLoadMetrics: serviceLoadMetrics[] //Indicates an array of the above-defined type
}

var partitionScheme interface = {
}

//Demonstrate interface inheritance
var namedPartitionScheme interface as partitionScheme = {
  names: string[]
}

var singletonPartitionScheme interface as partitionScheme = {
}

var uniformIntRangePartitionScheme interface as partitionScheme = {
  count: int
  highKey: int
  lowKey: int
}

var sfStatefulService interface = {
  targetReplicaSetSize = 3
  minReplicaSetSize: int = 2 //Again, ideally just infer the type if I don't specify it
  replicaRestartWaitDuration: string = '00:01:00.0'
  quorumLossWaitDuration: '00:02:00.0'
  hasPersistedState: bool = false
}

Now, in a separate file, I need only do something like the following:
var serviceA './my-modules/ServiceFabricApplicationDeployment.bicep': sfStatefulService = { //Like a resource type, this illustrates where to find the interface
name: 'ServiceA'
//minReplicaSetSize: Don't need to specify as it already has a default value of 2
replicaRestartWaitDuration: '00:05:00.0' //But I can specify it if I want to override the default
partitionScheme: { //Automatically type matches to a uniformIntRangePartitionScheme even without explicitly indicating it
count: 5
lowKey: 0
highKey: 100
}
}
module SFApp './my-modules/ServiceFabricApplicationDeployment.bicep'  = {
services: [serviceA] //Configured to accept serviceFabricService[], meaning either a stateful or stateless service
//Other properties
}
And then per the markup above, I get type-checking on any objects I pass around and know that even if I have several dozen files that all point into this module, I can be certain that they contain the necessarily typed properties to properly populate the parameters necessary.
Additional thoughts - for your benefit, it might be worth swapping var with interface when defining the interfaces themselves since these aren't actually variables that would be populated and exposed in the ARM template itself - thinking in a similar vein as TypeScript, these exist purely at development time and do not otherwise leak into the deployed files.

Wanted to jump in with another example relevant to today's task. I'm working on simplifying my various modules so I can both publish them for public consumption and simplify everything.
As is not uncommon when working with Azure resources, child resources tend to require more than one value to be set and they each require these values be set alongside each other. Bicep has neither an optional nor required keyword today, rather allowing all parameters to be set by a caller and allowing default values to be set in the module. This is generally fine, but doesn't assist module development for more complex scenarios like this.
Let's say I've got an Event Hub resource and I would like to optionally allow a caller to add some number of authorization rules to. Each of these authorization rules needs to specify a name and an array of "rights", spanning 1 to 3 of the following values: 'Send', 'Manage' and/or 'Listen'. I can attach these in a module like the following:
@description('The name of the authorization rule')
param Name string

@minLength(1)
@maxLength(3)
@allowed([
  'Send'
  'Manage'
  'Listen'
])
param Rights array

@description('The name of the Event Hub resource')
param EventHubName string

@description('The name of the Key Vault to save the ')
param KeyVaultName string = ''

resource EventHub 'Microsoft.EventHub/namespaces/eventhubs@2022-01-01-preview' existing = {
  name: EventHubName
}

resource AuthorizationRule 'Microsoft.EventHub/namespaces/eventhubs/authorizationRules@2022-01-01-preview' = {
  name: Name
  parent: EventHub
  properties: {
    rights: Rights
  }
}

As it's not intended that the authorization-rule module be accessed externally, I need to make these name and rights properties visible on the event-hub.bicep entry module then, but this is where I run into an issue without something like my interfaces concept.
I could do something like the following:
@param EventHubName string
@param AuthorizationRuleNames array = []
@param AuthorizationRuleRights array = []

//...

module EventHubAuthorizationRule './authorization-rule.bicep' = [for (authorizationRuleName, i) in AuthorizationRuleNames: {
  name: 'eh-${EventHubName}-AuthRule-${authorizationRuleName}'
  params: {
    Name: authorizationRuleName
    Rights: AuthorizationRuleRights[i]
  }
}]

But the problem with this is that as the module author, I'm limited only in describing in the description metadata that authorization rule rights really need to be an array of arrays and I lose out on being able to put the validation constraints on what the passed values actually are. Further, I have no way of dictating that the length of AuthorizationRuleRights should be the same length as AuthorizationRuleNames.
So I could just make it a single array and expect that the author provides me with an appropriately formatted JSON object, but I lose all validation capability again and will only know whether it works or not at runtime (where I can count on the error message being less than stellar per https://github.com/Azure/bicep/issues/4848).
@param EventHubName string
@param AuthorizationRules array

Going back to my proposal here, this could be quite easily remedied with some sort of strongly typed interface that I can decorate with the existing validation options:
interface authorizationRule = {
  name: string
  @minlength(1)
  @minlength(3)
  @allowed([
    'Listen'
    'Send'
    'Manage'
  ])
  rights: array
}

Now in my module, I can easily indicate that I'll accept zero or more of this interface:
@param EventHubName string
@param AuthorizationRules authorizationRule[] = [] //Indicates that members of the array should be typed with the interface

//...

module EventHubAuthorizationRule './authorization-rule.bicep' = [for (authorizationRules, i) in AuthorizationRules: {
  name: 'eh-${EventHubName}-AuthRule-${authorizationRule.name}'
  params: {
    Name: authorizationRule.name
    Rights: AuthorizationRule.rights
  }
}]

And when compiled, the interface drops away. Like that of a TypeScript interface, it provides only for the shape of the expected values and doesn't actually have any sort of runtime presence.
This would easily allow me, as the module developer, to clearly articulate the shape of complex objects without reliance on the consumer reading and understanding verbose in-lined documentation, trial and error through runtime failures or mismatching of sets of arrays that I'm enumerating through by index. Rather, I can define the shape of the expected arguments between modules (ideally both in via parameters and out via outputs) and make that available at development time via Intellisense, at build-time via the compiler and leave the runtime entirely oblivious to it all.

cc @jeskew as FYI
I'm not reopening only because I don't want multiple issues speaking to the same problem, not because this issue is not valuable. We will take a look at these examples and try to factor that into our design (which there will be ample opportunity for feedback on).
We are taking a close look at how to define custom types and perform operations on those types as you'd expect in other languages. You should be able to use custom types in other type definitions, and we should be able to validate that deeply, etc.

@alex-frankel It's fine not re-opening it, so long as it's visible as a possible solution to the problem. Encountered this again today in the example above so it's just top of mind for me. Reviewing the feedback in #4158 , I still find that proposal far too strictly bound to specific resources and not nearly flexible enough for larger, more modular deployments that span dozens of potentially inter-configurable resources (e.g. optionally create 1 or more authorization rules in an event hub, but optionally save each of their connection strings to a specified key vault).
The above is doable with today's typing, but only in a way that's not easily transferrable outside of a single developer's domain knowledge about the larger Bicep deployment philosophy, much less a public repository of such modules.
Thanks for the continued consideration!


Sync repository packagist.org

New version v1.9.1 is not on site
https://packagist.org/packages/cbschuld/browser.php

@visavi got that solved for now!  more updates coming in the next few days

need to reopen this to re-address the auto updates!

handled officially now!


fastjson version need to be upgraded?

Describe the bug(Please use English)
The fastjson version used by MySQL CDC has a deserialization vulnerability.
Additional Description
fastjson version need to be upgraded?
A thermal repair can be carried out if necessary.

@leonardBang


Windows unable to acquire crypt context with new keyset

I am not sure if this is variant of bug reported in #163, but when I tried to remove password on Windows with user log in domain I got error:
qpdf.exe --decrypt --password=TOP_SECRET --progress 09868_01.12.2018.pdf out.pdf
unable to acquire crypt context with new keyset

When I run the same comand on the same computer under local user it works.
I use the latest qpdf (8.40).

qpdf-8.4.0-bin-mingw64.zip produces same error message under domain user.

I don't have any way to diagnose or test this as I don't have access to any Windows systems that are part of a domain. Maybe someone who is familiar with how this code works can provide some assistance.

Can you please implement better error message/exception handling e.g. something like this:
diff --git a/libqpdf/SecureRandomDataProvider.cc b/libqpdf/SecureRandomDataProvider.cc
index fe9f1d4..ba99d64 100644
--- a/libqpdf/SecureRandomDataProvider.cc
+++ b/libqpdf/SecureRandomDataProvider.cc
@@ -66,6 +66,17 @@ class WindowsCryptProvider
                                          PROV_RSA_FULL,
                                          CRYPT_NEWKEYSET))
                 {
+                    DWORD errorMessageID = ::GetLastError();
+                    LPSTR messageBuffer = nullptr;
+                    size_t size = FormatMessageA(
+                                  FORMAT_MESSAGE_ALLOCATE_BUFFER | FORMAT_MESSAGE_FROM_SYSTEM |
+                                  FORMAT_MESSAGE_IGNORE_INSERTS, NULL, errorMessageID,
+                                  MAKELANGID(LANG_NEUTRAL, SUBLANG_DEFAULT),
+                                  (LPSTR)&messageBuffer, 0, NULL);
+                    std::string message(messageBuffer, size);
+                    LocalFree(messageBuffer);
+                    fprintf(stderr, "Error number %lx: %s", errorMessageID,  message.c_str());
+
                     throw std::runtime_error(
                         "unable to acquire crypt context with new keyset");
                 }

which now provided useful explanation of problem:
>qpdf.exe --decrypt --password=TOP_SECRET --progress 09868_01.12.2018.pdf out.pdf
Error number 80090345: The requested operation cannot be completed. The computer must be trusted for delegation and the current user account must be configured to allow delegation.
unable to acquire crypt context with new keyset

This will help also with #163 .

And based on error message I found solution:

I added the following value under:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Cryptography\Protect\Providers\df9d8cd0-1501-11d1-8c7a-00c04fc297eb

ProtectionPolicy = 1 (DWORD)

@zdenop Thanks -- yes, I will take this. This is a helpful change. Can you also suggest some text I might add to README-windows.md?

I will incorporate the error into qpdf's regular error reporting mechanism, but your code snippet shows me how to get the details.

I am not sure if to put something to README-windows.md:  when I get error text (error code does not help me - it was too general for searching on internet) - I found fix (windows policy) quickly.

By the way, I think it's really funny that a Windows-specific issue has number 286.

qpdf 9 will include this change, though I have no way of testing it. When this issue is closed, you will know the change is in master. Feel free to test if you'd like. If you're subscribed to the qpdf announce list, watch for an announcement of an alpha or other prerelease of qpdf 9. There are lots of changes, so I want to give people a chance to test before I do a full 9.0.0 release. Thanks again for the patch.

I ran into this problem when running qpdf from a normal command prompt in Windows 10. Using Run as administrator to elevate the command prompt solved the issue.


CPPISetOwner Error

Hello, here's the error:
[ERROR] gamemodes/basewars/gamemode/buymenu.lua:53: attempt to call method 'CPPISetOwner' (a nil value)

func - gamemodes/basewars/gamemode/buymenu.lua:53
unknown - lua/includes/extensions/net.lua:32

This gamemode requires a CPPI compliant prop protection addon to run. My recommendation is to use FPP, but ultimately It's up to you.


Multicore in assigning ACDC of ACDC

looking at
https://cmsweb.cern.ch/reqmgr2/config?name=pdmvserv_task_SMP-PhaseIITDRFall17wmLHEGS-00030__v1_T_171103_204004_8254
https://cmsweb.cern.ch/reqmgr2/config?name=mcremone_ACDC0_task_SMP-PhaseIITDRFall17wmLHEGS-00030__v1_T_171116_004028_825
https://cmsweb.cern.ch/reqmgr2/config?name=vlimant_ACDC1_task_SMP-PhaseIITDRFall17wmLHEGS-00030__v1_T_171122_180054_1020
https://cmsweb.cern.ch/reqmgr2/config?name=mcremone_ACDC2_task_SMP-PhaseIITDRFall17wmLHEGS-00030__v1_T_171213_210739_3479
it looks like ACDC0 maintained the numberOfCores = 8 in the spec, while the other ones switched to numberOfCores = 1
I cannot find any flaw in how things are assigned. From the logs of where the assignement was done, no Multicore parameter was passed on to the assign api, and yet numberOfCores = 1 even for the tasks that specify Multicore=8
somehow @ticoann @amaltaro the base directory value (set to 1 by default) is taken as the value that needs to go everywhere for ACDC of ACDC.
N.B. this is a showstopper for still cursed upgrade requests of highest priority.
FYI @prozober

I allow myself to put urgent ops until I figure out a way to circumvent the problem and be able to make the last acdc for >10 upgrade requests of top priority

Hmm, this should have been fixed by
https://github.com/dmwm/WMCore/pull/8286
Could you point me the script where the actual assignment is done. That will be helpful for debugging the issue.

Actually, now I see what is wrong. I can create the patch but this will require cmsweb update. Meanwhile I maybe find the temporary solution on this. How is ACDC of ACDC is created and assigned?
@prozober, Paola are you using wmstats/reqmgr2 web interface or some script from WmAgentScript?

Hi @ticoann, actually we are creating ACDCs using Unified scripts.
I think @areinsvo could please help us to provide the info we need, that is where the ACDCs are created and assigned.

my 2 cents is that we should not touch actor to patch for this and run second++ ACDC by hand and assign with the multicore option of assign.py until the fix propagates to reqmgr.

I think assign with mutilcore option would be fine. Any minimal changes will be preferred for temporary solution. We will fix the update in next reqmgr release.


The multi-scale setting

Thanks for your wonderful work.
I have a question.
Are all multi-scale results of DW in the paper with 1333x[480:960]  setting?

Yes.  We notice this setting is a little different from several papers and we just follow GFocal loss to use [480:960].

Yes. We notice this setting is a little different from several papers and we just follow GFocal loss to use [480:960].

Thanks for your reply!


Move inverter settings into backend settings

Lets abandon the bootstrap file for the inverter stuff and move this into the backend settings.
Will be tricky to handle the migration but I guess it can be done :P

This seems to be working now but must be tested on a gw in some different scenarios.


Is api_product_list claim required in tokens?

I am getting this logs, this seems to point to no valid products found. Is the api_product_list claim also required in JWT tokens?
2018-04-14T00:11:57.329004Z    info        Resolved api: meme.default.svc.cluster.local, path: /meme/v1/health, scopes: [] => []   {"adapter": "apigee-handler.apigee.istio-system"}
2018-04-14T00:11:57.329052Z    info        Resolved hints: {"adapter": "apigee-handler.apigee.istio-system"}

Yes, it is.

Thanks!


Node Red Basic HTTP Auth not working

When I enable Node Red with httpNodeAuth, the Hubitat MakerAPI webhook fails.  Is there a way to pass the header token from Hubitat? In general, how can we secure the NR Hubitat HTTP inbound endpoint for MakerAPI?  If I expose my NR instance using SSL, then the webhook is exposed inherently.
NR settings.js config below...
httpNodeAuth: {user:"user",pass:"$2a$08$zZWtXTja0fB1pzD4sHCMyOCMYz2Z6dNbM6tl8sJogENOMcxWV9DN."},

Did you try this procedure?`
https://community.hubitat.com/t/node-red-nodes-for-hubitat/34386#how-to-configure-webhook-after-configuring-node-red-authenticationhttpsnoderedorgdocsuser-guideruntimesecuring-node-red-21
I should add it to the README too

I tried this and it's not working.  I also tried in Postman just to see if I could get some info as to whether or not it would connect.
I'm using nginx with SSL on port 443 and forwarding into NR docker image on port 1880.  This setup works fine without NR auth enabled.  Below is the error message in Postman, it thinks I'm sending a port in the URL.
POST<EMAIL_ADDRESS>Error: Port should be >= 0 and < 65536. Received SX99.

Did you try this procedure?`
https://community.hubitat.com/t/node-red-nodes-for-hubitat/34386#how-to-configure-webhook-after-configuring-node-red-authenticationhttpsnoderedorgdocsuser-guideruntimesecuring-node-red-21
I should add it to the README too

I tried this and it's not working. I also tried in Postman just to see if I could get some info as to whether or not it would connect.
I'm using nginx with SSL on port 443 and forwarding into NR docker image on port 1880. This setup works fine without NR auth enabled. Below is the error message in Postman, it thinks I'm sending a port in the URL.
POST<EMAIL_ADDRESS>Error: Port should be >= 0 and < 65536. Received SX99.

UPDATE - I got this working.  Had to URL encode my password, works fine now, thanks!

UPDATE - I got this working. Had to URL encode my password, works fine now, thanks!


Toggle the presence of a class based on a boolean flag

html! {
    ul @for pony in &["Rarity", "Fluttershy", "Pinkie Pie"] {
        li.pink(pony == "Pinkie Pie") { (pony) }
    }
}

It can often be useful to toggle a class based on a boolean flag. Need to be careful with multiple classes.

Multiple classes would just repeat that structure wouldn't they?
html! {
    div.foo(foo).bar(bar) (content)
}

Where foo and bar are boolean values

Yeah I was thinking that as well.
My issue isn't with the syntax, it's with the implementation. To be specific, if you look at the HTML output, whether or not "bar" has a space before it would depend on the presence of "foo".
I guess that's enough of an edge case that we can ignore it -- having extra spaces in a class attribute is still valid after all. But it still feels kind of weird.

Whoops -- turns out this syntax is ambiguous. Given div.foo (bar), it'd be impossible to tell whether we want a foo class toggled by bar, or a <div> which contains the value of bar.
I'm open to other suggestions for syntax.

The obvious alternatives would be [] or {}.
html! {
    div.foo[foo].bar[bar] (content)
}

html! {
    div.foo{foo}.bar{bar} (content)
}

I don't like either much...
Some random ideas:
html! {
    div.foo@(foo).bar@(bar) (content)
}

html! {
    div.foo?(foo).bar?(bar) (content)
}

The last one works as it's . <name> ( <bool> ) which differs due to the dot with content-less attributes.

@TheNeikos Thanks for writing those out!
I'm actually growing fond of the first alternative, using div.foo[foo]. The expressions involved aren't Renderable -- they're boolean flags -- so it makes sense for them to look different. And since the empty attribute syntax will be changed anyway (#59), we don't need to be consistent with how they work now.
I've pushed the corresponding change to empty attributes as 68feeeff1bfa0e569beb8b38e7766cf2d29c4d2e, and I'd love to see how that turns out.


Slider Mousedown Mouseup Pause Play Video Feature

Lets say im watching a video and i wish to slide the rail.
But when im sliding, the player is still playing which is very annoying and downside usability.
So i edited the function and used forcedHandlePause bool to  pause on mousedown ( if player is playing). and then play on mouseup(if forcedHandlePause == true) .
But for some reasons the t.current is not changing position while sliding.(i,e when mouseIsDown= true)

	// handle clicks
		t.rail.on('mousedown touchstart', function (e) {
			forcedHandlePause = false;
			if (media.duration !== Infinity) {
				// only handle left clicks or touch
				if (e.which === 1 || e.which === 0) {
					t.container.find('.' + t.options.classPrefix + 'postroll-layer').hide();

					if(!media.paused){
						t.media.pause();
						forcedHandlePause = true;
					}
					mouseIsDown = true;
					handleMouseMove(e);
					t.globalBind('mousemove.dur touchmove.dur', function (e) {
						handleMouseMove(e);
					});
					t.globalBind('mouseup.dur touchend.dur', function () {
						mouseIsDown = false;
						if (t.timefloat !== undefined) {
							t.timefloat.hide();
						}
						t.globalUnbind('mousemove.dur touchmove.dur mouseup.dur touchend.dur');
					});
				}
			}
		}).on('mouseup', function (e) {
			 if(forcedHandlePause){
				 t.media.play();
				 forcedHandlePause = true;
			 }
		}).on('mouseenter', function (e) {
            //.......
  });

Let me take a look on this. This is a good enhancement since it's related to accessibility. I'll keep you posted

I checked your code, but I couldn't find what was wrong. I'd like to integrate it as a part of the core functionality, but if you can provide me a URL where I can see your issue and provide a solution I'd appreciate it

Here's the link https://jsfiddle.net/6qcvj1en/ with the modified build/player.js
If you look closely.
Whenever you slide the handle by keeping the cursor inside the rail progress bar its working fine.
The issue is : as soon as you hold left mouse button and slide handle and if your cursor moves out  of the progress rail (while youre still holding left mouse ) youll see a stop sign  . and thus current rail dosent moves anymore.
In short you have to keep your cursor inside that small rail (which is very difficult at times) in order to drag.

I tried using Chrome on OSX and I couldn't replicate it. Which OS and browser are you using? I know it is difficult but can you provide me a screen recording of what you are experiencing so I can see what's going on, please?

heres a screen capture https://sendvid.com/ku8xf3tl
im using windows 8.1 chrome

Thanks I'll check it later

Thanks for providing me the screen recording; I tried to replicate your bug on Chrome and FF on Mac OSX (since I don't have a Windows machine) and I wasn't able to do it. This might be more a behavior on Windows.  One thing I did do was to put the forcedHandlePause variable before the mouse events like this:
                var forcedHandlePause = false;

		// handle clicks
		t.rail.on('mousedown touchstart', (e) => {
                          forcedHandlePause = false;
          	          if (media.duration !== Infinity) {

I don't know if that helps or not but let me know so I can integrate this in the core. Thanks

ok so after doing a lil bit of diggin i found that changing the currentTime on handlemousemove was expensive on low ram(4gb) when there were lots of ui along with the videoplayer.
So instead of chaning the currentTime of video on handlemousemove im chaning the currentTime of video on mouseup when  forcedHandlePause is true. and also i modified the updateSlider/updateCurrentTime with a fakeTime variable to show the change.
The performance is improved alot and im no more experiencing the issue.

Yes please anything that is performance or accesibility related is always good to integrate. Create the PR and I'll take a look. Thanks


[Azure] Centralizing Environment Validation for Azure OpenAI LLM & Chat Model

Description
References Issue #3769. Ports over functionality from AzureChatOpenAI to AzureOpenAIMixin to standardize interaction experience with Azure models.
Changes

Added AzureOpenAIMixin
Changed AzureOpenAI to inherit from AzureOpenAIMixin
Changed AzureChatOpenAI to inherit from AzureOpenAIMixin
Added relevant docstrings

Testing

Ran on Jupyter Notebook :/

@hwchase17

can this also be used in embeddings?

Yes, it can. I'll adjust the PR tomorrow with an Azure class for embeddings. Might have to add a slight modification to the Mixin class to pass in the client (previous is openai.ChatCompletion) but embeddings is openai.Embedding.

can this also be used in embeddings?

Yes, it can. I'll adjust the PR tomorrow with an Azure class for embeddings. Might have to add a slight modification to the Mixin class to pass in the client (previous is openai.ChatCompletion) but embeddings is openai.Embedding.

well the LLM class should also not be ChatCompletion but Completion, right?

can this also be used in embeddings?

Yes, it can. I'll adjust the PR tomorrow with an Azure class for embeddings. Might have to add a slight modification to the Mixin class to pass in the client (previous is openai.ChatCompletion) but embeddings is openai.Embedding.

well the LLM class should also not be ChatCompletion but Completion, right?

Yes - good catch. Created a new commit that allocates the correct client per model type along with adding the AzureEmbeddings class.

@philipk19238 Hi , could you, please, resolve the merging issues and address the last comments (if needed)? After that, ping me and I push this PR for the review. Thanks!

This PR no longer lines up with the current directory structure of the library (would need to be in /libs/langchain/langchain instead of /langchain). Would you be interested in updating to the new format, or would it be better to close this and someone can work on a new PR off the current state of the library (using this as a reference implementation)?


tinygo: 0.16.0 -> 0.23.0

Notable changes:

Fix for missing version information in Go 1.17 and later.
Add -isystem to libclang includes.
nixpkgs-fmt TinyGo default.nix file.
Upgrade to LLVM 11.

TinyGo supports LLVM 12 and 13, but may result in weird errors when
building:
error: could not parse executable for stack size analysis: could not decode .debug_frame bytecode op 0x9 (for address 0x8012a4a)
Description of changes

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
22.05 Release Notes (or backporting 21.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

Superseded by https://github.com/NixOS/nixpkgs/pull/157129.


[QST] Interface for host, device, pinned and managed mdspan

What is the right way to expose vector search API that supports all of the following: host, device, pinned, managed arrays?

RAFT ivf_pq::build has a pointer based API that accepts all of these
RAFT cagra::build has a plain mdspan input, which can be any of the above.

In contrast cuVS APIs use explicitly host_mdspan and device_mdspan. This is verbose, and I am not sure if it is compatible with additional mdspan variants (pinned, managed, user defined). Could we use a similar API that RAFT cagra::build has?

Capturing here our offline discussion:
cuVS headers have explicit type parameters. That means we would need to have overloads for each mdspan variant that we support. In practice the host_mdspan and device_mdspan structures are sufficient to capture the intent for how the user is expected to interact with the library. If we need to support something like managed_mdspan, we would need to add it the headers, or just re-wrapped the array as a supported mdspan type.