Issue when calling gp.predictive_gradients()

I am running Python v2.7.12 using Anaconda 64bit on Windows 7. I installed GPy using 'pip install GPy' to get version 1.5.6 which I have been using for a while on small problems without any issues. More, recently I have wanted to make use of the predictive_gradients() function but I have run into some issues.
Firstly, when I import GPy I get the following warning: "warning in stationary: failed to import cython module: falling back to numpy" which I have not thought too much of in the past since everything else seemed to work okay. Secondly, when I try and run the following code,
import numpy as np
import GPy as gp

k = gp.kern.RBF(input_dim=1, variance=1., lengthscale=1.)

X = np.random.uniform(-3.,3., (20,1))
Y = np.sin(X) + np.random.randn(20,1)*0.05

m = gp.models.GPRegression(X, Y, kernel=k)

grad = m.predictive_gradients(X)

I get the following error,
NameError                                 Traceback (most recent call last)
<ipython-input-10-964a0d158030> in <module>()
----> 1 m.predictive_gradients(X)

C:\Program Files\Anaconda2\lib\site-packages\GPy\core\gp.pyc in predictive_gradi
ents(self, Xnew, kern)
    335
    336         for i in range(self.output_dim):
--> 337             mean_jac[:,:,i] = kern.gradients_X(self.posterior.woodbury_v
ector[:,i:i+1].T, Xnew, self._predictive_variable)
    338
    339         # gradients wrt the diagonal part k_{xx}

C:\Program Files\Anaconda2\lib\site-packages\GPy\kern\src\kernel_slice_operation
s.pyc in wrap(self, dL_dK, X, X2)
    116     def wrap(self, dL_dK, X, X2=None):
    117         with _Slice_wrap(self, X, X2) as s:
--> 118             ret = s.handle_return_array(f(self, dL_dK, s.X, s.X2))
    119         return ret
    120     return wrap

C:\Program Files\Anaconda2\lib\site-packages\GPy\kern\src\stationary.pyc in grad
ients_X(self, dL_dK, X, X2)
    234         """
    235         if config.getboolean('cython', 'working'):
--> 236             return self._gradients_X_cython(dL_dK, X, X2)
    237         else:
    238             return self._gradients_X_pure(dL_dK, X, X2)

C:\Program Files\Anaconda2\lib\site-packages\GPy\kern\src\stationary.pyc in _gra
dients_X_cython(self, dL_dK, X, X2)
    321         X, X2 = np.ascontiguousarray(X), np.ascontiguousarray(X2)
    322         grad = np.zeros(X.shape)
--> 323         stationary_cython.grad_X(X.shape[0], X.shape[1], X2.shape[0], X,
 X2, tmp, grad)
    324         return grad/self.lengthscale**2
    325

NameError: global name 'stationary_cython' is not defined

If I use a linear kernel I do not get the above error.
I wondered if this was related to the documented issues on using cython code with anaconda on Windows 64bit due to bundling with mingw64 (see: https://github.com/ContinuumIO/anaconda-issues/issues/175) but I am now using TDM-GCC and the issue persists.
Has this been documented before? Is there anything I have missed that I can try to get this to work?
Many thanks for your help.

Hmm this seems odd, indeed. If the cython module is missing, it should never get into that part of the if statement. As you can see above in the same file, it should write into the config, that cython is not available. as a sanity check could you try putting
[cython]
working = False
in your %userprofile%.config\GPy\user.cfg file and rerun the script?
I am more bamboozled about the fact, that cython is not working for you. We build the cython modules for windows and upload them to pypi, so it should install on windows without problems. Have you installed the cython module using anaconda?

Hi Max,
I actually re-installed Anaconda this morning to sanity check a few things, I don't recall explicitly installing cython so I'm not sure whether it was already there or pip took care of it. Nevertheless, I currently have cython v0.25.1 installed according to conda.
With the above flag in the user.cfg file I get the same errors,
warning in stationary: failed to import cython module: falling back to numpy
Traceback (most recent call last):
  File "H:\Projects\test\test\test.py", line 11, in <module>
    grad = m.predictive_gradients(X)
  File "C:\Program Files\Anaconda2\lib\site-packages\GPy\core\gp.py", line 337,
in predictive_gradients
    mean_jac[:,:,i] = kern.gradients_X(self.posterior.woodbury_vector[:,i:i+1].T
, Xnew, self._predictive_variable)
  File "C:\Program Files\Anaconda2\lib\site-packages\GPy\kern\src\kernel_slice_o
perations.py", line 118, in wrap
    ret = s.handle_return_array(f(self, dL_dK, s.X, s.X2))
  File "C:\Program Files\Anaconda2\lib\site-packages\GPy\kern\src\stationary.py"
, line 236, in gradients_X
    return self._gradients_X_cython(dL_dK, X, X2)
  File "C:\Program Files\Anaconda2\lib\site-packages\GPy\kern\src\stationary.py"
, line 323, in _gradients_X_cython
    stationary_cython.grad_X(X.shape[0], X.shape[1], X2.shape[0], X, X2, tmp, gr
ad)
NameError: global name 'stationary_cython' is not defined

Does the following report False?
from GPy.util.config import config
print(config.getboolean('cython', 'working'))

It reports True.

can you import
from GPy.kern.src import stationary_cython

?

No, I get the following error:
ImportError: DLL load failed: The application has failed to start because its si
de-by-side configuration is incorrect. Please see the application event log or u
se the command-line sxstrace.exe tool for more detail.

So the config reports True, although you have given cython working as false? This is getting more odd by the minute. Are you able to import cython?
On another note, the gradient computation for the ARD parameters makes use of cython, too. What happens if you use an ARD kernel:
k = gp.kern.RBF(input_dim=1, variance=1., lengthscale=1., ARD=True)

and run a normal GP as you described above, without the predictive gradients?

Hmmm, I can import cython with no errors. To double check I ran the cython hello world example here. When I first ran this tutorial there was an "Unable to find vcvarsall.bat" error which I have now resolved.
Without the predictive gradients, i.e., running the following code:
import numpy as np
import GPy as gp

k = gp.kern.RBF(input_dim=1, variance=1., lengthscale=1.)

X = np.random.uniform(-3.,3., (20,1))
Y = np.sin(X) + np.random.randn(20,1)*0.05

m = gp.models.GPRegression(X, Y, kernel=k)
print(m.predict(X))

I get the following output:
warning in stationary: failed to import cython module: falling back to numpy
(array([[-0.59422249],
       [ 0.37418019],
       [-0.55712291],
       [-0.23910571],
       [ 0.1361988 ]]), array([[ 1.34195044],
       [ 1.48095964],
       [ 1.35368817],
       [ 1.46978674],
       [ 1.49680015]]))

If I usethe RBF kernel with ARD=True, then the code fails again with a similar output to before:
warning in stationary: failed to import cython module: falling back to numpy
Traceback (most recent call last):
  File "test.py", line 9, in <module>
    m = gp.models.GPRegression(X, Y, kernel=k)
  File "C:\Program Files\Anaconda2\lib\site-packages\paramz\parameterized.py", l
ine 54, in __call__
    self.initialize_parameter()
  File "C:\Program Files\Anaconda2\lib\site-packages\paramz\core\parameter_core.
py", line 331, in initialize_parameter
    self.trigger_update()
  File "C:\Program Files\Anaconda2\lib\site-packages\paramz\core\updateable.py",
 line 79, in trigger_update
    self._trigger_params_changed(trigger_parent)
  File "C:\Program Files\Anaconda2\lib\site-packages\paramz\core\parameter_core.
py", line 128, in _trigger_params_changed
    self.notify_observers(None, None if trigger_parent else -np.inf)
  File "C:\Program Files\Anaconda2\lib\site-packages\paramz\core\observable.py",
 line 91, in notify_observers
    [callble(self, which=which) for _, _, callble in self.observers]
  File "C:\Program Files\Anaconda2\lib\site-packages\paramz\core\parameter_core.
py", line 498, in _parameters_changed_notification
    self.parameters_changed()
  File "C:\Program Files\Anaconda2\lib\site-packages\GPy\core\gp.py", line 195,
in parameters_changed
    self.kern.update_gradients_full(self.grad_dict['dL_dK'], self.X)
  File "C:\Program Files\Anaconda2\lib\site-packages\GPy\kern\src\kernel_slice_o
perations.py", line 102, in wrap
    ret = f(self, dL_dK, s.X, s.X2)
  File "C:\Program Files\Anaconda2\lib\site-packages\GPy\kern\src\rbf.py", line
107, in update_gradients_full
    super(RBF,self).update_gradients_full(dL_dK, X, X2)
  File "C:\Program Files\Anaconda2\lib\site-packages\GPy\kern\src\kernel_slice_o
perations.py", line 102, in wrap
    ret = f(self, dL_dK, s.X, s.X2)
  File "C:\Program Files\Anaconda2\lib\site-packages\GPy\kern\src\stationary.py"
, line 193, in update_gradients_full
    self.lengthscale.gradient = self._lengthscale_grads_cython(tmp, X, X2)
  File "C:\Program Files\Anaconda2\lib\site-packages\GPy\kern\src\stationary.py"
, line 228, in _lengthscale_grads_cython
    stationary_cython.lengthscale_grads(N, M, Q, tmp, X, X2, grads)
NameError: global name 'stationary_cython' is not defined

So you said you are using your own compiler. Can you download the source and run the setup code from the source directory with your compiler directly? That should compile the resources directly and make everything work.
You might have to look up the compilation options for the setup script (i only know the two options --compiler=msvc or mingw, not sure about tdm).
I think we have brought it down to the compilation, though. It might be windows 7, which is the most likely to me, but the compiled version seems to not work for you :(

Hmmm,  I've tried to install GPy from source. I uninstalled GPy using pip, then downloaded the source and ran 'python setup.py install' from the source directory with distutils.cfg pointing to msvc (I have that installed as well) and still ended up with the same errors when running the above scripts. Am I missing anything here?
Thanks for your help so far.

It seems the compilation is not working. You should be able to still use the numpy code by providing cython working = false in the config. Unfortunately I do not have Windows and cannot go deeper, I am afraid :/

On 22 Nov 2016, at 17:22, JM<EMAIL_ADDRESS>wrote:
Hmmm, I've tried to install GPy from source. I uninstalled GPy using pip, then downloaded the source and ran 'python setup.py install' from the source directory with distutils.cfg pointing to msvc (I have that installed as well) and still ended up with the same errors when running the above scripts. Am I missing anything here?
Thanks for your help so far.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

I provided cython working = False but that didn't help. This morning I uninstalled 64bit Anaconda and installed 32bit Anaconda and everything now works fine. It must be a 64bit compiler issue as I mentioned in the first post. I am not sure whether I am being naive or whether there is a bug with respect to the cython working = False as that seems to not have the desired effect on my machine.  Either way, I now have a work around and at least this issue Anaconda 64bit on windows is documented here in case anyone else comes across it. Thanks for your time and help with this.

On the contrary, thanks for reporting and using GPy!

On 23 Nov 2016, at 09:54, JM<EMAIL_ADDRESS>wrote:
I provided cython working = False but that didn't help. This morning I uninstalled 64bit Anaconda and installed 32bit Anaconda and everything now works fine. It must be a 64bit compiler issue as I mentioned in the first post. I am not sure whether I am being naive or whether there is a bug with respect to the cython working = False as that seems to not have the desired effect on my machine. Either way, I now have a work around and at least this issue Anaconda 64bit on windows is documented here in case anyone else comes across it. Thanks for your time and help with this.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

I'm still getting the same error. Also, I don't want to change everything to use 32-bit version.
@blaahhrrgg The reason why cython is still True is because during import process,
config.set('cython', 'working', 'True') is done in linalg.py

Hi!
I am running a python code (containing GPy and GPyOpt) from Matlab on Windows and I am facing with the following error:
warning in stationary: failed to import cython module: falling back to numpy
Error using stationary>_gradients_X_cython (line 323)
Python Error: NameError: global name 'stationary_cython' is not defined
When I run the code in python I do not have any problem, but the problem comes when I call the script from MATLAB (I have run the code from MATLAB a few months ago without any problem.)
I have to mention, recently for some reason, I have downgraded the numpy to Numpy=1.11.0.
The reason was Matlab has a bug with the latest version of numpy.
Could you please help me to resolve the issue?

I am facing the following issue. I am using python 2.7 and anaconda 32bit.
NameErrorTraceback (most recent call last)
 in ()
----> 1 myProblem = GPyOpt.methods.BayesianOptimization(myf,bounds)
C:\ProgramData\Anaconda2\lib\site-packages\GPyOpt\methods\bayesian_optimization.pyc in init(self, f, domain, constrains, cost_withGradients, model_type, X, Y, initial_design_numdata, initial_design_type, acquisition_type, normalize_Y, exact_feval, acquisition_optimizer_type, model_update_interval, evaluator_type, batch_size, num_cores, verbosity, verbosity_model, bounds, maximize, **kwargs)
242
243         # --- Initialize everything
--> 244         self.run_optimization(max_iter=0,verbosity=self.verbosity)
245
246     def _model_chooser(self):
C:\ProgramData\Anaconda2\lib\site-packages\GPyOpt\methods\bayesian_optimization.pyc in run_optimization(self, max_iter, max_time, eps, verbosity, save_models_parameters, report_file, evaluations_file, models_file, **kwargs)
456                 self.acquisition_optimizer.optimizer ='CMA'
457             print('WARNING: "acqu_optimize_method" will be deprecated in the next version!')
--> 458         super(BayesianOptimization, self).run_optimization(max_iter = max_iter, max_time = max_time,  eps = eps, verbosity=verbosity, save_models_parameters = save_models_parameters, report_file = report_file, evaluations_file= evaluations_file, models_file=models_file)
459
460     def _sign(self,f):
C:\ProgramData\Anaconda2\lib\site-packages\GPyOpt\core\bo.pyc in run_optimization(self, max_iter, max_time, eps, verbosity, save_models_parameters, report_file, evaluations_file, models_file)
106
107             # --- Update and optimize acquisition and compute the exploration level in the next evaluation
--> 108             self.suggested_sample = self._compute_next_evaluations()
109
110             if not ((self.num_acquisitions < self.max_iter) and (self._distance_last_evaluations() > self.eps)):
C:\ProgramData\Anaconda2\lib\site-packages\GPyOpt\core\bo.pyc in _compute_next_evaluations(self)
184         Computes the location of the new evaluation (optimizes the acquisition in the standard case).
185         """
--> 186         return self.evaluator.compute_batch()
187
188     def _update_model(self):
C:\ProgramData\Anaconda2\lib\site-packages\GPyOpt\core\evaluators\sequential.pyc in compute_batch(self)
19         Selects the new location to evaluate the objective.
20         """
---> 21         return self.acquisition.optimize()
22
23
C:\ProgramData\Anaconda2\lib\site-packages\GPyOpt\acquisitions\base.pyc in optimize(self)
57             out = self.optimizer.optimize(f=self.acquisition_function)[0]
58         else:
---> 59             out = self.optimizer.optimize(f=self.acquisition_function, f_df=self.acquisition_function_withGradients)[0]
60         return out
61
C:\ProgramData\Anaconda2\lib\site-packages\GPyOpt\optimization\acquisition_optimizer.pyc in optimize(self, f, df, f_df)
175             if self.search:
176                 if self.f_df == None: fp_dfp = None  # -- In case no gradients are available
--> 177                 x_min, f_min = self.optimizer.optimize(x0, f =fp, df=None, f_df=fp_dfp)
178                 return self._expand_vector(x_min), f_min
179             else:
C:\ProgramData\Anaconda2\lib\site-packages\GPyOpt\optimization\optimizer.pyc in optimize(self, x0, f, df, f_df)
62             res = scipy.optimize.fmin_l_bfgs_b(f, x0=x0, bounds=self.space.get_bounds(),approx_grad=True, maxiter=self.maxiter)
63         else:
---> 64             res = scipy.optimize.fmin_l_bfgs_b(_f_df, x0=x0, bounds=self.space.get_bounds(), maxiter=self.maxiter)
65         return np.atleast_2d(res[0]),np.atleast_2d(res[1])
66
C:\ProgramData\Anaconda2\lib\site-packages\scipy\optimize\lbfgsb.pyc in fmin_l_bfgs_b(func, x0, fprime, args, approx_grad, bounds, m, factr, pgtol, epsilon, iprint, maxfun, maxiter, disp, callback, maxls)
191
192     res = _minimize_lbfgsb(fun, x0, args=args, jac=jac, bounds=bounds,
--> 193                            **opts)
194     d = {'grad': res['jac'],
195          'task': res['message'],
C:\ProgramData\Anaconda2\lib\site-packages\scipy\optimize\lbfgsb.pyc in _minimize_lbfgsb(fun, x0, args, jac, bounds, disp, maxcor, ftol, gtol, eps, maxfun, maxiter, iprint, callback, maxls, **unknown_options)
326             # until the completion of the current minimization iteration.
327             # Overwrite f and g:
--> 328             f, g = func_and_grad(x)
329         elif task_str.startswith(b'NEW_X'):
330             # new iteration
C:\ProgramData\Anaconda2\lib\site-packages\scipy\optimize\lbfgsb.pyc in func_and_grad(x)
276     else:
277         def func_and_grad(x):
--> 278             f = fun(x, *args)
279             g = jac(x, *args)
280             return f, g
C:\ProgramData\Anaconda2\lib\site-packages\scipy\optimize\optimize.pyc in function_wrapper(*wrapper_args)
290     def function_wrapper(wrapper_args):
291         ncalls[0] += 1
--> 292         return function((wrapper_args + args))
293
294     return ncalls, function_wrapper
C:\ProgramData\Anaconda2\lib\site-packages\scipy\optimize\optimize.pyc in call(self, x, *args)
61     def call(self, x, *args):
62         self.x = numpy.asarray(x).copy()
---> 63         fg = self.fun(x, *args)
64         self.jac = fg[1]
65         return fg[0]
C:\ProgramData\Anaconda2\lib\site-packages\GPyOpt\optimization\optimizer.pyc in _f_df(x)
58         if f_df is not None:
59             def _f_df(x):
---> 60                 return f(x), f_df(x)[1][0]
61         if f_df is None and df is None:
62             res = scipy.optimize.fmin_l_bfgs_b(f, x0=x0, bounds=self.space.get_bounds(),approx_grad=True, maxiter=self.maxiter)
C:\ProgramData\Anaconda2\lib\site-packages\GPyOpt\optimization\acquisition_optimizer.pyc in fp_dfp(x)
163             xx = self._expand_vector(x)
164
--> 165             fp_xx , dfp_xx = f_df(xx)
166             dfp_xx = dfp_xx[:,np.array(self.free_dims)]
167             return fp_xx, dfp_xx
C:\ProgramData\Anaconda2\lib\site-packages\GPyOpt\acquisitions\base.pyc in acquisition_function_withGradients(self, x)
44         Takes an acquisition and it gradient and weights it so the domain and cost are taken into account.
45         """
---> 46         f_acqu,df_acqu = self._compute_acq_withGradients(x)
47         cost_x, cost_grad_x = self.cost_withGradients(x)
48         f_acq_cost = f_acqu/cost_x
C:\ProgramData\Anaconda2\lib\site-packages\GPyOpt\acquisitions\EI.pyc in _compute_acq_withGradients(self, x)
45         """
46         fmin = self.model.get_fmin()
---> 47         m, s, dmdx, dsdx = self.model.predict_withGradients(x)
48         phi, Phi, _ = get_quantiles(self.jitter, fmin, m, s)
49         f_acqu = (fmin - m + self.jitter) * Phi + s * phi
C:\ProgramData\Anaconda2\lib\site-packages\GPyOpt\models\gpmodel.pyc in predict_withGradients(self, X)
111         m, v = self.model.predict(X)
112         v = np.clip(v, 1e-10, np.inf)
--> 113         dmdx, dvdx = self.model.predictive_gradients(X)
114         dmdx = dmdx[:,:,0]
115         dsdx = dvdx / (2*np.sqrt(v))
C:\ProgramData\Anaconda2\lib\site-packages\GPy\core\gp.pyc in predictive_gradients(self, Xnew, kern)
335
336         for i in range(self.output_dim):
--> 337             mean_jac[:,:,i] = kern.gradients_X(self.posterior.woodbury_vector[:,i:i+1].T, Xnew, self.predictive_variable)
338
339         # gradients wrt the diagonal part k{xx}
C:\ProgramData\Anaconda2\lib\site-packages\GPy\kern\src\kernel_slice_operations.pyc in wrap(self, dL_dK, X, X2)
116     def wrap(self, dL_dK, X, X2=None):
117         with _Slice_wrap(self, X, X2) as s:
--> 118             ret = s.handle_return_array(f(self, dL_dK, s.X, s.X2))
119         return ret
120     return wrap
C:\ProgramData\Anaconda2\lib\site-packages\GPy\kern\src\stationary.pyc in gradients_X(self, dL_dK, X, X2)
234         """
235         if config.getboolean('cython', 'working'):
--> 236             return self._gradients_X_cython(dL_dK, X, X2)
237         else:
238             return self._gradients_X_pure(dL_dK, X, X2)
C:\ProgramData\Anaconda2\lib\site-packages\GPy\kern\src\stationary.pyc in _gradients_X_cython(self, dL_dK, X, X2)
321         X, X2 = np.ascontiguousarray(X), np.ascontiguousarray(X2)
322         grad = np.zeros(X.shape)
--> 323         stationary_cython.grad_X(X.shape[0], X.shape[1], X2.shape[0], X, X2, tmp, grad)
324         return grad/self.lengthscale**2
325
NameError: global name 'stationary_cython' is not defined

I was having a similar issue, which seems to be caused by not having gcc and gmp installed in Conda.
I was able to use:
conda install -y gcc gmp
To install the necessary libraries, and then I ran:
python setup --verbose build_ext
making sure that the cython modules were being rebuild.

I solved this by install the Microsoft Visual C++ redistributable 2008 SP1.  My guess is that the cythonized files in the GPy package were compiled with this version of MSVC.  This is kind of an old version, and newer Windows machines (e.g. fresh install of Windows 10) do not come with all of the redistributables for older MSVC++.


feat: add a new prop(adjustable) so that the drawer can adjust its width/height

Can you add a demo?

Can you add a demo?

ok, i will.


Float values for chromosome start and end

Is there a way to have float values for the chromosome start and end positions? E.g.:
chr,start,end
1,100.75,150.25
1,500.00,1000.987654321

No, but I have a version of the ncls that supports floats: https://github.com/biocore-ntnu/ncls

I think I can work with the code you have in examples. Thank you!

Feel free to open an issue in the NCLS repo if you have any questions :)

Feel free to open an issue in the NCLS repo if you have any questions :)


Skipping compressed file as an option?

Hi,
I understand why you skipped compressing jpgs and other files but the advantage of pre-compress is so your web server doesn't have to. If jpgs and other images aren't gzipped then nginx or your web server will be gzipping them on the fly which is the action we're trying to prevent.
Maybe that file type skip list could be made a configuration option which defaults to skipping nothing? This way it will pre-compress everything but if you want to omit files as an end user you can.

I'm certainly open to making it configurable. 😄
Most Nginx configurations I've seen use an include-list of mime-types to compress and, regarding images, usually only include image/svg+xml (or skip image/* entirely). It's usually similar for other webservers that I've seen, but I certainly haven't done an exhaustive survey or anything.
How do you usually configure Nginx's gzip module?

My last nginx config is shorter (fewer font types as I dropped eot,ttf,otf years ago), but about the same idea (and probably nearly as old!)
I don't use a lot of jpg's, but a limited test here of larger, photo-based .jpg's saves 3% with brotli & gzip, and gains(!) 1% with zstd. Would need more testing to see how that works on smaller files though.
Several woff's are ±1%. woff2's are all break-even to slight increases. (Both of these are compressed internally, so I'd expect that.)
You should be able to test compressing everything by adding Propshaft::Compressor::SKIP_EXTS.clear to application.rb (just as a test; we can still explore a proper config option if appropriate).

Thanks, do you want to close this?
I'll leave it up to you because I think you provided a few ways to either making this a non-issue or override that array (yay for Ruby for making this so easy).


Reorganize files to src folder.

This PR puts all circuits files into src folder for better project organization.

Reorganize files to src folder.


simpleJson endpoint is not presented by API (or at least is difficult to find!)

Summary
The simpleJson output described in the docs looks to be a useful output in simple use-cases. However, it is not implemented in the python client, and doesn't appear to be implemented in the API (or is at least it beats me how to find it!).
I can't see any other issues or PRs that reference simpleJson; apologies if this is a duplicate (or straight-up wrong!)
Steps To Reproduce
This seems to me to be regardless of what config file and input file is used, so the Jupyter notebook example with defaultConfig.json is sufficient to demonstrate.
Steps to reproduce the behaviour:

Run the Jupyter notebook example as here.
Send a document, e.g. the sample document provided, with defaultConfig.json or any config with simpleJson output requested and check other outputs are as expected.
Call e.g. parsr.get_status() and note that even if requested, the endpoint for simpleJson output is not listed in the response (example request hash):

{'request_id': '01cecaa5bd8624aa9d8a1d899dde73',
 'server_response': {'id': '01cecaa5bd8624aa9d8a1d899dde73',
  'json': '/api/v1/json/01cecaa5bd8624aa9d8a1d899dde73',
  'csv': '/api/v1/csv/01cecaa5bd8624aa9d8a1d899dde73',
  'text': '/api/v1/text/01cecaa5bd8624aa9d8a1d899dde73',
  'markdown': '/api/v1/markdown/01cecaa5bd8624aa9d8a1d899dde73'}}

Visit other endpoints with e.g. (substituting request hash)

http://localhost:3001/api/v1/json/01cecaa5bd8624aa9d8a1d899dde73

Find no such endpoint for /api/v1/simplejson or /api/v1/simple-json, api/v1/simple/json etc.

See in the API logs that the simpleJson output was in fact produced, e.g.

[2020-10-08T15:43:10] INFO  (parsr-api/7 on 9da727d13778): Exporting markdown...
[2020-10-08T15:43:10] INFO  (parsr-api/7 on 9da727d13778): Writing file: /opt/app-root/src/api/server/dist/output/sampleFile-01cecaa5bd8624aa9d8a1d899dde73/sampleFile.simple.json
[2020-10-08T15:43:11] INFO  (parsr-api/7 on 9da727d13778): Writing file: /opt/app-root/src/api/server/dist/output/sampleFile-01cecaa5bd8624aa9d8a1d899dde73/sampleFile.txt
[2020-10-08T15:43:11] INFO  (parsr-api/7 on 9da727d13778): Exporting markdown...
[2020-10-08T15:43:11] INFO  (parsr-api/7 on 9da727d13778): Writing file: /opt/app-root/src/api/server/dist/output/sampleFile-01cecaa5bd8624aa9d8a1d899dde73/sampleFile.md

but fails to be presented as a discoverable endpoint.
Expected behavior
An endpoint should be available for the useful simpleJson output. It'd also be nice to add a get_simpleJson() method to ParsrClient or add parsr.get_json([simple: bool=False]) option to the existing full JSON method.
Actual behavior
No endpoint found at some sensible addresses; no way that was apparent to me of accessing the simpleJSON output. Correspondingly no way to access simpleJSON from the py client.
Environment

Reference commit/version: Parsr==v1.2.0, parsr-client==3.0.0
Other platform details: Docker Engine - Community v19.03.5
Hardware: Mac air 2015
OS: MacOS High Sierra

Additional context
I would be happy to put together the py client implementation as a PR if you think it is a useful feature, but my knowledge of the API in js is absolutely minimal so I'm not sure I can fix this part. For the same reason sorry if this is a daft issue and I'm missing something obvious!

@ben-lavelle Did you end up extracting the file from the location (container/host) or did you find a way to hook an endpoint that serves the simple JSON?

Hi @kleag. I built using the docker-compose-build file and the API worked but some staged changes were not part of the above commit and are required. Here they are https://github.com/axa-group/Parsr/pull/593. My apologies.

Don't apologize @agarwal-nitesh . In the contrary, thanks a lot for your contribution and help! With your clone, I was able to build and use the docker composed images.
I had to copy ./docker/parsr-base/policy.xml to the root of the project and to create a user with UID 1001 on my system (my user is 1000) to make it work.
I also had to add "simpleJson":true in the config.json file. It would be more coherent if it would be simple-json as in the endpoint.
The status message should also contain an entry for the simple-json result.
Maybe the two above points should be new issues in fact.

The endpoint has been added by @agarwal-nitesh on PR #593


encrypted storage wrapper for web application

Hello world :-)
I am using RNES for the first time. I know that secure storage is only available on Android and iOS but I am also testing with expo start:web and here it would be really nice to have it working with underlying wrapper directly in react-native-encrypted-storage module. It may leverage browser Password Manager (i.e. [1]) or just simple plaintext store of some sort so it works out of the box :-)
I saw test mocker by @emeraldsanto at [2] but it works only for testing. Manual detection of runtime environment ans switching between AsyncStorage and ReactNativeEncryptedStorage is possible but it would be really nice to have it directly here in this module :-)
[1] https://support.mozilla.org/en-US/kb/password-manager-remember-delete-edit-logins
[2] https://github.com/emeraldsanto/react-native-encrypted-storage/blob/master/src/mocks/react-native.js

That would be fantastic.

I'm developing ios and web app. For storing credentials using encrypted storage, and in web I'm using local storage for now.
But at the time of compiling web app throws error.
rnw_blogpost.bundle.js:5540 Uncaught Error: RNEncryptedStorage is undefined     at eval (EncryptedStorage.js:1:862)     at Object../node_modules/react-native-encrypted-storage/lib/module/EncryptedStorage.js (rnw_blogpost.bundle.js:1766:1)     at __webpack_require__ (rnw_blogpost.bundle.js:5537:33)     at fn (rnw_blogpost.bundle.js:5736:21)     at eval (index.js:1:389)     at Object../node_modules/react-native-encrypted-storage/lib/module/index.js (rnw_blogpost.bundle.js:1776:1)     at __webpack_require__ (rnw_blogpost.bundle.js:5537:33)     at fn (rnw_blogpost.bundle.js:5736:21)     at eval (encrypted-storage.ts:1:565)     at Object../src/util/encrypted-storage.ts (rnw_blogpost.bundle.js:4098:1)
This is due to https://github.com/emeraldsanto/react-native-encrypted-storage/blob/master/src/EncryptedStorage.ts#L7

Is there any way, that in web I can handle it.

I think this is possible with a fairly low amount of user code until we are able to support this natively. See #69.


the rationals as an ordered field

The good news: we have the rationals and a discrete linear ordered field.
The bad news: the migrate command at the end of the file rat/order.lean fails on my computer, so I commented it out. The error messages is: "excessive memory consumption detected at 'whnf' (potential solution: increase memory consumption thresold)". I don't know how to increase the memory threshold.
(BTW: there is a typo, "thresold" -> "threshold".)
As far as the rat formalization, there is one more thing that needs to be done: write reduce (a : prerat), which reduces a fraction to lowest terms. There are two uses for this:
(1) For more efficient computation, we can reduce fractions to lowest terms after computing them.
(2) We can define numerator (a : rat) : int and denominator (a : rat) : int such that
a = numerator a / denominator a.
For this, I need to do some more work on int.div.

The bad news: the migrate command at the end of the file rat/order.lean fails on my computer, so I commented it out. The error messages is: "excessive memory consumption detected at 'whnf' (potential solution: increase memory consumption thresold)". I don't know how to increase the memory threshold.

The problem is that migrate tries to unfold definitions such as algebra.abs and algebra.sign in the migrated theorems. The unfolded versions are huge.
I improved the migrate.trace option. Now, it also covers "out-of-memory" situations.
set_option migrate.trace true

I fixed the issue by using:
  definition abs (n : rat) : rat := algebra.abs n
  definition sign (n : rat) : rat := algebra.sign n
  migrate from algebra with rat replacing abs → abs, sign → sign

It is possible to perform this kind of replacement automatically.
To be able to do that, we need to migrate algebra.abs before we migrate any theorem that uses algebra.abs. The declarations are being migrated in an arbitrary order. Moreover, the command should perform the replacement only if the user did not specify one.


OLS-211: Removed LLMLoader param override for OpenAI

Description
We are not able to send question to OpenAI model. This is due to an invalid parameter for ChatOpenAI has been passed. We are not updating any parameters for OpenAI with this fix. We have to see if there is a need to do override and then make a better way to ensure that the passed parameters are valid for specific model.
Type of change

[ ] Refactor
[ ] New feature
[x] Bug fix
[ ] CVE fix
[ ] Optimization
[ ] Documentation Update
[ ] Configuration Update
[ ] Bump-up dependent library

Related Tickets & Documents

Related Issue OLS-211

Checklist before requesting a review

[x] I have performed a self-review of my code.
[x] PR has passed all pre-merge test jobs.
[ ] If it is a core feature, I have added thorough tests.

Testing

Please provide detailed steps to perform tests related to this code change.
How were the fix/results from this change verified? Please provide relevant screenshots or results.

@xrajesh perfect pragmatic solution with followup, thanks!
One ask: can you create a jira story for us to follow up on how we allow/enable overriding of provider/model specific parameters and filtering out (or reporting an error for) invalid ones?
/lgtm
/approve


CSS Changes for Navigation Bar and Home

Update the user link (Top Right hand)
Add Message on Home page (before login)
Customise CSS for login modal and logout modal.

CSS for login modal and logout modal not done. Also - explore authZero for login.


Create sign up page

Issue: 8
Creates a signup page
-Added files index.html and styles.css
-Added image in the assets folder

@bhumi-cka please check the guidelines of project, you need to do the project in React.js not in html
if you can change it do it by 19th midnight


Testing framework

@mitni455 It will be great to share the PR on what you can contribute, instead of sharing emails.

mate, sorry to offend...


Error in creating new bot in bot framework composer

I am getting this error
Error calling npm to fetch template. Please ensure that node and npm are installed and available on your system. Full error message: Command failed with exit code 1: npm install -g --prefix /Users/nikhil/BotFrameworkComposer/.yo-repository --loglevel error --no-optional<EMAIL_ADDRESS>I have nodejs yarn and .net sdk installed still getting this error

Hi @testrepro,
I am unable to reproduce your issue with either a C# or Node template on Composer 2.1.2.
Perhaps you could try manually running the failing command and check what error message it may provide?

I also gets the same issue, Node version v18.16.0 and npm 9.5.1. Bot Framework composer version v2.12

@bk2012,
I am running the same versions and do not face this issue. Are you behind a restrictive proxy or firewall? Are you able to reach npm to download packages?

@anishprasad01
How do i check if npm is able to reach to download packages, i am running bot framework composer on macbook with m1 chip and I am getting problem on mac only, in windows it works perfectly

@testrepro,
Ah, that would be it. Composer does not support M1/M2 chips.


Missing About section.

OS: Linux
Issue Descriptions: Finding the either the homepage or github repo is harder that it needs to. The app name is generic enough that you get hits form all over the internet, from the google play store, to tutorials and comparisons of online services. About section is either missing or very hard to find.
Steps to Reproduce: Try to find an About Homepage.

@ccolorado I'm not sure what you are asking for here, or what you think the issue is,  If you google search

"google music desktop" the website is the 6th result
"google play music desktop" the website is the 3rd result
"google play music desktop player" the website is the 3rd result
"playmusic desktop" the website is the 3rd result

There is only so much SEO I can do, and in most cases I won't be able to beat Googles own websites in their search results... ???

Yeah I guess I could have been a little more descriptive.
Most desktop applications have "About *" Sections on their menu.
I didn't mean't to go to try to rate higher on google. Just a local way to reference the application's landing page.
Here is the skype section as an example.

Oh, you can access our about menu from the tray icon. :+1:


OpenTelemetry Tracer: only export OTLP trace span when sampled

Commit Message: Export OTLP Trace span only when sampled
Additional Description: Envoy's health checks rely on setting the sampled attributed to false for the span for the health check. This should prevent the spurious spans from being created, as we generally don't want to trace every time we health check a cluster.  Before this change, this Tracer just exported them, and it now will check the sampled flag before exporting each span. See https://github.com/envoyproxy/envoy/pull/21893#issuecomment-1167450850 for more context.
Risk Level: Low
Testing: Added a unit test
Part of https://github.com/envoyproxy/envoy/issues/9958

/assign @wbpcode


Mobile Hamburger configuration

Why
The mobile hamburger is currently in an unexpected location (bottom-left)
This makes it hard to find, and can also mean it is not clear when evidence is in "mobile" mode.
What is best practice?
When you look at most modern websites and apps, the mobile hamburger menu is either in the top left (eg Github - this page, https://www.adidas.ca/), or top right (eg https://getbootstrap.com/, https://www.nike.com).
Potential solution

This PR includes a top right positioning.

This is a pretty easy implementation as it doesn't block the current nav items.
This can be done with the slide-in as is, from the left, or altering the slide-in to be from the right. See gifs below.
Personally prefer the slide from the left as before, but keen to hear other views

From Left                                                       From Right

Other thoughts
Most implementations have a full top navbar which includes a hamburger, rather than a floating hamburger button. This might be a better option, but does reduce screen space?

@mcrascal this is related to your comment re the Stackblitz setup. I think if we had something more like this then we could remove the "you are in mobile mode" text.

After a bit of css shuffling, the menu now visually appears on top of the top breadcrumbs / header bar.
However - should clean this up in the future to actually put the hamburger inside the header component.


X509 Mutual TLS authentication

FEATURE REQUEST

Is there an open issue addressing this request? If it does, please add a "+1" reaction to the
existing issue, otherwise proceed to step 2.
No.

Describe the feature you are requesting, as well as the possible use case(s) for it.
Support for X509 mTLS. Reference #367

Indicate the importance of this feature to you (must-have, should-have, nice-to-have).
Must have.

Currently, we are doing PoC with OpenResty a well known & production ready platform which is built on top of Nginx server. Projects like Kong are built on top of OpenResty with the power of Lua.
Proxy
OpenResty is capable to do mTLS, to be precise Nginx is capable to do mTLS so with power of Lua and OpenResty API after successful mutual authentication, we can extract the client ID from the client X509 certificate and pass it to the adapters, who are then capable to proceed with AuthZ flow.
This way we completly offload mutual authentication to OpenResty which servers as our API Gateway.
PKI
Hashicorp Vault is serving as our PKI with PKI secret engine enabled.
Things to solve
Revokation is done on Vault side and Vault supports CRL's which can be fetched using HTTP API or Vault CLI.
Nginx supports usage of CRL with ssl_crl directive which reads static .pem file that you point to.
The problem is in sync between Vault and Nginx CRL. Every time when we revoke cert on Vault somehow we need to update revoked.pem on Nginx (which is the list of revoked ID's)  with this new revoked cert ID in order to stay in sync. Otherwise revoked cert will still be valid and things will be able to do successful mTLS.
Hint: Regular CRL approach is not used anymore, OCSP is prefered way, beside this Vault does not support it just CRL's approach.
Currently I am checking consule-template as possible solution for automation.

For the reference for improvement of nginx.conf: https://www.nginx.com/blog/pci-dss-best-practices-with-nginx-plus/


[FLOC-2488] Increase the timeout on test_move_container_with_dataset

Fixes https://clusterhq.atlassian.net/browse/FLOC-2488

Thanks @wallrj following your notes on https://clusterhq.atlassian.net/browse/FLOC-2488?focusedCommentId=18745&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-18745 this new timeout length seems to be sufficient. The same build Ubuntu 14.04 AWS is still failing the similar moving data acceptance test, though the API test is now passing. The other test is not strictly related to this issue and flocker.acceptance.test_moving_data.MovingDataTests.test_moving_data uses flocker-deploy at the moment (which in turn uses the "override configuration" _compose API endpoint) - but maybe you need to increase the timeout on that too in order for buildbot to be happy with this branch, in which case maybe file another issue and this branch can fix both.
Otherwise, good to merge.

The test_create_container_with_dataset and test_move_container_with_dataset tests are now failing with timeout on vagrant + zfs builder.
So I think I'll double the timeout and apply it to the entire TestCase.
[ERROR]
Traceback (most recent call last):
Failure: twisted.internet.defer.TimeoutError: <flocker.acceptance.test_api.ContainerAPITests testMethod=test_create_container_with_dataset> (test_create_container_with_dataset) still running at 120.0 secs

flocker.acceptance.test_api.ContainerAPITests.test_create_container_with_dataset
===============================================================================
[ERROR]
Traceback (most recent call last):
Failure: twisted.trial.util.DirtyReactorAggregateError: Reactor was unclean.
DelayedCalls: (set twisted.internet.base.DelayedCall.debug = True to debug)
<DelayedCall 0x2cf35a8 [0.0991139411926s] called=0 cancelled=0 Deferred.callback(None)>

flocker.acceptance.test_api.ContainerAPITests.test_create_container_with_dataset
===============================================================================
[ERROR]
Traceback (most recent call last):
Failure: twisted.internet.defer.TimeoutError: <flocker.acceptance.test_api.ContainerAPITests testMethod=test_move_container_with_dataset> (test_move_container_with_dataset) still running at 240.0 secs

flocker.acceptance.test_api.ContainerAPITests.test_move_container_with_dataset
===============================================================================
[ERROR]
Traceback (most recent call last):
Failure: twisted.trial.util.DirtyReactorAggregateError: Reactor was unclean.
DelayedCalls: (set twisted.internet.base.DelayedCall.debug = True to debug)
<DelayedCall 0x2cf34d0 [0.0995428562164s] called=0 cancelled=0 Deferred.callback(None)>

flocker.acceptance.test_api.ContainerAPITests.test_move_container_with_dataset
-------------------------------------------------------------------------------
Ran 20 tests in 1182.492s

FAILED (skips=1, errors=4, successes=17)
Unhandled error in Deferred:Unhandled Error
Traceback (most recent call last):
Failure: twisted.internet.error.ProcessTerminated: A process has ended with a probable error condition: process ended by signal 15.
program finished with exit code 1
elapsedTime=2102.071704

So it turns out that the ZFS builders were meant to be in the "expected failures" list:

https://clusterhq.atlassian.net/browse/FLOC-2544

Tom's fixed that now:

http://build.clusterhq.com/boxes-flocker?branch=acceptance-test-containerapi-timeout-FLOC-2488

...so I'll merge.


Rename stack commands to package commands

Fixes #54
Renames all kubectl crossplane stack commands to kubectl crossplane package commands.

Please wait for me to merge after reviews are complete so I can coordinate other changes :+1:

Is all feedback incorporated and we're ready to merge this one @hasheddan?

@jbw976 Yes, although the cicd and each of the stacks / apps need to follow in quick succession because I believe the cicd setup is periodic, so if it runs before it is updated or each of the stacks are updated then it will fail. It would not be a big problem, but I would rather minimize that window.


Mac crashes when searching on the sidebar search form.

Issue Type: Bug
Type validation-> in the sidebar search form.
When entering one character at a time
As soon as you enter the> character, the mac hangs and the mac restarts.
I tried it several times, but even if I didn't hang up, mac stopped and started moving for about 1 minute. In this case,> is input as many characters as validation->>>>>>>>>>>>>>>>>>>>.
When pasting validation-> string with copy and paste
Mac hangs up.
I think this bug did not occur before Ver1.40
Maybe related to #86913
VS Code version: Code 1.41.0 (9579eda04fdb3a9bba2750f15193e5fafe16b959, 2019-12-11T17:58:38.338Z)
OS version: Darwin x64 18.7.0

System Info

Item
Value

CPUs
Intel(R) Core(TM) i7-8700B CPU @ 3.20GHz (12 x 3200)

GPU Status
2d_canvas: enabledflash_3d: enabledflash_stage3d: enabledflash_stage3d_baseline: enabledgpu_compositing: enabledmetal: disabled_offmultiple_raster_threads: enabled_onoop_rasterization: disabled_offprotected_video_decode: unavailable_offrasterization: enabledskia_renderer: disabled_offsurface_control: disabled_offsurface_synchronization: enabled_onvideo_decode: enabledviz_display_compositor: enabled_onviz_hit_test_surface_layer: disabled_offwebgl: enabledwebgl2: enabled

Load (avg)
3, 35, 30

Memory (System)
16.00GB (0.58GB free)

Process Argv
-psn_0_520319

Screen Reader
no

VM
0%

Extensions (64)

Extension
Author (truncated)
Version

html-snippets
abu
0.2.1

vscode-caniuse
aka
0.5.3

Bookmarks
ale
10.6.0

project-manager
ale
10.9.1

vagrant
bbe
0.5.0

path-intellisense
chr
1.4.2

regex
chr
0.2.0

bracket-pair-colorizer-2
Coe
0.0.29

vscode-svgviewer
css
2.0.0

vscode-eslint
dba
2.0.6

vscode-dash
dee
2.3.0

githistory
don
0.4.11

jquerysnippets
don
0.0.1

gitlens
eam
10.2.0

EditorConfig
Edi
0.14.3

apacheconf-snippets
eim
1.3.0

LogFileHighlighter
emi
2.6.0

vscode-great-icons
emm
2.1.47

prettier-vscode
esb
3.16.1

vscode-todo-plus
fab
4.14.1

php-debug
fel
1.13.0

auto-close-tag
for
0.5.6

auto-rename-tag
for
0.1.1

code-runner
for
0.9.15

vscode-google-translate
fun
1.4.2

vue-snippets
hol
0.1.11

minify
Hoo
0.4.3

rest-client
hum
0.23.0

output-colorizer
IBM
0.1.2

plantuml
jeb
2.13.5

docthis
joe
0.7.1

markdownfromcsv
joj
1.0.0

vscode-insertdatestring
jsy
2.2.4

vscode-sshfs
Kel
1.16.3

vscode-regexp-preivew
le0
0.0.3

vue
liu
0.1.5

MagicPython
mag
1.1.0

rainbow-csv
mec
1.3.1

easy-less
mrc
1.5.1

vscode-apache
mrm
1.2.0

vscode-csscomb
mrm
5.3.0

vscode-docker
ms-
0.9.0

vscode-language-pack-ja
MS-
1.41.2

mssql
ms-
1.8.0

python
ms-
2019.11.50794

debugger-for-chrome
msj
4.12.3

indent-rainbow
ode
7.4.0

csv-to-table
php
1.2.1

material-icon-theme
PKi
3.9.2

partial-diff
ryu
1.4.0

vscode-sql-beautify
sen
0.0.4

code-settings-sync
Sha
3.4.3

addDocComments
ste
0.0.8

pretty-json
sup
0.0.4

ayu
tea
0.18.0

pdf
tom
0.5.1

highlight-matching-tag
vin
0.9.6

vscode-icons
vsc
9.6.0

quokka-vscode
Wal
1.0.267

jinja
who
0.0.8

change-case
wma
1.0.0

better-align
wwm
1.1.6

JavaScriptSnippets
xab
1.7.2

markdown-pdf
yza
1.4.1

(5 theme extensions excluded)

Additional Information
The search string will behave strangely with any string, not validation->.
I guess the search function on the left side is abnormal.
I downgraded vscode to ver1.40.2 and tried the same procedure, but no bugs occurred.
I confirmed that I can search normally with ver1.40.2.

Met the same issue.
Version: 1.41.0
Commit: 9579eda04fdb3a9bba2750f15193e5fafe16b959
Date: 2019-12-11T17:58:38.338Z
Electron: 6.1.5
Chrome: 76.0.3809.146
Node.js: 12.4.0
V8: 7.6.303.31-electron.0
OS: Darwin x64 17.7.0

Also experiencing this issue:
VSCode Version: 1.41.1
OS: Mac Mojave 10.14.6
Confirmed tarky17's experience that downgrading to 1.40.2 resolves the bug.
Note that this bug appears unaffected by modifications to User Preferences for Search (both "Exclude" and "Use Ignore Files") which are things I've seen people suggest changing on other similar bug reports.
Reporter alluded to this, but the bug feels related to debouncing the query - I don't encounter bugginess on the first search, but if I start to modify it before completion of search, VSC does not seem to wait for/honor pressing the "Enter" key but perhaps is searching immediately and after every keystroke.

Can you try disabling search.searchOnType and see whether that helps in 1.41?

I have confirmed that this bug does not occur with search. SearchOnType disabled in ver1.41.1.
My env below.
Version: 1.41.1
Commit: 26076a4de974ead31f97692a0d32f90d735645c0
Date: 2019-12-18T14:57:51.166Z
Electron: 6.1.5
Chrome: 76.0.3809.146
Node.js: 12.4.0
V8: 7.6.303.31-electron.0
OS: Darwin x64 18.7.0

@roblourens same for me.
I gather this is a new feature in this release, as I don't see that settings option in 1.40.2?
I wonder if changing the default setting might be advisable? Several people I know have switched to another IDE out of frustration with this behavior, which is a shame if it's actually a feature.

Yeah, it was new. So when you have that setting enabled and type your query, you get the freeze, but when you disable it and type the same query and press enter, it doesn't freeze?

Hi Rob,
That's correct.
I did a quick ABA test to confirm this - reinstalled<EMAIL_ADDRESS>confirmed that searchOnType is enabled (w a debounce of 300ms, which I assume is the default), began to type a search string, and beachballed my Mac.
There is definitely some nuance to this behavior - I see different behavior when using different query strings/folder inclusion settings.  I think the common thread is the total volume of results being generated by an in-flight query (or queries, not sure how that looks under the hood).
For example, in a TS project, typing "im" (on the way to searching for "import") is more likely to cause freezing than searching for "xyz"
Let me know if there's anything else I can do to help debug this.

So is it a really large project, like would "im" return a ton of results? And if you disable search on type, then type "im" and press enter, it doesn't freeze?
It might help if you do this

Run the command "Set log level" and change it to "Trace"
Repro the issue
Show the search-related messages in the "Log (Window)" output panel

That would show us how many times search is being triggered and why. That may or may not be useful info.

I don't have a sense of what you would consider a lot, but "import" returns 8,000 results across 1,500 files, so... maybe?
Just searching for "im" hits the 10,000 result limit/instruction to narrow the search, so presumably there are considerably more matches for that substring.
I went back and tested "im" on both 1.40.2, and 1.41.1 with searchOnType disabled, and I can sometimes* but not always cause freezes. With searchOnType enabled in 1.41.1 the freeze occurs 100% of the time.
So it seems that the issue is not a problem with the search on type feature, per se, but rather that it exposes an underlying limitation of the search functionality against a repo of our size. Still, not a great UX from our perspective :(

I can't figure out the pattern yet in why this behavior isn't consistent; maybe caching of some sort?

Sadly, while I can attempt to capture those logs, they don't survive the hard reboot required to recover my machine after reproducing the behavior 🤷♀

with searchOnType disabled, and I can sometimes* but not always cause freezes

Thanks for the investigation, this is really the root of it I think. Something about having ripgrep return a ton of results can cause this, and starting/cancelling multiple searches quickly probably makes it worse. One more thing, could you try disabling search.followSymlinks? It has led to perf issues before and if you have a lot of symlinks in your workspace then this might help.

search.followSymlinks  did not help

Version: 1.41.1
Commit: 26076a4de974ead31f97692a0d32f90d735645c0
Date: 2019-12-18T14:57:51.166Z
Electron: 6.1.5
Chrome: 76.0.3809.146
Node.js: 12.4.0
V8: 7.6.303.31-electron.0
OS: Darwin x64 19.2.0

@roblourens we don't do any symlinking that I'm aware of in this project, but I tested the combination of searchOnType: enabled & followSymlinks: disabled for the dreaded "im" search string, and still saw a freeze.
Also, something occurred to me with regards to the intermittency of the freeze that I was describing in yesterday's comment: would these kind of Settings changes require a restart of the app before they take effect?

Ok, it was just a guess.

would these kind of Settings changes require a restart of the app before they take effect?

No, not these

I have looked into this a few times and have never been able to repro any issue, I am able to use searchOnType and start multiple searches in a large workspace one after another with no issues at all, on my 2016 macbook.
There are some things that I think I am not sure about

When this is happening, do you see multiple rg processes? Would show up as /Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/node_modules.asar.unpacked/vscode-ripgrep/bin/rg in the Process Explorer or code --status
Do you see any other vscode-related processes using high cpu?
Also, the logs that I mentioned earlier might still be helpful - https://github.com/microsoft/vscode/issues/87254#issuecomment-571820487 if needed you can check the path of the log file before starting to reproduce the issue, it should survive a reboot if needed
I wonder whether there are any other factors, do you have low memory available, other processes busy when this is happening, is it an older Mac, etc?

I will attempt to repro again this week. Issue has not occurred again since disabling searchOnType. I am running on a new 16" mbp with hardly nothing else installed and I am working in a javascript react git repo with around 1800 files.

If you are still able to repro it, could you share the repo so I can try with the exact same setup?

I’ll have to try it against some of my public repos, the one I first hit this issue on is private

Hey Rob,
I've upgraded to 1.42.1, and even with searchOnType turned on, I've not been able to recreate the freeze. (I assume we don't expect anything to have fixed this between 1.41.1 and 1.42.1 but if you think it's worth rolling back to the original version let me know.)
My Mac is pretty modern (2017, 3.1 GHz Intel Core i7), but your question got me thinking - I do frequently run Docker, which has intermittent performance spiking problems on its own. I can certainly see the combo causing a total freeze, if they happened simultaneously.
I will leave 'searchOnType' turned on, in hopes of being able to raise it organically, but let me know if you think rolling back to 1.41.1 is also worth doing for testing purposes.

That's interesting, I'll try to repro this with some containers running.
No need to downgrade, but please let me know how it goes with searchOnType enabled, thanks!

Hey Rob,
I'm not sure what I'm currently seeing is the same root cause as this bug (because I have searchOnType turned off), but I'm now encountering multiple search-triggered beachballs per day.  I will add a follow-up comment with more data if/as I do a little more sleuthing and decide if this is relevant to the current thread.
Meanwhile - I am reviewing my logs in hopes of getting more data, but I'm curious what determines the number of simultaneous render logs/whether this is normal(ish)?

Hm, I don't think I've seen that format before but it's probably not a problem. We try to pick window IDs to be simple and increasing but can probably lose count somehow...

I have reliably encountered this when searching for callPackage in https://github.com/NixOS/nixpkgs master checkout with searchOnType enabled. I do not have any time to investigate this further.

Hi Rob,
The searchOnType is disabled but it is not taking effect in the file/folder filter. It still does search on type and freezes the Mac. Can you please enable that option for filter box as well? This workaround will mostly solve the freeze for me.

FYI, disabling searchOnType did not fix the issue for me. It can still cause a freeze once I do anything that interrupts/restarts a search (eg. hit Enter, change case or word options, click Clear to abort).

I uninstalled a couple plugins and haven't had a crash/freeze in a couple days. IIRC they were the Microsoft C/C++ plugin and the Deno plugin.

Exactly same issue here.
@roblourens I think it will be easier to reproduce by:

Add LLVM, Golang, ... many large projects to one workspace.
Reach a single character a in sidebar to produce many search results.
It will finally freeze the macOS.

I have 20 projects in a single workspace (incl. LLVM, Golang), it's 100% reproducible here.

I first started seeing this error after upgrading to the Microsoft C++ extension which recently jumped to a 1.0 release. I found that I can reproduce the hang consistently on one of my repo clones but not another clone. Anyone with some debug tips they'd like me to try I can take a moment and experiment on that. Since mine reproduces on command we might be able to find something out.

I have been running into this issue on my 16" MBP, and the cause is always the same: an unconstrained search of a workspace with 5+ projects each with between 6k - 15k files each. By unconstrained, I mean not limiting search results by using a filter/glob in the files to include input field.
It is not 100% reproducible (though given the impact, I don't spend any time trying to narrow things down).  Sometimes the unconstrained search works as expected, but usually I find that out by lucky accident. I know immediately when I've hit it, as the search quickly triggers the spinning beachball of death, the system is unresponsive (so no ability to use system/diagnostic tools to kill the proc), then around 30 seconds the fans kick in, and finally the automatic reboot.
I have never had the issue with a filter that constrains the search to one specific project in my workspaces (e.g. ./foo-project.
The reboot happens as a result of the macOS watchdog timing out on it's kernel health check. Seeing the timeout, it reboots the machine. This can be verified in the logs:
"macOSPanicString" : "panic(cpu 10 caller 0xffffff7f879a1a8d): watchdog timeout: no checkins from watchdogd in 96 seconds (309 total checkins since monitoring last enabled)

Per this thread, I had previously "Fast Graphics Switching" and hadn't encountered this issue in a while.

My environment is a npm-oriented monorepo and due to some architectural concerns, there are node_modules directories at various different depths, i.e. not all easily caught by a single globbing pattern.
I was able to stop the hangs entirely by taking the time to build out the necessary glob patterns to exclude all my node_modules directories, which generally gels with other observations around unconstrained search, size of search space, etc.

Have 1.49.1 fixed this issue? I am not so sure, but seems not happen again here. 🤔

It's probably just good luck, david9991   ;-)
It's not fixed for me in 1.49.1

Not fixed in Version: 1.50.1

Not sure about the vs code search, but I found that my macbook pro 16'' 2019, time from time freezing and I hear load fun's noise, and after few similar incidents I get to point that it's because of vs code and node_modules folder, last time I was faced with freez issues was when I was doing some surfing in node_modules folder, so I just start opening some embedded node_modules folders and after some time I've started hearing a noise and it almost immediately frozed my laptop and after hot reboot I was needed to wait for more than usual until my macbook will load mac os

So, that has happened again, my mac stuck totally and it was right after I was tried to open package.json file from the node_modules folder, so now I'm almost sure it is somehow related to my vs code insiders IDE. This time search Type was turned off. This issue too critical for me, it happens too often. And seems that issue make some negative effect on the laptop as well, can someone share some thought about that?

I had to downgrade to version 1.39 to be able to keep using VSCode due to this issue too.

Happening to me since last VS Code update on OSX Catalina. No specific search query that I can reproduce, seems "random". Has happened 4 times in last few days:
Version: 1.56.2
Commit: 054a9295330880ed74ceaedda236253b4f39a335
Date: 2021-05-12T17:44:30.902Z
Electron: 12.0.4
Chrome: 89.0.4389.114
Node.js: 14.16.0
V8: <IP_ADDRESS>-electron.0
OS: Darwin x64 19.6.0

VS code 1.62.3  on MacBook pro  2018 running Catalina 10.15.7 .. search crashes the system almost every time.. This has been an issue for as long as I have been using VScode on MacBook pro .. This is a super critical issue since as when dealing with large number of files and many thousands lines of code one can't rely on the VS search but has to use the OS find, or use ATOM!

For me the issue disappeared after removing some of my repos from the workspace, so it has something to do with the files that are being searched. I am working now for 3 weeks without a crash, while previously VS code crashed multiple times per day. I have yet to figure out what the problematic repo and file content is.

If you figure out where the problem is, and can share that folder/file with me, I'd really like to see it.

Just let you know that searching can also crash Windows 10.

I got the same issue for vscode 1.64.2 in MacBook pro 2018.

@roblourens, unfortunately I can no longer reproduce the problem, so I cannot locate the problematic file content. I have not seen a crash for weeks now despite working with repos that were always problematic for me. Not sure what changed.
These are my current specs:
VSCode Version: 1.65.1 (Universal)
Commit: 8908a9ca0f221f36507231afb39d2d8d1e182702
Date: 2022-03-08T02:20:11.670Z (1 wk ago)
Electron: 13.5.2
Chromium: 91.0.4472.164
Node.js: 14.16.0
V8: 9.1.269.39-electron.0
OS: Darwin x64 20.6.0

I've never reproduced this and don't know what the issue is. Is anybody still seeing it?

when I open less 5 projects(folds) in the workspace, the search-in-files works well!

It may be an issue with the number of simultaneous ripgrep processes, but I haven't seen an issue.

We closed this issue because we are unable to reproduce the problem with the steps you describe. Chances are we've already fixed your problem in a recent version of VS Code. If not, please ask us to reopen the issue and provide us with more detail. Our issue reporting guidelines might help you with that.
Happy Coding!


update pymatreader to version 0.0.26

this is the version of the pymatreader changes in #10005 as implemented in upstream pymatreader. the changes are only minor so i do not expect any problems.
thanks again @larsoner for proposing the fixes!

Thanks @thht !


Is there any plan to provide a double-ended queue implementation

Hi guys, is there any plan to provide a double-ended queue implementation in this package?

I'll make one.

It is at https://github.com/Rooster17678/go-datastructures
This is the first go program I have ever written so I recommend validating it yourself.


bump pack-repo to 0.4.1

manually tested and confirmed that this fixes draft init.
><> make
GOBIN=/home/bacongobbler/go/src/github.com/Azure/draft/bin go install  -tags 'kqueue' -ldflags ' -X github.com/Azure/draft/pkg/version.Version=v0.1.0 -X github.com/Azure/draft/pkg/version.GitCommit=e80d9f336e238dfb424c99c9c8b7fa86d118354b -X github.com/Azure/draft/pkg/version.GitTreeState=clean' github.com/Azure/draft/cmd/...
><> rm -rf ~/.draft/
><> ./bin/draft init
Installing default plugins...
Downloading https://azuredraft.blob.core.windows.net/draft/pack-repo-v0.4.1-linux-amd64.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 3283k  100 3283k    0     0  1811k      0  0:00:01  0:00:01 --:--:-- 1812k
Preparing to install into /home/bacongobbler/.draft/plugins/draft-pack-repo
draft-pack-repo installed into /home/bacongobbler/.draft/plugins/draft-pack-repo/draft-pack-repo
Installed plugin: pack-repo
Installation of default plugins complete
Installing default pack repositories...
Installing pack repo from https://github.com/Azure/draft
Installed pack repository github.com/Azure/draft
Installation of default pack repositories complete
$DRAFT_HOME has been configured at /home/bacongobbler/.draft.
Happy Sailing!


Windows Support

Wondering if there is a plan to support Windows. Is there is any technical problem preventing the support of windows desktop?

Just wait for this PR to land
https://github.com/flutter-webrtc/flutter-webrtc/pull/290

@cloudwebrtc Is there any plan to revisit this now that the PR has landed? Thanks for this library!

@cloudwebrtc is there any update for windows support on desktop ?

The current version of dart-sip-ua should work on webrtc plan-b, but it needs some testing. It fully supports unified-plan and still needs to wait https://github.com/flutter-webrtc/flutter-webrtc/pull/610 Landing


Dropin daemon reload

Fixes #234

I'm not sure this is correct. Puppet should already reload services that are managed. In https://github.com/voxpupuli/puppet-systemd/issues/234#issuecomment-962998689 I asked for some more info.

I'm not sure this is correct. Puppet should already reload services that are managed.

Which it does but the bug is for services that only dropin files are managed. I use those all the time to add a delta on top of what the distro/packager provided for example.

In https://github.com/voxpupuli/puppet-systemd/issues/234#issuecomment-964077912 I suggested to use a define to avoid multiple reloads for the same service, in case multiple drop ins are used.

* GDM with `hidepid=2` requires an updated to `systemd-logind` via a dropin file.

Sorry if that's not related to the core issue but I've been wanting to go back to hidepid=2 for sometimes but didn't know how to make it work. What's the systemd-logind tweak that's needed, please? :)

@simondeziel See https://github.com/simp/pupmod-simp-gdm.
Also, are you up for getting this changed now or should I refactor into another PR? At a minimum, we need to get the exec into its own defined type.

@simondeziel Can you try this? https://github.com/voxpupuli/puppet-systemd/tree/daemon-reload
It works for me, but I'm working on a section to fix all of the systems that got broken by the current 3.X branch (daemon-reload was never called)

@trevor-vaughan the daemon-reload branch works for me as well, thanks!

Closing in favor of #277 per the conversation above


Add support for EXT_mesh_gpu_instancing

Create UInstancedStaticMeshComponent objects for meshes that use the EXT_mesh_gpu_instancing extension. Nothing is done yet about metadata.
I used multiple inheritance to share the Cesium data that was in UCesiumGltfPrimitiveComponent with the new UCesiumGltfInstancedComponent class, but this is kind of awkward in the context of Unreal's automatically generated reflection support et al. I'd like to get feedback on whether it would be better to use a composition-based approach instead.

I'd like to get feedback on whether it would be better to use a composition-based approach instead.

While multiple inheritance doesn't seem to be explicitly disallowed in UE's coding standard, from reading discussions on UE forums, it seems that multiply inheriting from interfaces is the general accepted pattern.
I think I understand your approach and seems like it could work. If it was up to my preferences, I'd probably lean toward a single inheritance approach here

UCesiumGltfPrimitiveComponent derives from UStaticMeshComponent only
UCesiumGltfPrimitiveComponent contains an instance of CesiumGltfPrimitiveBase
Pros - More UE coding paradigm friendly, no multi-inheritance means no multi-inheritance complexities (problems)
Cons - Maybe more duplicated code than you want, but maybe not that bad in the end?

I'd like to get feedback on whether it would be better to use a composition-based approach instead.

While multiple inheritance doesn't seem to be explicitly disallowed in UE's coding standard, from reading discussions on UE forums, it seems that multiply inheriting from interfaces is the general accepted pattern.
I think I understand your approach and seems like it could work. If it was up to my preferences, I'd probably lean toward a single inheritance approach here

UCesiumGltfPrimitiveComponent derives from UStaticMeshComponent only
UCesiumGltfPrimitiveComponent contains an instance of CesiumGltfPrimitiveBase
UCesiumGltfInstancedComponent derives from UInstancedStaticMeshComponent only
UCesiumGltfInstancedComponent contains an instance of CesiumGltfPrimitiveBase
Pros - More UE coding paradigm friendly, no multi-inheritance means no multi-inheritance complexities (problems)
Cons - Maybe more duplicated code than you want, but maybe not that bad in the end?

Thanks for the feedback. I'm starting to look at this in detail. The one issue I have with moving to a single inheritance approach -- where UCesiumGltfPrimitiveComponent and UCesiumGltfInstancedComponent don't share a common Cesium base class -- is how to implement UpdateTransformFromCesium(). While it's nice that it's virtual in my current implementation, I suppose a combination of Cast and a template function could get around it. I'm going to continue to play it and try to figure out if multiple inheritance is really that bad here.

I noticed a crash while loading an i3dm tileset from the CesiumJS account on Cesium ion, asset ID 3877. Looks like it's dereferencing an empty optional.

After a little investigation, I think this is a glTF 1.0 tileset, which cesium-native doesn't support at all, so I don't expect it to work. But it shouldn't crash. It should warn about an unsupported version, like it does for the same situation in b3dm.

Your changes look good, thanks @timoore! It looks like you may have missed my comment about the crash above, though. I'll check if that's still an issue and fix it before the release if so. Merging!


Fix key ordering on document replacing

Description
Closes #1736.

Readiness checklist

[ ] I added/updated unit tests.
[x] I added/updated integration/compatibility tests.
[ ] I added/updated comments and checked rendering.
[x] I made spot refactorings.
[ ] I updated user documentation.
[ ] I ran task all, and it passed.
[x] I ensured that PR title is good enough for the changelog.
[x] (for maintainers only) I set Reviewers (@FerretDB/core), Assignee, Labels, Project and project's Sprint fields.
[x] I marked all done items in this checklist.

It seems that we need to unify field names we use for Tigris collections, so we don't need to extend this schema even more in the future.

It would be cool to make some helper function that would get schema as an argument, to make some test specific schemas if needed.


[Suggestion] Use ory stack for auth management

Yo guys, what if you will consider for the future to use ory stack for auth Z/N? - https://www.ory.sh
ory kratos for user management
ory oathkeeper for authorization

They offer as well an open-source IAM solution and Oauth solution which can be handy for developing your own public APIs.
No need to maintain own auth solution
Security issues tracking and hardening can be a hussle

would love to hear from you guys what do you think about this.

We're using Hasura Auth and we don't have any plans of moving to another auth solution.


Track 'Borlabs' cookie compliance

Example Site - https://getbyrd.com/
Vendor Site - https://borlabs.io/borlabs-cookie/
Category - Cookie Compliance
Detection - Look for JS object 'borlabsCookieConfig'

Closing this as this seems to be working.


feat!: Disaster recovery via gateway

This change proposes a disaster recovery mechanism where an embedding Noosphere client may take advantage of an existing gateway to restore a user's local sphere data.
In order to recover data from a gateway, the following information is needed:

URL of the gateway
The DID of the user's sphere
The name of an authorized device key
The user's recovery mnemonic

The Noosphere library then generates an ephemeral, one-time-use authorized credential and uses it to perform the fetch half of a gateway sync. After the sync completes, it ensures that the local storage layer is properly configured to use the original device key.

Continuing the trend of forcing lint checks for missing documentation on a crate-by-crate basis, I added #[warn(missing_docs)] to the noosphere crate and filled in the gaps where docs were missing.


Clean up some FAQ/Frequently asked question repetition in readme

Ah, yes, copypaste error. Thanks!


common/crypto/ed25519: Use curve25519-voi

[x] Switch the import
[x] Use the new batch verification API
[x] Use expanded form public keys, under the assumption that the set of public keys is relatively static.
[x] Add a SLIP-0010 implementation (moved from the voi repo)

This should eventually carve out the cache into somewhere else (maybe as part of voi, it seems generally useful), so that the tendermint side can also use it.  But what is there is ok as an illustration of how this will work.

Looks good. Just making sure -- this is not consensus breaking and these verification parameters are compatible with what we had before?

Yes.


HRRR weathermodel 'temp_air' calculation wrong

Describe the bug
The HRRR model 'temp_air' calculation uses the method:
pvlib/forecast.py:
'def isobaric_to_ambient_temperature(self, data)
The formula implemented by this method appears to produce incorrect 'temp_air' values.
To Reproduce
Steps to reproduce the behavior:

Run the attached Jupyter notebook 'hrrr_air_temp_testcase':

hrrr_air_temp_testcase.ipynb.txt

Compare the 'temperature_isobaric' values (in Kelvin)  with the resulting 'temp_air' values (in Celsius):

from the weather forecast:
                           Temperature_isobaric
2018-10-12 07:00:00-07:00            288.361938
2018-10-12 08:00:00-07:00            288.778656
2018-10-12 09:00:00-07:00            289.354309
2018-10-12 10:00:00-07:00            290.199005
2018-10-12 11:00:00-07:00            291.025238

after conversion with 'get_processed_data():
                           temp_air  wind_speed         ghi         dni  \
2018-10-12 07:00:00-07:00 -2.443878    3.547055    0.000000    0.000000
2018-10-12 08:00:00-07:00 -1.725800    3.856754   72.719945  177.219441
2018-10-12 09:00:00-07:00 -1.064240    4.183833  269.398844  587.731450
2018-10-12 10:00:00-07:00 -0.781128    3.085214  452.789030  745.781057
2018-10-12 11:00:00-07:00  1.528809    3.174382  594.728725  793.903707

Expected behavior
'temp_air' values are expected to be around 17 C, not around the freezing point (0 in C).
Screenshots
see attached Jupyter notebook testcase
Versions:

pvlib.__version__: v0.6.0
pandas.__version__: 0.23.4
python: Jupyter Python 3 kernel

Additional context

It would be best to pull the 2 m temperatures directly from the data feed. HRRR 2 m temps were not available on the Unidata feed at the time we wrote the code, so we went with what we had. Pull request welcome.


JRuby <IP_ADDRESS> produces a warning from rubygems, ignore it

Merging to get the builds green

Thanks


Add unit tests for InstallableItem.getDownloadStatus() method

Codecov Report

Merging #928 into master will decrease coverage by 0.04%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #928      +/-   ##
==========================================
- Coverage   95.29%   95.24%   -0.05%
==========================================
  Files          39       39
  Lines        3445     3620     +175
  Branches      294      527     +233
==========================================
+ Hits         3283     3448     +165
  Misses        162      162
- Partials        0       10      +10

Impacted Files
Coverage Δ

browser/services/credentialManager.js
96.87% <0%> (-3.13%)
:arrow_down:

browser/pages/location/controller.js
92.3% <0%> (-2.86%)
:arrow_down:

browser/pages/install/controller.js
78.84% <0%> (-2.75%)
:arrow_down:

browser/main.js
98.48% <0%> (-1.52%)
:arrow_down:

browser/model/devstudio.js
96.8% <0%> (-0.97%)
:arrow_down:

browser/model/cdk.js
99.21% <0%> (-0.79%)
:arrow_down:

browser/model/helpers/downloader.js
95.27% <0%> (-0.63%)
:arrow_down:

browser/services/data.js
97.06% <0%> (-0.59%)
:arrow_down:

browser/model/jdk-install.js
96.04% <0%> (-0.39%)
:arrow_down:

browser/services/openpgp.js
100% <0%> (ø)
:arrow_up:

... and 25 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 48e2cde...4a007ba. Read the comment docs.


Add getCurrentUser method

Get info about the currently logged in user (whos PAT is in use)

/makerelease


cmd/compile: ICE: panic during late deadcode while compiling (*ServeMux).shouldRedirectRLocked

What version of Go are you using (go version)?

$ go version
1.15.2

Does this issue reproduce with the latest release?
What operating system and processor architecture are you using (go env)?
go env Output
$ go env

What did you do?
When building packer on Debian armhf machines, we face an ICE.
https://buildd.debian.org/status/fetch.php?pkg=packer&arch=armhf&ver=1.6.4%2Bdfsg-2&stamp=1601746051&raw=0
# net/http
/usr/lib/go-1.15/src/net/http/server.go:2318:14: internal compiler error: '(*ServeMux).shouldRedirectRLocked': panic during late deadcode while compiling (*ServeMux).shouldRedirectRLocked:

runtime error: makeslice: cap out of range

goroutine 39 [running]:
cmd/compile/internal/ssa.Compile.func1(0x342d71c, 0x558e300)
	/usr/lib/go-1.15/src/cmd/compile/internal/ssa/compile.go:48 +0x7c
panic(0x760ce8, 0x89eae0)
	/usr/lib/go-1.15/src/runtime/panic.go:969 +0x118
cmd/compile/internal/ssa.ReachableBlocks(0x558e300, 0xa6b000b8, 0x1, 0x3)
	/usr/lib/go-1.15/src/cmd/compile/internal/ssa/deadcode.go:23 +0x38
cmd/compile/internal/ssa.deadcode(0x558e300)
	/usr/lib/go-1.15/src/cmd/compile/internal/ssa/deadcode.go:181 +0x40
cmd/compile/internal/ssa.Compile(0x558e300)
	/usr/lib/go-1.15/src/cmd/compile/internal/ssa/compile.go:93 +0x98c
cmd/compile/internal/gc.buildssa(0x1f395f0, 0x3, 0x0)
	/usr/lib/go-1.15/src/cmd/compile/internal/gc/ssa.go:460 +0xbf4
cmd/compile/internal/gc.compileSSA(0x1f395f0, 0x3)
	/usr/lib/go-1.15/src/cmd/compile/internal/gc/pgen.go:317 +0x3c
cmd/compile/internal/gc.compileFunctions.func2(0x30171c0, 0x1ae2e50, 0x3)
	/usr/lib/go-1.15/src/cmd/compile/internal/gc/pgen.go:382 +0x38
created by cmd/compile/internal/gc.compileFunctions
	/usr/lib/go-1.15/src/cmd/compile/internal/gc/pgen.go:380 +0x100

It uses external link, other settings are standard.
GOFLAGS := -ldflags=-linkmode=external

We choose to use external link, because previously(when we still use go1.14) we meet link error, but it's another issue.
https://buildd.debian.org/status/fetch.php?pkg=packer&arch=armhf&ver=1.6.0%2Bdfsg-2&stamp=1594662885&raw=0
# armel/armhf fail to link:
# os/user(.text): direct call too far: .plt 87db4e

It's hard for me to reproduce locally and to write down a detailed issue, since it happens on an armhf machine, . But since it's an ICE, so I report here anyway, maybe you will know something from the trace.
What did you expect to see?
What did you see instead?

cap out of range is a very strange error to get at that point.
	reachable := make([]bool, f.NumBlocks())

Sounds like some sort of corruption happened. I don't see any way f.NumBlocks() could return strange (negative, or very large) values.
Even stranger, we got an error on cap instead of len. makeslice is explicitly designed to report len in preference to cap. For the 2-arg make, we have len==cap, and in that scenario there's no path through makeslice that can generate a cap error.
How repeatable is this error? Can you get it to happen on two different machines? It would be nice to eliminate the machine itself.

How repeatable is this error? Can you get it to happen on two different machines? It would be nice to eliminate the machine itself.

I don't have access to these builders. When I trigger a rebuild, it goes to another machine, which is an arm64 hardware.
The log is https://buildd.debian.org/status/logs.php?pkg=packer&ver=1.6.4%2Bdfsg-2&arch=armhf
The failed machine is https://db.debian.org/machines.cgi?host=henze
I will try to see if I can get access to similar hardware.

@dr2chase is it related to #41780 ? This is also on ARM. Maybe the compiler itself is miscompiled?

I find a machine which I can access, which has same CPU(Marvell Armada 370/XP CPU @ 1.6GHz on a Marvell MV78460 SoC Development Board (ARM v7)) https://db.debian.org/machines.cgi?host=abel
But I can't reproduce the ICE.

Can't reproduce, make fails with:
common/bootcommand/config.go:1: running "struct-markdown": exec: "struct-markdown": executable file not found in $PATH

Where does struct-markdown come from?  I'm trying to see if the fix for #41780 ( https://go-review.googlesource.com/c/go/+/259450/ ) affects this.  I'm on an Odroid (arm64) with GOARCH=arm in my environment and running an arm-compiled compiler.

FYI, my current repro attempt for compilation on arm64 ODroid Linux:
git diff Makefile
diff --git a/Makefile b/Makefile
index 41bfd38c9..3f358d6cb 100644
--- a/Makefile
+++ b/Makefile
@@ -1,3 +1,5 @@
+GOFLAGS := -ldflags=-linkmode=external
+
 TEST?=$(shell go list ./...)
 VET?=$(shell go list ./...)
 ACC_TEST_BUILDERS?=all

Cross compilation for cgo:
sudo apt install gcc-arm-linux-gnueabihf libc6-armhf-armel-cross

Environment variables for the make command:
PATH=$HOME/work/go-linux-arm-bootstrap/bin:$GOPATH/bin:$PATH \
GOARCH=arm \
LD_LIBRARY_PATH=/usr/arm-linux-gnueabihf/lib \
CC=arm-linux-gnueabihf-gcc \
make

Unfortunately it still fails with the linking error mentioned above, despite making the recommended change to Makefile:
...
go run ./cmd/generate-fixer-deprecations
# github.com/hashicorp/packer
os/user(.text): direct call too far: .plt a6fb8c
os/user(.text): direct call too far: .plt a6fb84
os/user(.text): direct call too far: .plt a6fb83
...

Perhaps someone more familiar with the Makefile can help me along to the next roadblock.

@dr2chase try -ldflags=-linkmode=external.

Successful build (i.e., sadly I failed to reproduce the failure) with the following diff to the Makefile, which may not all be necessary:
git diff Makefile
diff --git a/Makefile b/Makefile
index 41bfd38c9..e94bcc84b 100644
--- a/Makefile
+++ b/Makefile
@@ -1,3 +1,5 @@
+GOFLAGS := -ldflags=-linkmode=external
+
 TEST?=$(shell go list ./...)
 VET?=$(shell go list ./...)
 ACC_TEST_BUILDERS?=all
@@ -17,7 +19,7 @@ GIT_DIRTY=$(shell test -n "`git status --porcelain`" && echo "+CHANGES" || true)
 GIT_COMMIT=$(shell git rev-parse --short HEAD)
 GIT_IMPORT=github.com/hashicorp/packer/version
 UNAME_S := $(shell uname -s)
-LDFLAGS=-s -w
+LDFLAGS=-linkmode=external
 GOLDFLAGS=-X $(GIT_IMPORT).GitCommit=$(GIT_COMMIT)$(GIT_DIRTY) $(LDFLAGS)

 export GOLDFLAGS

I notice that the failure actually occurs compiling part of net/http in go-1.15, so this ought to be happening rather more often.
Working from the assumption that this might be a compiler-compiled-on-Arm problem (because the bootstrap was compiled on another platform), I tried to provoke that:
LD_LIBRARY_PATH=/usr/arm-linux-gnueabihf/lib CC=arm-linux-gnueabihf-gcc  GOSSAHASH=n go install -a cmd/compile

(setting GOSSAHASH forces the compiler to run again) and then
LD_LIBRARY_PATH=/usr/arm-linux-gnueabihf/lib CC=arm-linux-gnueabihf-gcc go test net/http

but no failure.  I retried the packer bug with the self-compiled compiler, no luck (i.e., it built).


update is-empty

is-empty has some bug fixes. A looser version requirement will allow also is-empty to change
potentially the version requirement could be even looser

Thanks!

thanks for the quick response!


[AutoRelease] t2-devcenter-2022-08-15-55992(Do not merge)

https://github.com/Azure/sdk-release-request/issues/3029
Live test success
https://dev.azure.com/azure-sdk/internal/_build?definitionId=5089

issue link:https://github.com/Azure/sdk-release-request/issues/3029


Fix MTU adjustment in case of remote vlan mechanism

The MTU should be updated in connection context based on the host interface MTU in vpp instead of the sub-interface MTU
Related PR: networkservicemesh/cmd-nse-remote-vlan/pull/107
Signed-off-by: Laszlo Kiraly<EMAIL_ADDRESS>

LGTM, but needs a rebase :)


Add instagram to header

What kind of change does this PR introduce?

Does this PR introduce a breaking change?

What needs to be documented once your changes are merged?

👷 Deploy Preview for linneaslustgard processing.

Name
Link

🔨 Latest commit
4c783a6d9e70df41e78f1afc723485d83d687a1e

🔍 Latest deploy log
https://app.netlify.com/sites/linneaslustgard/deploys/627ce2e3ba7b6d000921b384


support symfony5 ?

it seems your package is not yet compatible with symfony5
Restricting packages listed in "symfony/symfony" to "5.*"
Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - dziki/monolog-sentry-bundle v1.0.2 requires symfony/dependency-injection ^3.4|^4.1 -> no matching package found.
    - dziki/monolog-sentry-bundle v1.0.1 requires symfony/dependency-injection ^3.4|^4.1 -> no matching package found.
    - dziki/monolog-sentry-bundle v1.0.0 requires symfony/dependency-injection ^3.4|^4.1 -> no matching package found.
    - Installation request for dziki/monolog-sentry-bundle ^1.0 -> satisfiable by dziki/monolog-sentry-bundle[v1.0.0, v1.0.1, v1.0.2].

Try version 1.1.0-RC1, please.

thanks for the quick reply then I got
    - Conclusion: remove monolog/monolog 2.1.0
    - Conclusion: don't install monolog/monolog 2.1.0
    - dziki/monolog-sentry-bundle v1.1.0-RC1 requires monolog/monolog ^1 -> satisfiable by monolog/monolog[1.0.0, 1.0.0-RC1, 1.0.1, 1.0.2, 1.1.0, 1.10.0, 1.11.0, 1.12.0, 1.13.0, 1.13.1, 1.14.0, 1.15.0, 1.16.0, 1.17.0, 1.17.1, 1.17.2, 1.18.0, 1.18.1, 1.18.2, 1.19.0, 1.2.0, 1.2.1, 1.20.0, 1.21.0, 1.22.0, 1.22.1, 1.23.0, 1.24.0, 1.25.0, 1.25.1, 1.25.2, 1.25.3, 1.25.4, 1.3.0, 1.3.1, 1.4.0, 1.4.1, 1.5.0, 1.6.0, 1.7.0, 1.8.0, 1.9.0, 1.9.1].

ATM this bundle does not support monolog 2.x, sentry handler changed a lot and I'm not sure if this improvements will be still needed - haven't checked yet.

currently i'm based on sf5 and latest version of the sentry sdk for symfony, and unfortunately I still have something very similar to your "before" screen :(

I'll try to add new implementation this weekend then :)

thanks a lot !

looks good to me :+1: thanks a lot