Strange random errors

I am trying to run a couple toy systems, for 100000 steps each.  They did fail/succeeded at different times though) In total: 5 completed, and 7 died (all more than 15000 steps) with the following errors:

2 times this stack trace, click to expand
Traceback (most recent call last):
 File "RETIS_analytical_0.py", line 190, in <module>
   calc.run(steps)
 File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/pathsimulators/path_sampling.py", line 258, in run
   self.sample_set.sanity_check()
 File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/sample.py", line 243, in sanity_check
   assert(sample.ensemble(sample.trajectory))
 File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 2868, in __call__
   return super(TISEnsemble, self).__call__(trajectory, trusted)
 File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 2328, in __call__
   return self._new_ensemble(self._alter(trajectory), trusted)
 File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 1339, in __call__
   fname="__call__"
 File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 1309, in _generalized_short_circuit
   a = f1(trajectory, trusted)
 File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 1585, in __call__
   transitions = self.transition_frames(trajectory, trusted)
 File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 1556, in transition_frames
   subtraj_first, ens_num)
 File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 1639, in _find_subtraj_final
   while ((ens.can_append(subtraj, trusted=True) or
 File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 2335, in can_append
   trusted)
 File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 1349, in can_append
   fname="can_append"
 File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 1327, in _generalized_short_circuit
   b = f2(trajectory, trusted)
 File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 2190, in can_append
   return self._trusted_call(trajectory, self._cache_can_append)
 File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 2180, in _trusted_call
   cache.contents['previous'] = self._volume(frame)
 File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/volume.py", line 204, in __call__
   return not self.volume(snapshot)
 File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/volume.py", line 135, in __call__
   a = self.volume1(snapshot)
 File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/volume.py", line 402, in __call__
   l = self.collectivevariable(snapshot).__float__()
 File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/volume.py", line 204, in __call__
   return not self.volume(snapshot)
AttributeError: 'FunctionCV' object has no attribute 'volume'

It looks like, somehow, the __call__ signature of a FunctionCV get's overwritten by the one of  NegatedVolume
or for 2 others it got overwritten by  the __call__ of CVDefinedVolume:

Stack trace for 2 others, click to expand
Working on Monte Carlo cycle number 15203
Running for 2 days 23 hours 59 minutes 54 seconds - 17.05 seconds per step
Estimated time remaining: 16 days 17.61 hours
Traceback (most recent call last):
  File "RETIS_langevin_0.py", line 213, in <module>
    calc.run(steps)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/pathsimulators/path_sampling.py", line 258, in run
    self.sample_set.sanity_check()
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/sample.py", line 243, in sanity_check
    assert(sample.ensemble(sample.trajectory))
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 2328, in __call__
    return self._new_ensemble(self._alter(trajectory), trusted)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 1339, in __call__
    fname="__call__"
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 1327, in _generalized_short_circuit
    b = f2(trajectory, trusted)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 2216, in __call__
    if not self._volume(frame):
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/volume.py", line 204, in __call__
    return not self.volume(snapshot)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/volume.py", line 402, in __call__
    l = self.collectivevariable(snapshot).__float__()
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/volume.py", line 402, in __call__
    l = self.collectivevariable(snapshot).__float__()
AttributeError: 'FunctionCV' object has no attribute 'collectivevariable'
Error in sys.excepthook:

Another error I encountered,  len() is deferred to snapshot instead of trajectory by netcdf:
Netcdf stacktrace
Working on Monte Carlo cycle number 14283
Running for 2 days 19 hours 16 minutes 42 seconds - 16.96 seconds per step
Estimated time remaining: 16 days 19.79 hours
Traceback (most recent call last):
  File "RETIS_langevin_45.py", line 212, in <module>
    calc.run(steps)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/pathsimulators/path_sampling.py", line 258, in run
    self.sample_set.sanity_check()
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/sample.py", line 243, in sanity_check
    assert(sample.ensemble(sample.trajectory))
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 2868, in __call__
    return super(TISEnsemble, self).__call__(trajectory, trusted)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 2328, in __call__
    return self._new_ensemble(self._alter(trajectory), trusted)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 1339, in __call__
    fname="__call__"
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 1309, in _generalized_short_circuit
    a = f1(trajectory, trusted)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 1585, in __call__
    transitions = self.transition_frames(trajectory, trusted)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 1556, in transition_frames
    subtraj_first, ens_num)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 1639, in _find_subtraj_final
    while ((ens.can_append(subtraj, trusted=True) or
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 2335, in can_append
    trusted)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 1349, in can_append
    fname="can_append"
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 1327, in _generalized_short_circuit
    b = f2(trajectory, trusted)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 2190, in can_append
    return self._trusted_call(trajectory, self._cache_can_append)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 2180, in _trusted_call
    cache.contents['previous'] = self._volume(frame)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/volume.py", line 204, in __call__
    return not self.volume(snapshot)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 2203, in __call__
    if len(trajectory) == 0:
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/netcdfplus/proxy.py", line 82, in __len__
    return len(self.__subject__)
TypeError: object of type 'ToySnapshot' has no len()
Error in sys.excepthook:

Original exception was:

Even more weird stuff: the call of a UnionVolume on a snapshot is somehow resulting in a __getitem__ call from chaindict
random chaindict call
Working on Monte Carlo cycle number 18580
Running for 1 day 17 hours 20 minutes 7 seconds -  8.01 seconds per step
Estimated time remaining: 7 days 13.15 hours
Traceback (most recent call last):
  File "TIS_langevin_0.py", line 210, in <module>
    calc.run(steps)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/pathsimulators/path_sampling.py", line 258, in run
    self.sample_set.sanity_check()
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/sample.py", line 243, in sanity_check
    assert(sample.ensemble(sample.trajectory))
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 2868, in __call__
    return super(TISEnsemble, self).__call__(trajectory, trusted)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 2328, in __call__
    return self._new_ensemble(self._alter(trajectory), trusted)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 1339, in __call__
    fname="__call__"
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 1309, in _generalized_short_circuit
    a = f1(trajectory, trusted)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 1601, in __call__
    if not self.ensembles[subtraj_i](subtraj):
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 2328, in __call__
    return self._new_ensemble(self._alter(trajectory), trusted)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 1339, in __call__
    fname="__call__"
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 1327, in _generalized_short_circuit
    b = f2(trajectory, trusted)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 2216, in __call__
    if not self._volume(frame):
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/volume.py", line 204, in __call__
    return not self.volume(snapshot)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/netcdfplus/chaindict.py", line 229, in __getitem__
    return self._post[items]
AttributeError: 'UnionVolume' object has no attribute '_post'
Error in sys.excepthook:

Original exception was:

And the same happened to NegatedVolume as well:
NegatedVolume._post error
Working on Monte Carlo cycle number 37098
Running for 1 day 13 minutes 28 seconds  -  2.35 seconds per step
Estimated time remaining: 1 day 17.08 hours
Traceback (most recent call last):
  File "2WTSRETIS_langevin_45.py", line 177, in <module>
    calc.run(steps)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/pathsimulators/path_sampling.py", line 228, in run
    movepath = self._mover.move(self.sample_set, step=self.step)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/pathmover.py", line 2559, in move
    self.mover.move(sample_set),
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/pathmover.py", line 1582, in move
    subchange = mover.move(sample_set)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/pathmover.py", line 1582, in move
    subchange = mover.move(sample_set)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/pathmover.py", line 1582, in move
    subchange = mover.move(sample_set)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/pathmover.py", line 613, in move
    change = self.move_core(samples)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/pathmover.py", line 640, in move_core
    trials, call_details = self(*samples)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/pathmover.py", line 786, in __call__
    shooting_index)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/pathmover.py", line 2360, in _run
    mid_traj_shoot_idx)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/pathmover.py", line 2260, in _make_forward_trajectory
    running=[fwd_ens.can_append])
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/engines/dynamics_engine.py", line 423, in generate
    for trajectory in it:
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/engines/dynamics_engine.py", line 610, in iter_generate
    continue_conditions=running)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/engines/dynamics_engine.py", line 377, in stop_conditions
    can_continue = condition(trajectory, trusted)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 2335, in can_append
    trusted)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 1349, in can_append
    fname="can_append"
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 1309, in _generalized_short_circuit
    a = f1(trajectory, trusted)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 1349, in can_append
    fname="can_append"
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 1327, in _generalized_short_circuit
    b = f2(trajectory, trusted)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 1859, in can_append
    return self._generic_can_append(trajectory, trusted, strict=False)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 1752, in _generic_can_append
    trajectory, subtraj_first, ens_num, last_checked
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 1639, in _find_subtraj_final
    while ((ens.can_append(subtraj, trusted=True) or
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 1349, in can_append
    fname="can_append"
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 1309, in _generalized_short_circuit
    a = f1(trajectory, trusted)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 2190, in can_append
    return self._trusted_call(trajectory, self._cache_can_append)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 2168, in _trusted_call
    reset_value = self(trajectory[:-1], trusted=False)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 2216, in __call__
    if not self._volume(frame):
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/netcdfplus/chaindict.py", line 229, in __getitem__
    return self._post[items]
AttributeError: 'NegatedVolume' object has no attribute '_post'
Error in sys.excepthook:

Original exception was:

Do you have any tips on how to debug/solve this further?

Alright a preliminary update:
I first tried

Run fewer simulations simultaneously (thinking that you're memory-limited)

That did not work out as a single simulation would balloon into using the full 64 GB of RAM
then I tried limiting the python process to 8 GB of RAM (by using  ulimit -v 8000000 before running python)
this led to corrupt netcdfs and the following errors:

Memory Error
  File "TIS_langevin_0.py", line 217, in <module>
    s = paths.Storage(save_name, 'a')
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/storage/storage.py", line 64, in __init__
    fallback=fallback)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/netcdfplus/netcdfplus.py", line 347, in __init__
    self._restore_storages()
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/netcdfplus/netcdfplus.py", line 580, in _restore_storages
    storage.restore()
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/storage/stores/snapshot_wrapper.py", line 296, in restore
    super(SnapshotWrapperStore, self).restore()
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/netcdfplus/stores/object.py", line 243, in restore
    self.load_indices()
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/netcdfplus/stores/object.py", line 247, in load_indices
    self.index.extend(self.vars['uuid'][:])
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/storage/stores/snapshot_wrapper.py", line 39, in extend
    self._list.extend(t)
MemoryError

HDF Error
Traceback (most recent call last):
  File "TIS_langevin_45.py", line 219, in <module>
    calc.run(1000)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/pathsimulators/path_sampling.py", line 246, in run
    self.save_current_step()
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/pathsimulators/path_sampling.py", line 112, in save_current_step
    self.storage.steps.save(self._current_step)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/netcdfplus/stores/object.py", line 894, in save
    self._save(obj, n_idx)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/netcdfplus/stores/variable.py", line 54, in _save
    self.write(var, idx, obj)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/netcdfplus/stores/object.py", line 359, in write
    var[int(idx)] = val
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/netcdfplus/netcdfplus.py", line 120, in __setitem__
    self.variable[key] = self.setter(value)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/netcdfplus/netcdfplus.py", line 913, in <lambda>
    '-' * 36 if v is None else str(UUID(int=store.save(v)))
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/netcdfplus/stores/object.py", line 894, in save
    self._save(obj, n_idx)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/storage/stores/movechange.py", line 23, in _save
    self.vars['subchanges'][idx] = movechange.subchanges
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/netcdfplus/netcdfplus.py", line 120, in __setitem__
    self.variable[key] = self.setter(value)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/netcdfplus/netcdfplus.py", line 912, in <lambda>
    if set_is_iterable(v) else \
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/netcdfplus/netcdfplus.py", line 911, in <listcomp>
    for w in list.__iter__(v)]) \
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/netcdfplus/stores/object.py", line 894, in save
    self._save(obj, n_idx)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/storage/stores/movechange.py", line 23, in _save
    self.vars['subchanges'][idx] = movechange.subchanges
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/netcdfplus/netcdfplus.py", line 120, in __setitem__
    self.variable[key] = self.setter(value)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/netcdfplus/netcdfplus.py", line 912, in <lambda>
    if set_is_iterable(v) else \
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/netcdfplus/netcdfplus.py", line 911, in <listcomp>
    for w in list.__iter__(v)]) \
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/netcdfplus/stores/object.py", line 894, in save
    self._save(obj, n_idx)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/storage/stores/movechange.py", line 23, in _save
    self.vars['subchanges'][idx] = movechange.subchanges
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/netcdfplus/netcdfplus.py", line 120, in __setitem__
    self.variable[key] = self.setter(value)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/netcdfplus/netcdfplus.py", line 912, in <lambda>
    if set_is_iterable(v) else \
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/netcdfplus/netcdfplus.py", line 911, in <listcomp>
    for w in list.__iter__(v)]) \
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/netcdfplus/stores/object.py", line 894, in save
    self._save(obj, n_idx)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/storage/stores/movechange.py", line 21, in _save
    self.vars['samples'][idx] = movechange.samples
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/netcdfplus/netcdfplus.py", line 120, in __setitem__
    self.variable[key] = self.setter(value)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/netcdfplus/netcdfplus.py", line 912, in <lambda>
    if set_is_iterable(v) else \
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/netcdfplus/netcdfplus.py", line 911, in <listcomp>
    for w in list.__iter__(v)]) \
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/netcdfplus/stores/object.py", line 894, in save
    self._save(obj, n_idx)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/netcdfplus/stores/variable.py", line 54, in _save
    self.write(var, idx, obj)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/netcdfplus/stores/object.py", line 359, in write
    var[int(idx)] = val
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/netcdfplus/netcdfplus.py", line 120, in __setitem__
    self.variable[key] = self.setter(value)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/netcdfplus/netcdfplus.py", line 913, in <lambda>
    '-' * 36 if v is None else str(UUID(int=store.save(v)))
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/netcdfplus/stores/object.py", line 894, in save
    self._save(obj, n_idx)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/storage/stores/trajectory.py", line 13, in _save
    self.vars['snapshots'][idx] = trajectory
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/netcdfplus/netcdfplus.py", line 120, in __setitem__
    self.variable[key] = self.setter(value)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/netcdfplus/netcdfplus.py", line 926, in <lambda>
    ]) if set_is_iterable(v) else \
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/netcdfplus/netcdfplus.py", line 925, in <listcomp>
    for w in list.__iter__(v)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/storage/stores/snapshot_wrapper.py", line 372, in save
    self._set_id(n_idx, obj)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/storage/stores/snapshot_wrapper.py", line 720, in _set_id
    self.vars['uuid'][idx // 2] = obj.__uuid__
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/netcdfplus/netcdfplus.py", line 120, in __setitem__
    self.variable[key] = self.setter(value)
  File "netCDF4/_netCDF4.pyx", line 4806, in netCDF4._netCDF4.Variable.__setitem__
  File "netCDF4/_netCDF4.pyx", line 4731, in netCDF4._netCDF4.Variable._assign_vlen
  File "netCDF4/_netCDF4.pyx", line 1887, in netCDF4._netCDF4._ensure_nc_success
RuntimeError: NetCDF: HDF error
Error in sys.excepthook:

Original exception was:

I am not familiar enough with 2. to try that. So I went for option 3.
So far (2 simulations running, 1 5000 steps in and one 3500 steps in) the simulations are still growing in memory, but a lot slower (each using about 320 MB so far)
Other than the monkleypatches I had to change one extra thing that might matter:
I had to set interfacesset.cv_max = None after initiating the object in order for it to be able to restart simulations.
I will keep you posted wether or not the new storage also dumps out random errors.

@sroet: Can this be closed? I think the solution was to switch to SimStore (which is now considerably more mature)?

You would think so, unfortunately the latest episode in this saga got triggered without even having a storage:
while running:
equilibration = paths.PathSampling(
    storage=None,
    sample_set=sset,
    move_scheme=testscheme
)

equilibration.run_until_decorrelated()

I hit
Step 54459: 4 of 44 trajectories still correlated
Traceback (most recent call last):
  File "test_new_storage_scripts_45_l.py", line 235, in <module>
    equilibration.run_until_decorrelated()
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/pathsimulators/path_sampling.py", line 225, in run_until_decorrelated
    self.run(1)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/pathsimulators/path_sampling.py", line 305, in run
    self.sample_set.sanity_check()
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/sample.py", line 253, in sanity_check
    assert(sample.ensemble(sample.trajectory))
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 2868, in __call__
    return super(TISEnsemble, self).__call__(trajectory, trusted)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 2328, in __call__
    return self._new_ensemble(self._alter(trajectory), trusted)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 1339, in __call__
    fname="__call__"
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 1309, in _generalized_short_circuit
    a = f1(trajectory, trusted)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 1601, in __call__
    if not self.ensembles[subtraj_i](subtraj):
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 2328, in __call__
    return self._new_ensemble(self._alter(trajectory), trusted)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 1339, in __call__
    fname="__call__"
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 1327, in _generalized_short_circuit
    b = f2(trajectory, trusted)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 2216, in __call__
    if not self._volume(frame):
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/volume.py", line 204, in __call__
    return not self.volume(snapshot)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/volume.py", line 135, in __call__
    a = self.volume1(snapshot)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/volume.py", line 401, in __call__
    l = self.collectivevariable(snapshot).__float__()
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/volume.py", line 135, in __call__
    a = self.volume1(snapshot)
AttributeError: 'CoordinateFunctionCV' object has no attribute 'volume1'

Weird. My gut it still that this might be a memory issue. Try using a SimStore-style CV and set cv.mode = 'no-caching'. (Your CVs aren't too expensive anyway, right?)

Yeah, I am pretty sure it is some form of a memory/caching issue. I already was using simstore-style cvs, but with the mode set to 'production'. I will restart them with no-caching and will let you know how that goes.
Can I alter the mode before starting the production run, or would you just keep them in no-caching mode (they are dirt cheap, just the x coordinate)

The mode should have zero effect on the results of the simulation, so you can modify it if you want. (This is why it isn't immutable.) For a CV that cheap, I honestly don't know how performance will be affected. The CV is still more expensive than a dictionary look-up, which makes production sound faster, but assuming on average about half your snapshots are new at each MC step, no-caching saves you the cost of the cache miss for those.
I would probably bet on production being a little faster. But I wouldn't put much money on it!

That (setting mode to "no-caching") didn't help, it died even earlier this time, any other suggestions on how to run this til completion?
Step 37263: 4 of 44 trajectories still correlated
Traceback (most recent call last):
  File "test_new_storage_scripts_45_l.py", line 235, in <module>
    equilibration.run_until_decorrelated()
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/pathsimulators/path_sampling.py", line 225, in run_until_decorrelated
    self.run(1)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/pathsimulators/path_sampling.py", line 305, in run
    self.sample_set.sanity_check()
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/sample.py", line 253, in sanity_check
    assert(sample.ensemble(sample.trajectory))
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 2868, in __call__
    return super(TISEnsemble, self).__call__(trajectory, trusted)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 2328, in __call__
    return self._new_ensemble(self._alter(trajectory), trusted)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 1339, in __call__
    fname="__call__"
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 1309, in _generalized_short_circuit
    a = f1(trajectory, trusted)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 1585, in __call__
    transitions = self.transition_frames(trajectory, trusted)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 1556, in transition_frames
    subtraj_first, ens_num)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 1639, in _find_subtraj_final
    while ((ens.can_append(subtraj, trusted=True) or
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 2335, in can_append
    trusted)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 1349, in can_append
    fname="can_append"
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 1327, in _generalized_short_circuit
    b = f2(trajectory, trusted)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 2190, in can_append
    return self._trusted_call(trajectory, self._cache_can_append)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/ensemble.py", line 2180, in _trusted_call
    cache.contents['previous'] = self._volume(frame)
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/experimental/simstore/storable_functions.py", line 371, in __call__
    if self.is_scalar(items):
  File "/home/sander/miniconda3/envs/tstis/lib/python3.6/site-packages/openpathsampling/experimental/storage/collective_variables.py", line 23, in is_scalar
    return super(CollectiveVariable, self).is_scalar(item)
TypeError: super(type, obj): obj must be an instance or subtype of type

I honestly have no idea what's going on. It's clear that your types are mutating for some reason. In this last function, the __call__ didn't error because they take the same signature (and kwargs weren't used), but it's clear that self._volume in the last line from ensemble.py is now a CollectiveVariable, not a Volume.
I could only imagine that happening in combination with memory pressure (or possibly UUID conflicts, but that would require something like 2^32 objects instantiated in one session, and you'd have to be regularly creating simulation objects, for these errors to occur -- not just data objects like snapshots). I would have thought we'd gotten rid of the memory pressure by not caching. However, now I think about it, it's possible that all data objects are still getting cached in the SimStore storage, which would still cause memory pressure.
Try adding an occasional storage.cache.clear() (maybe every 1000 steps?). This will only clear the temporary cache, not the simulation objects (which are in the fixed_cache). One approach to that would be to build off of @hejung's path sampling hooks in #911 (that will probably be the long-term way to modify things like this).

So, question; How would I add: storage.cache.clear() if there is no storage attached to my PathSimulator object?
This is running:
equilibration = paths.PathSampling(
    storage=None,
    sample_set=sset,
    move_scheme=testscheme
)

equilibration.run_until_decorrelated()

Ah, yes. Good point. So it definitely isn't caching in there.....
Is it memory pressure? Are you seeing increased memory usage?
I think the only approach here is a deep dive into seeing what's happening. You could do some sanity checking on your objects (recurse down to_dict and be sure nothing has changed compared to an original version). That could be implemented as a hook. You can also try some more detailed logging, combined with rotating log files so they don't take too much disk. Logging, plus maybe even modifying deeper parts of the code to do a sanity check (e.g., at the beginning of a call of can_append, check that everything that should be a volume is a volume) might help identify when this happens, which will hopefully give some insight into why it happens.
If you can use PDB that might also help, although you'll catch the error after the problem has occurred (which is why I suggest using logging).
I'd advise against trying to make any hook like that generically useful. This is one-off debugging; making it generic would be hard work.

If you can use PDB that might also help, although you'll catch the error after the problem has occurred (which is why I suggest using logging).

So the issue with this is that I can't keep a shell alive connected this python process (remote machine, I can keep the connection alive for a couple hours, but not the 3 days it requires for this to trigger)

Is it memory pressure? Are you seeing increased memory usage?

I ran that instance over the weekend, but I am rerunning it now and will keep an eye on it, but it seems rock solid under 0.5%  memory utilization (820 steps in, 32/44 correlated) (it increased by 13 MB between step 300 (41/44 correlated) and step 820 (32/44 correlated trajectories)), this is also on a machine with mutli-bit ECC, so it should not be due to bit-flipping

I think the only approach here is a deep dive into seeing what's happening. You could do some sanity checking on your objects (recurse down to_dict and be sure nothing has changed compared to an original version). That could be implemented as a hook. You can also try some more detailed logging, combined with rotating log files so they don't take too much disk. Logging, plus maybe even modifying deeper parts of the code to do a sanity check (e.g., at the beginning of a call of can_append, check that everything that should be a volume is a volume) might help identify when this happens, which will hopefully give some insight into why it happens.

I was afraid that was going to be the case, kinda hoped you had some clever insights into this. I will start implementing those when/if the current try (monitoring memory) crashes out.

So the issue with this is that I can't keep a shell alive connected this python process (remote machine, I can keep the connection alive for a couple hours, but not the 3 days it requires for this to trigger)

Not sure how details of how it works with the remote machine, but if you can keep a Jupyter process alive, you can use the %pdb magic to turn on pdb debugging, and it'll catch the error. You'll probably want to check in regularly to see if it has crashed, but it might work.

Not sure how details of how it works with the remote machine, but if you can keep a Jupyter process alive, you can use the %pdb magic to turn on pdb debugging, and it'll catch the error. You'll probably want to check in regularly to see if it has crashed, but it might work.

Yeah, so this runs into the same issue when there is no browser open to drop the shell into, if you run the following in jupyter and close the browser window before the crash is triggered and open it up afterward. It just opens up without any output from the cell:
import time
def crash():
    a = 10
    time.sleep(10) #Have time to close the browserwindow
    raise RuntimeError("crash!")

%pdb 1
crash()

Keeping a ssh tunnel alive to do the browser forwarding runs into the same issues as keeping a shell alive in the first place, so this is unfortunately not an option

Memory update: at step 3640, 21/44 trajectories still correlated, memory footprint grew with 184 MB since step 300, still only using 0.6% of total system memory (I will keep updating this comment throughout the day)

Given the update at 3640 (which is about 10% of the way to your last crash), I can't imagine that memory is a problem if you're at 0.6% of total system memory.
Another thing you might try: set up the simulation, save the simulation objects to a file, then run your simulation by reading in the simulation objects and running those.
The idea here would be to create a new UUID space. The UUIDs that @jhprinz designed are split into what is essentially a session ID, plus a counter. (This is why all UUIDs tend to start with the same numbers -- that's the session ID). If somehow your UUIDs for the session are being reused, this would avoid that problem.
I can't image that's actually the issue, because the counter is at least 32 bits (although we only use even numbers of it). Also, I don't see how that would change your simulation objects (more likely, you'd just not save the already-existing UUIDs to storage).

Alright, I have a theory, will run a test to see if it right :P
First, I added a print statement to StorableObject.init() to see why there were so many objects being created. (Please never actually do this, this dumped 100 MB of text for 10 mc steps xD).  Now for 10 mc steps this generated 759_796 uuids (this is only from the point where the simulation actually started (and already divided by 2 to account for the only even numbers)
Now there is no way this can reach the required number of 2^62 uuids before the hashes become equal. Now there is something interesting about these hashes. The uuids are essentially generated in the following way (for other readers, ACTIVE_LONG is used as the initial seed for UUIDs and are incremented by 2 every time a new object is initialized):
import uuid

INSTANCE_UUID = list(uuid.uuid1().fields[:-1])
CREATION_COUNT = 0
ACTIVE_LONG = int(uuid.UUID(
            fields=tuple(
                INSTANCE_UUID +
                [CREATION_COUNT]
            )
        ))

hashed = ACTIVE_LONG &<PHONE_NUMBER>606846975

Now, somehow @jhprinz figured out a way that no matter how many times I run that code (in the same python kernel), the initial hash is identical between the runs, even though ACTIVE_LONG is different every time. And these hashes are actually used to compare if two objects have the same uuid.
If you start a new python proces it does change that hash, so something about the PID or  is seeded in there
Now here is where it goes into the "theory" part, if ever the import_cache from python gets reset/invalidated (say after a couple days of running), so the class StorableObject has to be reimported, it generates a new ACTIVE_LONG seed, but it effectively resets the hash.
I will rerun the code with a print in the part of that class that sets the seed, just to see if it pops up more than once.

Try #975. It should at least get rid of (most of) those NegatedVolume creations.

Try #975. It should at least get rid of (most of) those NegatedVolume creations.

Will try that one if/when the current try (with the extra print statement) crashes out

Just curious, since it's been a few days: Any evidence that #975 #976 #978 are doing anything to help with this?
If so (meaning this was a UUID issue), one thing you should do is to validate that everything is storing correctly later on. My fear is that if, for some reason, our UUID space isn't as large as we thought, then later snapshots might just not save. The way to check this would something like this:
from openpathsampling.experimental.simstore import get_uuid
all_snapshots = set([])
old_uuids = set(get_uuid(snap) for samp in storage.steps[0].active
                for snap in samp.trajectory)
all_snapshots = set(old_uuids)  # start with a copy
for step in storage.steps:
    trial_uuids = set(get_uuid(snap) for samp in step.chance.trials
                      for snap in samp.trajectory)
    new_uuids = trial_uuids - old_uuids
    if any(new in all_snapshots for new in new_uuids):
        print("Crap. That's not good.")

    all_snapshots.update(trial_uuids)
    old_uuids = set(get_uuid(snap) for samp in step.active
                    for snap in samp.trajectory)

Not tested at all, but basic idea is, keep a list of all the UUIDs of snapshots you've ever seen. At any MC step, any newly created snapshots can be id'd by the fact that they weren't in the previous active. If a new snapshot has the same UUID as something you've ever seen before, then there's a problem.
Only checks UUIDs for snapshots, but if it's a problem, that's the most likely place for it to show up.

So, the code that is currently running for 4.5 days (only has #975 and #976) is currently:
Step 67160: 2 of 44 trajectories still correlated

at 10% memory utilization .
So it at least seems to help with that so far :smiley: which means that it is probably an UUID issue indeed. I am pretty sure I saw at least something show at some point that said something like: "Trajectory is not valid for ensemble", but I guess that still might be part of what #978 fixes.
Also, I am pretty sure that the uuid space is big enough (2^128), but that the limited hash space (2^62) might be tripping up. As storage is done based on the full uuid (if I remember correctly), I am reasonably sure that storage should be ok.
I will run that code once the current code starts producing a storage (it is not during the decorrelation steps)

I would still think that 2^62 is a big enough space that we wouldn't have problems. But if somehow we were actually looking at 2^32 (or 2^30), then I could imagine conflicts.

As storage is done based on the full uuid (if I remember correctly)

In SimStore, the UUID is converted to a string, so this should be the case. (This was under the assumption that we may use longer UUIDs are some point; I didn't want to tie it to an integer representation.)
I've been debating whether to tag the PRs resulting from this as "bugfix" (fixing this) or "enhancement" (performance effects). Any thoughts? Mainly makes a difference in release notes, although the precedent might be important: enhancements count more strongly toward potential milestone paper coauthorship. That's already decided for authors of these PRs, but it's worth thinking about how it might affect future contributions.

I would still think that 2^62 is a big enough space that we wouldn't have problems. But if somehow we were actually looking at 2^32 (or 2^30), then I could imagine conflicts.

Yeah, I do agree and 2^32 is suspiciously close to what is observed (2^32 / 37263 (steps in last crash) = 57 632 (definitely reachable, looking at #978). I still have no real idea where that comes from (My latest guess was that I was maybe running 32 bit python, but len(bin(sys.maxsize)) is 65 -> 63 bits so that is not it.)

I've been debating whether to tag the PRs resulting from this as "bugfix" (fixing this) or "enhancement" (performance effects). Any thoughts? Mainly makes a difference in release notes, although the precedent might be important: enhancements count more strongly toward potential milestone paper coauthorship. That's already decided for authors of these PRs, but it's worth thinking about how it might affect future contributions.

So far I might even classify them as misc, this is not 'fixing' the bug (UUID space seems smaller than 64 bit 'somehow', but neither of us really has a clue why), it is just making 'smarter' use of the available UUID space and pushes the triggering of these bugs further away. Also, it does not really change the performance (as far as I could see).

Alright, (hopefully) final update on this from me, a simulation that only had #975 and #976 ran 166_770 steps without a storage, followed by a 100_000 step production run (with a storage). I will run 10 more (similar) simulations over the following weeks, which are run with #978 as well.
I feel fairly certain those PRs pushed the occurrences of these bugs far enough back that it should not matter for most simulations.
It seems (from the PRs that fixed this) to have something to do with the UUID/hash space not being as big as we think it is, but I have no clue why, and no bandwidth to investigate this further.
A summary of things that weren't it in my case:

storage did not matter as these bugs surfaced without any storage attached to the simulations
It was not a memory issue, as the bug surface at ~10% memory utilization with only a single process running
I wasn't running python in 32 bit precision (which will cut hashing values to 32 bits), can be checked by running len(bin(sys.maxsize))
the __uuid__ and  __hash__ space should be big enough for all intents and purposes.

So, this bug seems fixed enough for me that I can close this issue. Someone searching for these errors should end up here at some point. I will reopen this if any of the other 10 simulations fail. @dwhswenson Thanks for all the feedback helping debugging here.

Glad to hear it seems to be working, although I'm also still perplexed. I agree it's likely an issue with UUIDs, although it seems to also involve some sort of caching (to explain a lookup by UUID), though I don't see how.
I'm open to modifications of the UUID generation algorithm, although there's a lot to like about the version @jhprinz wrote. In particular, our UUIDs are defined by high-order bits that are set by session information and low-order bits that are just a counter. This means that (1) you can immediately identify objects from the same session by the first few digits of the UUID, which can be convenient when debugging; and (2) UUID creation is fast, since it is just incrementing an integer.


feat: electron splash screen

Description
Adds splash screen to composer.
Task Item
closes #4047
Screenshots
Windows 10

Mac OS (Catalina)

Linux (Ubuntu 20.04 LTS)

This is awesome! 👍👍👍

Coverage decreased (-0.1%) to 55.377% when pulling 1ae085fabde247ae28c20b1b0cb5ed53927f5485 on hatpick:sorgh/splash_screen into 2315eeb9ad92551c7c1d3ff3fa4c6a50aba58cf1 on microsoft:main.

Very nice @hatpick

This looks so branded and official. Love it!

@mareekuh I talked to Sangwoo about the "brand family" (Microsoft Azure), and he wasn't sure if Composer falls under that family. What do you think? Should I remove the "Microsoft Azure" wording?

@hatpick let's take this into the nightly and rev the UI if need be. Please move out of Draft when you are ready. Thanks!


DataProtection: XmlNamespace, Configuration, Options

After review the following error and the associated assembly source, I'm led to believe this error is associated with configuring, the equivalent machineKey section found in the machine or web.config, DataProtection.
For example the CngCbcAuthenticatedEncryptorConfiguration.cs references the http://www.asp.net/2014/dataProtection/cng xmlnamespace. I also found the CngCbcAuthenticatedEncryptorConfigurationOptions.cs which implies configuring the option in Startup.cs.
One final thought, I aslo found https://support.microsoft.com/kb/2915218 which contains powershell scripts for Generate-MachineKey or Provision-AutoGenKeys.
So, what is the preferred way to configure the DataProtection to avoid the error below?
An unhandled exception occurred while processing the request.
CryptographicException: Key not valid for use in specified state.

Microsoft.AspNet.Security.DataProtection.Cng.DpapiSecretSerializerHelper.UnprotectWithDpapiImpl(Byte* pbProtectedData, UInt32 cbProtectedData, Byte* pbOptionalEntropy, UInt32 cbOptionalEntropy)

Stack
Query
Cookies
400 Headers
Environment
CryptographicException: Key not valid for use in specified state.
Microsoft.AspNet.Security.DataProtection.Cng.DpapiSecretSerializerHelper.UnprotectWithDpapiImpl(Byte* pbProtectedData, UInt32 cbProtectedData, Byte* pbOptionalEntropy, UInt32 cbOptionalEntropy)
Microsoft.AspNet.Security.DataProtection.Cng.DpapiSecretSerializerHelper.UnprotectWithDpapi(Byte[] protectedSecret)
400
Microsoft.AspNet.Security.DataProtection.XmlEncryption.DpapiXmlDecryptor.Decrypt(XElement encryptedElement)
Microsoft.AspNet.Security.DataProtection.AuthenticatedEncryption.CngCbcAuthenticatedEncryptorConfigurationXmlReader.FromXml(XElement element)
Microsoft.AspNet.Security.DataProtection.KeyManagement.XmlKeyManager.ParseKeyElement(XElement keyElement)
Microsoft.AspNet.Security.DataProtection.KeyManagem 400 ent.XmlKeyManager.GetAllKeys()
Microsoft.AspNet.Security.DataProtection.KeyManagement.KeyRingProvider.CreateCachedKeyRingInstanceUnderLock(DateTime utcNow, CachedKeyRing existingCachedKeyRing)
Microsoft.AspNet.Security.DataProtection.KeyManagement.KeyRingProvider.GetCurrentKeyRing()
Microsoft.AspNet.Security.DataProtection.KeyManagement.KeyRingBasedDataProtector.Protect(Byte[] unprotectedData)
400
Microsoft.AspNet.Security.DataHandler.SecureDataFormat`1.Protect(TData data)
Microsoft.AspNet.Security.Google.GoogleAuthenticationHandler.BuildChallengeUrl(AuthenticationProperties properties, String redirectUri)
Microsoft.AspNet.Security.OAuth.OAuthAuthenticationHandler`2.ApplyResponseChallenge()
Microsoft.AspNet.Security.Infrastructure.AuthenticationHandler.ApplyResponseChallengeAsync()
400
Microsoft.AspNet.Security.Infrastructure.AuthenticationHandler.<ApplyResponseCoreAsync>d__58.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
Microsoft.AspNet.Se 400 curity.Infrastructure.AuthenticationHandler.<ApplyResponseAsync>d__57.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
Microsoft.AspNet.Security.Infrastructure.Authentication 400 Handler.<TeardownAsync>d__45.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
Microsoft.AspNet.Security.Infrastructure.AuthenticationHandler.<TeardownAsync>d__45.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
400
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
Microsoft.AspNet.Security.Infrastructure.AuthenticationMiddleware`1.<Invoke>d__11.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
400
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
Microsoft.AspNet.Security.Infrastructure.AuthenticationMiddleware`1.<Invoke>d__11.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
Microsoft.AspNet.Security.Infrastructure.AuthenticationMiddleware`1.<Invoke>d__11.MoveNext()
400
--- End of stack trace from previous location where exception was thrown ---
System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
Microsoft.AspNet.Security.Infrastructure.AuthenticationMiddleware`1.<Invoke>d__11.MoveNext()
400
--- End of stack trace from previous location where exception was thrown ---
Microsoft.AspNet.Security.Infrastructure.AuthenticationMiddleware`1.<Invoke>d__11.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
System.Runtime.Comp 400 ilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
Microsoft.AspNet.Security.Infrastructure.AuthenticationMiddleware`1.<Invoke>d__11.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
Microsoft.AspNet.Security.Infrastructure.AuthenticationMiddleware`1.<Invoke>d__11.MoveNext()
--- End of stack trace from previous 400 location where exception was thrown ---
System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
Microsoft.AspNet.Security.Infrastructure.AuthenticationMiddleware`1.<Invoke>d__11.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
Microsoft.AspNet.Security.Infrastructure.AuthenticationMiddleware`1.<Invoke>d__11.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
400
Microsoft.AspNet.Diagnostics.ErrorPageMiddleware.<Invoke>d__4.MoveNext()
Variable Value
authenticationType Google

< 400 div id="headerspage" class="page"> Variable Value
Accept application/json, text/plain, */*
Accept-Encoding gzip, deflate, sdch
Accept-Language en-US,en;q=0.8
400  Connection keep-alive
Cookie .AspNet.Correlation.Google=VpRqdBW-jM7ZSLM-GqS91j-s58vHN7CAotDK9IsNMYs
Host localhost:8080
Referer http://localhost:8080/
User-Agent Mozilla/5.0 (W 400 indows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.111 Safari/537.36

Created a module for  https://support.microsoft.com/kb/2915218
Generate-MachineKeys
Provision-AutoGenKeys (modified to pull SID from WindowsIdentity.GetCurrent() instead of using UPN)
Run Generate-MachineKeys to create the  machinekey xml needed for web.config.
Updated the web.config in root and wwwroot.
Not sure however why the AppData\local\ASP.NET\ folder started to appear with key. It could have been for executing the Provision-AutoGenKeys cmdlet but I had to modified this cmdlet to not use UPN.
Any ideas?

The <machineKey> element from previous versions of ASP.NET doesn't apply to ASP.NET 5. DataProtection uses its own independent configuration mechanism. Can you elaborate further on your environment? For instance, are you seeing this in IIS Express or full IIS? If it's in full IIS, what user account (app pool identity, network service, etc.) is the application pool running under?

BTW, to immediately unblock yourself, you can delete the contents of the %LOCALAPPDATA%\ASP.NET\ folder.

A few things I know I did before it started working. I granted my id ownerhsip of my hard drive, ran the Provision-AutoGenKey, and deleted the contents of the %LocalAppData%\AspNet folder.


Flesh out README instructions

Hellooo, Justiiiiiin. I tried to flesh out the README more since documentation is something I'm actively trying to practice. Lemme know if you want me to change or clarify anything!

Just the one typo in line 45 of the README. I will need to read over the whole file once more to ensure my understanding, but it all looks more than fine to me in my current understanding!


Docs: Add color palettes in use across OpenSearch Dashboards

Description
and plugin visualizations.
Seeing these color palettes alongside the OUI color palettes and usage guidelines will help us consolidate and make visualizations more cohesive.
Includes link to various hard-coded color values and programmatic color generators.
Reviewers: It should be generally safe to ignore all the new files in the src-docs/src/views/color_palette/osd_legacy_color_maps/ directory, except for the README - these were directly copied from other OpenSearch Dashboard projects for ease of collecting the palettes.

Next steps
To avoid confusion, I may refactor this to be a cloned page that isn't linked from the navigation, so that it can be safely merged and deployed as a reference, but won't confuse users just looking for the current guidance.
I also need to investigate colors in other plugins such as Alerting, Anomaly Detection, and Maps.
Issues Resolved
fixes https://github.com/opensearch-project/OpenSearch-Dashboards/issues/2586
Check List

[ ] New functionality includes testing.
[ ] New functionality has been documented.
[ ] All tests pass

[ ] yarn lint
[ ] yarn test-unit

[ ] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@joshpalis Want to finish/close this?
[Catch All Triage w/ 1, 2, 3]


Basic authorship pending txs limiter

Motivation
According to this hack https://github.com/PureStake/substrate/commit/fdedb87add4f0215f9c0fd6c75ac817669dc486b
https://github.com/PureStake/substrate/blob/fdedb87add4f0215f9c0fd6c75ac817669dc486b/client/basic-authorship/src/basic_authorship.rs#L358-L366
Do you mind that block producer requires some limiter in pending transaction queue?

I'm giving a bit more context.
What we observed was that some collators didn't include many tx in the block even having a full pool. The timing of the block production was under the 333ms allocated, and the weight was always < 30%.
We did some logging and observed that even if the pool ready queue contained 1000 items, only few were pushed to the block.
On top of that we discovered that the re-validation was triggered always during the block production (even if we changed the 200ms loop delay to 20s).
I suspected that somehow the revalidation impacted the pending tx iterator. So getting all the txs quickly before processing them would allow more tx in the block (and we confirmed that after doing that, we had stable and faster txs/block)

So getting all the txs quickly before processing them would allow more tx in the block (and we confirmed that after doing that, we had stable and faster txs/block)

I already had some testing branch to make the iterator more independent and not being influenced by a potential revalidation. However, I first need a proper benchmark to ensure that this really improves the situation.

@bkchr So far, my only benchmark is to send 1000tx in the pool and see how many get into the block, with the bug, you will see inconsistent result: a block at 250, another one at 20, and ....
With the fix I did on our branch, it stays consistent (250, 243, 265...)
Here is my script (slightly modified):
// This script is expected to run against a parachain network (using launch.ts script)

import { Keyring } from "@polkadot/api";
import { SubmittableExtrinsic } from "@polkadot/api/promise/types";

import yargs from "yargs";

const argv = yargs(process.argv.slice(2))
  .usage("Usage: $0")
  .version("1.0.0")
  .options({
    url: {
      type: "string",
      description: "Websocket url",
      conflicts: ["network"],
      string: true,
    },
    from: {
      type: "string",
      description: "Private key to transfer from",
      demandOption: true,
    },
    count: {
      type: "number",
      default: 1000,
      description: "Number of transactions",
      demandOption: true,
    },
  }).argv;

const sendAllAndWaitLast = async (extrinsics: SubmittableExtrinsic[]) => {
    return new Promise(async (resolve, reject) => {
      console.log(`Preparing to send ${extrinsics.length} extrinsics`);
      for (let i = 0; i < extrinsics.length; i++) {
        if (i == extrinsics.length - 1) {
          const unsub = await extrinsics[i].send((result) => {
            if (result.isError) {
              reject(result.toHuman());
            }
            if (result.isInBlock) {
              console.log(`Last extrinsic submitted`);
              unsub();
              resolve(null);
            }
          });
        } else {
          await extrinsics[i].send();
        }
        if (i % 100 == 0) {
          console.log(`Sending extrinsic: ${i}...`);
        }
      }
      console.log(`Waiting for last extrinsic...`);
    });
  };

const main = async () => {
  const wsProvider = new WsProvider("wss://wss.moonriver.moonbeam.network");
  const polkadotApi = await ApiPromise.create({
    provider: wsProvider,
    typesBundle: typesBundle as any,
  });

  const keyring = new Keyring({ type: "ethereum" });
  const fromAccount = await keyring.addFromUri(argv.from);
  console.log(`Using funds from ${fromAccount.address}`);

  let fromNonce = (await polkadotApi.rpc.system.accountNextIndex(fromAccount.address)).toNumber();

  console.log(`Creating ${argv.count} balance tranfers of ${argv.count} Tokens...`);
  const transferTxs = await Promise.all(
    new Array(argv.count).fill(0).map((_, index) => {
      return polkadotApi.tx.system
        .remark("ok")
        .signAsync(fromAccount, { nonce: fromNonce + index });
    })
  );

  // Send the transfer transactions and wait for the last one to finish
  await sendAllAndWaitLast(transferTxs);

  await polkadotApi.disconnect();
  console.log(`Finished`);
};

main();

Should be solved now.


Add Nodejs changes to development branch

Minimum accepted port should be 1024

Move bigTest.json to a folder named test inside the root src

validateJSON should not be aware of visualise : the function comment and invalid error message should be modified

validatePort should not be aware of setPort: the function comment should be modified

for the standard JSON tests, the code for each test is similar and could avoid repetition with the use of a helper function

add a code coverage tool eg. Coveralls

Issues:
Coveralls token is not in an environment variable. No solution found online so far for storing environment variables in github. Secrets seems to only work for workflows

@halcon-blanco Nice!


Camera motion blur not working in appleseed.studio

When opening a project containing camera motion blur in appleseed.studio,
the camera animated transform is reset at some point.
Rendering using appleseed.cli or the API is not affected.

Fixed.


Creating a group for decomposition-based methods and adding decomposition-based methods.

Types of changes

[ ] Docs change / refactoring / dependency upgrade
[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Motivation and Context / Related issue

How Has This Been Tested (if it applies)

Checklist

[x] The documentation is up-to-date with the changes I made.
[x] I have read the CONTRIBUTING document and completed the CLA (see CLA).
[x] All tests passed, and additional code has been covered with new tests.

Rewrittten in 923 and 927.


Issue 167: Adjust top Makefile dependency ordering in submodule initialization

This Pull Request resolves the bug described in #167 .
Coordination

[x] Pull Request is against correct branch
[x] Pull Request is started in Draft status
[x] CI passes in CASE feature branch against develop
[x] CI passes in CASE current unstable branch tracking UCO's unstable as submodule (d2e81b1)
[x] Impact on SHACL validation reviewed for CASE-Corpora
[x] Impact on SHACL validation remediated for CASE-Corpora (N/A)
[x] Impact on SHACL validation reviewed for CASE-Examples
[x] Impact on SHACL validation remediated for CASE-Examples (N/A)
[x] Impact on SHACL validation reviewed for casework.github.io
[x] Impact on SHACL validation remediated for casework.github.io (N/A)
[x] Milestone linked
[x] Solutions Approval vote logged on corresponding Issue (N/A)

This PR is ready for review and merge.


No license

Hello, could you please provide a license? Technically code/models without a license are not open source and one cannot use/build upon them

Thanks, I forgot


Auto-detect src and/or test directories

Some people keep their source in a src directory and some people keep their tests in test. This seems to up often enough that we should either detect this and automatically do something or warn people they should create a .env file that manipulates their PYTHONPATH to include the appropriate directories.

and some people keep their tests in test.

We already have some code that looks for directories named test (or similar) and we display a prompt asking the user whether they'd like to configure VS Code to discover and run the tests.

There is no problem with detection of the test! I can run the test by simply clicking on "Run Test" or typing "pytest" from shell and there is no need of adding any .env file.
The problem arises when you want to debug your test:

If you just use the default "Python" lanucher and press F5, suddenly it cannot import the target module.
This can be solved by the following launcher, but the problem is that it will run all the test cases that you have until it reaches the one you are trying to debug.

{
            "name": "PyTest",
            "type": "python",
            "request": "launch",
            "stopOnEntry": false,
            "pythonPath": "${config:python.pythonPath}",
            "module": "pytest",
            "args": [
                "-sv"
            ],
            "program": "${file}",
            "cwd": "${workspaceFolder}",
            "env": {},
            "envFile": "${workspaceFolder}/.env",
            "debugOptions": [
                "RedirectOutput"
            ]
},

You can still use the "Debug test" (link/button) on top of the test case (using default python launcher) and it will end up on your break point without adding .env file. (Do you see the inconsistency? it is some how able to import the module unlike pressing F5)
But of course it still has problems (you can see it in python test log):

Exception ignored in: <async_generator object _ag at 0x04B81BF0>
Traceback (most recent call last):
  File "C:\Python36\lib\types.py", line 27, in _ag
  File "C:\Users\Amir\.vscode\extensions\ms-python.python-2018.1.0\pythonFiles\PythonTools\ptvsd\visualstudio_py_debugger.py", line 971, in trace_func
  File "C:\Users\Amir\.vscode\extensions\ms-python.python-2018.1.0\pythonFiles\PythonTools\ptvsd\visualstudio_py_debugger.py", line 1014, in handle_call
  File "C:\Users\Amir\.vscode\extensions\ms-python.python-2018.1.0\pythonFiles\PythonTools\ptvsd\visualstudio_py_debugger.py", line 576, in should_debug_code
TypeError: 'NoneType' object is not callable

If you try it several times you will end up in a dangling debugger that you cannot even proceed with debugging!

I have attached my simple try out structure along with my lanuch.json so you can reproduce this:
I am using Windows 10 and latest VScode and ms-python extension.
testing_pytest_debugger.zip

@WindsOfHades please create a separate issue. This issue is not about bugs, it gets very difficult to manage issues when there's a lot of unrelated information in the same place.

We might want to update our documentation too (see #1328)

We already have some code that looks for directories named test (or similar) and we display a prompt asking the user whether they'd like to configure VS Code to discover and run the tests.

I take that back, the auto detection of tests was take out of the extension https://github.com/Microsoft/vscode-python/commit/99ff84cd09e683cee9588545476cbfd3aa899775
I guess this was removed as users found the prompts too annoying.

If we do go back with the auto-prompt, we might want to make the prompt more subtle. As it is we have a linter prompt. I guess it would be a review of the extension startup (setup) experience.

https://twitter.com/r1chardj0n3s/status/1019521759156924416 suggests at least on Twitter people like to use src.

All our projects have src and test folders, and now intellisense failing to discover these code. Is there any "work around" to make it work?

All our projects have src and test folders, and now intellisense failing to discover these code. Is there any "work around" to make it work?

Please create a separate issue for your.

I use src/ directories for my libraries. I think the addressing https://github.com/microsoft/vscode-python/issues/4954 would also fix this issue more directly than workarounds like suggesting PYTHONPATH modification.

@sloria I'm working on something that will improve import detection when editable installs are used; i.e. if a package has been installed as editable, then the language server will
recognize that and add it to its list of user import paths automatically, as opposed to the situation right now where editable installs work but still require extraPaths/PYTHONPATH to be set.
It has the downside of only working if something has been editable installed at all (so not if you just open any project), but would that be helpful/in the vein of what you were interested in?

Please also include the workspace folder directory itself. (the one referred using PYTHONPATH=.)

If you are talking to me, that's actually already done in the LS; we always have the workspace root as an import path (non-optional).
Also, linking this, since it involves auto-configuration for src directories: microsoft/python-language-server#1440

I was referring to the person who works on this in general, not just LS. I just noticed that I had to explicitly add PYTHONPATH=.. for tests to be discovered, and it was not auto-detecting modules in the root directory.

https://github.com/microsoft/vscode-python/issues/6544

Why close https://github.com/microsoft/vscode-python/issues/5710 for this one?
Nobody is going to use src and test directories all the time, it forces an unrealistic directory structure on Python projects.
Seems more like we need the app to recognize local modules the same way Python itself does without going through hoops of setting up additional, unnecessary files.

Nobody is going to use src and test directories all the time, it forces an unrealistic directory structure on Python projects.

This issue is about auto-detection. If you do not use this structure (most projects do not use src, many use test), you are not affected. The language server has done this for about 8 months to good effect. Jedi's import resolution is looser, and may resolve things more forgivingly. Other tooling (like linters) may or may not be able to do this yet.

Seems more like we need the app to recognize local modules the same way Python itself does without going through hoops of setting up additional, unnecessary files.

This is a pretty difficult thing to get correct in all cases. The biggest offenders are scripts; it's not generically possible to know if any given .py file is intended to be imported from somewhere else or run as a script, nor can we really know from where a user is intending to run their code (where imports are rooted, which we typically assume to be the workspace root). If it's a script, then the interpreter will add the script's directory as a search path. But the editor doesn't know this is going to happen 100% of the time. Tools like pytest may also modify the search path. And if the user has a src directory, or some other format, it can't know for certain if a given import is going to resolve or not.
Anything automatic/default is a heuristic to be improved, but the notion of configuring search paths is not uncommon in editors that need to handle Python, as there is no perfect algorithm that can both provide good feedback about errors and handle the wide range of cases.

When you follow the same import rules as Python, it is really simple.
A file or package in the same directory should be importable normally in the current file.
It’s really not much to ask to silence an incorrect warning that all other editors seem to get right.

On Jun 5, 2020, at 13:35, Jake Bailey<EMAIL_ADDRESS>wrote:

Nobody is going to use src and test directories all the time, it forces an unrealistic directory structure on Python projects.
This issue is about auto-detection. If you do not use this structure (most projects do not use src, many use test), you are not affected. The language server has done this for about 8 months to good effect. Jedi's import resolution is looser, and may resolve things more forgivingly. Other tooling (like linters) may or may not be able to do this yet.
Seems more like we need the app to recognize local modules the same way Python itself does without going through hoops of setting up additional, unnecessary files.
This is a pretty difficult thing to get correct in all cases. The biggest offenders are scripts; it's not generically possible to know if any given .py file is intended to be imported from somewhere else or run as a script, nor can we really know from where a user is intending to run their code (where imports are rooted, which we typically assume to be the workspace root). If it's a script, then the interpreter will add the script's directory as a search path. But the editor doesn't know this is going to happen 100% of the time. Tools like pytest may also modify the search path. And if the user has a src directory, or some other format, it can't know for certain if a given import is going to resolve or not.
Anything automatic/default is a heuristic to be improved, but the notion of configuring search paths is not uncommon in editors that need to handle Python, as there is no perfect algorithm that can both provide good feedback about errors and handle the wide range of cases.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or unsubscribe.

Pylance picks up on src/ directories already, so closing as outdated.


[Frontend][VLM] Add support for multiple multi-modal items in the OpenAI frontend

This PR is an iteration of #7923 and in particular simplifies the multimodal placeholder padding logic.
Note that now we use a dictionary to keep track of unique placeholders and their counts, and the order of insertion is still preserved when we add them to the input prompt if they're missing. This is important for models that have specific ordered image tokens (e.g., <|image_1|><|image_2|>).
Co-authored-by: @petersalas

 PR Checklist (Click to Expand)
Thank you for your contribution to vLLM! Before submitting the pull request, please ensure the PR meets the following criteria. This helps vLLM maintain the code quality and improve the efficiency of the review process.
PR Title and Classification
Only specific types of PRs will be reviewed. The PR title is prefixed appropriately to indicate the type of change. Please use one of the following:

[Bugfix] for bug fixes.
[CI/Build] for build or continuous integration improvements.
[Doc] for documentation fixes and improvements.
[Model] for adding a new model or improving an existing model. Model name should appear in the title.
[Frontend] For changes on the vLLM frontend (e.g., OpenAI API server, LLM class, etc.)
[Kernel] for changes affecting CUDA kernels or other compute kernels.
[Core] for changes in the core vLLM logic (e.g., LLMEngine, AsyncLLMEngine, Scheduler, etc.)
[Hardware][Vendor] for hardware-specific changes. Vendor name should appear in the prefix (e.g., [Hardware][AMD]).
[Misc] for PRs that do not fit the above categories. Please use this sparingly.

Note: If the PR spans more than one category, please include all relevant prefixes.
Code Quality
The PR need to meet the following code quality standards:

We adhere to Google Python style guide and Google C++ style guide.
Pass all linter checks. Please use format.sh to format your code.
The code need to be well-documented to ensure future contributors can easily understand the code.
Include sufficient tests to ensure the project to stay correct and robust. This includes both unit tests and integration tests.
Please add documentation to docs/source/ if the PR modifies the user-facing behaviors of vLLM. It helps vLLM user understand and utilize the new features or changes.

Notes for Large Changes
Please keep the changes as concise as possible. For major architectural changes (>500 LOC excluding kernel/data/config/test), we would expect a GitHub issue (RFC) discussing the technical design and justification. Otherwise, we will tag it with rfc-required and might not go through the PR.
What to Expect for the Reviews
The goal of the vLLM team is to be a transparent reviewing machine. We would like to make the review process transparent and efficient and make sure no contributor feel confused or frustrated. However, the vLLM team is small, so we need to prioritize some PRs over others. Here is what you can expect from the review process:

 After the PR is submitted, the PR will be assigned to a reviewer. Every reviewer will pick up the PRs based on their expertise and availability.
 After the PR is assigned, the reviewer will provide status update every 2-3 days. If the PR is not reviewed within 7 days, please feel free to ping the reviewer or the vLLM team.
 After the review, the reviewer will put an  action-required label on the PR if there are changes required. The contributor should address the comments and ping the reviewer to re-review the PR.
 Please respond to all comments within a reasonable time frame. If a comment isn't clear or you disagree with a suggestion, feel free to ask for clarification or discuss the suggestion.

Thank You
 Finally, thank you for taking the time to read these guidelines and for your interest in contributing to vLLM. Your contributions make vLLM a great tool for everyone!

I've added an additional test to cover too many images across messages, so this PR should be good to go.


Math magicians: events

I did the following in Math magicians: events

Copy the files calculate.js and operate.js into a logic/ directory in your project.
Analyze the files calculate.js and operate.js, be sure you understand what they do (note: you need to add big.js as dependency).
Import the files in your Calculator component.
Implement the event handlers you need to use the math logic from calculate.js and operate.js in your React component.
Practice what you have learned until now: use state and props, pass props to child components, use lifecycle methods, lift state up.
Test your app in the browser. It should be able to perform math operations.

@OLIPLICHE

Checked if I have met requirements and Approved.


Incopatibility with @storybook/builder-vite

Describe the bug
Using storybook-addon-next in combination with builder-vite produces an error once storybook server started.
Reproduce repo link.
I've added instruction to start.
All packages have latest versions.
A warning message:
Warning: ReactDOM.render is no longer supported in React 18. Use createRoot instead. Until you switch to the new API, your app will behave as if it's running React 17. Learn more:

Error message is somehow related with next-image-stub.js:
Uncaught TypeError: Invalid Version: undefined
    at new SemVer (semver.js:19:13)
    at compare (compare.js:3:3)
    at Object.gt (gt.js:2:29)
    at node_modules/storybook-addon-next/dist/images/next-image-stub.js (next-image-stub.js:7:22)
    at __require2 (chunk-S5KM4IGW.js?v=7ad206d8:18:50)
    at node_modules/storybook-addon-next/dist/preview.js (preview.js:7:1)
    at __require2 (chunk-S5KM4IGW.js?v=7ad206d8:18:50)
    at dep:storybook-addon-next_dist_preview__js:1:16

Your minimal, reproducible example
Branch bug:
https://github.com/dvakatsiienko/next-vite-storybook/tree/bug
Steps to reproduce

Clone repository
yarn
yarn storybook:start
Once Vite bundled, go to http://localhost:6006/
Inspect an error in console

Expected behavior
As a user I expect a correctly rendered page.
How often does this bug happen?
Every time
Screenshots or Videos
https://user-images.githubusercontent.com/15727958/192000483-00c4dd09-4123-46cc-8772-d5f12be5a7ba.mp4
Platform

OS MacOS
All packages latest versions

storybook-addon-next version
1.6.9
Additional context
No response

Thanks for opening this.
As of right now this addon doesn't support the vite builder because nextjs doesn't use vite and it's a lot for just me to maintain just the webpack version :(
Until I can get more help with this package, I can't take on that work :( sorry


🔧 MAINTAIN: fix test_tube emoji typo

This PR updated :testube: emoji to :test_tube:.

thanks for catchign this! totally missed the PR, sorry!


Theme on live site referencing files from local server. #2

Problem
working with slate breaks the live site.
I know there is another closed thread #736
But I need to reconfirm or get some tipps:
How do you deal with this if a website is already live with customers and you want to change things, the site always brakes for visitors? — or you are not even live yet, but the clients uploads products and wants to see them on the frontend while you work on it?
— a feature?
Replication steps
npm run slate

Chris from our team wrote a really useful articles on how we work with Themes at Shopify here : https://www.shopify.ca/partners/blog/development-workflow
The Working with theme IDs and Configuring one or multiple environments parts should be useful for your case

Oh okay, great thanks!
So I would get one production theme and one dev theme (for example):
work live on the dev theme and deploy to the production theme?

Here is our typical workflow...

We have a separate development shop (shopname-dev.myshopify.com) with most of the same products and setup as our live store.  However, there we can run tests with obscure product variant combinations, site settings and more.  We're ok if things break there, or we mess something up. This might not be necessary for your situation.
Once we've moved on from the bulk development and testing, we will duplicate our live shop's theme (shopname.myshopify.com), and run Slate to test the changes on the unpublished theme.
After verifying that everything works well we deploy (Slate uploads all assets to the server) to the unpublished theme.
Test again.
Publish the theme update.

thanks. now everything is clear.
i chose to have two themes on the shop, one production and one as a duplicate for development.
i was wondering why
npm run start --env=development
is not working, but
yarn start --evn=development
does?
i think it's not a big problem, but a bit confusing, because the documentation states that you can use either npm or yarn.

@fabianwohlfart It would depend which tool you used to install it, if that's what you mean? If you installed Slate with Yarn, then you'll have to use Yarn because npm doesn't have access to the program.


Navigator view seems to have a rather large range for allowing users to "pull down" on the view.

Is there any way to configure the height (or bounding box dimensions) of the drag area to allow users to pull a view down?  This seems to get in the way for things like list views.
Demo:

Sorry, this API is pretty lame and undocumented. We are currently overhauling the API to configure gestures at the navigator level and use the new Animated API to animate scenes.
The edge touch distance is specified in the scene config:
https://github.com/facebook/react-native/blob/master/Libraries/CustomComponents/Navigator/NavigatorSceneConfigs.js#L528
To override it:
<Navigator
  configureScene={scene => ({
    ...Navigator.SceneConfigs.FloatFromBottom,
    gestures: {
      pop: {
        ...Navigator.SceneConfigs.FloatFromBottom.gestures.pop,
        edgeHitWidth: 60,
      },
    },
  })}
  ...
/>
```


smack: remove "User" domain

The entire "User" domain support has been tentative. It depends a lot
on the actual application framework and thus is neither necessarily
the same everywhere nor absolutely needed. For example, Ostro OS does
not need it.
Support for it also was incomplete, probably even in Tizen, leading to
problems even when a distro has no use for the "User" domain: in Ostro
OS, the PAM plugin sets the process label to "User" even when logging
in as root, and then when "passwd" is used to modify /etc/passwd, the
new file has "User" as Smack label because there is no transmute Smack
rule for User -> System::Shared and thus the transmute flag on /etc
gets ignored.
Instead of trying to fix this, it is easier to simplify the entire
layers by removing the "User" labels and rules. What remains is
hard-coded in the upstream recipes.
If someone need support for a "User" domain, they can add that to
their custom distro together with the code that depends on it or (when
that is not easy, for example in the smacknet tool) propose patches
which add back what is needed in a way that a) it is properly
documented and b) optional.
@jobol @ipuustin please review. I also need to do some more testing
myself.

If we plan on not using User domain anymore by default, maybe we should also remove that from meta-security-smack/recipes-core/base-files/base-files_%.bbappend

Thinking about this since yesterday...
I think that it's cool to have User context when logged. But if you don't need it....
From your comment I read that

it is not proven to be fully functional
it bother humans when editing files

Killing the User domain only for the second reason doesn't make sense.
But telling that nobody wants to work on that topic to make it work or that it works without it can be heard. What is your meaning @cschaufler?

On Thu, 2016-04-21 at 05:09 -0700, acornea wrote:

If we plan on not using User domain anymore by default, maybe we
should also remove that from
meta-security-smack/recipes-core/base-files/base-files_%.bbappend

What is left from User there? The commit removes "User" also from that
file.

Apologies, missed that in between the many red lines :)

On Thu, 2016-04-21 at 05:21 -0700, jobol wrote:

From your comment I read that
 1. it is not proven to be fully functional
 2. it bother humans when editing files

The problem which broke the camel's back is that calling simple commands
like "passwd" leads to unexpected results.

Killing the User domain only for the second reason doesn't make sense.

It does. Why fix a problem in something that isn't even needed?

But telling that nobody wants to work on that topic to make it work or
that it works without it can be heard. What is your meaning
@cschaufler?

Casey was the one who suggested that the "User" domain should be removed
because it is not used in Ostro.
I'll probably go one step further and also remove unused components like
Cynara and SecurityManager. They are just a maintenance burden at this
point, and if someone needs them, they can copy the recipes and continue
maintaining them elsewhere.

Okay. I would like to have a PAM module for smack that has the configuration to set a label to User or to anything else. It is possible I think. For the remaining, it is up to the distro to set its rules.

@jobol if you need such a PAM plugin and can't find someone to maintain it for you, then you will have to maintain it yourself in your own layer.

retest this please

Force pushed a fixed "smack: remove "User" domain" patch (smack label test failed because now /etc/skel is no longer special and inherits the transmute flag).
Also added much better error output when the smack label test fails. Perhaps that'll help understand why /var/log ends up with a different label than expected (seen elsewhere, for example in https://ostroproject.org/jenkins/job/dsl-meta-ostro_pull-requests/295/).


[Question] Is there a way to stop publishing the compressed image topics?

Hello,
I would like to know if there is a way (ideally through the parameter configuration file) to stop publishing the compressed topics for each component. I know that depthai is using the image_transport package for the publishing of the imaging, but I could not figure out how to disable the compression.
/oak/nn/passthrough/image_raw/compressed
/oak/nn/passthrough/image_raw/compressedDepth
/oak/nn/passthrough/image_raw/theora
/oak/rgb/image_raw/compressed
/oak/rgb/image_raw/compressedDepth
/oak/rgb/image_raw/theora

I need this information because every time I subscribe to one of those topics (for example when opening Foxglove where I can not bypass some of the topics), my terminal is flooded with errors, such as:
[ERROR] [1704743237.583933472] [compressed_depth_image_transport]: Compressed Depth Image Transport - Compression requires single-channel 32bit-floating point or 16bit raw depth images (input format is: bgr8).
[ERROR] [1704743343.808687872] [CompressedPublisher]: [16UC1] is not a color format. but [mono8] is. The conversion does not make sense
[ERROR] [1704743421.184808416] [TheoraPublisher]: cv_bridge exception: '[16UC1] is not a color format. but [bgr8] is. The conversion does not make sense'

My initial approach was like this:
/oak:
  ros__parameters:
    rgb:
      image_raw:
        disable_pub_plugins: 'image_transport/compressed'

but from what I understand the "disable_pub_plugins" is not yet available in ROS2 Humble.
If you have any suggestions or approaches for such a problem, please let me know! Thank you!

Hi Adam,
Thanks for the answer, I will try the intra-process mode for now.


sysDescr remove from hardcoded base collection and use it from profil…

…e definitions. This partially fix #461 by avoiding "sysDescr" OID from the list of hardcoded OIDs used as a base list for polling. There is no need to hardcode it here, when it can be used from profile, like system-mib.yml which is anyway used for every device.

Any thoughts on this @RyanKadri

What's the reason behind taking just sysdesc out here? I guess I'm confused why you would just do this and not the rest too? For example, look at https://github.com/kentik/ktranslate/compare/kentik?
Can you give a little more color as to what you're hopping to achieve here? Is is bad to poll sysdesc?

I need to perform "conversion" with "regex" on the sysDescr and extract the information which is inside the sysDescr. In order to do it, I need to be able to specify sysDescr in the profiles and have it "usable" from profiles. At the moment, any configuration of the sysDescr OID inside the profiles is ignored, since it is added in the "hardcoded" way.
I removed only sysDescr as I don't have any specific needs from other hardcoded values and for the moment I can used them the way they are exposed. But in general case, it would be preferable using profiles instead of hardcoded behavior. As I don't understand the code fully and I don't know where this is used or important on some other places or behaviors,I just wanted to minimize the potential impact on other functionalities relying on this code.

https://github.com/kentik/ktranslate/compare/kentik
Does this get you more what you are looking for?


DO NOT MERGE - Stop sending empty files to EBI

Still need to test this out, but I think this will do the trick
Fix #1070

Is this really the best way of checking for empty files? I think it will be better if when we are generating the per sample fastq we detect that a sample do not have sequences and we don't write that file out. Does that makes sense?

@josenavas, it may depend on how that code is written as to if it is easier

@josenavas yes, I think that would be better, but I thought the per-sample fastq files were being generated by QIIME.
@wasade I think using os.stat(file_path).st_size would be fine.

@adamrp if you don't add the filter, in the live system the warning will be raised only once, (the first time), since this is the default behavior for warnings in python.

Okay, thanks! I added the filter.

Actually, checking st_size == 0 is not sufficient, since a gzipped empty file is not zero bytes:
(python276)adam:test@ebi_empty_files-issue-1070$ touch hello
(python276)adam:test@ebi_empty_files-issue-1070$ du -h hello
  0B	hello
(python276)adam:test@ebi_empty_files-issue-1070$ gzip hello
(python276)adam:test@ebi_empty_files-issue-1070$ du -h hello.gz
4.0K	hello.gz

And not all empty files have the same md5 (the name of the file is encoded):
(python276)adam:test@ebi_empty_files-issue-1070$ touch test test2
(python276)adam:test@ebi_empty_files-issue-1070$ gzip test test2
(python276)adam:test@ebi_empty_files-issue-1070$ md5 test.gz test2.gz
MD5 (test.gz) = 6976ece38ea13638c2c4724da1fdd640
MD5 (test2.gz) = a84c25cf07c3c8fbd22e6c6d69c3f615
(python276)adam:test@ebi_empty_files-issue-1070$ xxd test.gz
0000000: 1f8b 0808 684a 4d55 0003 7465 7374 0003  ....hJMU..test..
0000010: 0000 0000 0000 0000 00                   .........
(python276)adam:test@ebi_empty_files-issue-1070$ xxd test2.gz
0000000: 1f8b 0808 684a 4d55 0003 7465 7374 3200  ....hJMU..test2.
0000010: 0300 0000 0000 0000 0000                 ..........

So I think the best we can do is open (or gzip.open) the file, and check if calling read on it returns the empty string

Perhaps it is worth checking if its easy to just not produce the files?
On Fri, May 8, 2015 at 5:45 PM, adamrp<EMAIL_ADDRESS>wrote:

And not all empty files have the same md5 (the name of the file is
encoded):
(python276)adam:test@ebi_empty_files-issue-1070$ touch test test2
(python276)adam:test@ebi_empty_files-issue-1070$ gzip test test2
(python276)adam:test@ebi_empty_files-issue-1070$ md5 test.gz test2.gz
MD5 (test.gz) = 6976ece38ea13638c2c4724da1fdd640
MD5 (test2.gz) = a84c25cf07c3c8fbd22e6c6d69c3f615
(python276)adam:test@ebi_empty_files-issue-1070$ xxd test.gz
0000000: 1f8b 0808 684a 4d55 0003 7465 7374 0003  ....hJMU..test..
0000010: 0000 0000 0000 0000 00                   .........
(python276)adam:test@ebi_empty_files-issue-1070$ xxd test2.gz
0000000: 1f8b 0808 684a 4d55 0003 7465 7374 3200  ....hJMU..test2.
0000010: 0300 0000 0000 0000 0000                 ..........
So I think the best we can do is open (or gzip.open) the file, and check
if calling read on it returns the empty string
—
Reply to this email directly or view it on GitHub
https://github.com/biocore/qiita/pull/1169#issuecomment-100395100.

Thought that was coming from QIIME, but I might be mistaken. I'll have a look.

I thought we were generating from the demux files?
On May 8, 2015 5:54 PM, "adamrp"<EMAIL_ADDRESS>wrote:

Thought that was coming from QIIME, but I might be mistaken. I'll have a
look.
—
Reply to this email directly or view it on GitHub
https://github.com/biocore/qiita/pull/1169#issuecomment-100395752.

Yep, just found that in qiita_ware/commands.py. Workig on it

I think it will be easier just to change that code, so I'm going to close this PR.


Remove use of private deprecate util from ember-cli

Wasn't aware this wasn't meant to be used by addons when implementing https://github.com/steveszc/ember-cli-memory-leak-detector/pull/48.

@steveszc Not sure if you are still maintaining this addon, but if you do ever make a new release, it's best to include this PR as the deprecate util has now been removed from ember-cli.

@bertdeblock i haven't had much time to spend on this addon lately but I'll try to spend some time on this next week. Thanks!


[CHANGE ME] Re-generated  to pick up changes in the API or client library generator.

This PR was generated using Autosynth. :rainbow:
Synth log will be available here:
https://source.cloud.google.com/results/invocations/0685d92c-6b89-42e8-b103-469a98901c3c/targets

If https://github.com/googleapis/synthtool/tree/master/synthtool, we relaxed the regex replacement for fixing the grpc headers. We no longer need to do the replacement twice. Addressing in #85