On very large dask-backed dataset, huge graphs can be transmitted to dask crashing the processing

What is your issue?
Chaining many processing on a huge (200GB+) dask-backed datasets lead to huge graphs (500MB+) being passed. More data, bigger the graph is, to the point where the graph is so huge (31GB at my maximum) that the .compute() fails, with a "error to serialize" error in msgpack.
This is a problem that started when we started to use xarray to process climate experiments. The amount of data that we load is huge (200GB+ on my initial tests. Several TB in the real case). I do not have this problem for regular basic processing (e.g., data selection & plotting with very few processing), but in this case, we chained quite a lot of different operations (expansions of dimensions, dataset concatenations, data selections, means, std, min/max, new dimension expansion...).  Using the exact same processing on less data (e.g., one year) will only trigger a warning from Dask, telling me that the graph is huge (between 800MB and 1.3GB) and that it will be a performance issue, suggesting some good practice as well, but it will run. So, my current workaround is to just to do that: reduce the amount of data I'm processing at the same time (every year). I guess several intermediate .compute() would help as well, but considering the amount of data we're talking about, it's not an option.
I don't think it's a bug... but I also don't think it's the behaviour we want from xarray. We should be able to transmit to dask whatever dataset and get it processed. Xarray should be able to split the graphs better so it doesn't reach the limitation from dask or msgpack.
How to reproduce the problem
Below is the minimum code. Be aware that to make this code represent reality, it has to generate a huge amount of (random) data (200GB+).  The compute at the end will use a lot of memory. This code is made to run on a HPC with 768GB of ram. I cannot really make it smaller, as I think that the core of the problem is that I'm processing a huge amount of data.

Minimum code
import xarray as xr
import numpy as np
import pandas as pd
from distributed import Client
import os

client = Client(n_workers=64, threads_per_worker=2)
# Generate fake data
generateData = False
data_structure = xr.open_dataset('data_structure.nc')
x = data_structure.x.values
y = data_structure.y.values
z = data_structure.z.values

for ids, source in enumerate(['exp1_case1', 'exp2_case1', 'exp1_case2', 'exp2_case2', 'exp1_case3', 'exp2_case3']):
    # As the problem comes with lazy operation, a sampling dataset needs to be generated...
    # This generates about 200GB of random data. The dimensions are the same as what I use in the real case
    if generateData:
        os.makedirs(source, exist_ok=True)
        for idt, tt in enumerate(pd.date_range('1971-01-01','1972-12-31', freq='1SME')):
            ds = xr.Dataset(
                coords = {'z': z,
                          'x': x,
                          'y': y,
                          't': [tt],
                },
                data_vars = {
                          'po4': (('t', 'z', 'y', 'x'), np.random.rand(1, len(z), len(y), len(x))),
                          'no3': (('t', 'z', 'y', 'x'), np.random.rand(1, len(z), len(y), len(x))),
                          'nh4': (('t', 'z', 'y', 'x'), np.random.rand(1, len(z), len(y), len(x))),
                          'oxy': (('t', 'z', 'y', 'x'), np.random.rand(1, len(z), len(y), len(x))),
                          'si': (('t', 'z', 'y', 'x'), np.random.rand(1, len(z), len(y), len(x))),
                },
            )
            outname = f'{source}/data_{idt}.nc'
            if os.path.exists(outname): continue
            ds.to_netcdf(outname)
    # .. So it can be loaded with xr.open_mfdataset()
    ds = xr.open_mfdataset(f'{source}/data_*.nc').assign_coords(suffix='suffix').assign_coords(source=source).expand_dims('suffix').expand_dims('source')
    if ids == 0:
        data = ds
    else:
        data = xr.concat([data, ds], dim='source')

# Start processing on data
# We get the sub-surface and bottom data. The index of the bottom varies in space.
out = xr.Dataset()
lastindex = xr.DataArray(name='index',
                        coords={'x': x, 'y': y},
                        data=np.random.randint(0,10, size=(len(x),len(y))),
                        )
for varname in data.data_vars:
    print(f'Extracting the data for {varname}')
    if varname in [ 'no3']:
        out['din'] = ((data['nh4'].rename('din') + data['no3'].rename('din'))).isel(z=[0,1,2]).mean('z')
        out['dinb'] = (data['nh4'].rename('dinb').isel(z=lastindex) + data['no3'].rename('dinb').isel(z=lastindex))
    elif varname in ['po4']:
        out['dip'] = (data['po4']).isel(z=[0,1,2]).mean('z')
        out['dipb'] = data['po4'].isel(z=lastindex)
    elif varname in ['oxy']:
        out['o2b'] = data['oxy'].isel(z=lastindex)
    elif varname in ['si']:
        out['si'] = (data['si']).isel(z=[0,1,2]).mean('z')
        out['sib'] = data['si'].isel(z=lastindex)

# Now we start the computing ensemble means on the data
for idx, scenario in enumerate(['case1', 'case2', 'case3']):
    ds_mean = out.mean('source', skipna=True)
    ds_std  = out.std('source', skipna=True)
    ds_min  = out.min('source', skipna=True)
    ds_max  = out.max('source', skipna=True)
    da_tokeep = xr.concat( (ds_mean, ds_std, ds_min, ds_max), dim='stats').assign_coords(stats=['mean','std','min','max']).assign_coords(source=[scenario])
    print('working on da to be kept for scenario', scenario)
    if idx == 0:
        ds_ens = da_tokeep
    else:
        ds_ens = xr.concat( [ds_ens, da_tokeep], dim = 'source')

# And the final compute() that crashes
ds_ens.compute()

Error messages
/home/XXXXXX/.conda/envs/d312xup/lib/python3.12/site-packages/distributed/client.py:3371: UserWarning: Sending large graph of size 8.6
0 GiB.
This may cause some slowdown.
Consider loading the data with Dask directly
 or using futures or delayed objects to embed the data into the graph without repetition.
See also https://docs.dask.org/en/stable/best-practices.html#load-data-with-dask for more information.
  warnings.warn(
2024-11-20 15:11:21,623 - distributed.protocol.core - CRITICAL - Failed to Serialize
Traceback (most recent call last):
  File "/home/XXXXXX/.conda/envs/d312xup/lib/python3.12/site-packages/distributed/protocol/core.py", line 109, in dumps
    frames[0] = msgpack.dumps(msg, default=_encode_default, use_bin_type=True)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/XXXXXX/.conda/envs/d312xup/lib/python3.12/site-packages/msgpack/__init__.py", line 36, in packb
    return Packer(**kwargs).pack(o)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "msgpack/_packer.pyx", line 279, in msgpack._cmsgpack.Packer.pack
  File "msgpack/_packer.pyx", line 276, in msgpack._cmsgpack.Packer.pack
  File "msgpack/_packer.pyx", line 265, in msgpack._cmsgpack.Packer._pack
  File "msgpack/_packer.pyx", line 232, in msgpack._cmsgpack.Packer._pack_inner
  File "msgpack/_packer.pyx", line 265, in msgpack._cmsgpack.Packer._pack
  File "msgpack/_packer.pyx", line 213, in msgpack._cmsgpack.Packer._pack_inner
  File "msgpack/_packer.pyx", line 265, in msgpack._cmsgpack.Packer._pack
  File "msgpack/_packer.pyx", line 232, in msgpack._cmsgpack.Packer._pack_inner
  File "msgpack/_packer.pyx", line 265, in msgpack._cmsgpack.Packer._pack
  File "msgpack/_packer.pyx", line 189, in msgpack._cmsgpack.Packer._pack_inner
ValueError: bytes object is too large
2024-11-20 15:11:21,626 - distributed.comm.utils - ERROR - bytes object is too large
Traceback (most recent call last):
  File "/home/XXXXXX/.conda/envs/d312xup/lib/python3.12/site-packages/distributed/comm/utils.py", line 34, in _to_frames
    return list(protocol.dumps(msg, **kwargs))
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/XXXXXX/.conda/envs/d312xup/lib/python3.12/site-packages/distributed/protocol/core.py", line 109, in dumps
    frames[0] = msgpack.dumps(msg, default=_encode_default, use_bin_type=True)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/XXXXXX/.conda/envs/d312xup/lib/python3.12/site-packages/msgpack/__init__.py", line 36, in packb
    return Packer(**kwargs).pack(o)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "msgpack/_packer.pyx", line 279, in msgpack._cmsgpack.Packer.pack
  File "msgpack/_packer.pyx", line 276, in msgpack._cmsgpack.Packer.pack
  File "msgpack/_packer.pyx", line 265, in msgpack._cmsgpack.Packer._pack
  File "msgpack/_packer.pyx", line 232, in msgpack._cmsgpack.Packer._pack_inner
  File "msgpack/_packer.pyx", line 265, in msgpack._cmsgpack.Packer._pack
  File "msgpack/_packer.pyx", line 213, in msgpack._cmsgpack.Packer._pack_inner
  File "msgpack/_packer.pyx", line 265, in msgpack._cmsgpack.Packer._pack
  File "msgpack/_packer.pyx", line 232, in msgpack._cmsgpack.Packer._pack_inner
  File "msgpack/_packer.pyx", line 265, in msgpack._cmsgpack.Packer._pack
  File "msgpack/_packer.pyx", line 189, in msgpack._cmsgpack.Packer._pack_inner
ValueError: bytes object is too large
2024-11-20 15:11:21,631 - distributed.batched - ERROR - Error in batched write
Traceback (most recent call last):
  File "/home/XXXXXX/.conda/envs/d312xup/lib/python3.12/site-packages/distributed/batched.py", line 115, in _background_send
    nbytes = yield coro
             ^^^^^^^^^^
  File "/home/XXXXXX/.conda/envs/d312xup/lib/python3.12/site-packages/tornado/gen.py", line 766, in run
    value = future.result()
            ^^^^^^^^^^^^^^^
  File "/home/XXXXXX/.conda/envs/d312xup/lib/python3.12/site-packages/distributed/comm/tcp.py", line 264, in write
    frames = await to_frames(
             ^^^^^^^^^^^^^^^^
  File "/home/XXXXXX/.conda/envs/d312xup/lib/python3.12/site-packages/distributed/comm/utils.py", line 48, in to_frames
    return await offload(_to_frames)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/XXXXXX/.conda/envs/d312xup/lib/python3.12/site-packages/distributed/utils.py", line 1507, in run_in_executor_with_contex
t
    return await loop.run_in_executor(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/XXXXXX/.conda/envs/d312xup/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/XXXXXX/.conda/envs/d312xup/lib/python3.12/site-packages/distributed/utils.py", line 1508, in <lambda>
    executor, lambda: context.run(func, *args, **kwargs)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/XXXXXX/.conda/envs/d312xup/lib/python3.12/site-packages/distributed/comm/utils.py", line 34, in _to_frames
    return list(protocol.dumps(msg, **kwargs))
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/XXXXXX/.conda/envs/d312xup/lib/python3.12/site-packages/distributed/protocol/core.py", line 109, in dumps
    frames[0] = msgpack.dumps(msg, default=_encode_default, use_bin_type=True)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/XXXXXX/.conda/envs/d312xup/lib/python3.12/site-packages/msgpack/__init__.py", line 36, in packb
    return Packer(**kwargs).pack(o)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "msgpack/_packer.pyx", line 279, in msgpack._cmsgpack.Packer.pack
  File "msgpack/_packer.pyx", line 276, in msgpack._cmsgpack.Packer.pack
  File "msgpack/_packer.pyx", line 265, in msgpack._cmsgpack.Packer._pack
  File "msgpack/_packer.pyx", line 232, in msgpack._cmsgpack.Packer._pack_inner
  File "msgpack/_packer.pyx", line 265, in msgpack._cmsgpack.Packer._pack
  File "msgpack/_packer.pyx", line 213, in msgpack._cmsgpack.Packer._pack_inner
  File "msgpack/_packer.pyx", line 265, in msgpack._cmsgpack.Packer._pack
  File "msgpack/_packer.pyx", line 232, in msgpack._cmsgpack.Packer._pack_inner
  File "msgpack/_packer.pyx", line 265, in msgpack._cmsgpack.Packer._pack
  File "msgpack/_packer.pyx", line 189, in msgpack._cmsgpack.Packer._pack_inner
ValueError: bytes object is too large
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/XXXXXX/.conda/envs/d312xup/lib/python3.12/site-packages/xarray/core/dataset.py", line 1046, in compute
    return new.load(**kwargs)
           ^^^^^^^^^^^^^^^^^^
  File "/home/XXXXXX/.conda/envs/d312xup/lib/python3.12/site-packages/xarray/core/dataset.py", line 873, in load
    evaluated_data: tuple[np.ndarray[Any, Any], ...] = chunkmanager.compute(
                                                       ^^^^^^^^^^^^^^^^^^^^^
  File "/home/XXXXXX/.conda/envs/d312xup/lib/python3.12/site-packages/xarray/namedarray/daskmanager.py", line 86, in compute
    return compute(*data, **kwargs)  # type: ignore[no-untyped-call, no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/XXXXXX/.conda/envs/d312xup/lib/python3.12/site-packages/dask/base.py", line 660, in compute
    results = schedule(dsk, keys, **kwargs)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/XXXXXX/.conda/envs/d312xup/lib/python3.12/site-packages/distributed/client.py", line 2427, in _gather
    raise exception.with_traceback(traceback)
distributed.client.FutureCancelledError: ('concatenate-3bab6b8738b6ee66b3adcbd4114d6fae', 2, 2, 0, 31, 0, 0) cancelled for reason: scheduler-connection-lost.
Client lost the connection to the scheduler. Please check your connection and re-run your work.
>>> 2024-11-20 15:11:36,180 - tornado.application - ERROR - Uncaught exception GET /status/ws (<IP_ADDRESS>)
HTTPServerRequest(protocol='http', host='localhost:8787', method='GET', uri='/status/ws', version='HTTP/1.1', remote_ip='<IP_ADDRESS>')
Traceback (most recent call last):
  File "/home/XXXXXX/.conda/envs/d312xup/lib/python3.12/site-packages/tornado/web.py", line 1790, in _execute
    result = await result
             ^^^^^^^^^^^^
  File "/home/XXXXXX/.conda/envs/d312xup/lib/python3.12/site-packages/tornado/websocket.py", line 273, in get
    await self.ws_connection.accept_connection(self)
  File "/home/XXXXXX/.conda/envs/d312xup/lib/python3.12/site-packages/tornado/websocket.py", line 863, in accept_connection
    await self._accept_connection(handler)
  File "/home/XXXXXX/.conda/envs/d312xup/lib/python3.12/site-packages/tornado/websocket.py", line 946, in _accept_connection
    await self._receive_frame_loop()
  File "/home/XXXXXX/.conda/envs/d312xup/lib/python3.12/site-packages/tornado/websocket.py", line 1105, in _receive_frame_loop
    self.handler.on_ws_connection_close(self.close_code, self.close_reason)
  File "/home/XXXXXX/.conda/envs/d312xup/lib/python3.12/site-packages/tornado/websocket.py", line 571, in on_ws_connection_close
    self.on_connection_close()
  File "/home/XXXXXX/.conda/envs/d312xup/lib/python3.12/site-packages/tornado/websocket.py", line 563, in on_connection_close
    self.on_close()
  File "/home/XXXXXX/.conda/envs/d312xup/lib/python3.12/site-packages/bokeh/server/views/ws.py", line 308, in on_close
    self.connection.session.notify_connection_lost()
    ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/XXXXXX/.conda/envs/d312xup/lib/python3.12/site-packages/bokeh/server/connection.py", line 65, in session
    assert self._session is not None
           ^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError

Environment
```
INSTALLED VERSIONS
------------------
commit: None
python: 3.12.7 | packaged by conda-forge | (main, Oct  4 2024, 16:05:46) [GCC 13.3.0]
python-bits: 64
OS: Linux
OS-release: 5.14.0-427.37.1.el9_4.x86_64
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: ('en_US', 'UTF-8')
libhdf5: 1.14.3
libnetcdf: 4.9.2
xarray: 2024.10.0
pandas: 2.2.3
numpy: 2.1.3
scipy: 1.14.1
netCDF4: 1.7.1
pydap: None
h5netcdf: 1.4.1
h5py: 3.12.1
zarr: None
cftime: 1.6.4
nc_time_axis: None
iris: None
bottleneck: None
dask: 2024.11.2
distributed: 2024.11.2
matplotlib: 3.9.2
cartopy: 0.24.0
seaborn: 0.13.2
numbagg: None
fsspec: 2024.10.0
cupy: None
pint: None
sparse: None
flox: None
numpy_groupies: None
setuptools: 75.5.0
pip: 24.3.1
conda: None
pytest: None
mypy: None
IPython: None
sphinx: None
</details>

telling me that the graph is huge (between 800MB and 1.3GB)

something is very wrong here but we can't reproduce your issue without datastructure.nc. Can you show us the repr for ds (after the open_mfdataset call).

Sorry,  there is the same code without the data_structure. I'll make the file available tomorrow, GiHub doesn't let me attach netcdf files.
With this code, the error is slightly different, but it still crash on the msgpack code (see below)

Minimum updated code
import xarray as xr
import numpy as np
import pandas as pd
from distributed import Client
import os
import time

client = Client(n_workers=64, threads_per_worker=2)
# Generate fake data
generateData = False
y = [ f for f in range(0,523) ]
x = [ f for f in range(0,619) ]
z = [ f for f in range(0,56) ]

for ids, source in enumerate(['exp1_case1', 'exp2_case1', 'exp1_case2', 'exp2_case2', 'exp1_case3', 'exp2_case3']):
    # As the problem comes with lazy operation, a sampling dataset needs to be generated...
    # This generates about 200GB of random data. The dimensions are the same as what I use in the real case
    if generateData:
        os.makedirs(source, exist_ok=True)
        for idt, tt in enumerate(pd.date_range('1971-01-01','1972-12-31', freq='1SME')):
            ds = xr.Dataset(
                coords = {'z': z,
                          'x': x,
                          'y': y,
                          't': [tt],
                },
                data_vars = {
                          'po4': (('t', 'z', 'y', 'x'), np.random.rand(1, len(z), len(y), len(x))),
                          'no3': (('t', 'z', 'y', 'x'), np.random.rand(1, len(z), len(y), len(x))),
                          'nh4': (('t', 'z', 'y', 'x'), np.random.rand(1, len(z), len(y), len(x))),
                          'oxy': (('t', 'z', 'y', 'x'), np.random.rand(1, len(z), len(y), len(x))),
                          'si': (('t', 'z', 'y', 'x'), np.random.rand(1, len(z), len(y), len(x))),
                },
            )
            outname = f'{source}/data_{idt}.nc'
            if os.path.exists(outname): continue
            ds.to_netcdf(outname)
    # .. So it can be loaded with xr.open_mfdataset()
    ds = xr.open_mfdataset(f'{source}/data_*.nc').assign_coords(suffix='suffix').assign_coords(source=source).expand_dims('suffix').expand_dims('source')
    if ids == 0:
        data = ds
    else:
        data = xr.concat([data, ds], dim='source')

# Start processing on data
# We get the sub-surface and bottom data. The index of the bottom varies in space.
out = xr.Dataset()
lastindex = xr.DataArray(name='index',
                        coords={'x': x, 'y': y},
                        data=np.random.randint(0,10, size=(len(x),len(y))),
                        )
for varname in data.data_vars:
    print(f'Extracting the data for {varname}')
    if varname in [ 'no3']:
        out['din'] = ((data['nh4'].rename('din') + data['no3'].rename('din'))).isel(z=[0,1,2]).mean('z')
        out['dinb'] = (data['nh4'].rename('dinb').isel(z=lastindex) + data['no3'].rename('dinb').isel(z=lastindex))
    elif varname in ['po4']:
        out['dip'] = (data['po4']).isel(z=[0,1,2]).mean('z')
        out['dipb'] = data['po4'].isel(z=lastindex)
    elif varname in ['oxy']:
        out['o2b'] = data['oxy'].isel(z=lastindex)
    elif varname in ['si']:
        out['si'] = (data['si']).isel(z=[0,1,2]).mean('z')
        out['sib'] = data['si'].isel(z=lastindex)

# Now we start the computing ensemble means on the data
for idx, scenario in enumerate(['case1', 'case2', 'case3']):
    ds_mean = out.mean('source', skipna=True)
    ds_std  = out.std('source', skipna=True)
    ds_min  = out.min('source', skipna=True)
    ds_max  = out.max('source', skipna=True)
    da_tokeep = xr.concat( (ds_mean, ds_std, ds_min, ds_max), dim='stats').assign_coords(stats=['mean','std','min','max']).assign_coords(source=[scenario])
    print('working on da to be kept for scenario', scenario)
    if idx == 0:
        ds_ens = da_tokeep
    else:
        ds_ens = xr.concat( [ds_ens, da_tokeep], dim = 'source')

ds_ens.compute()

Associated error

/home/XXXXXX/.conda/envs/d312/lib/python3.12/site-packages/distributed/client.py:3161: UserWarning: Sending large graph of size 5.08 GiB.
This may cause some slowdown.
Consider scattering data ahead of time and using futures.
  warnings.warn(
2024-11-20 17:17:37,647 - distributed.protocol.core - CRITICAL - Failed to Serialize
Traceback (most recent call last):
  File "/home/XXXXXX/.conda/envs/d312/lib/python3.12/site-packages/distributed/protocol/core.py", line 109, in dumps
    frames[0] = msgpack.dumps(msg, default=_encode_default, use_bin_type=True)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/XXXXXX/.conda/envs/d312/lib/python3.12/site-packages/msgpack/__init__.py", line 36, in packb
    return Packer(**kwargs).pack(o)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "msgpack/_packer.pyx", line 279, in msgpack._cmsgpack.Packer.pack
  File "msgpack/_packer.pyx", line 276, in msgpack._cmsgpack.Packer.pack
  File "msgpack/_packer.pyx", line 265, in msgpack._cmsgpack.Packer._pack
  File "msgpack/_packer.pyx", line 232, in msgpack._cmsgpack.Packer._pack_inner
  File "msgpack/_packer.pyx", line 265, in msgpack._cmsgpack.Packer._pack
  File "msgpack/_packer.pyx", line 213, in msgpack._cmsgpack.Packer._pack_inner
  File "msgpack/_packer.pyx", line 265, in msgpack._cmsgpack.Packer._pack
  File "msgpack/_packer.pyx", line 232, in msgpack._cmsgpack.Packer._pack_inner
  File "msgpack/_packer.pyx", line 265, in msgpack._cmsgpack.Packer._pack
  File "msgpack/_packer.pyx", line 189, in msgpack._cmsgpack.Packer._pack_inner
ValueError: bytes object is too large
2024-11-20 17:17:43,253 - distributed.comm.utils - ERROR - bytes object is too large
Traceback (most recent call last):
  File "/home/XXXXXX/.conda/envs/d312/lib/python3.12/site-packages/distributed/comm/utils.py", line 55, in _to_frames
    return list(protocol.dumps(msg, **kwargs))
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/XXXXXX/.conda/envs/d312/lib/python3.12/site-packages/distributed/protocol/core.py", line 109, in dumps
    frames[0] = msgpack.dumps(msg, default=_encode_default, use_bin_type=True)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/XXXXXX/.conda/envs/d312/lib/python3.12/site-packages/msgpack/__init__.py", line 36, in packb
    return Packer(**kwargs).pack(o)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "msgpack/_packer.pyx", line 279, in msgpack._cmsgpack.Packer.pack
  File "msgpack/_packer.pyx", line 276, in msgpack._cmsgpack.Packer.pack
  File "msgpack/_packer.pyx", line 265, in msgpack._cmsgpack.Packer._pack
  File "msgpack/_packer.pyx", line 232, in msgpack._cmsgpack.Packer._pack_inner
  File "msgpack/_packer.pyx", line 265, in msgpack._cmsgpack.Packer._pack
  File "msgpack/_packer.pyx", line 213, in msgpack._cmsgpack.Packer._pack_inner
  File "msgpack/_packer.pyx", line 265, in msgpack._cmsgpack.Packer._pack
  File "msgpack/_packer.pyx", line 232, in msgpack._cmsgpack.Packer._pack_inner
  File "msgpack/_packer.pyx", line 265, in msgpack._cmsgpack.Packer._pack
  File "msgpack/_packer.pyx", line 189, in msgpack._cmsgpack.Packer._pack_inner
ValueError: bytes object is too large
2024-11-20 17:17:43,271 - distributed.batched - ERROR - Error in batched write
Traceback (most recent call last):
  File "/home/XXXXXX/.conda/envs/d312/lib/python3.12/site-packages/distributed/batched.py", line 115, in _background_send
    nbytes = yield coro
             ^^^^^^^^^^
  File "/home/XXXXXX/.conda/envs/d312/lib/python3.12/site-packages/tornado/gen.py", line 766, in run
    value = future.result()
            ^^^^^^^^^^^^^^^
  File "/home/XXXXXX/.conda/envs/d312/lib/python3.12/site-packages/distributed/comm/tcp.py", line 270, in write
    frames = await to_frames(
             ^^^^^^^^^^^^^^^^
  File "/home/XXXXXX/.conda/envs/d312/lib/python3.12/site-packages/distributed/comm/utils.py", line 70, in to_frames
    return await offload(_to_frames)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/XXXXXX/.conda/envs/d312/lib/python3.12/site-packages/distributed/utils.py", line 1510, in run_in_executor_with_context
    return await loop.run_in_executor(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/XXXXXX/.conda/envs/d312/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/XXXXXX/.conda/envs/d312/lib/python3.12/site-packages/distributed/utils.py", line 1511, in <lambda>
    executor, lambda: context.run(func, *args, **kwargs)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/XXXXXX/.conda/envs/d312/lib/python3.12/site-packages/distributed/comm/utils.py", line 55, in _to_frames
    return list(protocol.dumps(msg, **kwargs))
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/XXXXXX/.conda/envs/d312/lib/python3.12/site-packages/distributed/protocol/core.py", line 109, in dumps
    frames[0] = msgpack.dumps(msg, default=_encode_default, use_bin_type=True)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/XXXXXX/.conda/envs/d312/lib/python3.12/site-packages/msgpack/__init__.py", line 36, in packb
    return Packer(**kwargs).pack(o)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "msgpack/_packer.pyx", line 279, in msgpack._cmsgpack.Packer.pack
  File "msgpack/_packer.pyx", line 276, in msgpack._cmsgpack.Packer.pack
  File "msgpack/_packer.pyx", line 265, in msgpack._cmsgpack.Packer._pack
  File "msgpack/_packer.pyx", line 232, in msgpack._cmsgpack.Packer._pack_inner
  File "msgpack/_packer.pyx", line 265, in msgpack._cmsgpack.Packer._pack
  File "msgpack/_packer.pyx", line 213, in msgpack._cmsgpack.Packer._pack_inner
  File "msgpack/_packer.pyx", line 265, in msgpack._cmsgpack.Packer._pack
  File "msgpack/_packer.pyx", line 232, in msgpack._cmsgpack.Packer._pack_inner
  File "msgpack/_packer.pyx", line 265, in msgpack._cmsgpack.Packer._pack
  File "msgpack/_packer.pyx", line 189, in msgpack._cmsgpack.Packer._pack_inner
ValueError: bytes object is too large
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/XXXXXX/.conda/envs/d312/lib/python3.12/site-packages/xarray/core/dataset.py", line 1014, in compute
    return new.load(**kwargs)
           ^^^^^^^^^^^^^^^^^^
  File "/home/XXXXXX/.conda/envs/d312/lib/python3.12/site-packages/xarray/core/dataset.py", line 848, in load
    evaluated_data = chunkmanager.compute(*lazy_data.values(), **kwargs)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/XXXXXX/.conda/envs/d312/lib/python3.12/site-packages/xarray/core/daskmanager.py", line 70, in compute
    return compute(*data, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/XXXXXX/.conda/envs/d312/lib/python3.12/site-packages/dask/base.py", line 628, in compute
    results = schedule(dsk, keys, **kwargs)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/XXXXXX/.conda/envs/d312/lib/python3.12/site-packages/distributed/client.py", line 2244, in _gather
    raise exc
concurrent.futures._base.CancelledError: ('concatenate-19fce72c9dafb4bec765a940adc69ee7', 1, 2, 0, 32, 0, 0)

I bet at least one issue is this recursive concat
data = xr.concat([data, ds], dim='source')

I encourage you to use a single open_mfdataset call to open and concatenate all datasets.

After a bit more testing, my workaround performances utterly awful. I thought that taking processing the dataset year by year would allow to process the whole dataset, despite the dask performance warning, but not really. It worked on my 2 years test data (~ 200GB), but when applied on the whole dataset (120 yrs, several TB), it takes more than 24h to process a single year, for a single dataarray, and it's still running.
As a comparison, I wrote a version of the same processing without using dask-backed dataset, and the processing took 4h30 on the whole 6 climate experiment (several TB).
There is the associated workround code. This replace the .compute() from the original code.

Dask backed workaround to replace ds_ens.compute()
# Replace ds_ens.compute() by:
for nyr, yr in enumerate(np.unique(ds_ens.t.dt.year)):
                suffix = ds_ens.suffix.values
                source = ds_ens.source.values
                stats = ds_ens.stats.values
                x = ds_ens.x.values
                y = ds_ens.y.values
                t = ds_ens.t.sel(t=str(yr)).values
            tmp = xr.Dataset(
                coords = {'x': x,
                          'y': y,
                          't': t,
                          'suffix': suffix,
                          'source': source,
                          'stats': stats,
                          },
                data_vars = { ds_ens[da].name: (ds_ens[da].dims, ds_ens[da].sel(t=str(yr)).values) for da in ds_ens.data_vars }
                )
            if nyr == 0:
                full_ds = tmp
            else:
                full_ds = xr.concat([full_ds, tmp], dim="t")


codemirror widths and lefts are so large they push codemirror to right edge of browser

using babel and webpack-dev-server to bundle and serve the the rendered GraphiQL component. For some reason the margin-left and certain width style properties are set to almost the entire width of the screen so all i see is a blank space and then the first character of each code line up against the right side of the browser window
<div class="CodeMirror-sizer" style="margin-left: 1904px; margin-bottom: 0px; border-right-width: 30px; min-height: 675px; min-width: 619px; padding-right: 0px; padding-bottom: 0px;">
   <div ....>....
  </di>
</div>

Whoa this is crazy, what browser/OS are you using? Can you also verify that you have the graphiql.css file loaded into the browser?

nothing crazy. chrome on osx.
I guess I took the readme example a little to literally and used just
import React from 'react';
import GraphiQL from 'graphiql';
import fetch from 'isomorphic-fetch';

function graphQLFetcher(graphQLParams) {
  return fetch(window.location.origin + '/graphql', {
    method: 'post',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify(graphQLParams),
  }).then(response => response.json());
}

React.render(<GraphiQL fetcher={graphQLFetcher} />, document.body);

and an index.html of
<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <title>GraphiQL on localhost</title>
</head>
<body>
  <script src="/static/bundle.js"></script>
</body>
</html>

and served the above with webpack-dev-server
I now see you have examples using static asset css files. I'll introduce that to my index and I'm sure it'll fix things... I'll confirm in a bit

I was assuming all inline styles in the <GraphiQL /> component :)

Ok just needed to add css-loader to webpack.config and all is good again.

@philiptzou do you have recommendations as to how you ensure the stylesheet is loaded first? Or could you share how you went about this?

@mikefowler
I used setTimeout() to delay the rendering of <GraphiQL /> element. I know it's not 100% guaranteed but it is enough for my case. If you need to guaranteed this, I think you can use setInterval() to check if the codemirror.css was loaded, then setState() to render <GraphiQL /> element and clearInterval().

@philiptzou any guidance on what value for setTimeout worked for you?


GH-3502: Add sources attribute to DotNetCoreRunSettings

Add sources attribute to DotNetCoreRunSettings
Close #3502

@eoehen your changes have been merged, thanks for your contribution :+1:


Setup travis

This Merge, adds Travis CI integration testing framework.  Also .floo and .flooignore have been added to .gitignore

Cancel that, I will fix the current setup so that it passes the test, before merging main.


[NEW GAME][GSSOC'23] Rabbit Rush Game

A retro styled game testing your reflexes and precision on the keyboard or mouse. Your main aim is to save the rabbit from the wolf running behind him to catch him. You can do it by jumping the rabbit over the hedgehogs that come in his path and by grabbing the carrots that increase its speed.
Screenshots (optional)

I am a GSSoC23 contributor please assign me this issue.
@pranjay-poddar @Saurabh-kumar09 please assign me this issue.

Please add the labels of gssoc23


Timestamp override

Added option to specify timestamp from field in event.
use the timestamp_override field to define the field to use when specifying a timestamp instead of using the timestamp of the event. Useful if parsing historical log files.

Thanks a lot for your contribution @lingster , in order to move forward with your PR is going to be necessary to sign the CLA agreement, you can find more information at https://www.elastic.co/contributor-agreement.
On the other side, it would be super nice if you can add test for this change. We try to enforce that all PR introduce some kind of testing, so we're able to increase the overall quality. Don't hesitate to ask any question regarding you might have, more than looking forward to help.
Looking forward to get this in,

purbon

Jenkins standing by to test this. If you aren't a maintainer, you can ignore this comment. Someone with commit access, please review this and clear it for Jenkins to run; then  say 'jenkins, test it'.

Opened #19 because of the long stalled CLA request not fulfilled, more than welcome to reopen and get this merged if the CLA gets signed.
Thanks a lot for your time.


no NvidiaOpticalFlow

Expected behaviour
I am using Navidia optical flow following this demo: https://developer.nvidia.com/blog/opencv-optical-flow-algorithms-with-nvidia-turing-gpus/
Actual behaviour
However, with nvof = cv2.cuda_NvidiaOpticalFlow_1_0.create(frame1.shape[1], frame1.shape[0], 5, False, False, False, 0), I got an error:
AttributeError: module 'cv2.cv2' has no attribute 'cuda_NvidiaOpticalFlow_1_0'
Steps to reproduce
I am using custom compiled opencv with cuda on:
cmake -D CMAKE_BUILD_TYPE=RELEASE
-D CMAKE_INSTALL_PREFIX=/usr/local
-D WITH_CUDA=ON
-D ENABLE_FAST_MATH=1
-D CUDA_FAST_MATH=1
-D WITH_CUBLAS=1
-D PYTHON3_EXECUTABLE=$CONDA_ENV_PATH/$CONDA_ENV_NAME/bin/python
-D INSTALL_C_EXAMPLES=ON
-D INSTALL_PYTHON_EXAMPLES=ON
-D OPENCV_EXTRA_MODULES_PATH=$WHERE_OPENCV_CONTRIB/modules
-D BUILD_EXAMPLES=ON $WHERE_OPENCV
-D PYTHON3_PACKAGES_PATH=$CONDA_ENV_PATH/$CONDA_ENV_NAME/lib/python3.6/site-packages
-D BUILD_opencv_python2=OFF
-D BUILD_opencv_python3=ON \
Following related item could be found:
/usr/local/lib/libopencv_cudaoptflow.so.4.5.2
/usr/local/lib/libopencv_cudaoptflow.so.4.5
/usr/local/lib/libopencv_cudaoptflow.so
/usr/local/include/opencv4/opencv2/cudaoptflow.hpp
/usr/local/share/opencv4/samples/cudaoptflow/nvidia_optical_flow.cpp
/usr/local/share/opencv4/samples/cudaoptflow/optical_flow.cpp
And following cuda modules could be found with python:
cv2.cuda_BufferPool
cv2.cuda_DeviceInfo
cv2.cuda_Event
cv2.cuda_GpuMat
cv2.cuda_GpuMat_Allocator
cv2.cuda_HostMem
cv2.cuda_Stream
cv2.cuda_TargetArchs
The Nvidia optical flow SDK was compiled already.
Issue submission checklist

[x] This is not a generic OpenCV usage question (looking for help for coding, other usage questions, homework etc.)

[x] I have read the README of this repository and understand that this repository provides only an automated build toolchain for OpenCV Python packages (there is no actual OpenCV code here)

[x] The issue is related to the build scripts in this repository, to the pre-built binaries or is a feature request (such as "please enable this additional dependency")

[x] I'm using the latest version of opencv-python

Optical flow is dedicated module in OpenCV Contrib. Please check your OpenCV build options with print(cv2.getBuildInformation()). I use pre-built OpenCV installed with Pypi (pip) It's built without CUDA support and Optical Flow module is in list of unavailable modules:
General configuration for OpenCV 4.5.2 =====================================
...
  OpenCV modules:
    To be built:                 aruco bgsegm bioinspired calib3d ccalib core cvv datasets dnn dnn_objdetect dnn_superres dpm face features2d flann fuzzy gapi hfs highgui img_hash imgcodecs imgproc intensity_transform line_descriptor mcc ml objdetect optflow phase_unwrapping photo plot python3 quality rapid reg rgbd saliency shape stereo stitching structured_light superres surface_matching text tracking video videoio videostab wechat_qrcode xfeatures2d ximgproc xobjdetect xphoto
    Disabled:                    world
    Disabled by dependency:      -
    Unavailable:                 alphamat cnn_3dobj cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters cudaimgproc cudalegacy cudaobjdetect cudaoptflow cudastereo cudawarping cudev freetype hdf java julia matlab ovis python2 sfm ts viz
...

The ticket has been closed, because Pypi distro does not support CUDA for now.  opencv-python and  opencv-python-contrib packages are self contained and use internal version OpenCV. So /usr/local/lib/libopencv_cudaoptflow.so.4.5 are not used if you install OpenCV with pip.


Integrated terminal not working, VSC doesn't flush through integrated terminal

Does this issue occur when all extensions are disabled?: Don't know how to debug/run my file without them.

VS Code Version: 1.74.3
OS Version: Windows 11

Steps to Reproduce:

Download version 1.74.3 of VSCode
Download gdb debugger (MinGW)
In JSON choose: "externalConsole": false
Create printf("Hello World");, c-file.
Try to debug or run the C file.

Terminal prints:  *  Executing task: C/C++: gcc.exe build active file

Starting build...
C:\MinGW\bin\gcc.exe -fdiagnostics-color=always -g src/*.c -I include -o C:\Users\mahmu\code\swe\ChessProjectClass/bin/main.exe

Build finished successfully.
 *  Terminal will be reused by tasks, press any key to close it.

Debugger shows:
Kill the program being debugged? (y or n) [answered Y; input not from terminal]
error return ../../gdb-7.6.1/gdb/windows-nat.c:1275 was 5
The program 'C:\Users\mahmu\code\swe\ChessProjectClass/bin/main.exe' has exited with code 0 (0x00000000).

=thread-group-added,id="i1"
GNU gdb (GDB) 7.6.1
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "mingw32".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Warning: Debuggee TargetArchitecture not detected, assuming x86_64.
=cmd-param-changed,param="pagination",value="off"
[New Thread 15952.0x2e94]
[New Thread 15952.0x2188]
Loaded 'C:\WINDOWS\SysWOW64\kernel32.dll'. Symbols loaded.
Loaded 'C:\WINDOWS\SysWOW64\KernelBase.dll'. Symbols loaded.
Loaded 'C:\WINDOWS\SysWOW64\apphelp.dll'. Symbols loaded.
Loaded 'C:\WINDOWS\SysWOW64\msvcrt.dll'. Symbols loaded.
Execute debugger commands using "-exec <command>", for example "-exec info registers" will list registers in use (when GDB is the debugger)

Please file this on the c++ debug extension repo. I'd expect it to be an issue with your code or configuration though.


moving packages to common system-tools. also alphabetical order.

:)

Can one of the admins verify this patch?

"ok to test" to accept this pull request for testing
"test this please" for a one time test run
"add to whitelist" to add the author to the whitelist

LGTM

hey @epwn lets get @rcwong added to the jenkins whitelist ok thanks bi


Lots of improvements

Hi!
I really like this cool mod/utility. I've been using it in my own singleplayer modpack for some time now. I've noticed a few issues while playing, and wanted to fix them myself. I've kept this "version" private, but I've decided to open a PR, since I now feel they would be a great improvement.
Here's a list of what I've changed (or at least what I can remember):

Improved Heat sources: each source has a 'heatLevel' (starting from 1 and counting up). This heatLevel determines the level required to melt something (from the recipe). I would like this to be configurable and not hard-coded, but I didn't have much time to work on.
Added some new heat sources (fire, soul fire, soultourch, soul campfire, magma block)
I've added a 'Creative Heat Source' block. Placing it under the Melter will turn it purple and insta-melt anything (with a valid recipe). This block does not have a default recipe, this is upto a modpack maker to define (if they want).
JEI shows all the possible heat sources in the recipe viewer for a specific level. It is not perfect, as displaying both blocks, items or fluids was challenging (I would like to refactor this maybe sometime :)). If no valid source is found (eg. level 8 needed, but that level is not defined), a barrier block will be shown with this info. Also, fire and soulfire do not have a texture, so I made a compromise to display flint and steel and a fire charge with appropriate text color and names.
Melter interaction improvements: Added a way to take out the items form the Melter (not possible before) by right-clicking, added a way to place items into a melter by right-clicking with the item/block.
Fixed processing time: now the time is actualy correct for the minimum heat level; for each level above minimum, there is a 2 tick bonus so it will melt faster
Fixed an issue where a different fluid will start melting even though there is already a different fluid present in the Melter
Default minimum heat level is now 1
Improved the way items look inside of a Melter. They now lay flat inside. Blocks are the same as before.
Improved liquid position inside of Melter for small amounts (<200mB). I haven't tested less than 50mB.
Improved JEI Melter recipe look (centering, added slots, decimal format, etc.)
Minor improvements, check the commits.

I'll try and post some screenshots below, but I suggest you clone the repo and try the mod for yourself.
I haven't modded in almost 7 years, so this mini-project reminded me of the days I did modding :)
I'm open for any questions and discussion.
All the best.

Since we can't get fire and soulfire textures from a block (like we use to), these two sources are displayed like so:

Here is the look of lava

Any liquid should work (eg. Superheated Sodium from Mekanism)

If no valid source is present

Heat level 0 (no heat)

Heat level 1

Heat level 2

Heat level 3

Heat level 4

Heat level 5 (Create superheated)

Creative

This is how the item use to render inside of the Melter

This is how it is rendered now

Small amounts of liquid were not visible (inside of the Melter)

They should be visible now (I haven't tested below 50mB but it should be an easy fix if it is still not)

Also, here's the improved JEI look and an info tab for the creative heat source block

Thanks so much for this PR!!
It looks very promising!

I also had some free time to work on a way to configure heat sources, instead of hard-codding them. I'm having some issues with JEI, but it shouldn't be too difficult to fix. I'll probably add this feature to this PR, since it overhauls the HeatSource.

Configurable heat sources are done.
Resolves #5

Heat source list is configured in each save file, under serverconfig.
Each heat source list entry must be a list of strings, in a pre-defined order. This is really a limitation of the config TOML parser (ref: https://github.com/TheElectronWill/night-config/blob/master/toml/src/main/java/com/electronwill/nightconfig/toml/ValueWriter.java#L35)
Format of each entry:

type (block or fluid) - we need this information mostly because of JEI, because block and fluid type textures are passed to the renderer differently
name - this is the name (or id) of block/fluid
heat level (from 1 to 10 only) - the limitation in levels come from the HEAT_SOURCE property in MelterBlock
description - Any additional info about the heat source. This will be displayed in JEI. Mostly used for blocks with different states.

Example entry: ["block", "minecraft:dirt", "2", ""]
Better check for heat source below melter
Added 'Melter Heat Sources' in JEI, which shows all valid heat sources and level values
Changed the way creative melter works. It is not a property of Melter block.
Few other fixes

These are the default heat sources (same as before)

Here's the description part of the config. It is displayed in both JEI recipes

Here, I've changed the heat sources to dirt and a fluid from Mekanism mod.

It is then reflected in the game.

Merged! and uploaded new version!!
I still have to add documentation and proper credit to the curseforge, modrinth and README.
Thanks so much!!!!

You're welcome, cheers :)


Improving control of colorize

Fixes #458.
While the internal representation of the :colorize config key has changed from a string to a keyword, this should be fully backwards compatible with whatever users have in MIDJE_COLORIZE or .midje.clj.

@philomates I haven't seen that arity error before, but I also haven't tried the autotest functionality.
I'll look into it.

@philomates I was able to reproduce the issue and fix it. lein midje and lein midje :autotest now both work correctly with true, reverse, and false for MIDJE_COLORIZE.

I've released this under Midje 1.9.7 and updated the docs a little: https://github.com/marick/Midje/wiki/Configuration-files and https://github.com/marick/Midje/wiki/Colorizing


Line charts with dataZoom draw vertical line after a value of 0 on refresh when zooming

One-line summary [问题简述]
Line charts with dataZoom draw vertical line on refresh when zooming with a value of 0 before the next value. If i replace the zoom, the chart is drawn perfectly. It's only on refresh.
Version & Environment [版本及环境]

ECharts version [ECharts 版本]: 4.0.4
Browser version [浏览器类型和版本]: IE 11
OS Version [操作系统类型和版本]: Windows 7

Expected behaviour [期望结果]
No vertical line
Other comments [其他信息]
Before refresh :

After refresh :

After replacing the zoom :

May I have the echarts option to reproduce the issue?

@100pah


Bevy 0.12 Update

This updates to Bevy 0.12 and fixes the deprecation warning by changing configure_set to configure_sets. Additionally, this exposes the steamworks dependency's serde feature via a feature of the same name in this crate disabled by default, fixing #21.

---- src/lib.rs - (line 43) stdout ----
error[E0599]: no method named `add_startup_system` found for mutable reference `&mut bevy::prelude::App` in the current scope
  --> src/lib.rs:59:8
   |
15 | /   App::new()
16 | |       .add_plugins(DefaultPlugins)
17 | |       .add_plugins(SteamworksPlugin::new(AppId(480)))
18 | |       .add_startup_system(steam_system)
   | |       -^^^^^^^^^^^^^^^^^^ help: there is a method with a similar name: `add_state`
   | |_______|
   |

Should be:
.add_startup_system(Startup, steam_system)


🐛 [firebase_messaging] Native crash in Android for v14.7.10

Bug report
Describe the bug
It doesn't ask notification permission (some device and Android versions) and getting crash in native code in Android. Normally it asks for both Android&iOS
Steps to reproduce
Standart plugin implementation for Android. It crashed in native when I call  firebaseRequestPermission() which is in below.
initializeFirebase() async {
  await Firebase.initializeApp(
    options: DefaultFirebaseOptions.currentPlatform,
  );

  // Pass all uncaught "fatal" errors from the framework to Crashlytics
  FlutterError.onError = FirebaseCrashlytics.instance.recordFlutterFatalError;
  FlutterError.onError = (errorDetails) {
    FirebaseCrashlytics.instance.recordFlutterFatalError(errorDetails);
  };
  // Pass all uncaught asynchronous errors that aren't handled by the Flutter framework to Crashlytics
  PlatformDispatcher.instance.onError = (error, stack) {
    FirebaseCrashlytics.instance.recordError(error, stack, fatal: true);
    return true;
  };

  FirebaseMessaging.onBackgroundMessage(_firebaseMessagingBackgroundHandler);

  FirebaseAnalytics analytics = FirebaseAnalytics.instance;
  analytics.setAnalyticsCollectionEnabled(true);
}

void firebaseForegroundReceiver(BuildContext context) {
  FirebaseMessaging.onMessage.listen((RemoteMessage message) {
    print('Got a message whilst in the foreground!');
    print('Message data: ${message.data}');

    if (message.notification != null) {
      print('Message also contained a notification: ${message.notification}');
      sendNotification(context, message.notification!.body.toString());
    } else {
      sendNotification(context, message.data.toString());

      print(message.data);
    }
  });
}

Future<void> firebaseRequestPermission() async {
  FirebaseMessaging messaging = FirebaseMessaging.instance;
  await messaging.requestPermission(
    alert: true,
    announcement: true,
    badge: true,
    carPlay: false,
    criticalAlert: false,
    provisional: false,
    sound: true,
  );
}

@pragma('vm:entry-point')
Future<void> _firebaseMessagingBackgroundHandler(RemoteMessage message) async {
  // If you're going to use other Firebase services in the background, such as Firestore,
  // make sure you call `initializeApp` before using other Firebase services.
  await Firebase.initializeApp();

  print("Handling a background message: ${message.messageId}");
}

Expected behavior
Working without problem

Flutter doctor
Run flutter doctor and paste the output below:
Click To Expand
Doctor summary (to see all details, run flutter doctor -v):
[√] Flutter (Channel stable, 3.16.0, on Microsoft Windows [Version 10.0.22621.2861], locale en-US)
[√] Windows Version (Installed version of Windows is version 10 or higher)
[√] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
[√] Chrome - develop for the web
[√] Visual Studio - develop Windows apps (Visual Studio Build Tools 2019 16.11.25)
[√] Android Studio (version 2022.3)
[√] VS Code (version 1.79.2)
[√] Connected device (3 available)
[√] Network resources

• No issues found!

Flutter dependencies
Run flutter pub deps -- --style=compact and paste the output below:
Click To Expand
Dart SDK 3.2.0
Flutter SDK 3.16.0

dependencies:
- alarm 3.0.2 [flutter flutter_fgbg shared_preferences]
- app_tracking_transparency 2.0.4 [flutter]
- firebase_analytics 10.8.0 [firebase_analytics_platform_interface firebase_analytics_web firebase_core firebase_core_platform_interface flutter]
- firebase_core 2.24.2 [firebase_core_platform_interface firebase_core_web flutter meta]
- firebase_crashlytics 3.4.9 [firebase_core firebase_core_platform_interface firebase_crashlytics_platform_interface flutter stack_trace]
- firebase_messaging 14.7.10 [firebase_core firebase_core_platform_interface firebase_messaging_platform_interface firebase_messaging_web flutter meta]
- flutter 0.0.0 [characters collection material_color_utilities meta vector_math web sky_engine]
- flutter_inappwebview 6.0.0 [flutter flutter_inappwebview_platform_interface flutter_inappwebview_android flutter_inappwebview_ios flutter_inappwebview_macos flutter_inappwebview_web]
- flutter_localizations 0.0.0 [flutter intl characters clock collection material_color_utilities meta path vector_math web]
- flutter_native_splash 2.3.9 [args flutter flutter_web_plugins js html image meta path universal_io xml yaml ansicolor]
- flutter_timezone 1.0.8 [flutter_web_plugins flutter js]
- fluttertoast 8.2.4 [flutter flutter_web_plugins]
- font_awesome_flutter 10.6.0 [flutter]
- google_fonts 6.1.0 [flutter http path_provider crypto]
- google_mobile_ads 4.0.0 [meta flutter webview_flutter_android webview_flutter_wkwebview webview_flutter]
- in_app_purchase 3.1.12 [flutter in_app_purchase_android in_app_purchase_platform_interface in_app_purchase_storekit]
- intl 0.18.1 [clock meta path]
- share_plus 7.2.1 [cross_file meta mime flutter flutter_web_plugins share_plus_platform_interface file url_launcher_web url_launcher_windows url_launcher_linux url_launcher_platform_interface ffi win32]
- shared_preferences 2.2.2 [flutter shared_preferences_android shared_preferences_foundation shared_preferences_linux shared_preferences_platform_interface shared_preferences_web shared_preferences_windows]
- simple_shadow 0.3.1 [flutter]
- url_launcher 6.2.2 [flutter url_launcher_android url_launcher_ios url_launcher_linux url_launcher_macos url_launcher_platform_interface url_launcher_web url_launcher_windows]

dev dependencies:
- flutter_lints 3.0.1 [lints]
- flutter_test 0.0.0 [flutter test_api matcher path fake_async clock stack_trace vector_math async boolean_selector characters collection material_color_utilities meta source_span stream_channel string_scanner term_glyph web]

transitive dependencies:
- _flutterfire_internals 1.3.16 [collection firebase_core firebase_core_platform_interface flutter meta]
- ansicolor 2.0.2
- archive 3.4.10 [crypto path pointycastle]
- args 2.4.2
- async 2.11.0 [collection meta]
- boolean_selector 2.1.1 [source_span string_scanner]
- characters 1.3.0
- clock 1.1.1
- collection 1.18.0
- convert 3.1.1 [typed_data]
- cross_file 0.3.3+8 [meta web]
- crypto 3.0.3 [typed_data]
- csslib 1.0.0 [source_span]
- fake_async 1.3.1 [clock collection]
- ffi 2.1.0
- file 7.0.0 [meta path]
- firebase_analytics_platform_interface 3.9.0 [_flutterfire_internals firebase_core flutter meta plugin_platform_interface]
- firebase_analytics_web 0.5.5+12 [_flutterfire_internals firebase_analytics_platform_interface firebase_core firebase_core_web flutter flutter_web_plugins js]
- firebase_core_platform_interface 5.0.0 [collection flutter flutter_test meta plugin_platform_interface]
- firebase_core_web 2.10.0 [firebase_core_platform_interface flutter flutter_web_plugins js meta]
- firebase_crashlytics_platform_interface 3.6.16 [_flutterfire_internals collection firebase_core flutter meta plugin_platform_interface]
- firebase_messaging_platform_interface 4.5.18 [_flutterfire_internals firebase_core flutter meta plugin_platform_interface]
- firebase_messaging_web 3.5.18 [_flutterfire_internals firebase_core firebase_core_web firebase_messaging_platform_interface flutter flutter_web_plugins js meta]
- flutter_fgbg 0.3.0 [flutter]
- flutter_inappwebview_android 1.0.12 [flutter flutter_inappwebview_platform_interface]
- flutter_inappwebview_internal_annotations 1.1.1
- flutter_inappwebview_ios 1.0.13 [flutter flutter_inappwebview_platform_interface]
- flutter_inappwebview_macos 1.0.11 [flutter flutter_inappwebview_platform_interface]
- flutter_inappwebview_platform_interface 1.0.10 [flutter flutter_inappwebview_internal_annotations plugin_platform_interface]
- flutter_inappwebview_web 1.0.8 [flutter flutter_web_plugins js flutter_inappwebview_platform_interface]
- flutter_web_plugins 0.0.0 [flutter characters collection material_color_utilities meta vector_math web]
- html 0.15.4 [csslib source_span]
- http 1.1.2 [async http_parser meta web]
- http_parser 4.0.2 [collection source_span string_scanner typed_data]
- image 4.1.3 [archive meta xml]
- in_app_purchase_android 0.3.0+16 [collection flutter in_app_purchase_platform_interface json_annotation]
- in_app_purchase_platform_interface 1.3.6 [flutter plugin_platform_interface]
- in_app_purchase_storekit 0.3.8 [collection flutter in_app_purchase_platform_interface json_annotation]
- js 0.6.7 [meta]
- json_annotation 4.8.1 [meta]
- lints 3.0.0
- matcher 0.12.16 [async meta stack_trace term_glyph test_api]
- material_color_utilities 0.5.0 [collection]
- meta 1.10.0
- mime 1.0.4
- path 1.8.3
- path_provider 2.1.1 [flutter path_provider_android path_provider_foundation path_provider_linux path_provider_platform_interface path_provider_windows]
- path_provider_android 2.2.2 [flutter path_provider_platform_interface]
- path_provider_foundation 2.3.1 [flutter path_provider_platform_interface]
- path_provider_linux 2.2.1 [ffi flutter path path_provider_platform_interface xdg_directories]
- path_provider_platform_interface 2.1.1 [flutter platform plugin_platform_interface]
- path_provider_windows 2.2.1 [ffi flutter path path_provider_platform_interface win32]
- petitparser 6.0.2 [meta]
- platform 3.1.4
- plugin_platform_interface 2.1.8 [meta]
- pointycastle 3.7.3 [collection convert js]
- share_plus_platform_interface 3.3.1 [cross_file flutter meta mime plugin_platform_interface path_provider uuid]
- shared_preferences_android 2.2.1 [flutter shared_preferences_platform_interface]
- shared_preferences_foundation 2.3.4 [flutter shared_preferences_platform_interface]
- shared_preferences_linux 2.3.2 [file flutter path path_provider_linux path_provider_platform_interface shared_preferences_platform_interface]
- shared_preferences_platform_interface 2.3.1 [flutter plugin_platform_interface]
- shared_preferences_web 2.2.2 [flutter flutter_web_plugins shared_preferences_platform_interface web]
- shared_preferences_windows 2.3.2 [file flutter path path_provider_platform_interface path_provider_windows shared_preferences_platform_interface]
- sky_engine 0.0.99
- source_span 1.10.0 [collection path term_glyph]
- sprintf 7.0.0
- stack_trace 1.11.1 [path]
- stream_channel 2.1.2 [async]
- string_scanner 1.2.0 [source_span]
- term_glyph 1.2.1
- test_api 0.6.1 [async boolean_selector collection meta source_span stack_trace stream_channel string_scanner term_glyph]
- typed_data 1.3.2 [collection]
- universal_io 2.2.2 [collection meta typed_data]
- url_launcher_android 6.2.1 [flutter url_launcher_platform_interface]
- url_launcher_ios 6.2.2 [flutter url_launcher_platform_interface]
- url_launcher_linux 3.1.1 [flutter url_launcher_platform_interface]
- url_launcher_macos 3.1.0 [flutter url_launcher_platform_interface]
- url_launcher_platform_interface 2.3.0 [flutter plugin_platform_interface]
- url_launcher_web 2.2.3 [flutter flutter_web_plugins url_launcher_platform_interface web]
- url_launcher_windows 3.1.1 [flutter url_launcher_platform_interface]
- uuid 4.2.2 [crypto sprintf meta]
- vector_math 2.1.4
- web 0.3.0
- webview_flutter 4.4.3 [flutter webview_flutter_android webview_flutter_platform_interface webview_flutter_wkwebview]
- webview_flutter_android 3.13.2 [flutter webview_flutter_platform_interface]
- webview_flutter_platform_interface 2.9.0 [flutter meta plugin_platform_interface]
- webview_flutter_wkwebview 3.10.1 [flutter path webview_flutter_platform_interface]
- win32 5.2.0 [ffi]
- xdg_directories 1.0.4 [meta path]
- xml 6.5.0 [collection meta petitparser]
- yaml 3.1.2 [collection source_span string_scanner]

Fatal Exception: io.flutter.plugins.firebase.crashlytics.FlutterError: PlatformException(error, Attempt to invoke virtual method 'android.content.SharedPreferences android.content.Context.getSharedPreferences(java.lang.String, int)' on a null object reference, null, java.lang.NullPointerException: Attempt to invoke virtual method 'android.content.SharedPreferences android.content.Context.getSharedPreferences(java.lang.String, int)' on a null object reference
	at io.flutter.plugins.firebase.messaging.b.m(SourceFile:1)
	at io.flutter.plugins.firebase.messaging.FlutterFirebaseMessagingBackgroundService.n(SourceFile:1)
	at io.flutter.plugins.firebase.messaging.e.onMethodCall(SourceFile:1)
	at i6.k$a.a(SourceFile:1)
	at a6.c.l(SourceFile:1)
	at a6.c.m(SourceFile:1)
	at a6.c.f(SourceFile:1)
	at a6.b.run(SourceFile:1)
	at android.os.Handler.handleCallback(Handler.java:938)
	at android.os.Handler.dispatchMessage(Handler.java:99)
	at android.os.Looper.loop(Looper.java:246)
	at android.app.ActivityThread.main(ActivityThread.java:8653)
	at java.lang.reflect.Method.invoke(Native Method)
	at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:602)
	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1130)
)
       at StandardMethodCodec.decodeEnvelope(message_codecs.dart:651)
       at MethodChannel._invokeMethod(platform_channel.dart:322)
       at MethodChannel.invokeMapMethod(platform_channel.dart:522)
       at MethodChannelFirebaseMessaging.registerBackgroundMessageHandler(method_channel_messaging.dart:202)

Another bug:

Fatal Exception: io.flutter.plugins.firebase.crashlytics.FlutterError: [firebase_messaging/unknown] Attempt to invoke virtual method 'int android.content.Context.checkSelfPermission(java.lang.String)' on a null object reference
       at StandardMethodCodec.decodeEnvelope(message_codecs.dart:651)
       at MethodChannel._invokeMethod(platform_channel.dart:322)
       at MethodChannel.invokeMapMethod(platform_channel.dart:522)
       at MethodChannelFirebaseMessaging.requestPermission(method_channel_messaging.dart:295)
       at .firebaseRequestPermission(firebase_utils.dart:52)

Thanks for the report @aykutuludag
Can you take a look at this and check if it helps in your case or not ?

It doesn't ask notification permission (some device and Android versions)

See https://firebase.google.com/docs/cloud-messaging/flutter/receive which is applicable for Android 13 and newer and check if it helps or not.

this

I don't think so because I downgrade to firebase messaging v14.7.4 now everything seems to working perfectly. There is no crash and it ask notification permission again. Problem is seems to be in v14.7.10

Another bug in Android.

Please file a new issue for this so that we can track and address it separately.

I am unable to replicate the reported crash using the plugin example running on latest versions and on Android emulator, as it requested the permissions properly.  @aykutuludag  Can you use the same plugin example and check at your end if you still get the reported behavior ?

I'm having the same problem. Please help @aykutuludag  @darshankawar

I'm having the same problem. onBackgroundMessage is triggered when app is in foreground on android, then there is a crash Please help @aykutuludag @darshankawar

For now, I'm using 14.7.4 and there is no error. Just downgrade the plugin.

Thanks for the update. Based on the report and as I was unable to replicate it, keeping the issue open for team's attention / tracking.

Hello @aykutuludag, I've tried to reproduce on the latest version without success. Can you try to create a complete example that we could git clone easily to see what's going on.
You have a couple of other dependencies, and one of them might interact unexpectedly with messaging


Can only view ars0n dashboard when localhost

I recently made a docker container for ars0n framework so I can run it on one of my servers. The problem is the website is not fully accessible unless you are accessing it on localhost. Same goes for just a regular install. Switch to bridged adapter in vbox and try to access the website from the host box and you will see what I mean.

Yep, that's actually how the tool is designed.  At least for this phase of development.  I may choose to refactor the code so it can be run in the way you are describing, but not for at least a year or two.


Initial Migration Failing

Netbox 3.7.0
Plugin 0.6.4
Python 3.10.12

Installing the plugin - the following error when attempting to run the initial migration. Not all that familiar with Django is someone is able to assist.
django.db.migrations.exceptions.NodeNotFoundError: Migration netbox_documents.0007_circuitproviderdocument dependencies reference nonexistent parent node ('extras', '0107_cachedvalue_extras_cachedvalue_object')

Make sure you're netbox instance is really up to date. This dependency got added in the last update

Make sure you're netbox instance is really up to date. This dependency got added in the last update

But officially 3.7 isnt even supported or mentioned in the readme. Is 0.6.4 usable with 3.7?

This plugin should work fine with netbox < 4.0. The referenced migration is only 3 months old: https://github.com/netbox-community/netbox/blame/develop/netbox/extras/migrations/0107_cachedvalue_extras_cachedvalue_object.py


wslu: 4.1.0 -> 4.1.1

Description of changes

Also sets PREFIX to ensure scripts are installed under /bin and not /usr/bin (the default):
https://github.com/wslutilities/wslu/blob/dev/master/Makefile#L3
Things done

Built on platform(s)

[X] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[X] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[X] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 208895 run on x86_64-linux 1

1 package built:

wslu

@JamieMagee anything holding it up from merging?

Nope!

Replacing with #211396 because force pushing here is broken


Question about Relation

Laravel Version: 5.4
Voyager Version: 0.11

Description:
I want to show referer relationship in users edit(add) page.
User can be affiliated by other user, and i need to show who affiliate him.
Now i create relation in dropdown select, but there is showing the name of current user like

Self registered - need to show when the user registered himself
Admin - everything is fine here
User - exclude current user name from relation , need to show only other users
My relation in user model
public function referredBy()
{
     return $this->hasOne($this, 'affiliate_id', 'referred_by');
}

Can anyone do this, can someone help?

Auuuuuuu is there anyone can halp me?))))
@fletch3555 ?

Labels: question, enhancement

Please share the BREAD settings for this field.


Update FAQ Page

https://simtk-confluence.stanford.edu:8443/display/OpenSim/Frequently+Asked+Questions

I took a pass and made a few updates and deleted a few questions that were no longer relevant. @aymanhab It would be helpful if you could take a pass, as well.

@jenhicks I updated the sections related to installation, visualization. Not sure Scripting is up-todate and import/export data still has TODOs Let me know how you want to handle the remaining sections.

I took a first pass at updating the Scripting section and addressing the TODOs.

I'll move this to Needs Verifying.

Currently, the OpenSim Plotter can load and plot data from a C3D file.

IIRC, the Plotter was able to access only the first table in the C3D file; if still the case, perhaps mention here or direct the reader to more information (e.g., where known limitations and plans for future development are listed)?

Updated.

This is ready for verifying.

Looks good ... moving to verified. Thanks!


Proto - Remote executor: instance recovery

On startup:

Checks the remote tfstate:

If we have the job running and the id is diff of what we have another executor must have been launched. Die.

On execution:

Update cloud state with logs

On termination:

Job completed:

Updates cloud state with logs
Updates remote state logs with machine logs and completion time and destroying instance details
updates remote state with completed

Detects SIGTERM or cloud spot termination:

launches a new Instance
updates the remote state logs with termination and new request logs
updates the remote state with provisioning

Acceptance tests
WIP

We can safely skip logs for answering the question about whether we can support spot recovery in the Terraform client approach.
At a high level, the acceptance tests for this should be something like:

The instance terminates during task execution. A new instance spins up and completes the task.
A user requests an instance with a spot price that's too low. After waiting for an instance to become available, the instance spins up and the task is executed.
The task fails with a non-zero exit code. The instance is terminated or at least stops.

Closed with #237


Not able to see upcoming matches

BUG REPORT
Bug: {Not able to see upcoming matches for Argentina}

SUGGESTION
Suggestion: {Should be able to see, upcoming matches of team}

Versions

NodeJS version: v10.5.0
Football CLI version:  Not able to get it.

Error
Was not able to see upcoming match of argentina, which according to my timezone is today

Expected behaviour
Should be able to see upcoming match
Steps to recreate
Have added screenshot

Hi @murarisumit,
There is a --next flag which needs to be invoked for upcoming fixtures. The command defaults to past fixtures.
Here is the updated command:
$ football fixtures --next

Regarding version issue, I've raised a PR (#40) for it which should fix it.


”Homoroade” is listed in the drop-down menu

Description:
"Homoroade" is listed in the drop-down menu alongside its corresponding county and commune, according to the law.
Precondition:
The website is up an running.

Step 1
Type "Homoroade" in the search bar.

Expected results
The location is listed in the drop-down menu as "Comuna Homoroade, SATU MARE".


Release 1.2.0

close #200 #193

How would one fix the deprecation introduced in this release? I feel like I'm missing a step here.

@pichfl the fixes has already being made. This PR only releases it.
@snewcomer any expectations to release that?

Hence why I'm asking. Installing this release will result in a deprecation warnings without a clear path on how to fix them. The text inside the warning is not helpful for consumers of the add-on.
If I'm wrong, please point me and other people running into the same deprecation into the right direction.

@pichfl is right... the following deprecation warning is confusing. Help appreciated!
DEPRECATION: If you are overriding ember-router-scroll's implementation of "pushState", then you can subclass and override a new location object from: import HistoryLocation from '@ember/routing/history-location'; [deprecation id: ember-router-scroll]
https://github.com/DockYard/ember-router-scroll/blob/695d89485927f6637a9c2631904b67bbc3f3087a/addon/locations/router-scroll.js#L11-L41


"Multiple" playout mode isn't working for me

I've set up a Collection of Nickelodeon shows:

I created my Schedule with the "Multiple" option enabled and set to 2:

But the Playout is only playing 1 episode of each show before moving to a new one. The XML seems to be the same:

The playout modes are described here https://ersatztv.org/docs/user-guide/create-channels#schedule-items

Multiple: Play the specified Multiple Count of media items from the collection before advancing to the next schedule item.

In this case, your collection "Nickelodeon" has multiple shows that are all shuffled together, and ETV will play 2 items from that shuffle before advancing to the next schedule item. Since there is only one schedule item, it is effectively the same as flood.
I think you're touching on a scheduling scenario that is not yet supported very well. The current way to achieve what you're looking for is to create a schedule item for each show, configured as chronological and multiple 2 (or whatever is appropriate for that show). You can then edit the schedule itself to shuffle schedule items (meaning it would randomize the order of shows). This is done in the same place that you would edit the schedule name.
With that configuration, the order of shows would be shuffled, and each show would play its configured number of episodes in chronological order before advancing to the next (randomly-selected) show.

@fmr-iv the latest develop release has a new feature Fill With Group Mode that I think will be closer to your expectations.

See #1534


Stage 3 Review

This doesn't appear to have changed much since my last review so it is fairly simple:
https://github.com/tc39/proposal-json-modules/blob/master/spec.html#L81

if { type: 'json' } should be assert { type: 'json' }

per updated import assertions spec.

The above text implies that is recommended but not required that hosts do not use moduleRequest.[[Assertions]] as part of the module cache key. In either case, an exception thrown from an import with a given assertion list does not rule out success of another import with the same specifier but a different assertion list.

We probably don't want to imply a recommendation (since web and compatible envs itself won't do this exactly); the text above it isn't necessarily a recommendation but is an observation . Maybe a reframing:

The above text implies that is recommended but not required clarifies that hosts may or may not do not use moduleRequest.[[Assertions]] as part of the module cache key. In either case, an exception thrown from an import with a given assertion list does not rule out success of another import with the same specifier but a different assertion list.

Rest seems fine. 👍🏼

It was pretty intentional that it be phrased as a recommendation - that the web can't guarantee complying with the recommendation in all cases doesn't change what's recommended.

It was pretty intentional that it be phrased as a recommendation - that the web can't guarantee complying with the recommendation in all cases doesn't change what's recommended.

@ljharb it should not be framed that way as no env I know of would follow it.

@ljharb it should not be framed that way as no env I know of would follow it.

I think it needs to be framed that way even if no env follows it - but also, browsers already will be following it except where servers prevent them from doing so; i hope node does; and bundlers and transpilers certainly will.

I think it needs to be framed that way even if no env follows it - but also, browsers already will be following it except where servers prevent them from doing so; i hope node does; and bundlers and transpilers certainly will.

@ljharb I think it must not be framed that way. It does not match even Node's fs usage if fs manipulation occurs during the loading process. If we frame it as a recommendation we must frame it in such a way that explains that it almost certainly can never match a real env.

@ljharb I think it must not be framed that way. It does not match even Node's fs usage if fs manipulation occurs during the loading process. If we frame it as a recommendation we must frame it in such a way that explains that it almost certainly can never match a real env.

Perhaps we could introduce our first "intention" note into the spec here? I know we had a similar issue where graph re-ordering was intended to be banned but was unable to actually be enforced.

Perhaps we could introduce our first "intention" note into the spec here? I know we had a similar issue where graph re-ordering was intended to be banned but was unable to actually be enforced.

It matches a real env under normal circumstances - which includes no server shenanigans and no filesystem manipulation. It’s not a requirement because these things can’t be guaranteed not to happen, but it’s a recommendation because in their absence, the guarantees should be kept (and will be, by browsers, and hopefully by node).

It matches a real env under normal circumstances - which includes no server shenanigans and no filesystem manipulation. It’s not a requirement because these things can’t be guaranteed not to happen, but it’s a recommendation because in their absence, the guarantees should be kept (and will be, by browsers, and hopefully by node).

@ljharb I disagree, a host does not control things and they are real possibilities, not shenanigans as you so claim. If it is a recommendation to assert guarantees and the guarantees cannot be achieved it should not be a recommendation. This sounds like an intention.

@ljharb I disagree, a host does not control things and they are real possibilities, not shenanigans as you so claim. If it is a recommendation to assert guarantees and the guarantees cannot be achieved it should not be a recommendation. This sounds like an intention.

To be clear, this comes from the fact that we are a standards body and the goals of interoperability both through documentation and creation of behaviors is not being done with a recommendation here. Stating something about behavior outside of the intention not being considered breaking if altered seems apt, but we cannot recommend something here if it cannot be actually achieved by anyone.

To be clear, this comes from the fact that we are a standards body and the goals of interoperability both through documentation and creation of behaviors is not being done with a recommendation here. Stating something about behavior outside of the intention not being considered breaking if altered seems apt, but we cannot recommend something here if it cannot be actually achieved by anyone.

I’m confused; this was a requirement for me for stage 3, and i thought we’d resolved it months ago.
A host does not control these things which is why it’s not a requirement. However, in the absence of things a host does not control, the host is recommended to employ the stronger guarantee. It is very important to me that the spec language do whatever it can to maximize occurrences of the stronger guarantee (which matches what browsers will be doing in the absence of these things).

I’m confused; this was a requirement for me for stage 3, and i thought we’d resolved it months ago.
A host does not control these things which is why it’s not a requirement. However, in the absence of things a host does not control, the host is recommended to employ the stronger guarantee. It is very important to me that the spec language do whatever it can to maximize occurrences of the stronger guarantee (which matches what browsers will be doing in the absence of these things).

@ljharb we had several calls about this in which I did state that it should not be a recommendation but was fine with a rewording.

@ljharb we had several calls about this in which I did state that it should not be a recommendation but was fine with a rewording.

which matches what browsers will be doing in the absence of these thing

The point of our standard is to define the behaviors and stating that something is outside of behaviors we will be defining seems fine to me. Stating that a real behavior in all our implementations shouldn't happen doesn't make sense.

which matches what browsers will be doing in the absence of these thing

The point of our standard is to define the behaviors and stating that something is outside of behaviors we will be defining seems fine to me. Stating that a real behavior in all our implementations shouldn't happen doesn't make sense.

As i recall, you stated it must not be a requirement - a recommendation is an editorial status which has no normative weight, and i was under the impression that this was acceptable. How would you suggest rewording it?
It’s a rare behavior, and stating that hosts should strive, outside a rare behavior, to behave in a stronger way, makes perfect sense to me.

As i recall, you stated it must not be a requirement - a recommendation is an editorial status which has no normative weight, and i was under the impression that this was acceptable. How would you suggest rewording it?
It’s a rare behavior, and stating that hosts should strive, outside a rare behavior, to behave in a stronger way, makes perfect sense to me.

How would you suggest rewording it?

Various possibilities from RFCs seems good to use, but we don't use that RFCs for wording

"REALLY SHOULD NOT use the assertion in the cache key" from RFC 6919 seems the best fit.
"MUST (BUT WE KNOW YOU WON'T)" from RFC 6919 seems possible? It is strictly stronger than a recommendation but explaining the why is complex and likely not useful to the spec text itself.
"MAY WISH TO use the assertion in the cache key" from RFC 6919 seems strange / implies the wrong emphasis to me as it lacks a negative association with the behavior.
"WOULD PROBABLY" from RFC 6919 seems strange to me.
"SHOULD"  is what RFC 2119 uses to alias "recommended", likely it wouldn't make sense here since it states there are valid or even potentially useful behaviors (to my knowledge no useful/valid behavior is known to use this)

"Intention" seems fine, but I would prefer we actually document somewhere what it means.

"Clarification" seemed fine to me above but my text above doesn't state anything about intent of host implementations.

It’s a rare behavior, and stating that hosts should strive, outside a rare behavior, to behave in a stronger way, makes perfect sense to me.

Strive for and recommendation are very different.
Recommendation implies it is possible since recommending something that is not possible doesn't make sense; to my knowledge the only way this is possible is with bundling an entire codebase and pre-linking the module graph in some way to make all modules static and unable to be altered during runtime.

How would you suggest rewording it?

Various possibilities from RFCs seems good to use, but we don't use that RFCs for wording

"REALLY SHOULD NOT use the assertion in the cache key" from RFC 6919 seems the best fit.
"MUST (BUT WE KNOW YOU WON'T)" from RFC 6919 seems possible? It is strictly stronger than a recommendation but explaining the why is complex and likely not useful to the spec text itself.
"MAY WISH TO use the assertion in the cache key" from RFC 6919 seems strange / implies the wrong emphasis to me as it lacks a negative association with the behavior.
"WOULD PROBABLY" from RFC 6919 seems strange to me.
"SHOULD"  is what RFC 2119 uses to alias "recommended", likely it wouldn't make sense here since it states there are valid or even potentially useful behaviors (to my knowledge no useful/valid behavior is known to use this)

"Intention" seems fine, but I would prefer we actually document somewhere what it means.

"Clarification" seemed fine to me above but my text above doesn't state anything about intent of host implementations.

It’s a rare behavior, and stating that hosts should strive, outside a rare behavior, to behave in a stronger way, makes perfect sense to me.

Strive for and recommendation are very different.
Recommendation implies it is possible since recommending something that is not possible doesn't make sense; to my knowledge the only way this is possible is with bundling an entire codebase and pre-linking the module graph in some way to make all modules static and unable to be altered during runtime.

Browsers intend to use the stronger guarantee most of the time, as would, presumably, node - that’s why i find “should” appropriate. In other words, it’s quite possible often, just not always, which is the only reason it’s not a requirement.

Browsers intend to use the stronger guarantee most of the time, as would, presumably, node - that’s why i find “should” appropriate. In other words, it’s quite possible often, just not always, which is the only reason it’s not a requirement.

@ljharb it isn't able to be controlled by browsers or node, how can they make a guarantee?

@ljharb it isn't able to be controlled by browsers or node, how can they make a guarantee?

They can’t, which is why when it happens, the weaker guarantee will kick in. Until it does happen, the stronger guarantee would hold.

They can’t, which is why when it happens, the weaker guarantee will kick in. Until it does happen, the stronger guarantee would hold.

@ljharb the point is they can't make the stronger guarantee.

@ljharb the point is they can't make the stronger guarantee.

@ljharb what is wrong with "REALLY SHOULD NOT"?

@ljharb what is wrong with "REALLY SHOULD NOT"?

They can make them in the common case, just not always.
“REALLY SHOULD NOT”’s definition seems like a good fit (altho “OUGHT NOT” would fit my intentions better), but I’m not sure 262 follows that RFC so far.

They can make them in the common case, just not always.
“REALLY SHOULD NOT”’s definition seems like a good fit (altho “OUGHT NOT” would fit my intentions better), but I’m not sure 262 follows that RFC so far.

@ljharb they cannot guarantee even the common case since it is the server which isn't an ECMAScript implementation that causes the behavior to be observed due to the cache infrastructure. Browsers cannot assert guarantees on server behaviors.

@ljharb they cannot guarantee even the common case since it is the server which isn't an ECMAScript implementation that causes the behavior to be observed due to the cache infrastructure. Browsers cannot assert guarantees on server behaviors.

@ljharb "OUGHT NOT" seems fine to me but the usage of moral implications is uncomfortable since we are not here to make moral judgements.

@ljharb "OUGHT NOT" seems fine to me but the usage of moral implications is uncomfortable since we are not here to make moral judgements.

re using RFC6919 keywords, the publication date of that RFC is not a coincidence

lol thanks, I’d totally missed that.

I'm fine w/ whatever wording is chosen even if it is from what was originally a joke. We cannot recommend impossibilities since they cannot be done.

They aren’t impossible tho, that’s the crux i think of the disagreement here. They’re impossible for all cases, which is why it’s not required, but possible in many cases, which is why it’s what’s recommended.

@ljharb they are not possible for the majority of cases. Any case where there is a non-bundled form it has a race which cannot be done. If IO is considered the non-majority that would be one thing, but the vast majority of cases are subject to this. I would once again move to block if the spec is trying to state the impossible.

Even if a recommendation couldn't be followed in the majority case, we could still recommend it. The spec permits handling the cases you're concerned about - why is this recommendation so problematic for you?

@ljharb because you can't recommend someone do something that cannot be done. Reframing it to be recommended for when it is possible seems fine, but for the vast majority of cases that hosts are doing it isn't. I wouldn't want to state that I recommend people stop eating for example if they need to eat and then back peddle to say that they don't need to eat all the time.

The current phrasing says "It is recommended but not required" which already implies "when possible", would adding the words "when possible" address your concern?

Likely that would work. To my knowledge there is no implication of "when possible" in the current phrasing.

Great! I just filed #11; hopefully we can land that and move forward.

Just for the record, the following point has been fixed:

if { type: 'json' } should be assert { type: 'json' }

LGTM now since that has merged

Since we addressed the two @bmeck points, could we close this issue? and merge #12 ?

Wasn't @codehag another reviewer? Have we gotten signoff from her?

I am a reviewer. I haven't had time yet, the week since TC39 has been very busy.

I read it all, it looks good to me, I didn't find any outstanding issues. but I am also brain dead due to a really long week.
I can do another pass the week after next and should be able to give a definitive thumbs up.

@xtuc sounds good to close this and wait on the other review separately

@codehag edited her message and it looks we have a :+1: too.  Thanks, i'm going ahead and close the issue.


Beta Release 0.3.0-dev.4

[0.3.0-dev.4] - Refactoring, Bug Fix and Dependencies update: 2020-12-XX.

Breaking Changes (Not backward compatiable)
Bug Fix

Webview auto dispose (Fixed by webview_flutter)
Closing media content on widget dispose (Fixed by webview_flutter)
Pausing/Stop media content on Widget State Change (limited to Youtube Embed)

Remove Suport of Facebook Video/Post (Comming back soon)
Dependencies Verion Update

webview_flutter: 0.3.22+1 -> 1.0.7
url_launcher: 5.5.1 -> 5.7.10

#4 As flutter official team has updated webview_flutter, possibly that could have already fixed this issue.


RFC: Add small function ignore_result

This RFC talks about adding a small method ignore_result, similar to unwrap but that ignores Err / None values on Result and Option
Rendered

What is the advantage of
foo.insert("maybe an already inserted key", "bar").ignore_result();

over
let _ = foo.insert("maybe an already inserted key", "bar");

? The latter is shorter and more general (and doesn't require a new method).

It's much clearer to newer Rustaceans to include a method explicitly for this, that indicates the writer's intention to any readers, no matter their level of expertise in Rust. It also benefits from IDE implementations for auto-completion.

I agree that .ignore_result() would be easier to understand for beginners than let _ =.
However,

the latter already exists
is suggested by the compiler
changing all code in existence to use the proposed method instead would be a lot of work for very little benefit
I don't think let _ = is so unclear or confusing that it must be changed

In addition, an IDE could diagnose this and offer a quick fix to add the let _ =.

You can already ignore an Option, and you can make a Result into an Option by calling ok on it.

Imho let _ =  seems clearer, and better pedagogically, given the point of #[must_use]

Alternatively, one could use core::mem::drop, which does what it say on the tin and is conveniently available in the prelude.

I find that let _ = ... isn't intuitive for a lot of the Rust userbase. That's why there are posts on the Rust Forums and StackOverflow on this matter. But I guess that it is also possible and maybe adding a new method would be too complex for just this.
I'll close the issue, goodbye and thanks for giving your feedback!


What if window closes while logged in

Just a question.
I'm using localStorageService to store user credentials so the session persists on multiple tabs in same window and different windows of the same browser, offering persistent session even if the window closes; only close session button or timing out using this library effectively closes the session.
Is it possible to keep on watching the idle time while there is no window or tab? I log in, close the window or tab and wait the time set for the timeout. When I open the window again, should the session be closed or open?
I think that maybe because there is no context, no browser data at all? This library would start the countdown again, right?
Thanks.

You are correct, it's session-based so when the browser closes, all the idle information goes again. When going back to the site, you'll have to start watching again and it'll be starting a new session.
There currently isn't support for a permanent session. I suppose it could be possible by letting you choose to use localStorage instead of sessionStorage.


feat(grid): add usage page

Closes #3619
Changelog
New

2x Grid Usage page
Add redirect from Implementation to Usage

Removed

2x Grid Implementation page

related files/images

TODO

[x] Add updated security link (first resource card on the page)

https://carbondesignsystem-git-fork-alisonj-999ba2-carbon-design-system.vercel.app/guidelines/2x-grid/usage

Thanks for taking the time to proof-read @tw15egan I just copy/pasted the content from Figma without reading all of it. 😅

Aside from the copy changes I just committed @alisonjoseph the only visual thing I noticed was that the Do/Don't image here gets too close to the check mark/x mark in the large breakpoint...
It looks like these svgs are floating in the Do/Don't component, is there a way to add a slight bit more padding like 8px or 16px max?

@jeanservaas the padding will need to be added to the images

OK @alisonjoseph let's try this!
grid_img_updates.zip

Still maybe not enough padding, or I don't see an improvement... let's see what these do.
grid_img_updates.zip

@jeanservaas


How share a Singleton that implement 2 different interfaces in to different scopes ?

Sorry, this is not easy to find a good title.
I'm trying to build a multi-module app with TP3 ^^ (3.0.2)
Problem is simple:
I have 2 interfaces:
interface IFooWrite {
    fun write(n: Int)
}

interface IFooPrint {
    fun print()
}

I implemented those 2 interface into a single class
@Singleton
class Foo @Inject constructor() : IFooWrite, IFooPrint {
    var number: Int? = null

    override fun write(n: Int) {
        number = n
    }

    override fun print() {
        Log.d("##", number.toString())
    }
}

I want to first define a scope, inject IFooWrite in a first module. This feature is responsible to get a data and save it
Then in a second module, I need this data back, so I inject IFooRead, to then print the data.
So I defined the instance Foo as a Singleton, so I expected that both injection will provide me the same instance, that is not the case.
class MyApp : Application() {
    override fun onCreate() {
        super.onCreate()

        KTP.openScope("ROOT")
            .installModules(object : Module() {
                init {
                    bind(IFooWrite::class.java).to(Foo::class.java)
                }
            })
            .getInstance(IFooWrite::class.java)
            .write(42)

        KTP.openScope("ROOT")
            .openSubScope("SUB")
            .installModules(object : Module() {
                init {
                    bind(IFooPrint::class.java).to(Foo::class.java)
                }
            })
            .getInstance(IFooPrint::class.java)
            .print()
    }
}

This print "##: null"
I tried various try/hack but failed at the end !

There are no aliases in TP, which is the feature you are looking for: #248 .
You can still create the instance of your singleton manually and bind it twice using bindToInstance.
For now, it's the official workaround.

Ok now I have a name !
Could you tell us why this is not the normal behavior ?
From a descriptive perspective, I would expect such behavior, don't you ?
Maybe I didn't get the full picture, but defining alias doesn t make the description clearer (even if that would solve the implementation problem)

@stephanenicolas please have a look on my previous question, because I'm still don't understand why such code doesn't offer the behavior I expect (as mentioned https://github.com/stephanenicolas/toothpick/issues/248#issuecomment-330939778)
It seems I found a viable solution, would that make sens for you ?
Here are the providers:
class IFooWriteProvider : Provider<IFooWrite> {

    @Inject
    lateinit var foo: Foo

    override fun get(): IFooWrite {
        return foo
    }
}

class IFooPrintProvider : Provider<IFooPrint> {

    @Inject
    lateinit var foo: Foo

    override fun get(): IFooPrint {
        return foo
    }
}

I inject the singleton via TP inside the provider.
And the bindings:
        KTP.openScope("ROOT")
            .installModules(object : Module() {
                init {
                    bind(IFooWrite::class.java).toProvider(IFooWriteProvider::class.java)
                }
            })
            .getInstance(IFooWrite::class.java)
            .write(42)

        KTP.openScope("ROOT")
            .openSubScope("SUB")
            .installModules(object : Module() {
                init {
                    bind(IFooPrint::class.java).toProvider(IFooPrintProvider::class.java)
                }
            })
            .getInstance(IFooPrint::class.java)
            .print()

This is working well, and ok from my point of view.
However I thing you had another idea in mind, did you ? (I'm not sure if I understood the workaround)

Another linked topic. If I add this code (bind(Foo***), the expected behavior does'n work anymore. I had a look on the Factory and MemberInjector, but nothing seems to get changed. Any idea ?
     KTP.openScope("ROOT")
            .openSubScope("SUB")
            .installModules(object : Module() {
                init {
                    bind(IFooPrint::class.java).toProvider(IFooPrintProvider::class.java)
                    bind(Foo::class.java).singleton()
                    // bind(Foo::class.java) or this one
                }
            })
            .getInstance(IFooPrint::class.java)
            .print()

@afaucogney your workaround is also valid. As long as it works ! :)
But for your last comment, you should open a new ticket, this is not normal. Please close this one and open a new ticket.

We are looking at ways of make it work out of the box @afaucogney :)


Ignored trait parameters in where clauses

Allow specifying _ in where clauses when you don't care about the type used by the trait. For example, count itself doesn't care what the element type is, so Element can be replaced by _:
-- before
count :: Collection Element where (Iterate Collection Element) => Collection -> Natural

-- after
count :: Collection where (Iterate Collection _) => Collection -> Natural

The compiler will just synthesize a new unnamed type parameter.

Implemented in eeab040.


Better implementation of at_least

The old one used two calls of times()

I don't have the repository set up localy, I just fixed the bug on github and hoped it'd work straightaway... Could you do that for me instead?

I'm afraid I'm extremely unlikely to merge changes that have not even been tested, sorry.


How can I use middleware for admin in my route in laravel 5.4?

Hi,
I am using this code in laravel 5.4
`Route::prefix('admin')->middleware(['web', 'admin', 'auth:admin'])->group(function(){
Route::get('/{id}', 'AdminController@index');
});`
but it uses basic laravel user login not admin login from the admins table.
How can I check If the user is not admin go to /admin/login?

@marn65
All routes defined  in your  routes/admin.php is protected by admin middle-ware already. It means no other than admin user can access these routes.

thank you