Retained Phantom Mailboxes do not show up in Active Mailboxes

Unlike online mailboxes when a retention policy is applied to a Phantom Mailbox and the account is deleted the mailbox does not appear in the Inactive Mailbox section in the Compliance Center. Additionally, these accounts are no available to place on Holds or resolve using Content Search.
In Content Search you can use the participant action to return Teams Chats however there is now way to put a hold on this content at this in the UI that I could find.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: ab0bb4d7-d9fa-ae92-d3ca-246147d1434a
Version Independent ID: 7ba20db3-55e2-3d29-2d6c-6f68a6b577a8
Content: Searching cloud-based mailboxes for on-premises users - Microsoft 365 Compliance
Content Source: microsoft-365/compliance/search-cloud-based-mailboxes-for-on-premises-users.md
Service: o365-seccomp
GitHub Login: @markjjo
Microsoft Alias: markjjo

@Mlavieri First of all I think you should open a ticket with MS Support to investigate these issues/questions.
Having said that, engineering says that an inactive mailbox would be created if you deleted the on-premises user (and that deletion is synced to AAD). They also said that on-premises users whose identities are synced to AAD should be returned in the user picker to assign a retention policy or eDiscovery hold to. How did you apply the retention policy on the on-premises user before the account was deleted? Also note that to be licensing compliance, the on-premises user would need an EXO Plan 2 license, which is the requirement to make any mailbox inactive.
Also, I'm a little confused about a few things:

"...these accounts are no available to place on Holds" If by accounts you mean the on-premises user whose account was deleted, then this would be expected behavior. You can't place a hold on an "inactive mailbox".

Have you tried using the Get-Mailbox -InactiveMailboxOnly command in EXO PowerShell? Curious if it will return the inactive shadow mailbox.

Have you tried using the New-ComplianceSearch -IncludeUserAppContent command in SCC PowerShell to search the inactive phantom mailbox?

Not sure I understand what you mean by "put a hold on this content". I assume that you are searching the inactive phantom mailbox, whose content is already on hold because a retention policy was applied to it before you deleted the on-premises user.

Thanks again Mark, there may have been a licensing issue when we were testing this all out but the tenant has EXO Plan 2, I'll follow up with a ticket if we see that issue again. With regards to your other notes:

I was not aware that an Inactive Mailbox could not be placed on hold which in all honestly renders the less of my questions moot. The scenario that I was attempting to solve for is when there is a Global Retention policy applied to all Teams Chats for the tenant and an employee leaves the firm and on a later date there is a litigation matter where that user's data needs to be put on hold. The Retention Policy would retain that chat until the end of the the retention policy but the litigation hold may end up being longer than that retention policy. While there are other ways to solve for this we were hoping it could be accomplished via a hold.
The mailbox does not appear with the EXO command
Even in compliance search or holds I am unable to search the mailbox
Yes the phantom mailbox had a retention policy assigned to it before it was deleted, I believe this is related to the first point about not being able to place an Inactive Mailbox on hold or an issue with the licensing which is causing the deleted phantom mailbox not to show up but I can follow up with a specific ticket.

Thanks for the answer and your help on this!


Unified error handling, comments and naming.

@mesosphere-ci run all integration tests

@mesosphere-ci run all integration tests


autopilot  rewrite rs function

@autopilot please rewrite the rust function in elf-extract/src/lib.rs into c++ functions

@autopilot write_vtable

Hey @zhangstevenunity I'm the creator of codeautopilot. Just reaching out to say that autopilot was being very stubborn on your request. I noticed that from my logs and adjusted it to allow tasks like the one you created in this issue.
Since the repo is public I thought about reaching out here directly. Hope that it's not too invasive. Cheers


Does RepeatedField need a len field?

I'm a bit confused by the implementation of RepeatedField. It seems to me that for all means and purposes it could be a newtype:
struct RepeatedField<T>(Vec<T>);

impl<T> Deref for RepeatedField<T> {
    type Target = Vec<T>;
    fn deref(&self) -> &Vec<T> { &self.0 }
}

impl<T> DerefMut for RepeatedField<T> {
    fn deref_mut(&mut self) -> &mut Vec<T> { &mut self.0 }
}

with Deref and DerefMut implementations for Vec rather than [T] (so as to forward method calls like .push, .extend etc to the inner vec).
I'm sure I'm missing some clever trick with the len field, could anyone explain why it's implemented the way it is?

To reuse previously allocated memory.
I've explained it there: #84.

Thank you! Sorry I didn't see the previous issue


core-providers (audio): removes deprecated status property

#8177

Maintainer merge checklist

[x] Title is descriptive/clear for inclusion in release notes.
[x] Applied a Component: xxx label.
[ ] Applied the api-deprecation or api-break label.
[x] Applied the release-highlight label to be highlighted in release notes.
[x] Added to the milestone version it was merged into.
[ ] Unittests are included in PR.
[ ] Properly documented, including versionadded, versionchanged as needed.

Hi @Zerus97 !
We're using this deprecated property in one of our examples 😅
Can you also change the following in order to not use the deprecated property, so we are sure to not break examples / demos?
https://github.com/kivy/kivy/blob/ee040926edd6e68d13e8799484c25d6e2911397b/examples/audio/main.py#L36

Hi @misl6 , how are you doing?
Ive just commited the change you asked for using the state property as indicated.

Hi @Zerus97 !
We now have a conflict, can you resolve it? (make sure to rebase on upstream/master)


OpenID requires SQL in farms

It should really be mentioned in the prerequisites that OpenID will not work with using WID in an ADFS farm with multiple servers. Those ADFS servers all have to point to a centralized SQL server for OpenID to function. I discovered this the hard way, and I'm sure it would be helpful for others to know. Thanks

OpenID actually works with WID as database in an ADFS farm

At least with my installation that has three load balanced servers in it, OpenID/Oath would not function correctly against the two secondary servers. It worked fine against the primary but that meant it failed 2/3rds of the time. We believe it had something to do with some artifacts not replicating from the primary server's WID to the others. Once we converted all three to use a centralized database all of those problems cleared up instantly.

Artifact is a part of SAML protocol, so that's not the problem.
Actually, there are an issue with WID and ADFS farm where some keys were not synchronized, but this is a known issue and a KB was performed. See KB4038801
On my side, I have a farm with 3 ADFS servers and WID, and OpenID works well ! :)

Ah makes sense. We ran into that wall in June of last year, and the patch hit in September. If I build another farm I'll give it a spin. Thanks for the info.


Add itla

URL: https://itla.edu.do/
Curriculum: https://itla.edu.do/images/pdf/pensums-2020/pensum-2020-sotfware.pdf
URL Showing Email Address in USE: https://itla.edu.do/#:~:text=809-793-2557-,info%40itla.edu.do,-TÉRMINOS DE USO

@DannyFeliz I am sorry, the domain itla.edu.do is removed from our trusted domains list because it was abused in the past. If you are the student of this school, please enroll to GitHub Student Developer Program. Then apply for JetBrains educational subscription with your GitHub account (GitHub tab at https://www.jetbrains.com/shop/eform/students).


Trouble testing react-redux connected components

Hello,
I'm trying to test a connected component. For some reason, I'm receiving the following error only when testing connected components which throws me off a little...
'ReferenceError: navigator is not defined'
import React from 'react';
import { Provider } from 'react-redux';
import { Row } from 'react-bootstrap';
import chai, { expect } from 'chai';
import dirtyChai from 'dirty-chai';
import chaiEnzyme from 'chai-enzyme';
import { mount } from 'enzyme';
import sinon from 'sinon';
import jsdom from 'jsdom';
import configureMockStore from 'redux-mock-store';
import thunk from 'redux-thunk';
import promiseMiddleware from 'redux-promise-middleware';
import PAGE_INITIAL_STATE from '../../reducers/page_reducer';

const middlewares = [thunk, promiseMiddleware()];
const mockStore = configureMockStore(middlewares);

chai.use(dirtyChai);
chai.use(chaiEnzyme());

import KbbSuperHero from './kbb_super_hero.jsx';

const doc = jsdom.jsdom('<!doctype html>');
global.document = doc;
global.window = doc.defaultView;

describe('<KbbSuperHero /> = Displayed on KBB Landing Page', () => {
  const mStore = mockStore({app: {kbbToken: 'xxxxxxxxxxx'}, page: PAGE_INITIAL_STATE});
  const spy = sinon.spy();
  const wrapper = mount(
    <Provider store={mStore}>
      <KbbSuperHero showModal={spy} />
    </Provider>
  );

  it('should display "Get Your Instant Cash Offer"', () => {
    expect(wrapper.find('h1').first().text()).to.equal('Get Your Instant Cash Offer');
  });

});

what happens if you do this line after assignign global.window: is it undefined?
console.log(window.navigator)

@blainekasten
Ended up including the following which seems to have resolved the issue.
global.navigator = {
  userAgent: 'node.js'
};

@michael-iglesias our docs outline what we recommend for setting up a jsdom environment for use with enzyme: http://airbnb.io/enzyme/docs/guides/jsdom.html


Doesn't work in Intellij version 2021.1

Plugin complains with error:
"Incompatible: requires IDE build 203.* or older"
when starting up Intellij version 2021.1.

Fixed by #12 and released just now as v0.2.1 (publish workflow is still running).


Shell command files not found

According to https://github.com/opennars/opennars,
Command Shell Usage
You can launch NARS in several ways from command window (shell):

empty reasoner (reads from stdin and writes to stdout):
./nars.sh

gui (Swing)
./gui.sh [.nal input file]

reasoner loaded from an experience file:
./nars.sh Examples/Example-NAL1-edited.txt --silence 90

web server start for HTML GUI usage:
./web.sh 9999 localhost 9100 1000
listen port^     ^IP    ^port ^time for each reasoning step in ms

However, these files were not included in the git distribution.
and
nars_core - reasoning engine
nars_gui - java.swing GUI
nars_web - web server
nars_lab - experiments & demos
nars_test - unit tests
nal - examples
structure also not present.

This is outdated.
We just support maven for now

https://github.com/opennars/opennars/pull/299 Pullrequest(PR) is pending

I close this now because the PR should make it today or this early week.


Add a HEALTHCHECK to the Dockerfile

curl isn't the greatest thing to use here because with -f it doesn't output
anything on failure, so if the healthcheck fails you don't get to see the JSON
from the healthcheck endpoint in 'docker inspect' (you will see it for a
successful healthcheck)
This directive is respected by Docker, Docker Compose and AWS ECS.
'docker inspect' output will look something like this for a failing healthcheck:
   [
       {
   [...]
           "State": {
   [...]
               "Health": {
                   "Status": "unhealthy",
                   "FailingStreak": 0,
                   "Log": [
                       {
                           "Start": "2019-08-10T10:50:39.9300904Z",
                           "End": "2019-08-10T10:50:40.3198947Z",
                           "ExitCode": 1,
                           "Output": ""
                       }
                   ]

Closing as this branch has gone stale


'pytorch_lightning.loggers' has no attribute 'TestTubeLogger'

continuing from my previous attempts i now get this error, apparently pytorch has removed this module 'TestTubeLogger'.
also tried installing the setup.py requirements for the torch  version
pytorch-lightning==1.5.9
pytorch=1.10.2
tried both:
python3 main.py --base configs/stable-diffusion/v1-finetune.yaml -t --actual_resume models\ldm\stable-diffusion-v1\model.ckpt -n FireballRun --data_root object\artstyle\scene --gpus 1 --init-word TrainedFireball
python3 main.py --base configs/latent-diffusion/txt2img-1p4B-finetune.yaml -t --actual_resume models/ldm/text2img-large/model.ckpt -n FireballRun --data_root object\artstyle\scene --gpus 1 --init-word TrainedFireball
Restored from models\ldm\stable-diffusion-v1\model.ckpt with 0 missing and 2 unexpected keys
Unexpected Keys: ['model_ema.decay', 'model_ema.num_updates']
Traceback (most recent call last):
  File "D:\Workspace\textual_inversion-main\main.py", line 646, in <module>
    trainer_kwargs["logger"] = instantiate_from_config(logger_cfg)
  File "D:\Workspace\textual_inversion-main\ldm\util.py", line 85, in instantiate_from_config
    return get_obj_from_str(config["target"])(**config.get("params", dict()), **kwargs)
  File "D:\Workspace\textual_inversion-main\ldm\util.py", line 93, in get_obj_from_str
    return getattr(importlib.import_module(module, package=None), cls)
AttributeError: module 'pytorch_lightning.loggers' has no attribute 'TestTubeLogger'. Did you mean: 'NeptuneLogger'?

did a total fresh env install with the same error as result
conda env create -f environment.yaml  naming the env different.

Ok, so the python at C:\Users\MyName.conda\envs\ldmt\python.exe is what you actually want, yes.
On the gpu errors, per the readme, --gpus xxxx this is a list of GPU indices that you want to use, not the number of gpus. So if you have only a single GPU, you should be using --gpus 0,.
I don't have a good guide for windows errors, but you can try to follow the steps here (I wouldn't comment out the assert torch.count_nonzero(tokens - 49407) == 2... line).
You can also have a look here: #12. And here: #14

added these lines
import signal import platform if platform.system() == 'Windows':     os.environ["PL_TORCH_DISTRIBUTED_BACKEND"] = "gloo"     signal.signal(signal.SIGTERM, melk)     signal.signal(signal.SIGTERM, divein) else:     signal.signal(signal.SIGUSR1, melk)     signal.signal(signal.SIGUSR2, divein)
Then this code did run:
python main.py --base configs/latent-diffusion/txt2img-1p4B-finetune.yaml -t --actual_resume models/ldm/text2img-large/model.ckpt -n FireballRun --data_root object\artstyle\scene --gpus 1 --init-word TrainedFireball
can i use this output with stable diffusion?
This did not run and gave me the following error:
python main.py --base configs/stable-diffusion/v1-finetune.yaml -t --actual_resume models\ldm\stable-diffusion-v1\model.ckpt -n FireballRun --data_root object\artstyle\scene --gpus 0 --init-word TrainedFireball
`Project config
model:
base_learning_rate: 0.005
target: ldm.models.diffusion.ddpm.LatentDiffusion
params:
linear_start: 0.00085
linear_end: 0.012
num_timesteps_cond: 1
log_every_t: 200
timesteps: 1000
first_stage_key: image
cond_stage_key: caption
image_size: 64
channels: 4
cond_stage_trainable: true
conditioning_key: crossattn
monitor: val/loss_simple_ema
scale_factor: 0.18215
use_ema: false
embedding_reg_weight: 0.0
unfreeze_model: false
model_lr: 0.0
personalization_config:
target: ldm.modules.embedding_manager.EmbeddingManager
params:
placeholder_strings:
- ''
initializer_words:
- sculpture
per_image_tokens: false
num_vectors_per_token: 1
progressive_words: false
embedding_manager_ckpt: ''
placeholder_tokens:
- ''
unet_config:
target: ldm.modules.diffusionmodules.openaimodel.UNetModel
params:
image_size: 32
in_channels: 4
out_channels: 4
model_channels: 320
attention_resolutions:
- 4
- 2
- 1
num_res_blocks: 2
channel_mult:
- 1
- 2
- 4
- 4
num_heads: 8
use_spatial_transformer: true
transformer_depth: 1
context_dim: 768
use_checkpoint: true
legacy: false
first_stage_config:
target: ldm.models.autoencoder.AutoencoderKL
params:
embed_dim: 4
monitor: val/rec_loss
ddconfig:
double_z: true
z_channels: 4
resolution: 512
in_channels: 3
out_ch: 3
ch: 128
ch_mult:
- 1
- 2
- 4
- 4
num_res_blocks: 2
attn_resolutions: []
dropout: 0.0
lossconfig:
target: torch.nn.Identity
cond_stage_config:
target: ldm.modules.encoders.modules.FrozenCLIPEmbedder
ckpt_path: models\ldm\stable-diffusion-v1\model.ckpt
data:
target: main.DataModuleFromConfig
params:
batch_size: 2
num_workers: 2
wrap: false
train:
target: ldm.data.personalized.PersonalizedBase
params:
size: 512
set: train
per_image_tokens: false
repeats: 100
validation:
target: ldm.data.personalized.PersonalizedBase
params:
size: 512
set: val
per_image_tokens: false
repeats: 10
--init-word: null
TrainedFireball: null
Lightning config
modelcheckpoint:
params:
every_n_train_steps: 500
callbacks:
image_logger:
target: main.ImageLogger
params:
batch_frequency: 500
max_images: 8
increase_log_steps: false
trainer:
benchmark: true
max_steps: 6100
accelerator: ddp
gpus: 0
| Name              | Type               | Params
0 | model             | DiffusionWrapper   | 859 M
1 | first_stage_model | AutoencoderKL      | 83.7 M
2 | cond_stage_model  | FrozenCLIPEmbedder | 123 M
3 | embedding_manager | EmbeddingManager   | 1.5 K
768       Trainable params
1.1 B     Non-trainable params
1.1 B     Total params
4,264.947 Total estimated model params size (MB)
Validation sanity check: 0it [00:00, ?it/s]C:\Users\Tristan.conda\envs\ldmt\lib\site-packages\pytorch_lightning\trainer\data_loading.py:132: UserWarning: The dataloader, val_dataloader 0, does not have many workers which may be a bottleneck. Consider increasing the value of the num_workers argument(try 32 which is the number of cpus on this machine) in theDataLoaderinit to improve performance.   rank_zero_warn( Validation sanity check:   0%|
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument index in method wrapper__index_select)

Looks like this is the same error as the person here, however they did not respond.
I don't have any environment where this happens, but I'd be happy to try debugging it with you for future users if you have the patience for this.
Could you try printing the device of the token embedder (with something like print(next(self.token_embedding.parameters()).device)) and the device of the input_ids (print(input_ids.device)) before line 181 in modules.py and letting me know what you get?

Looks like the model isn't running on your GPU for some reason. For example if you look at your latest output, it says:
GPU available: True, used: False
What parts of the gpu argument code did you modify?
Also - if you prefer to do this on discord so we can have quicker turnaround times, my handle is Sylvver#2478

Ok, so first of all with your change in the code, if you pass --gpus 0 then this code bit will have ngpu = 0 which means no GPU. In the above commands you posted that you used --gpus 1 for the LDM run, which is indeed what you want with this change.
The alternative is restoring the code in that block to the old version and running with --gpus 0, including the comma.
So to summerize:
Either keep the current code and use --gpus 1 or return to old code and use --gpus 0,.

Yep. You can reduce the batch size in the config (set batch_size: 1 both places it appears in).
If you do this, you might want to also add: accumulate_grad_batches: 2 to the trainer block at the bottom.

ok so in "textual_inversion-main\configs\stable-diffusion\v1-finetune.yaml" i set the following:
model:
  base_learning_rate: 5.0e-03
  target: ldm.models.diffusion.ddpm.LatentDiffusion
  params:
    linear_start: 0.00085
    linear_end: 0.0120
    num_timesteps_cond: 1
    log_every_t: 200
    timesteps: 1000
    first_stage_key: image
    cond_stage_key: caption
    image_size: 64
    channels: 4
    cond_stage_trainable: true   # Note: different from the one we trained before
    conditioning_key: crossattn
    monitor: val/loss_simple_ema
    scale_factor: 0.18215
    use_ema: False
    embedding_reg_weight: 0.0
    unfreeze_model: False
    model_lr: 0.0

    personalization_config:
      target: ldm.modules.embedding_manager.EmbeddingManager
      params:
        placeholder_strings: ["*"]
        initializer_words: ["sculpture"]
        per_image_tokens: false
        num_vectors_per_token: 1
        progressive_words: False

    unet_config:
      target: ldm.modules.diffusionmodules.openaimodel.UNetModel
      params:
        image_size: 32 # unused
        in_channels: 4
        out_channels: 4
        model_channels: 320
        attention_resolutions: [ 4, 2, 1 ]
        num_res_blocks: 2
        channel_mult: [ 1, 2, 4, 4 ]
        num_heads: 8
        use_spatial_transformer: True
        transformer_depth: 1
        context_dim: 768
        use_checkpoint: True
        legacy: False

    first_stage_config:
      target: ldm.models.autoencoder.AutoencoderKL
      params:
        embed_dim: 4
        monitor: val/rec_loss
        ddconfig:
          double_z: true
          z_channels: 4
          resolution: 512
          in_channels: 3
          out_ch: 3
          ch: 128
          ch_mult:
          - 1
          - 2
          - 4
          - 4
          num_res_blocks: 2
          attn_resolutions: []
          dropout: 0.0
        lossconfig:
          target: torch.nn.Identity

    cond_stage_config:
      target: ldm.modules.encoders.modules.FrozenCLIPEmbedder

data:
  target: main.DataModuleFromConfig
  params:
    batch_size: 1
    num_workers: 2
    wrap: false
    train:
      target: ldm.data.personalized.PersonalizedBase
      params:
        size: 512
        set: train
        per_image_tokens: false
        repeats: 100
    validation:
      target: ldm.data.personalized.PersonalizedBase
      params:
        size: 512
        set: val
        per_image_tokens: false
        repeats: 10

lightning:
  modelcheckpoint:
    params:
      every_n_train_steps: 500
  callbacks:
    image_logger:
      target: main.ImageLogger
      params:
        batch_frequency: 500
        max_images: 8
        increase_log_steps: False

  trainer:
    benchmark: True
    max_steps: 6100
    accumulate_grad_batches: 2

Not sure I have anything immediate to reduce it further if that still doesn't fit.
What GPU do you have? You could maybe try mixed-precision training if your GPU supports it.
Try adding trainer_kwargs['precision'] = 16 before line 740 in main.py and see if it helps.

i have a 3080 gpu
added to the main.py
      trainer_kwargs["callbacks"] = [instantiate_from_config(callbacks_cfg[k]) for k in callbacks_cfg]
      trainer_kwargs["max_steps"] = trainer_opt.max_steps
      trainer_kwargs['precision'] = 16

RuntimeError: CUDA out of memory. Tried to allocate 512.00 MiB (GPU 0; 12.00 GiB total capacity; 10.98 GiB already allocated; 0 bytes free; 11.01 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
if it doesn't fit then i guess it ends here :)
No way the latent model works together with stable diffusion? i wanted to use it with this ui thing.
https://github.com/hlky/stable-diffusion-webui

Oof, long way to come only to get stuck here :(
I'll try searching for ways to reduce this further and let you know if I have a good solution.
In principal the LDM model is not very different from the stable diffusion one, but you'll have to do a bit of tinkering with the web-ui to adapt it and I'm not sure you want to go there. The embeddings for one model will not work with the other, unfortunately.
If you have a specific concept or two that you want to train on, you can also send me your images and I can optimize an embedding here and send it to you. Not really a long-term solution though.

i guess if you add those windows changes, other ppl will have less issues :p
and me i've learned a thing or 2 now.
still exploring all of it and not yet something concrete, will send you on discord if something comes up.
poke me if you find something for the memory thing!
thanks again and have a good weekend!

@TristanMorelle @rinongal I was running into these errors too and found a way to get through this.
https://github.com/hlky/sd-enable-textual-inversion#q-why-am-i-receiving-a-sigusr1-error-why-am-i-receiving-an-ncnn-error-why-am-i-receiving-oserror-cannot-open-resource

This may address that user warning I shared above, in the steps listed by hlky it says to https://github.com/Lightning-AI/lightning/issues/10349
I've not tried it as I am just letting this run for a bit in case it's just a false alarm

Final Update: The model seems to be training on my images fine after the fixed shared in the hlky repo

I've got my 3080ti running through steps with these settings after a lot of head banging  today https://github.com/rinongal/textual_inversion/issues/50#issuecomment-1236284867

The initial problem here appears to be resolved, and the remaining problems appear to be covered by other threads, so I'm closing this. Feel free to reopen if you need more help.

AttributeError: module 'pytorch_lightning.loggers' has no attribute 'TestTubeLogger'. Did you mean: 'NeptuneLogger'?
this problem is how  to solve?everyone could tell me .Thanks.

pytorch_lightning = 1.7.2
python =3.7

AttributeError: module 'pytorch_lightning.loggers' has no attribute 'TestTubeLogger'. Did you mean: 'NeptuneLogger'?
this problem is how to solve?everyone could tell me .Thanks.

Use pytorch_lightning 1.6.5 if you could.
pip install pytorch_lightning==1.6.5
pip install test_tube


flatMap performance

The following filtering of an ObservableArray is extremely slow and resource hungry:
myobserver
  .flatMap(el => {return el; })
  .filter(el => {return !el.property; })
  .subscribe(validEl => {
    console.log(validEl);
  });

However, this isn't:
myobserver
  .subscribe(arr => {
    let filteredArr = arr.filter(el => { return !el.property; });
    console.log(filteredArr);
  });

A quick look at my laptop resources shows that the former takes CPU usage over the roof. Having tried different variations this only happens when I use the flatMap operator. switchMap has same results. I'm puzzled about this as the array's length is only 300
myobserver is an ObservableR returned by a BehaviourSubjectT
issue wrongly originally reported as https://github.com/Reactive-Extensions/RxJS/issues/1238

@cortopy could you provide more information to reproduce this behavior (sample data, etc.), or maybe a PR with a new performance test? I can't replicate your issue.

Same. I'm unable to replicate this issue. Do you have sample code or a JSBin or something?

@cortopy Also, what browser/Nodejs version are you using?

This is an Electron app developed in Angular 2. I'm not sure I could easily replicate it on JSBin
However, I have just been profiling CPU usage and it seems to point in the direction of subscriber methods. As mentioned earlier this is a small array of 200-250 elements, and the results are very puzzling to me.

In addition to this I've done some benchmarking as suggested. Unfortunately the --csv option didn't seem to work, so can only post txt format
rxjs_perf.txt
In regards to env, this is the stack:
Electron: 1.2.0 (includes Chrome 51)
Node: 6.1.0
Angular 2: rc1

@cortopy it's difficult to discern whether this is an issue with Rx, Electron, or the example you're profiling.
The presence of an observeOn operation in the profiler screenshot seems like a red flag. Do you have Chrome's async callstacks turned off? And it seems like you're mixing Rx 4 and Rx 5, which isn't necessarily advisable. Additionally, the mergeMap perf numbers in the text file are reporting a net gain against Rx 4, which seems to indicate Rx 5 mergeMap is working as expected.
I'd be happy to investigate further if you can distill the issue down to a bare-bones test case that I can load into Electron and profile myself.

Also, try choosing switchMap over flatMap as much as possible. Use flatMap only when you know that you need concurrency.

Since we can't replicate this perf bug, I'm going to close the issue for now. @cortopy if you're able to present a minimal example showing the perf bug, please do reopen this, because it would definitely be something we'd want to fix.
Thanks for filing the issue, sorry we couldn't locate any problems on our end.

Considering the culprits could be many, I'll just keep an eye and see if I can reproduce this somehow somewhere. Thanks for the input


Android 9.0 pie Api=28

i add MagicalCamera to my project and it doesnt work, is there any problem whit api 28? or is there something extra to do?

I also meet this problem in my app. Could you tell us a solution?


Helper SSH pod is not removed upon pod exit

Our documentation shares:
Remove SSH access
When done, exit the SSH session, stop any port forwarding, and then exit the interactive container session. After the interactive container session closes, the pod used for SSH access from the AKS cluster is deleted.
Although, this is not the case. Upon exiting the Pod it can either end up staying with Error/Completed status indefinitely, but will not be deleted automatically. Reproduced in my own environment.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 3b81a147-db1c-b92f-88d1-6dd0991a72fe
Version Independent ID: f5a2c949-498f-5848-d89c-031d1a757120
Content: SSH into Azure Kubernetes Service (AKS) cluster nodes - Azure Kubernetes Service
Content Source: articles/aks/ssh.md
Service: container-service
GitHub Login: @mlearned
Microsoft Alias: mlearned

@Lucas-MSFT Thanks for the feedback! I have assigned the issue to content author to check and update the document as appropriate. We will update you accordingly.

@mlearned Can you please check and add your comments on this doc update request as applicable

Thank you for the feedback.
We've updated the article to address your feedback and the changes should be available soon.
#please-close


Testing classes does not work well when combined with Puppet::Parser::Functions.function()

Given the following setup:

custom module
custom (rvalue) function in the module's lib/puppet/parser/functions/
a class in that module calling the function

Now, when rspec-testing that the function exists using a call to Puppet::Parser::Functions.function('foo') any class tests that include a call to the function will fail with "Function 'foo' does not return a value". This only happens if the function test is executed before the class test:
rspec spec/functions/foo_spec.rb spec/classes/foo_spec.rb # -> fails
rspec spec/classes/foo_spec.rb spec/functions/foo_spec.rb # -> works
Apparently, executing the function check has some side effects affecting the Puppet parser that persist throughout multiple checks.
The following tarball includes a minimal sample triggering the issue: http://osprey.tokkee.org/~tokkee/tmp/rspec-test.tgz
I'm not sure if this is due to some Puppet parser issue. However, imho, rspec tests should behave in the same way no matter in which order they are executed. Note, that when simply running "rspec" (without arguments) in the module's directory, the behavior is totally random; apparently depending on the order in which readdir() returns the "functions" and "classes" subdirectories.

I had to adapt the test case a little bit to get it working with rspec-puppet 2.x, but the problem then was not reproducible. Therefore I close this.
Please open a new issue with any further problems you have.


Serializing JObject results with NullReferenceException

I created a PR with failing unit test exposing the problem: https://github.com/akkadotnet/Hyperion/pull/65
Starting test execution, please wait...
[xUnit.net 00:00:00.5834487]   Discovering: Hyperion.Tests
[xUnit.net 00:00:00.8800229]   Discovered:  Hyperion.Tests
[xUnit.net 00:00:01.5499118]   Starting:    Hyperion.Tests
[xUnit.net 00:00:01.9438699]     Hyperion.Tests.ISerializableTests.SkipNonSerializedAttributeIfNoSerilizedAttribute [SKIP]
[xUnit.net 00:00:01.9442333]       Not implemented yet
[xUnit.net 00:00:01.9531298]     Hyperion.Tests.ISerializableTests.ShouldNotThrowIfNoConstructorWithSerializationInfo [SKIP]
[xUnit.net 00:00:01.9533130]       Not implemented yet
Skipped  Hyperion.Tests.ISerializableTests.SkipNonSerializedAttributeIfNoSerilizedAttribute
Skipped  Hyperion.Tests.ISerializableTests.ShouldNotThrowIfNoConstructorWithSerializationInfo
[xUnit.net 00:00:02.4549861]     Hyperion.Tests.JsonObjectTests.CanSerializeSimpleJObject [FAIL]
[xUnit.net 00:00:02.4565330]       System.NullReferenceException : Object reference not set to an instance of an object.
[xUnit.net 00:00:02.4572192]       Stack Trace:
[xUnit.net 00:00:02.4578435]            at Hyperion.SerializerFactories.DictionarySerializerFactory.<>c__DisplayClass3_0.<BuildSerializer>b__1(Stream stream, Object obj, Serializer
Session session)
[xUnit.net 00:00:02.4590671]            at Hyperion.ValueSerializers.ObjectSerializer.WriteValue(Stream stream, Object value, SerializerSession session)
[xUnit.net 00:00:02.4591752]            at Hyperion.Serializer.Serialize(Object obj, Stream stream)
[xUnit.net 00:00:02.4592541]         JsonObjectTests.cs(25,0): at Hyperion.Tests.JsonObjectTests.CanSerializeSimpleJObject()
Failed   Hyperion.Tests.JsonObjectTests.CanSerializeSimpleJObject
Error Message:
 System.NullReferenceException : Object reference not set to an instance of an object.
Stack Trace:
   at Hyperion.SerializerFactories.DictionarySerializerFactory.<>c__DisplayClass3_0.<BuildSerializer>b__1(Stream stream, Object obj, SerializerSession session)
   at Hyperion.ValueSerializers.ObjectSerializer.WriteValue(Stream stream, Object value, SerializerSession session)
   at Hyperion.Serializer.Serialize(Object obj, Stream stream)
   at Hyperion.Tests.JsonObjectTests.CanSerializeSimpleJObject() in C:\repos\Hyperion\Hyperion.Tests\JsonObjectTests.cs:line 25
[xUnit.net 00:00:03.0441936]     Hyperion.Tests.CollectionTests.CanSerializeMultiDimentionalArray [SKIP]
[xUnit.net 00:00:03.0444410]       add support for multi dimentional arrays
Skipped  Hyperion.Tests.CollectionTests.CanSerializeMultiDimentionalArray
[xUnit.net 00:00:03.0831862]   Finished:    Hyperion.Tests

Total tests: 138. Passed: 134. Failed: 1. Skipped: 3.
Test execution time: 3.5277 Seconds
Test Run Failed.

I wanted to dig into the problem, but I don't know how to attach a debugger ;)

opening sln in VS 2017 fails because of

C:\repos\Hyperion\Hyperion.FSharpTestTypes\Hyperion.FSharpTestTypes.fsproj : error MSB4019: The imported project "C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\MSBuild\Sdks\FSharp.NET.Sdk\Sdk\Sdk.props" was not found. Confirm that the path in the <Import> declaration is correct, and that the file exists on disk.```

2) Trying to debug in VSCode fails... I don't really know why (yet).

Any guidance would be appreciated.

The other question is, of course, whether I should expect JObject to be serializable by Hyperion at all :)

Ok, so I managed to get the debugger running (by removing all FSharp related stuff) and found out that:

JObject is a Dictionary<string, JToken> so Hyperion tries to serialize it using DictionarySerializerFactory
Support for dictionaries is unfinished  https://github.com/akkadotnet/Hyperion/issues/46

I feel like I simply should not expect JObject to be serializable by Hyperion at all. Is that correct?

Any update on this? I'm having a very similar problem

Any update on this? I'm having a very similar problem to #72


Universally loaded images

To embed an image it should be placed in public folder. Then, it can be loaded using a webpack loader:
// webpack.config.js
var imageLoader = {
  test: /\.(png|jpg)$/,
  loader: 'url-loader?limit=8192',
  exclude: /node_modules/
}

And in the file where it is used, it is imported with:
import someImg from './public/someImg.png'

This will put the path of someImg.png into the variable someImg. In case the image is small, it will put a data-uri string as the value of someImg.
In case the js-file importing the image is universal (used by the client and the server), it will be loaded twice by webpack. Once for the client and once for the server configuration. The client will generate build/public/assets/0845a58fef22e11981898cfb8d7ed119.png which is correct. Since the server outputs files directly to build it will be something like build/0845a58fef22e11981898cfb8d7ed119.png, which differs from the client and is wrong.  Now, we need some way to let the server "load" the file to get its correct final path, without actually generating it. The mismatch of paths happens because output.path currently differs for the client and the server.
why the server can't simply ignore images
If the server would ignore images with the IgnorePlugin, the server would not be able to get their paths or data-uri strings. It must be able to read them, but it should not emit the files. The client should do that.
what about images the server imports, but the client does not import
These images would be missing in case the server does not emit any files.

This is a very entangled situation. Not sure about the best way. I have a proof of concept where the client and the server both emit image files, potentially doing the work twice.
To get this to work, backend.js needs to be generated into build/public/assets/, because the images have to go into this directory as well. After this is done, backend.js is moved automatically into build/.
Maybe something is fundamentally wrong about the configured output paths that's causing this.

Solved with 19557ae2f93f7a345e1504223a0d6f309211ad5b. The solution was to add a relative path to output.filename that causes backend.js to go into build/.
    output: {
      path: outputPath, // put images into build/public/assets
      filename: '../../backend.js', // put backend.js into 'build/backend.js'
      publicPath: '/assets/'
    },

Now none of that manual moving of backend.js has to be done.


ceph-aio-no-vagrant.sh: check if ansible is installed

Since we don't provide install-ansible.sh anymore we just check if it
exists. If not we fail and ask for installation.
Signed-off-by: Sébastien Han<EMAIL_ADDRESS>

It looks like there are more things going on in this commit?

@ktdreyer yes some shell lint, want me to separate the commits?

It's ok to leave this PR as is to keep it moving.
I think the future it would be great to break these out so it's easier to review and bisect.

@ktdreyer sure, thanks!


How to deserialize JSON to new Date?

Thanks for your gem!
I want to make the serializer to create date object from date fields when returned from the server, but serializer doesn't seem to help.
How can I convert created_at to new Date(created_at)?
Am I missing something? Seems serializer only works to convert to JSON, but not to js?

ok, let's use interceptors. But afterResponse interceptor is called with array/instance result, maybe it would be better to add separate interceptors for different types of queries?

Solution (temporary i hope)
  class CertificateStatus extends RailsResource
    @configure
      url: "/certificate_statuses",
      name: "certificate_status",
      interceptors: [
        afterResponse: (result)->
          if angular.isArray result
            angular.forEach result, (item)->
              item.prepare()
          else
            item.prepare()
      ]
    prepare: ->
      @created_at = new Date(@created_at)
      this

Check out the answer to #154.  Instead of the DateTimeUtilities which is a separate utility class it was using you can use new Date.

Great, thanks!


helm ls output as JSON or YAML

Then the output could more easily be used in scripting.
My use case is orphaned CI runs that timed out and didn't clean up after themselves. To be able to list all helm releases, look at the dates, and delete the obviously orphaned ones. JSON and YAML are much easier to work with than parsing the text response.

https://github.com/kubernetes/helm/pull/2950 :)

Closing since its a duplicate of #1534 and PR #2950 is in-flight.


[CAMPAIGN] Bump jackson and snakeyaml dependencies

Core has bumped jackson and snakeyaml dependencies to 2.13.4 and 1.32 respectively here for versions 2.x, 2.3 , 2.2 , 2.1 , 2.0, 1.3 and 1.x. Plugin and client teams should do the same to align with core for avoiding version conflicts.
What are you proposing? What do you suggest we do to solve the problem or improve the existing situation?

[x] https://github.com/opensearch-project/data-prepper/issues/1839
[x] https://github.com/opensearch-project/alerting/issues/580
[x] https://github.com/opensearch-project/anomaly-detection/issues/676
[x] https://github.com/opensearch-project/asynchronous-search/issues/189
[x] https://github.com/opensearch-project/index-management/issues/525
[x] https://github.com/opensearch-project/job-scheduler/issues/244
[x] https://github.com/opensearch-project/k-NN/issues/566
[x] https://github.com/opensearch-project/performance-analyzer/issues/288
[x] https://github.com/opensearch-project/security/issues/2124
[x] https://github.com/opensearch-project/sql/issues/869
[x] https://github.com/opensearch-project/observability/issues/1053
[x] https://github.com/opensearch-project/cross-cluster-replication/issues/569
[x] https://github.com/opensearch-project/common-utils/issues/257
[x] https://github.com/opensearch-project/opensearch-java/issues/227
[x] https://github.com/opensearch-project/dashboards-reports/issues/478

@sean-zheng-amazon @joshuali925 Can you please update the status of this issue?

Looks like all the tasks have been taken care of. Closing this campaign.


'step' works like 'next'

'step' does not step into subexpressions on the same line. That is a major problem as I have to modify the source to be able to step through it. 'step' seems to behave almost like 'next'.

Please reopen if you have time to provide me with a way to reproduce this issue. Thanks!


Adding test with Simbad check

This adds a test to address issue #20. It will query Simbad for all sources by their Source.source name and then loop over the results to verify that they are not present in the database for more than one source. It handles the Simbad query properly to avoid hundreds of individual calls to their service.
Matching is done by exact name, so binaries will generally not match against each other (ie, TWA 27 does not match TWA 27B).

Correct.
The example I was hoping to catch was TWA 27, which was listed as two independent sources:
TWA 27 and 2MASS J12073346-3932539
Instead, they should have been two names for the same single source.

Great! Happy to merge once conflict is resolved.


fix: Store raw metadata only if cip36 data is present

Description
Fix cip36_registration::generate_from_tx_metadata function, store raw_metadata only if cip36 data is present.

Actually, we don't really need the raw data if the registration is valid.
Certainly not for production.
The purpose of the Rawdata is to save it in case we think there is a registration BUT its invalid, so that we can show categorically what the raw data was that we rejected.
Also, lets hold of on merging changes to sync while @FelipeRosa is working on the sync speed issues.


win: more reliable uv__hrtime precision

Eliminates floating-point operations that can cause precision loss.
Thanks to @Arnavion for suggesting the fix:
https://github.com/iojs/io.js/pull/1272#issuecomment-86644307
(And IRC discussion in #io.js: http://logs.libuv.org/io.js/latest#18:25:27.988)
Note: first time writing any C.
Also, I don't have a windows machine configured to test this on.

Possibly related to https://github.com/joyent/node/issues/8960

/cc @piscisaureus or @orangemocha care to review?

Ideally, uv__hrtime would just take (uint64_t scale) also, but I'm not sure that change can be made?

Updated, PTAL. (sorry, forgot about the whole long and longlong deal. :3)

-1
There is no guarantee on the range of scale (or frequency). That's why floating point was used in the first place.

@orangemocha perhaps it would be better to use a separate htime impl for timeouts then? https://github.com/libuv/libuv/blob/v1.x/src/win/timer.c#L32

Updated to use (counter.QuadPart / hrtime_interval_) * scale Which is better than what was there.
To actually do it with 128 bit, I think it would have to be one of these two to do the multiplication:
https://msdn.microsoft.com/en-us/library/windows/desktop/hh802934(v=vs.85).aspx
vs
https://msdn.microsoft.com/en-us/library/windows/desktop/jj635748(v=vs.85).aspx
But I really don't grok how they work. Also, it looks like bits would be lost in the conversion back to 64bit anyways, since windows does not have a 128bit divide.


example: add a CI pipeline for gradle

Add an example workflow in the examples/ci/ folder that exemplifies how to make use of Popper to create a CI pipeline for java projects that use gradle as their build system. The image used by the workflow can be the official one.

i will take this


Synergy profile provisioned from template using Python SDK is not consistent

Scenario/Intent
Provisioning a Synergy profile from template using Python SDK should result in a profile containing the OS Deployment Plan, connections, and local storage that was defined in the template.
Environment Details

OneView SDK Version: 3.3.0
OneView Appliance Version: 3.10.04-0299553
OneView Client API Version: 500
Python Version: 3.4.5
Platform: CentOS 7.3 x86_64

Steps to Reproduce
Create a template in OneView containing an OS Deployment Plan, connections, and local storage
Provision a profile in Python:
oneview_client = OneViewClient.from_json_file('connection.json')
new_profile_options = dict(
name=PROFILE_NAME,
serverProfileTemplateUri=TEMPLATE_URI,
serverHardwareUri=SRVR_HWARE_URI
)
new_profile = oneview_client.server_profiles.create(new_profile_options)
Expected Result
New profile contains same OS Deployment Plan, connections, and local storage as the template
Actual Result
New profile has OS Deployment Plan set to "None", no connections, and no local storage

This has been resolved by using the following combination of steps:
First, create a new profile from the template, which will be unassigned
new_profile = oneview_client.server_profile_templates.get_new_profile(TEMPLATE_URI)
Now update the serverHardwareUri, name, and custom attributes before assigning the new profile to hardware
new_profile['serverHardwareUri'] = SRVR_HWARE_URI
new_profile['name'] = PROFILE_NAME
new_profile['osDeploymentSettings']['osCustomAttributes'] = dict(name='key1',value=VALUE1),dict(name='key2',value=VALUE2),dict(name='key3',value=VALUE3)
oneview_client.server_profiles.update(resource=new_profile, id_or_uri=new_profile['uri'])
Create the profile (assign it to hardware)
created_profile = oneview_client.server_profiles.create(new_profile)

I'd like to make a suggestion that oneview_client.server_profile_templates.get_new_profile should perhaps be renamed to oneview_client.server_profile_templates.create_new_profile?  The verbage "get" is not intuitive since this is actually used to "create" a new profile from a template.

Hello @kmullican , the "get_new_profile" doesn't actually create a new profile.
It is a binding to the GET endpoint /rest/server-profile-templates/{id}/new-profile on the Server Profile Template.
What this does is actually just return a dictionary which is based on the server profile template, but ready to be used as base for a server profile.
In this case, when you run:
new_profile = oneview_client.server_profile_templates.get_new_profile(TEMPLATE_URI)

You have not yet created the Server Profile in OneView, you've just created a local dictionary containing the info you require to create that profile, and assigned it to the variable new_profile.
The profile is actually being created when you call:
created_profile = oneview_client.server_profiles.create(new_profile)

Which is using all of the info on the variable new_profile, both what was retrieved and what you edited, to create a Server Profile on the OneView appliance.
Hope this helps clear things up a bit. :octocat:

@fgbulsoni yes, true, thx.  It's a "get in preparation to create", lol.   would it be best for me to close this issue since it has been resolved?  just wanted to leave some breadcrumbs in the notes in case anyone else would like example of getting the profile defined and created from template.

I'll update the example and try to make this path more clear, as that should help as well.
These breadcrumbs are definitely important and will be searchable for anyone that gets in the repo, so that is great as well. :octocat:
I'd wait for that example file update to close this though.


[General] We need to add users to the Bacalhau team for Grafana

Just so folks can get access and make changes.
Who is the DRI?
aronchick
Team and command structure
Bacalhau - aronchick
Estimated monthly cost
$0
What else do we need to know?
Here's the list of folks
@aronchick
David Aronchick
aronchick
2FA
Owner
1 team
@binocarlos
Kai Davenport
binocarlos
2FA
Private
Member
1 team
@DeveloperAlly
Alison Haire
DeveloperAlly
2FA
Private
Member
1 team
@enricorotundo
Enrico Rotundo
enricorotundo
2FA
Private
Owner
1 team
@fakela
Favour Kelvin
fakela
2FA
Private
Owner
1 team
@it09
it09
it09
2FA
Private
Owner
1 team
@lukemarsden
lukemarsden
2FA
Private
Owner
1 team
@philwinder
Phil Winder
philwinder
2FA
Private
Owner
1 team
@ricmk
ricmk
2FA
Private
Owner
1 team
@simonwo
Simon Worthington
simonwo
2FA
Private
Member
1 team
@wdbaruni
Walid Baruni
wdbaruni
2FA
Private
Owner
1 team
@wesfloyd
Wes Floyd
wesfloyd
2FA
Private
Owner
1 team
@wjam
Will May
wjam
2FA
Private
Member
1 team

For completeness, this conversation was had:
There are two bacalhau teams in Grafana.

The 1 role on the first team is "Public Dashboard Writer".

Do you know which team you want them added to?
Do you think the other team should be destroyed?

@aronchick responded: Oh! yeah, we only need the one - we just need a bunch of writers

I've made @aronchick the admin of the bacalhau team (which @enricorotundo already appears to be admin of). This team is admin to RW in the newly created Bacalhau dashboard folder.
Please provide the email addresses of these users. I checked these and I cannot find them and probably need to be invited. (I can only invite protocol.ai emails. External emails will need additional approval that I'm not comfortable providing.

@binocarlos Kai Davenport
@DeveloperAlly Alison Haire
@fakela Favour Kelvin
@it09 it09
@lukemarsden lukemarsden
@philwinder Phil Winder

OK - so this looks great, but the question is can i add these folks as admins
or do you have to add them in the system and then i can add them to this

Enrico Rotundo<EMAIL_ADDRESS><EMAIL_ADDRESS>Iryna Tsimashenka<EMAIL_ADDRESS>Kai Davenport<EMAIL_ADDRESS>Luke Marsden<EMAIL_ADDRESS>Phil Winder<EMAIL_ADDRESS>Richard McKnight<EMAIL_ADDRESS><EMAIL_ADDRESS>Walid Baruni<EMAIL_ADDRESS>Wes Floyd<EMAIL_ADDRESS><EMAIL_ADDRESS>

@aronchick I've sent the following invitations. The ones indicated as "(already a member)" did not need an invite and were added to the Bacalhau team. Once these members accept their invite, you could try to add them to the team yourself.

<EMAIL_ADDRESS>(already a member)
[x]<EMAIL_ADDRESS>[x]<EMAIL_ADDRESS>[x]<EMAIL_ADDRESS><EMAIL_ADDRESS>(already a member)
[x]<EMAIL_ADDRESS>[x]<EMAIL_ADDRESS>[x]<EMAIL_ADDRESS><EMAIL_ADDRESS>(already a member)
[x]<EMAIL_ADDRESS>[x]<EMAIL_ADDRESS>
I don't think you need anything else form me at this point, but please re-open if there's anything I missed or you can't do. 🙇

Hi Mike, I do see that I have a viewer role, but that has very limited permissions related to our work, specifically not being able to access the Explore feature
https://grafana.com/docs/grafana/latest/administration/roles-and-permissions/
Is it possible to grant us editor permissions as well?


[macos]Crash when opening pdf files

    <img width="860" alt="image" src="https://user-images.githubusercontent.com/28723827/195575271-8058a508-06ea-4605-bf2a-6d1e459562b0.png">

您好 用你的demo在mac上打开pdf 闪退了
Originally posted by @iOSluozhiping in https://github.com/hlwhl/webview_cef/issues/20#issuecomment-1277403993

After triage, it's caused by single-process mode of cef. By avoid this, we should integrate helper bundles into the app and disable single-process mode.


RPi3: cannot install Numba due to issue with LLVM toolchain

Reporting a bug

system information:

Raspberry pi 3 Model B
os is Rapsbian 10 (buster)
Python 3.7.3
attempting to install Librosa through pip 20.0.1
gcc version 8.3.0

Hello,
I've been trying to install Librosa on my raspberry pi 3 model B.
the line I used in order to install Librosa was: pip3 install --upgrade --user librosa
after a really long download the installation got into infinite backtracking of the pip resolver trying to solve version conflicts over numba so I decided to  run the line: pip3 install --upgrade --user -no--deps librosa instead and it did work and librosa was installed successfully according to pip.
right now when I attempt to import Librosa in the python shell I get the following Traceback:
`pi@raspberrypi:~ $ python3
Python 3.7.3 (default, Jul 25 2020, 13:03:44)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.

import librosa
Traceback (most recent call last):
File "", line 1, in
File "/home/pi/.local/lib/python3.7/site-packages/librosa/init.py", line 211, in
from . import core
File "/home/pi/.local/lib/python3.7/site-packages/librosa/core/init.py", line 5, in
from .convert import *  # pylint: disable=wildcard-import
File "/home/pi/.local/lib/python3.7/site-packages/librosa/core/convert.py", line 7, in
from . import notation
File "/home/pi/.local/lib/python3.7/site-packages/librosa/core/notation.py", line 8, in
from ..util.exceptions import ParameterError
File "/home/pi/.local/lib/python3.7/site-packages/librosa/util/init.py", line 83, in
from .utils import *  # pylint: disable=wildcard-import
File "/home/pi/.local/lib/python3.7/site-packages/librosa/util/utils.py", line 10, in
import numba
File "/home/pi/.local/lib/python3.7/site-packages/numba/init.py", line 4, in
from numba import module_type_inference
File "/home/pi/.local/lib/python3.7/site-packages/numba/module_type_inference.py", line 8, in
from numba import typesystem, symtab
File "/home/pi/.local/lib/python3.7/site-packages/numba/typesystem/init.py", line 6, in
from .typemapper import *
File "/home/pi/.local/lib/python3.7/site-packages/numba/typesystem/typemapper.py", line 3, in
import llvm.core
ModuleNotFoundError: No module named 'llvm'
`

I've tried to run sudo apt-get install llvm and it did install llvm but didn't solve the issue.
also tried running pip3 instal llvmlite but got the following result:
sudo pip3 install llvmlite Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple Collecting llvmlite   Could not find a version that satisfies the requirement llvmlite (from versions: ) No matching distribution found for llvmlite
another attempt at installing librosa with pip resulted in this error:

ERROR: Command errored out with exit status 1:
   command: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-0gw3_ve1/llvmlite_1d4974bee4c74364ac4e473f80edfd7d/setup.py'"'"'; __file__='"'"'/tmp/pip-install-0gw3_ve1/llvmlite_1d4974bee4c74364ac4e473f80edfd7d/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-s9gczj9k
       cwd: /tmp/pip-install-0gw3_ve1/llvmlite_1d4974bee4c74364ac4e473f80edfd7d/
  Complete output (15 lines):
  running bdist_wheel
  /usr/bin/python3 /tmp/pip-install-0gw3_ve1/llvmlite_1d4974bee4c74364ac4e473f80edfd7d/ffi/build.py
  LLVM version... 7.0.1

  Traceback (most recent call last):
    File "/tmp/pip-install-0gw3_ve1/llvmlite_1d4974bee4c74364ac4e473f80edfd7d/ffi/build.py", line 191, in <module>
      main()
    File "/tmp/pip-install-0gw3_ve1/llvmlite_1d4974bee4c74364ac4e473f80edfd7d/ffi/build.py", line 181, in main
      main_posix('linux', '.so')
    File "/tmp/pip-install-0gw3_ve1/llvmlite_1d4974bee4c74364ac4e473f80edfd7d/ffi/build.py", line 143, in main_posix
      raise RuntimeError(msg)
  RuntimeError: Building llvmlite requires LLVM 10.0.x or 9.0.x, got '7.0.1'. Be sure to set LLVM_CONFIG to the right executable path.
  Read the documentation at http://llvmlite.pydata.org/ for more information about building llvmlite.

  error: command '/usr/bin/python3' failed with exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for llvmlite
  Running setup.py clean for llvmlite
Failed to build llvmlite
Installing collected packages: pycparser, llvmlite, numba, cffi, soundfile, resampy, audioread
    Running setup.py install for llvmlite ... error
    ERROR: Command errored out with exit status 1:
     command: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-0gw3_ve1/llvmlite_1d4974bee4c74364ac4e473f80edfd7d/setup.py'"'"'; __file__='"'"'/tmp/pip-install-0gw3_ve1/llvmlite_1d4974bee4c74364ac4e473f80edfd7d/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-c8uz7phd/install-record.txt --single-version-externally-managed --user --prefix= --compile --install-headers /home/pi/.local/include/python3.7m/llvmlite
         cwd: /tmp/pip-install-0gw3_ve1/llvmlite_1d4974bee4c74364ac4e473f80edfd7d/
    Complete output (18 lines):
    running install
    running build
    got version from file /tmp/pip-install-0gw3_ve1/llvmlite_1d4974bee4c74364ac4e473f80edfd7d/llvmlite/_version.py {'version': '0.35.0', 'full': 'ea23b026930cc00824c907172383f54c9d438e6b'}
    running build_ext
    /usr/bin/python3 /tmp/pip-install-0gw3_ve1/llvmlite_1d4974bee4c74364ac4e473f80edfd7d/ffi/build.py
    LLVM version... 7.0.1

    Traceback (most recent call last):
      File "/tmp/pip-install-0gw3_ve1/llvmlite_1d4974bee4c74364ac4e473f80edfd7d/ffi/build.py", line 191, in <module>
        main()
      File "/tmp/pip-install-0gw3_ve1/llvmlite_1d4974bee4c74364ac4e473f80edfd7d/ffi/build.py", line 181, in main
        main_posix('linux', '.so')
      File "/tmp/pip-install-0gw3_ve1/llvmlite_1d4974bee4c74364ac4e473f80edfd7d/ffi/build.py", line 143, in main_posix
        raise RuntimeError(msg)
    RuntimeError: Building llvmlite requires LLVM 10.0.x or 9.0.x, got '7.0.1'. Be sure to set LLVM_CONFIG to the right executable path.
    Read the documentation at http://llvmlite.pydata.org/ for more information about building llvmlite.

    error: command '/usr/bin/python3' failed with exit status 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-0gw3_ve1/llvmlite_1d4974bee4c74364ac4e473f80edfd7d/setup.py'"'"'; __file__='"'"'/tmp/pip-install-0gw3_ve1/llvmlite_1d4974bee4c74364ac4e473f80edfd7d/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-c8uz7phd/install-record.txt --single-version-externally-managed --user --prefix= --compile --install-headers /home/pi/.local/include/python3.7m/llvmlite Check the logs for full command output.

@COfek thank you for asking about this on the Numba issue tracker. Unfortunately we don't currently support binary wheels for llvmlite for Raspberry Pi yet. Hopefully in the next release. In the meantime you can either use conda to install a binary package or resort to compiling llvmlite from scratch. Instructions and more information can be found at:
https://llvmlite.readthedocs.io/en/latest/admin-guide/install.html#building-manually

@esc Thanks for responding, I am unable to use conda to install binary package because of conda not including some of the packages I need for raspberry pi like pyaudio, kivy and librosa itself seems to not install properly through conda as well I gave up on even trying to install tensorflow on a conda environment.
so my last resort is to try compile it from scratch, as I am not very experienced with this, may I ask what do I need to execute in command line to complete step 3 in the installation guide you gave me
Decompress the LLVM tar file and apply the following patches from the llvmlite/conda-recipes/ directory. You can apply each patch using the Linux patch -p1 -i {patch-file} command:

@COfek thank you for following up on this. As far as I understand the librosa package is available from the conda-forge repositories as a conda package: https://anaconda.org/conda-forge/librosa -- perhaps is is worth giving this a shot?
Lastly, regarding the issue when compiling: Unfortunately the llvmlite documentation assumes a certain familiarity with the compilation of libraries from sources in general. What the documentation is referring to with patch -p1 -i {patch-file} is to run the command patch -p1 -i on each {patch-file} then listed below.

@esc, just curious: is Raspberry pi 3 a valid target for numba?

@abebeos are you suggesting that numba would be valid on raspberry pi 4 ?

@abebeos are you suggesting that numba would be valid on raspberry pi 4 ?

No. Just wondering what the numba system-requirements are, and if raspberry (and similar boards) are within the limits.

To add some clarification here:
Numba works and is tested on armv7l (32-bit ARM) and aarch64 (64-bit ARM) architectures.  The Raspberry Pi 2-4 are 64-bit ARM chips, but for backward compatibility reasons Raspberry Pi OS (aka "Raspbian", but not called that now) has been shipped primarily as a 32-bit OS, with 64-bit versions still considered beta.  Ubuntu 64-bit works fine (and is what we use for 64-bit ARM testing).
The main obstacle here is that we have not shipped wheels for ARM before.  Now that manylinux2014 (and pip) has a spec for 32-bit and 64-bit ARM wheels, we can start doing that once we can get it into our automation.  That should simplify installation on Raspberry Pis.  Building LLVM is painful on all platforms, and we should provide prebuilt binaries whenever we can.
As an aside: Does anyone know if manylinux2014 wheels (which are based on Centos 7 on all hardware platforms) work on recent versions of Pi OS?  I assume they have a new enough glibc, but haven't tested yet.

@COfek how did it go with the install? Or did you maybe even get the conda-forge package to work? Is there anything else we can help with?

@esc I did give conda-forge a try and it didn't work,
I didn't try to build llvm manually yet because I'm not sure as for how I should do it.
I never worked with these sort of files tar.xz / gz and have little experience as for what lines I should do in the terminal in order to build it manually.
would really appreciate it if you could translate the guide into command lines I need to execute.
Thanks

@esc, I had myself many trouble with manual llvmlite build. What needs to be done here, to help COfek and other folks in similar situation:
=> Simplify "build llvmlite from source" (ideally: a few cli instructions), revise docs (related: numba/llvmlite#686)
@COfek , not sure if this helps in your setup:
the simplest sequence to build llvmlite would be something like this:
conda install -c numba llvmdev  # llvmdev=<specific version:>
git clone https://github.com/numba/llvmlite.git
# git checkout <specific version>
cd llvmlite
python setup.py build
python runtests.py
python setup.py install

@esc should know about <specific version> needed.

@abebeos sorry for the rude question but,
could I replace conda with anything else ?
berryconda only supports python 3.6 so I'm not able to use the 3.7 env where I was able to setup other problematic packages.

@COfek i
As for compiling the Numba/llvmlite stack without conda may also mean building it without conda-build. This is something I have only had to do once and it was reasonably involved. I don't think anyone has yet published step by step instructions on how to do this. Having said that, I would continue to recommend trying to obtain a suitable conda distribution and try the pre-compiled packages. I think that getting this setup to work properly on a RaspberryPi will yield more mileage for you in the long-term. It is my understanding that miniforge ships a suitable distribution for using recent Pythons on Raspberry PIs:
https://github.com/conda-forge/miniforge
But as far as I know, you will need the replacement of Raspbian with a 64 bit OS (ubuntu 64?) This may solve issues if you are unable to install miniforge.

sorry for the rude question

I see no rudeness!
As for conda: I recently came in contact with numba/llvmlite, and my summary is more or less: one needs to use conda, otherwise it can become difficult, even for experienced devs.
curiosity: can you please give me the result of your uname -a?

@abebeos
uname -r returns 5.10.14-v7+, so kernel v7?
also I think that I may just resort to formatting the sd card and download everything on an ubuntu since everything did work as a whole on 64bit os.

my understanding of https://github.com/numba/numba/issues/6723#issuecomment-779303209 tells me that you'll have similar problems.
uname -a ? (not -r)

@abebeos
uname -a Linux raspberrypi 5.10.14-v7+ #1401 SMP Mon Feb 8 14:23:24 GMT 2021 armv7l GNU/Linux

uname -a Linux raspberrypi 5.10.14-v7+ #1401 SMP Mon Feb 8 14:23:24 GMT 2021 armv7l GNU/Linux

nice!

by the way I didn't get a clear answer, do you think that if I run ubuntu on my RPI instead of raspbian everything would work fine ?

Yes, I think (according to comments made by seibert & esc, 64bit ubuntu works on RPI, and you could use miniforge (instead of miniconda3) , should be this one:
https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-aarch64.sh

@abebeos Thanks a bunch I will try it tomorrow and update you guys on the situation.

@COfek , I did not dig out my pi, but was curious enough to try on an fresh emulated ubuntu:20.04, arm64/aarch64
# use ubuntu aarch64, as suggested by @seibert
# on a fresh arm64 ubuntu:20.04

apt update && apt install  git wget gcc g++ make -y

# use conda-forge, as suggested by @esc
wget https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-aarch64.sh
chmod +x Miniforge3-Linux-aarch64.sh
./Miniforge3-Linux-aarch64.sh # follow through manually, usually  [ENTER] or "yes"
bash # start new shell to activate conda

#scikit-learn not working
conda install -c conda-forge numpy numba scikit-learn

# your initial  installation line:
pip3 install --upgrade --user librosa

# => should install fine

@abebeos
so I formatted my new SD card and installed the recommended Ubuntu Desktop 20.10 with the official imager of raspberry pi.
first thing I noticed is that the raspberry pi is extremely slow when using ubuntu compared to Raspbian,
as for trying to execute the following.
the python version installed by default is 3.8.6 and when I attempted to check pip version by pip3 --version pip didn't even seem to exist which already seems weird to me.
so I just apt installed pip3 and now pip3 --version returns pip 20.1.1
I was just casually trying to install numpy just to see if basic libraries would still install.
and got back no matching distribution found for numpy.
I haven't used ubuntu before so I didn't really know what to expect.
then I followed your instructions and it worked fine so far but I'll have to ask how using bash opened a conda environment ?
before I had to activate the relevant environment so why now opening a new shell automatically considers it to be a conda environment?
Thanks again

opening a shell from within a shell is more or less like restarting a shell (close / open).
As for your problem: whenever you see a sequence of instructions (https://github.com/numba/numba/issues/6723#issuecomment-780708001) , follow them strictly: start with a fresh ubuntu 20.04, execute one by one the instructions without anything else before (maybe add a sudo now and then when necessary).
If all this fails, then sorry, but this would be the point where I give up (as this needs to be verified on a real RPI then).

@abebeos
I see thanks for answering.
I will try again, thank you very much for your help.

Installing Numba via pip w/o compilation is currently not supported on RaspberryPi as we don't release binary wheels for aarch64 platform yet. The current recommended workaround is to use conda.

@COfek , one last try:
you could replace ubuntu64 with Raspberry Pi OS 64bit:
https://www.raspberrypi.org/forums/viewtopic.php?f=117&t=275370
and then apply the same steps: https://github.com/numba/numba/issues/6723#issuecomment-780708001

I just had another look at anaconda.org -- and it seems you may be able to install librosa using:
conda install -c conda-forge librosa
As the package is available here: https://anaconda.org/conda-forge/librosa/files?type=conda

@esc @abebeos just wanted to update you guys,
I got a new image for ubuntu 20.10 server and managed to install llvmlite and librosa successfully through pip.
the reason pip didn't work initially is because by default for some reason the pi used a ipv6 address instead of ipv4 so it had some communication issues with pypi.
after getting that figured out I just had to pip3 install llvmlite and following that librosa.

Glad you did it.

On my (tiny)'machine", I switched from umbutu to (the newly available) RPI OS 64:

https://www.raspberrypi.org/forums/viewtopic.php?t=275370
https://downloads.raspberrypi.org/raspios_arm64/images/raspios_arm64-2020-08-24/2020-08-20-raspios-buster-arm64.zip

faster and looks funnier!
(note that you can close this issue if you feel it is solved)

@esc, you mentioned miniforge, and it looks like conda-forge is anaconda.org related (https://conda-forge.org/#about).
It looks like conda-forge is kind of more complete (context: architectures) than miniconda, why is it not used for numba (e.g. within the buildscripts) ?

follow-up:  #120

@esc, you mentioned miniforge, and it looks like conda-forge is anaconda.org related (https://conda-forge.org/#about).
It looks like conda-forge is kind of more complete (context: architectures) than miniconda, why is it not used for numba (e.g. within the buildscripts) ?

In my experience conda-forge is not quite as stable. While they do make new and fresh packages available very quickly, these are often still buggy. Using defaults/main has been the more stable alternative historically.

EDIT: found this https://stackoverflow.com/questions/60532678/what-is-the-difference-between-miniconda-and-miniforge, but I'm now even more confused (mostly by Anaconda not including the quite relevant arm64 architecture into miniconda).

I believe it is simply not available yet.

@COfek thank you for following up and closing this issue. I am glad you managed to get things up and running.


Feature: Scene Transition

We can use a transition between the scenes after the next scene's loading.
The transitition we can create it with an image with the script SceneTransition. We can control in the inspector:

transitionMove: The direction of the transition.
To put it in the inspector, I have to do it with a integer which come from an enum.
-- 0: Left to Right
-- 1: Right to Left
-- 2: Top to Bottom
-- 3: Bottom to Top
-- 4: Fade In
-- 5: Fade Out
speedTransition: The speed of the transition.

Also I control the possibility to change of resolution by the user, resizing and positioning the object.
The only thing to improve is if the scene is like the level1 o 2 (in other worlds, big to load), the feeling the first time to load the scene is that the game brokes.
I create a prefab CanvasTransition.prefab with two example of transitions:

ImageStart: Goes a image in black from the left ot the right
ImageFadeOut_Exit: Does a fade out who ends with a black screen.

I put also the scenes to testing the feature (I already tested in ReleaseEditor):
Scenes_Transitions.zip
GIFS:

Task linked: CU-kdargf Scene transition - Black fade for example


[Removal]: Please help me remove from the rejection list, thank you, hard work!

Hotspot Name
Suave Berry Koala
Hotspot b58 Address
11UjvWDpGHk9NZsuhJV4ReEg9H81mKC6cCzNqxfNP44eXoUpbiK
Discord Handle
No response
Hotspot Manufacturer
Bobcat
Removal Reason
My equipment was blacklisted for no reason. This happened when I moved and reinstalled it. The installation was carried out in accordance with the official requirements.
Modifications
No
Extra forwarders
No
Extra antennas
No
Additional Information
No response

We have taken into account this removal request in the next iteration of the denylist. It may take some time before it's processed. This is an automated comment due to the volume of addition and removal requests. A future tool is in the works to provide more insight into the analysis approach made by the Helium team and community members that maintain this list.


Web server configuration

Please provide sample web server configuration or hints how to set it up.

What do you mean with web server configuration?
There is an web.config for ISS (I will add them later to flox):
https://github.com/laravel/laravel/blob/master/public/web.config
Here is an good Tutorial for nginx:
https://www.digitalocean.com/community/tutorials/how-to-install-laravel-with-an-nginx-web-server-on-ubuntu-14-04

By server configuration, I mean what are consideration to take when installing on root domain or in sub folder and how it will impact the backend/.env file.

It doens't matter in how many subfolders you put flox. Nothing will impact the .env  file if you have the public and backend folder on the same level.
# Therefor you need set the CLIENT_URI in .env to '/subfolder/for/flox/public'
http://mydomain.com/subfolder/for/flox/public

# If you map your domain directly to public. Here is CLIENT_URI only '/'
http://mydomain.com

Is still somewhat unclear? (:

Thank you.


Process hang if preferred local display language doesn't support by OS

VSCode Version: Current 1.5.2
OS Version: Windows 10

Don't know if this is a bug.
Background: Microsoft banned installing extra language packs on Windows 10 Chinese version.
Steps to Reproduce:

Change the local display language in locale.json to "en_US"
Since there's no language pack supported in the system, processes hang after start
Find locale.json and change local display language back to "zh_CN", problem solved.

I would be surprised if those steps can cause a hang, but @dbaeumer moving to you for comment first.

@zhangbolong th --locale switch doesn't rely on any language pack installed. It simply relies on language bundles shipped with VS Code. How does VS Code hand. Can you open the Developer Tools (Help -> Toogle Developer Tools) to see if there are any errors in the console.


fix: thorchain savers shenanigans

Description

Pull Request Type

[ ] :bug: Bug fix (Non-breaking Change: Fixes an issue)
[ ] :hammer_and_wrench: Chore (Non-breaking Change: Doc updates, pkg upgrades, typos, etc..)
[ ] :nail_care: New Feature (Breaking/Non-breaking Change)

Issue (if applicable)

closes #
Risk

Testing

Engineering

Operations

Screenshots (if applicable)

Current dependencies on/for this PR:

develop

PR #5735   👈

This stack of pull requests is managed by Graphite.


Repeated dot-sourcing of a script quietly discards parameter variable attributes

Steps to reproduce
Create script ./script.ps1 as follows:
param([ValidateSet('bar', 'baz')] [string] $Foo)

# Output count of attributes.
(Get-Variable Foo).Attributes.Count

Then dot-source the script twice:
> . ./script.ps1; . ./script.ps1

Expected behavior
3
3

Actual behavior
3
0

Environment data

PowerShell Core v6.0.0-alpha (v6.0.0-alpha.16) on Darwin Kernel Version 16.4.0: Thu Dec 22 22:53:21 PST 2016; root:xnu-3789.41.3~3/RELEASE_X86_64

Don't have to dot-source
PS /home/steve> param([ValidateSet('bar', 'baz')] [string] $Foo)
PS /home/steve>
PS /home/steve> # Output count of attributes.
PS /home/steve> (Get-Variable Foo).Attributes.Count
3
PS /home/steve> param([ValidateSet('bar', 'baz')] [string] $Foo)
PS /home/steve>
PS /home/steve> # Output count of attributes.
PS /home/steve> (Get-Variable Foo).Attributes.Count
0

@SteveL-MSFT - the interactive prompt is equivalent to dot sourcing. Dot sourcing just means - don't create a new scope. The interactive prompt executes everything in global scope which also means no new scope is created.


Add support for var in refinements

Closes #19809

This has been decided to be included in the 3.5.0 release.


Sorry to say this, but New Moon has slowed Brackets

I really, really love this theme, but after some playing with disabling/enabling extensions, I've seen a significant improvement in performance when using the default dark theme over your theme.
I was working on a project with a lot of files, feeling a major Brackets slow down, swiching between files, saving files, etc. Everything was slow.
Then I first reloaded without extensions, all smooth again. Then I went by disabling one extension after another and came to the concluison: New Moon is slowing down Brackets :((
It may not be felt with just a few files, but with a lot of files opened at the same time, New Moon is definitely slowing it down.

Hmm, that's odd. I use it on a daily basis with no performance issue. But I'll go through and make a clean copy and update it.

Hi Tania, I am having seconds thoughts also now.
Maybe it is just due to the fact that I am having many files opened as working files(either in the sidepanel or with some toolbar extension). It was very late yesterday and I was already freaking out with the overall chopiness, in one moment when I disabled New Moon it seemed that it may be the reason.
But after playing again with disabling/enabling in the last half an hour, it seems that New Moon is ok :)


Bug: Wrong type cast leads to crash when starting

At this line https://github.com/electric-sql/electric/blob/7765d50cbe57c9990245802366caee8bcd35d6ff/packages/sync-service/lib/electric/connection_manager.ex#L409 electric is reading the pg_setting server_version and casting it to a number.
However this leads to crashes as many postgres as a service providers are setting this value to e.g. 16.4 (Debian 16.4-1.pgdg1123+1).
Also it's quite wrong to expect a number as it is defined as string by postgres:
https://www.postgresql.org/docs/current/runtime-config-preset.html#GUC-SERVER-VERSION

ah good catch! Thanks for the detailed bug report.

Hi @dustin-H,
Thanks for the report!
I was expecting the version number string to always be in the format x.y but as you explain this may not always be the case. As per your link to the documentation, the server_version_num setting seems more appropriate as it is guaranteed to be an integer (although stored as text). Thus, it seems safe to read server_version_num and cast it to an integer.


add deprecation decorator

Adds a new decorator @deprecated
Adds deprecated parameter to headers and queries
Also adds the required parameter to headers and queries (Btw: Why are required query parameters explicitly disallowed? The openapi specs allows this.)

Thanks, I've squashed and merged this as cdacd124dc94c1ea66f8b90f5d384cb184f55a01


VPN status incorrect and disconnect not working in certain scenarios

On Android, when you connect to the VPN and then close the app, if the VPN disconnects in the background (imagine you have a periodic function), the VPN does not disconnect. However, when you reopen the app, the VPN status in the Android settings shows as connected, but the app shows the VPN as disconnected. You cannot do anything to disconnect it. Please fix this. Thank you.

i run your example code and test it again this is the logs in that situation
can you help me with this problem i wrote the app with flutter but sometimes in that situation i explained vpn says is disconnect but it connects in setting the logs are like this
E/NVPN    ( 9495): Connect - Can't connect to tunnel: java.lang.Exception: Permissions are not given
E/NVPN    ( 9495): java.lang.Exception: Permissions are not given
E/NVPN    ( 9495):      at billion.group.wireguard_flutter.WireguardFlutterPlugin$connect$1.invokeSuspend(WireguardFlutterPlugin.kt:236)
E/NVPN    ( 9495):      at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
E/NVPN    ( 9495):      at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108)
E/NVPN    ( 9495):      at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:115)
E/NVPN    ( 9495):      at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:103)
E/NVPN    ( 9495):      at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:584)
E/NVPN    ( 9495):      at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:793)
E/NVPN    ( 9495):      at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:697)
E/NVPN    ( 9495):      at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:684)
I/flutter ( 9495): failed to start PlatformException(Permissions are not given, null, null, null)
I/flutter ( 9495): #0      StandardMethodCodec.decodeEnvelope (package:flutter/src/services/message_codecs.dart:651:7)
I/flutter ( 9495): #1      MethodChannel._invokeMethod (package:flutter/src/services/platform_channel.dart:334:18)
I/flutter ( 9495):
I/flutter ( 9495): #2      _MyAppState.startVpn (package:testvpn/main.dart:57:7)
I/flutter ( 9495):
I/flutter ( 9495):
I/flutter ( 9495): getting stage
I/flutter ( 9495): stage: VpnStage.disconnected
I/flutter ( 9495): getting stage
I/flutter ( 9495): stage: VpnStage.disconnected
D/WireGuard/GoBackend/my_wg_vpn( 9495): peer(6uZg…HliQ) - Handshake did not complete after 5 seconds, retrying (try 13)
D/WireGuard/GoBackend/my_wg_vpn( 9495): peer(6uZg…HliQ) - Sending handshake initiation
I/flutter ( 9495): getting stage
D/WireGuard/GoBackend/my_wg_vpn( 9495): peer(6uZg…HliQ) - Handshake did not complete after 5 seconds, retrying (try 3)
D/WireGuard/GoBackend/my_wg_vpn( 9495): peer(6uZg…HliQ) - Sending handshake initiation
I/flutter ( 9495): stage: VpnStage.disconnected
D/WireGuard/GoBackend/my_wg_vpn( 9495): peer(6uZg…HliQ) - Handshake did not complete after 5 seconds, retrying (try 14)
D/WireGuard/GoBackend/my_wg_vpn( 9495): peer(6uZg…HliQ) - Sending handshake initiation
I/flutter ( 9495): getting stage
I/flutter ( 9495): stage: VpnStage.disconnected
D/WireGuard/GoBackend/my_wg_vpn( 9495): peer(6uZg…HliQ) - Handshake did not complete after 5 seconds, retrying (try 4)
D/WireGuard/GoBackend/my_wg_vpn( 9495): peer(6uZg…HliQ) - Sending handshake initiation
D/WireGuard/GoBackend/my_wg_vpn( 9495): peer(6uZg…HliQ) - Handshake did not complete after 5 seconds, retrying (try 15)
D/WireGuard/GoBackend/my_wg_vpn( 9495): peer(6uZg…HliQ) - Sending handshake initiation
I/flutter ( 9495): initialize success my_wg_vpn
D/WireGuard/GoBackend/my_wg_vpn( 9495): peer(6uZg…HliQ) - Handshake did not complete after 5 seconds, retrying (try 2)
D/WireGuard/GoBackend/my_wg_vpn( 9495): peer(6uZg…HliQ) - Sending handshake initiation
D/WireGuard/GoBackend/my_wg_vpn( 9495): peer(6uZg…HliQ) - Handshake did not complete after 5 seconds, retrying (try 16)
D/WireGuard/GoBackend/my_wg_vpn( 9495): peer(6uZg…HliQ) - Sending handshake initiation
D/WireGuard/GoBackend/my_wg_vpn( 9495): peer(6uZg…HliQ) - Handshake did not complete after 5 seconds, retrying (try 2)
D/WireGuard/GoBackend/my_wg_vpn( 9495): peer(6uZg…HliQ) - Sending handshake initiation
D/WireGuard/GoBackend/my_wg_vpn( 9495): peer(6uZg…HliQ) - Handshake did not complete after 5 seconds, retrying (try 17)
D/WireGuard/GoBackend/my_wg_vpn( 9495): peer(6uZg…HliQ) - Sending handshake initiation
D/WireGuard/GoBackend/my_wg_vpn( 9495): peer(6uZg…HliQ) - Sending handshake


Suggestion: New table-filter plugin

Hi @lelegard ,
I have a trouble with the psi-merge plugin after a FILE input with --interleave.
Here the example:
tsp -I file --interleave=1 --label-base 2 --first-terminate file1.ts file2.ts file3.ts file4.ts \
 -P psimerge --main-label 2 --merge-label 3 \
 -P psimerge --main-label 2 --merge-label 4 \
 -P psimerge --main-label 2 --merge-label 5 \
 -O file output.ts

All input files are equal: CBR generated with FFmpeg, and without overlaped pids/services. So only PAT, SDT, PMT and A/V pids are present.
Then the expected behaviour is to merge all PSI/SI tables. Correct? Not at all!

For the PAT table, one initial ocurrence with version 0 appears in the output with only the services of file1.ts. After that a correct table with version 2 is found (with all services from the four input files).
For the SDT, two incorrect occurrence appears. First one with version 0 and services only from file1.ts. After another occurrence with version 1 and services from file1.ts and file4.ts. And after that the correct table with version 2 and all services.

Is this the expected behaviour? How I can remove these "ancient" tables?
Please help me.
Thank you!

Hi @lelegard ,
Regarding this trouble, my first trial to overcome this problem is to remove tables with the old version. However, I don't know how to select packets of an specific version. It exists one filter to target this functionality?
Regards.

Hi @lelegard ,

Is this the expected behaviour?

After thinking on your comments about other functionalities in the TSDuck, I feel this is the expected behaviour. That's because it works focused on realtime and after the initial tables, the values are correct. So don't try to fix this.

How I can remove these "ancient" tables?

However, I feel will be good to have a method to remove tables with specific version numbers. So I suggest to add a new plugin called table-filter. The functionality will be in pair with the filter plugin but focused on tables and not in packets. Examples:

table-filter --pmt --version 0-2: This filter out all packets of a PMT table with versions 0, 1 or 2.
table-filter -n --pat --version 2 --version 5: This pass only packets of PAT tables with version 2 or 5.
table-filter --any --version 3: Filter out all tables with version 3.
table-filter --eit --table_id 0x50: Remove tables with schedule info from EIT.

What you think about this proposal?

Hi @lelegard ,
Until this enhancement can be partially implemented, any idea about how to do table-filter --eit --table_id 0x50 ?
I want to remove the schedule EIT and leave the p/f EIT. It's possible to do it at time?
Thank you!

What about using plugin sections ?

What about using plugin sections ?

Thank you! :+1:
Then to filter out (aka remove) the schedule EIT of the service 0x6f81, this will work?
tsp ... -P sections -e  0x506f81 -p 18
As the documentation the parameter -t 0x50-0x59 removes all schedule info, but for all services. The parameter -e can be used to filter out only for one service?
Regards.

Then to filter out (aka remove) the schedule EIT of the service 0x6f81, this will work?
tsp ... -P sections -e 0x506f81 -p 18

Yes, except that you have to enumerate all variants with table ids from 0x50 to 0x5F (16 values): -e 0x506f81 -e 0x516f81 ... -e 0x5f6f81

As the documentation the parameter -t 0x50-0x59 removes all schedule info, but for all services. The parameter -e can be used to filter out only for one service?

Yes, each -e option applies to one pair of table id / table id extension, here a pair of EITs / service id. You must enumerate all services to remove, for all possible EIT schedule id. As usual with TSDuck, you'd better use a shell script, for instance:
opt=""
for srv in 0xDEAD 0xBEEF 0x0F00; do
    for ((tid=0x50; $tid<=0x5F; tid++)); do
        opt="$opt$(printf ' -e 0x%X' $(( ($(($tid)) << 16) | $(($srv)) )))"
    done
done

tsp ... -P sections $opt ...

Hi @lelegard ,
Only for convenience I suggest a small change in the parameters of the filter sections for overcome the use of a very large commands (I know you don't want to reinvent the wheel, but...):
Usage
tsp -P sections [options]

Options
  -e id1[-id2]
  --etid-remove id1[-id2]
    Remove all sections with the corresponding extended table id values. The value is a combination
    of the table id and the table id extension.
    For example, the option -e 0x4A1234 removes all BAT sections (table id 0x4A) for bouquet id
    0x1234 (table id extension).
    Several options --etid-remove can be specified.

  --etid-remove-reverse id1[-id2]
    Remove all sections with the corresponding extended table id values in reversed order. The value
    is a combination of the table id extension and the table id.
    For example, the option -e 0x12344A removes all BAT sections (table id 0x4A) for bouquet id
    0x1234 (table id extension). And -e 0x123450-0x12345F removes all scheduled EIT (0x50-0x5F) for
    service id 0x1234.
    Several options --etid-remove-reverse can be specified.

I hope you agree with this small enhancement.
Regards.

Added option --reverse-etid

Added option --reverse-etid

Thank you, @lelegard ! :+1:
I appreciate very highly that you listen to the suggestions we make.
So finally, and before closing this this issue, any idea to do what I commented time ago?

I feel will be good to have a method to remove tables with specific version numbers.

Added option --version-remove

Hi @lelegard ,

Added option --version-remove

I see that you've implemented a lot more, including "and" operators. So I need to check it before close this issue.
Thank you for all your effort!


#bug plugin is not available at "Obsidian Community plugins"

We are working on publishing the plugin

Hi @amarjt ,
Askify is available on the obsidian community plugin store, so closing this issue


Feature/submit data types

Feature: Form to submit new types
Description

For is a text box in which the user can submit new Types to the back-end. This PR also updates the Types Table to include any changes.

Commits

[x] Commits are well named.
[x] Commits have descriptive messages.
[x] Each commit introduces atomic changes (optional)

PR Scope

[x] PR is appropriately small.
[x] PR is focused on single problem.

PR Review

[x] Reviewers:

Vlad H.

Code Quality

[x] Consistent formatting.
[x] Functions are relatively small.
[x] Few instances of duplicate code.
[x] Few instances of dead code.

I resolved the merge conflicts. Code previously approved.


Improve skysports extractor

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2017.05.14. If it's not read this FAQ entry and update. Issues with outdated version will be rejected.

[x] I've verified and I assure that I'm running youtube-dl 2017.05.14

Before submitting an issue make sure you have:

[x] At least skimmed through README and most notably FAQ and BUGS sections
[x] Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

[ ] Bug report (encountered problems with youtube-dl)
[x] Site support request (request for adding support for a new site)
[ ] Feature request (request for a new functionality)
[ ] Question
[ ] Other

The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your issue

If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:
Add -v flag to your command line you run youtube-dl with, copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'http://www.skysports.com/watch/video/sports/cricket/10880748/rl-cup-derbys-v-worcs', u'-v']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2017.05.14
[debug] Python version 2.7.9 - Linux-3.16.0-4-amd64-x86_64-with-debian-8.8
[debug] exe versions: avconv 11.9-6, avprobe 11.9-6, rtmpdump 2.4
[debug] Proxy map: {}
[generic] rl-cup-derbys-v-worcs: Requesting header
WARNING: Falling back on generic information extractor.
[generic] rl-cup-derbys-v-worcs: Downloading webpage
[generic] rl-cup-derbys-v-worcs: Extracting information
ERROR: Unsupported URL: http://www.skysports.com/watch/video/sports/cricket/10880748/rl-cup-derbys-v-worcs
Traceback (most recent call last):
  File "./youtube-dl/youtube_dl/extractor/generic.py", line 1970, in _real_extract
    doc = compat_etree_fromstring(webpage.encode('utf-8'))
  File "./youtube-dl/youtube_dl/compat.py", line 2526, in compat_etree_fromstring
    doc = _XML(text, parser=etree.XMLParser(target=_TreeBuilder(element_factory=_element_factory)))
  File "./youtube-dl/youtube_dl/compat.py", line 2515, in _XML
    parser.feed(text)
  File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1642, in feed
    self._raiseerror(v)
  File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1506, in _raiseerror
    raise err
ParseError: syntax error: line 1, column 0
Traceback (most recent call last):
  File "./youtube-dl/youtube_dl/YoutubeDL.py", line 760, in extract_info
    ie_result = ie.extract(url)
  File "./youtube-dl/youtube_dl/extractor/common.py", line 433, in extract
    ie_result = self._real_extract(url)
  File "./youtube-dl/youtube_dl/extractor/generic.py", line 2795, in _real_extract
    raise UnsupportedError(url)
UnsupportedError: Unsupported URL: http://www.skysports.com/watch/video/sports/cricket/10880748/rl-cup-derbys-v-worcs

If the purpose of this issue is a site support request please provide all kinds of example URLs support for which should be included (replace following example URLs by yours):

Single video: http://www.skysports.com/watch/video/sports/cricket/10880748/rl-cup-derbys-v-worcs
Single video: http://www.skysports.com/watch/video/sports/cricket/10880556/ipl-mumbai-v-pune-highlights

Note that youtube-dl does not support sites dedicated to copyright infringement. In order for site support request to be accepted all provided example URLs should not violate any copyrights.

Description of your issue, suggested solution and other information
Skysports is supposed to be supported since df43389ade6e7a6394521ae91c0640508dceb4dc but it does not work with the videos provided above.

duplicate of #13054.


Implant whitelist/blacklisting

About the PR
fixes #16900 and the dupes

pai cannot be given any implants as it has no body
mice cannot use uplink implants as they have no hands
other sensible whitelists

also added admeme implant so admemes can give plushies microbombs like a boss
mouse can be given mindshield for funny
monkey can be given mindshield for functionality
Why / Balance
bug
Technical details

moved implanter whitelist and blacklist into yaml instead of hardcoded
whitelist BodyComponent instead of MobStateComponent since pai has the latter but not the former
added implant whitelist and blacklist
gave everything implant related whitelists when needed

Media
everything from tator uplink still works

mouse not allowed to use most implants:

but emp and m*cro bombs can be :) (this is on a mouse)

[X] I have added screenshots/videos to this PR showcasing its changes ingame, or this PR does not require an ingame showcase

Breaking changes
no
Changelog
:cl:

fix: Fixed being able to give pAIs implants.

It's intended to allow simplemobs (like mice, fish, etc) to be implanted.

I suggest: mouse can be given uplink implant, and people can use it by picking up the mouse

allow mr dips and mr chips to have all implants :trollface:

just have mr dips say "time to blow up", then blow up yourself


missing file "xodr_export.h"

Thanks a lot for your helpful project. I've cloned the repository and its submodules, but there is no file like "xodr_export.h" detectable within the project (but it is used in "xodr.h"). Is this an issue from cloning or is there an other workaround to get access to this file?

It should be generated when you build with CMake , it should be in binary directory

yeah that's right, thanks a lot


Create Github release for 1.11.1

The 1.11.1 release has been correctly tagged, however a proper Github release has not been created: https://github.com/kivy/kivy/releases.
This causes issues for me, as I have a script that automatically gets the source of the latest release using the Github API:
curl -s https://api.github.com/repos/kivy/kivy/releases/latest | python -mjson.tool | grep 'zipball_url' | cut -d'"' -f4

Thus it will get the 1.11.0 release instead of 1.11.1:
$ curl -s https://api.github.com/repos/kivy/kivy/releases/latest | python -mjson.tool | grep 'zipball_url' | cut -d'"' -f4
https://api.github.com/repos/kivy/kivy/zipball/1.11.0

What happened with this issue? Current release in GitHub is 1.11.0 yet.

I think I was waiting on figuring out how to get osx dmg/app artifacts for 1.11.1 but it never materialized. I just made the release, without them.


unable to create ingress

I am following the quick start guide and would like to setup the ingress. I am attempting to run the following command :
helm install incubator/sparkoperator --namespace spark-operator --set ingress-url-format={{$appName}}.ingress.cluster.com
and i get the Error: failed parsing --set data: key map "}" has no value
Instructions says it can be added via cli flag. Can anyone help.
Thanks

Looks like the Helm chart does not yet support that flag. Flag ingress-url-format is used by the operator. But the Helm chart needs a way of populating value for that flag with a Chart level parameter. See https://github.com/helm/charts/blob/master/incubator/sparkoperator/values.yaml.

Got it, are you able to point me to instructions to install the spark operator manually so I can utilize the ingress, or hopefully helm chart will be updated soon?

We will update the Helm Chart. @yuchaoran2011.

The Helm chart has been updated to support the flag. Please check it out.

This issue seems resurfacing in my. Is there any workaround?
value.yaml:
operatorImageName: gcr.io/spark-operator/spark-operator
operatorVersion: v1beta2-1.0.1-2.4.4
Helm:
"v2.14.3", GitCommit:"0e7f3b663
K8S:
"v1.16.1", GitCommit:"d647ddbd


Fix slow property test

https://github.com/IntersectMBO/plutus/pull/6062#issuecomment-2120645814
        parser ignores whitespace:                                                    OK (687.93s)
            ✓ propIgnores whitespace passed 50000 tests.
        parser ignores comments:                                                      OK (2314.43s)
            ✓ propIgnores comments passed 50000 tests.

The whitespace parser tests are also fragile: its easy to break them when changing name representations, e.g. ` symbols etc.

I've fixed those in #6286.