Add Bfloat16 optimizer with Kahan summation option for high precision updates

What does this PR do? Please describe:
This adds a pure BFloat16 AdamW optimizer (BFF_Optimizer) with user controllable dtypes for momentum and variance states, and available Kahan summation for high precision weight updates for training in pure BFloat16.  The Kahan summation buffer also has user selectable dtype.
All states and buffers default to BFloat16.
This allows for experimentation with optimizer states in various dtypes (notably BF16, but also a mix of dtypes) and allows high precision updates via Kahan summation for running with pure BFloat16 training.
Running with momentum and variation = torch.float32 and Kahan_summation=False reverts you to traditional AdamW optimizer for easy comparisons.
Fixes #{issue number}
Not a fix, but related to https://github.com/pytorch/pytorch/issues/82513
Does your PR introduce any breaking changes? If yes, please list them:
No, it is a drop in replacement for AdamW optimizer but with pure BF16 / customizable dtypes and precision.
Check list:

[ ] Was this discussed and approved via a GitHub issue? (not for typos or docs)
[X ] Did you read the contributor guideline?
[X ] Did you make sure that your PR does only one thing instead of bundling different changes together?
[ ] Did you make sure to update the documentation with your changes? (if necessary)
[ ] Did you write any new necessary tests?
[ ] Did you verify new and existing tests pass locally with your changes?
[ ] Did you update the CHANGELOG? (not for typos, docs, or minor internal changes)

This is very awesome, @lessw2020!
would it be possible to use some more intuitive mnemonic for its name - I'm not sure how BFF came to be - perhaps call it BF16Adam? What does BFF stand for?
Unless this is an attempt to be future proof and down the road easily adapt to even lower precision dtypes and thus not wanting to hardcoded BF16? so FP8 Adam next?

This is very awesome, @lessw2020!
would it be possible to use some more intuitive mnemonic for its name - I'm not sure how BFF came to be - perhaps call it BF16Adam? What does BFF stand for?
Unless this is an attempt to be future proof and down the road easily adapt to even lower precision dtypes and thus not wanting to hardcoded BF16? so FP8 Adam next?

Hi @stas00 - thanks for the kind feedback!
Yes, your intuition is correct - I didn't want to name it BF16 specifically b/c I'm keeping the relevant dtypes flexible and controllable for future work on lower precision. I suspect the variance for example will be amenable to FP8 and/or int8 dynamic quant, and also possible we could reduce the Kahan compensation buffer to FP8 in the future.
(note I did change from BFF_Optimizer to BFF_AdamW to note that the core algo is still AdamW, just with Kahan  summation and flexible dtypes. Have also tested a bit with stochastic rounding but results were not impressive enough to move forward with so far.)
Re BFF = it was just a play on Best Friends Forever and BFloat lol, meaning the optimizer would continue to improve over time as a platform optimizer with additional iterations.
But as noted, the main reason for the more generic name was to avoid imputing that it was running only in BF16, as it is intended to serve as a platform for future lower precision datatype use.

Thank for sharing the commentary, @lessw2020. That helps to understand the choices.
If I may propose FlexibleDtypeAdam or FlexiblePrecisionAdam? A bit longer but much more telling the user what it is.

@pytorchbot merge

Is the usage documentation missing on purpose? i.e. not in https://pytorch.org/torchdistx/latest/ - if you want feedback and users don't know new features were added how would they know to try them out?
Also, won't it be good to move the source from src to src/torchdistx? So that the import would be:
python -c "from torchdistx.optimizers.anyprecision_optimizer import AnyPrecisionAdamW"

and not the headless:
python -c "from optimizers.anyprecision_optimizer import AnyPrecisionAdamW"

Thank you.

Just checking out this PR. Thanks @stas00, it is certainly a bug. All Python files should reside under src/python/torchdistx which is not the case here. @rohan-varma or @lessw2020 do you have time to fix this?

If I'm not mistaken it should be just src/torchdistx

The full path where it should reside is https://github.com/pytorch/torchdistx/tree/main/src/python/torchdistx. If I don't hear back from Rohan or Less by EOD, I can fix this quickly. I will keep you posted.

Apologies for late response - it looks like @cbalioglu already moved the directory. There is a follow up task to add unittests: https://github.com/pytorch/torchdistx/issues/57 - @lessw2020 would you have time to work on this?

and docs please if it's not too much trouble. Thank you!

ok, so I  first checked that I got the math right for opt-1.3B

Theoretical memory allocation for optim states, weights, grads

breakdown:       n_params*(optim + grad + weights)
bf16 mixed precision: 1.3*(8     +   2  +  4+2   ) = 1.3*16 = 20.8GB
bf16 pure:            1.3*(4+2   +   2  +    2   ) = 1.3*10 = 13.0GB
-----------------------------------------------------
diff:                                           7.8GB

Real memory allocation: (got by adding `--skip_memory_metrics 0` flag to get memory usage reports)

bf16 mixed precision:
  before_init_mem_gpu        =        0MB
  init_mem_gpu_alloc_delta   =     5019MB
  init_mem_gpu_peaked_delta  =        0MB
  train_mem_gpu_alloc_delta  =    20076MB
  train_mem_gpu_peaked_delta =      123MB
-----------------------------------------
  total                      =    25218MB

bf16 pure:
  before_init_mem_gpu        =        0MB
  init_mem_gpu_alloc_delta   =     5019MB
  init_mem_gpu_peaked_delta  =        0MB
  train_mem_gpu_alloc_delta  =    12548MB
  train_mem_gpu_peaked_delta =      124MB
-----------------------------------------
  total                      =    17691MB

diff: 7.53GB

So the numbers check out

Now the problem is that I get the same loss curve with a and b. Less, you suggested that pure-bf16 should do better than mixed bf16, but I get identical outcomes. fp16/mixed precision learns much faster than bf16 here on opt-1.3b

@lessw2020 @stas00 did either of you manage to get pure bf16 training working as well as mixed prec? I see above that Stas was finding it less effective in his tests last year, although I see that pure bf16 training is part of llama-recipes nowadays so I figure it must have had some successful runs?

I haven't retried since my attempt since I haven't heard back from Less since then.
My hunch is that pure bf16 should train at slower pace given the same number of samples as the steps are less precise, but since lots of memory and gpu resources are freed it can go overall faster and perhaps arrive at the finish line at about the same speed (or even faster?)?
Also see https://github.com/NVIDIA/nccl/issues/1026 where we have been discussing the lossy nccl reduction of grads which I guess fits well with training in pure bf16 as everything is lossier there. Because if you do mixed precision you most likely want to reduce in fp32 when using bf16. Which adds even more overhead.

Thank you for your reply, Stas. Do you know how mixed precision interacts with lora -- is it keeping fp32 copies only of the adaptor weights (which AFAICT is all that's needed); if so, then I guess for lora mixed precision might end up quite a bit better.

I don't know, but let me tag the LORA expert @younesbelkada.
Younes could you please help answer Jeremy's question.
Thank you!

Hi @stas00 and @jph00 !
Regarding mixed precision with LoRA it should keep fp32 copies only of the adapter weights
Also discussing offline with @stas00 I exposed that in my opinion it might be not very "efficient" to perform pure bf16 fine-tuning with LoRA as most of the cases the optimizer states end up being very small compared to model weights.
I also want to hear thoughts from @BenjaminBossan @pacman100 to correct me and/or complete my statement! 🙏

Thanks a lot, Younes!

Hello @jph00 and @stas00,
By default, the weights of lora layers are in FP32. In practice, when using QLoRA with BF16, they are cast to BF16, which works fine. The relevant code snippet is here:
https://github.com/artidoro/qlora/blob/7f4e95a68dc076bea9b3a413d2b512eca6d004e5/qlora.py#L396-L405

Important: as communicated by Less elsewhere AnyPrecisionAdamW has moved to its new home:
https://github.com/facebookresearch/multimodal/blob/acc421ed6b7d60274fbd00e558bafe40567e5045/torchmultimodal/modules/optimizers/anyprecision.py#L17
so look for updates in there.

Message ID: @.***>Whilst it's true that it it should keep fp32 copies only of the adapter weights, I wonder if there's some problem happening, because in practice when I use llama-recipes, with fsdp cpu offloading, going from pure bf16 to mixed precision nearly doubles CPU RAM usage from 46->83GB, and per-GPU RAM usage increases from 27->32GB.
This is training a 34B model on 8 A6000 GPUs.


pip-tools 3.0.0 incompatible with pip 18.1

pip 18.1 did some internal changes that break pip-tools.
Traceback example:
Traceback (most recent call last):
  File "/usr/local/bin/pip-compile", line 11, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/piptools/scripts/compile.py", line 191, in cli
    results = resolver.resolve(max_rounds=max_rounds)
  File "/usr/local/lib/python2.7/site-packages/piptools/resolver.py", line 101, in resolve
    has_changed, best_matches = self._resolve_one_round()
  File "/usr/local/lib/python2.7/site-packages/piptools/resolver.py", line 190, in _resolve_one_round
    best_matches = {self.get_best_match(ireq) for ireq in constraints}
  File "/usr/local/lib/python2.7/site-packages/piptools/resolver.py", line 190, in <setcomp>
    best_matches = {self.get_best_match(ireq) for ireq in constraints}
  File "/usr/local/lib/python2.7/site-packages/piptools/resolver.py", line 255, in get_best_match
    best_match = self.repository.find_best_match(ireq, prereleases=self.prereleases)
  File "/usr/local/lib/python2.7/site-packages/piptools/repositories/pypi.py", line 129, in find_best_match
    best_candidate.project, best_candidate.version, ireq.extras, constraint=ireq.constraint
  File "/usr/local/lib/python2.7/site-packages/piptools/utils.py", line 51, in make_install_requirement
    return InstallRequirement.from_line(
AttributeError: type object 'InstallRequirement' has no attribute 'from_line'

Seems like InstallRequirement.from_line is gone. We'll have to check if there's an equivalent that can be used.
Environment Versions

OS Type: Any (test done on Linux)
Python version: Any (test done on 2.7.x)
pip version: 18.1
pip-tools version: 3.0.0

Steps to replicate
With any requirements.in file

pip-compile

Expected result
Success
Actual result
See traceback above.

I guess this is related to commit https://github.com/pypa/pip/commit/a5a07fe61c4861c58fded7420c929d56cde11a89, which moved pip._internal.req.req_install.InstallRequirement.from_line to pip._internal.req.constructors.install_req_from_line.

Thanks @eseifert, that helped, We might be able to fix this today.

No problem. That's great news!
pip._internal.req.req_install.InstallRequirement.from_editable was also moved to pip._internal.req.constructors.install_req_from_editable by the way (see https://github.com/pypa/pip/commit/69b494aa29096838addad0735359f9406d74252a).

Will be released in the next hours unless something terrible happens.


Extension causes high cpu load

Issue Type: Performance
Extension Name: vscodefy
Extension Version: 0.0.11
OS Version: Darwin x64 18.5.0
VSCode version: 1.33.1

Find more details here: https://github.com/Microsoft/vscode/wiki/Explain:-extension-causes-high-cpu-load

Thanks for open this issue.
I will look up this, but do not think I can improve this because I need to constantly polling the API to get the status of the player

Don't know if you're still looking into this, but here is the CPU-Profile log for VSCodefy:
iagolaguna.vscodefy-unresponsive.cpuprofile.txt

https://github.com/iagolaguna/vscodefy/blob/d9a3ecdd36cfb1f4b7f2823c673961fb98e6a4d1/src/commands/commands.js#L54
Maybe increase this timeout? I think increasing it to 1000, would be fine.


New Extension OPENAI Extension

PR fo issue: https://github.com/poozlehq/engine/issues/66

@AdityaSanthosh can you share the link from where the openapi schema is taken.

@AdityaSanthosh Thanks for contributing!!


Upgrade MediaPlayer.create to conform with dash#~2.0.0

SEE: https://github.com/Dash-Industry-Forum/dash.js/issues/1317#issuecomment-207627034

Reading the open PRs and issues, it is difficult to understand where is the Dash.js 2.x support for this project headed ? What needs to be done ? How can one help ? Thanks !

Dash 2.x works as it stands from the initial upgrade in https://github.com/videojs/videojs-contrib-dash/commit/3e7258e88e8a65855263548607cfb28a9e4378ed (single and common manifests).
However, I have had playback issues if the license server 500's on key exchange. If that issue occurs, you can no longer use the player until you refresh the page (probably something to do with disposing).
One of the dash members recommended fixing how dash is disposed (only create once instead of recreating every time). This may or may not solve the issue...

Once I get back to player work I will address this issue. Feel free to open PR.
As for the support, I am not sure I haven't spoken to @forbesjo.

ok thanks for the update

videojs-contrib-dash implementation doesn't seem to have an effect on the outstanding 500 license issue. The dash player dispose issue probably should be considered before closing this issue.
Regarding the 500 uncaught exception plaguing both video.js and dash.js I opened up another issue with them.

@chemoish I believe #79 should fix this

:tada: thanks.


Avoid using both flags and /etc/docker/daemon.json

Is this a BUG REPORT or FEATURE REQUEST? (choose one):
Bug Report

Environment:

**Cloud provider or hardware configuration: ** CentOS 7 bare metal

docker[7988]: unable to configure the Docker daemon with file /etc/docker/daemon.json: the following directives are specified both as a flag and in the

Got the same issue on debian bare metal.
It seams my previous installation of docker is conflucting with installation of kubespray. I delete the file /etc/docker/daemon.json to fix the issue.

/etc/systemd/system/docker.service.d/* should use least options as possible,and other configurations shoud be used in /etc/docker/daemon.json ,such as  insecure-registries,  storage-driver,storage-opts,live-restore,oom-score-adjust,debug ...

Hello,
Fresh CentOS 7.4 install.
Just installed docker with "yum install docker" (got 2:1.13.1-53.git774336d.el7.centos).
File "/usr/lib/systemd/system/docker.service" contains:
EnvironmentFile=-/etc/sysconfig/docker-storage
called line 25:
$DOCKER_STORAGE_OPTIONS
which contains:
DOCKER_STORAGE_OPTIONS="--storage-driver overlay2 "
Removed content of file /etc/sysconfig/docker-storage and could start docker without problem using devmapper as specified in /etc/docker/daemon.json

WIP for fixing this with #4286


Bump to fix master

[23:17:03] FAIL  ./dist/v0.js: 83.46KB > maxSize 83.45KB (gzip)
[23:17:03] ERROR: bundlesize found that ./dist/v0.js has exceeded its size cap of 83.45KB (Δ +0.01KB)

https://travis-ci.org/ampproject/amphtml/jobs/466246880

@jridgewell Merging to unbork master.


Gap between toggle switch buttons and container

We're using angular for an android app. On my local, there is no gap. On my Samsung Galaxy S3, there is a 1px-3px gap between the toggle switch and container (see screenshots).  I've tried 3 times to close the gap; As soon as I close one gap on the off button a gap will appear on the off button.
Attempt No 1
.toggle-switch .switch-off {
  left: -49%;
}

Attempt No 2
.toggle-switch .knob {
  border: none;
}

Attempt No 3
.toggle-switch .knob {
  border: none;
}

My current CSS:
     .toggle-switch {
    border: 2px solid #626262;
    float: right;
    }
    .toggle-switch span {
    height: 100%;
    font-size: 12px;
    text-transform: uppercase;
    font-weight: normal;
    }
    .toggle-switch .switch-left {
    color: #fff;
    background: #33b5e5;
    text-transform: uppercase;
    }
    .toggle-switch .switch-right {
    color: #000;
    background: #1e6681;
    text-transform: uppercase;
    }

Any ideas on how to fix it so there's no gap on both buttons? Thanks.

I am currently having the same issue, i think it is related to left: -49%; as there could be a pixel or two difference.


Upload failed with "io: read/write on closed pipe" on Apple Mac Mini

Describe the bug
We frequently end up with failed builds due to read/write errors on a closed pipe. With frequently I mean, 1 out of 5 build encounter some kind of read/write error. The most frequent one being during upload / publish step for our builds. These errors seems to be emitted from the jf binary itself and not the Jenkins plugin or Jenkins remoting API itself - which is why I am filing this against the JFrog CLI project.
Setup

Jenkins 2.414.1
JFrog Plugin 1.5.0 (not really relevant here)
JFrog CLI 2.46
Artifactory Version 7.41.14 running on AWS over a "Direct Link". There is a load-balancer in front of it with 2 IPs
Host / Agent is running OS-X 12.6.5 on an ARM64 based Mac Mini

Notes
We did experience problems with jfrog-cli on Linux when our WAN link(s) was overloaded and started dropping traffic. Linux (and Linux networking stack) seemed to be somewhat more resilient and in the case where multiple retries occurred, the upload  / publish step took 2-3 times as long as the normal ones. Still not ideal, but it sort of worked. Even in that case, we'd like the retry to happen sooner than e.g. 60 seconds (especially if you have many small artifacts being uploaded). We are thinking the unreliable network is making communication to the Artifactory server difficult and have seen that other operations such as build-publish, etc. also fails with long timeouts.
Current behavior
Scrubbed the following logs for internal IP/addresses and filenames ...
07:36:48 [Error] io: read/write on closed pipe
07:36:48 [Error] io: read/write on closed pipe
07:36:48 [Error] io: read/write on closed pipe
07:36:48 [Error] io: read/write on closed pipe
07:36:48 [Error] io: read/write on closed pipe
07:36:48 [Warn] [Thread 2] (Attempt 1) - Failure occurred while uploading to https://artifactory/filename1;property1=value1;...: Put "https://artifactory/filename1;property1=value1;...": write tcp <IP_ADDRESS>:54732-><IP_ADDRESS>:443: use of closed network connection
07:36:48 [Warn] [Thread 1] (Attempt 1) - Failure occurred while uploading to https://artifactory/filename2;property1=value1;...: Put "https://artifactory/filename2;property1=value1;...": write tcp <IP_ADDRESS>:54725-><IP_ADDRESS>:443: use of closed network connection
07:36:48 [Error] io: read/write on closed pipe
07:36:48 [Error] io: read/write on closed pipe
07:36:48 [Error] io: read/write on closed pipe

.. 50 more entries of read/write on closed pipe messages ...

07:36:48 [Error] io: read/write on closed pipe
07:36:49 [Warn] [Thread 2] (Attempt 2) - Failure occurred while uploading to https://artifactory/filename3;property1=value1;...: Put "https://artifactory/filename3;property1=value1;...: net/http: HTTP/1.x transport connection broken: http: ContentLength=7219518 with Body length 18771

We are uploading either 7 or 10 artifacts from the Apple platform dependent on the build variant.
Reproduction steps
To reproduce this, I think I'd explore options to use Linux as a traffic shaper as a gateway for a Mac Mini (Arm64) running recent version of OS-X and use the traffic shaper to introduce packet loss.
AWS Artifactory
        |
   Load Balancer
        |
       WAN
        |
     Gateway
        |
Linux Traffic Shaper
        |
Mac Mini Arm64/OS-X 12.6.5 (or similar)

I used latest version of jfrog-cli version 2.46
Expected behavior
The connection(s) in case it receives a connection reset or otherwise tries to do a read/write on a closed pipe should not fail, it should retry. And retries should occur in a "reasonable" time frame, preferably configurable?
JFrog CLI version
2.46
Operating system type and version
Apple Mac Mini, Arm64 based, OS-X 12.6.5
JFrog Artifactory version
7.41.14
JFrog Xray version
N/A
Additional
We've also tried to capture packets on the firewalls in between and not really been successful in isolating the problem.

Still seeing this on Apple even when adding --retries 5 --retry-wait-time 1s. Additionally, the jf binary returns error code -1 when this occurs despite the detailed summary states the upload succeeded???
12:06:16 [Error] io: read/write on closed pipe
{
  "status": "failure",
  "totals": {
    "success": 7,
    "failure": 0
  }
}

I would expect the status would be success unless you had failures in it. Right now, there are 0 failures, but status is still configured to "failure"??


Hard to set options without providing the filled routes variable

let router = new Router5({}, {useHash: false});

// Uncaught Error: Route constructor expects routes to have an name and a path defined.

let router = new Router5(null, {useHash: false});

// Uncaught Error: Route constructor expects routes to have an name and a path defined.

let router = new Router5(undefined, {useHash: false});

// this works

Looks like a minor API quirk.

Use .setOption('useHash, false | true)`

Don't you think the current API is not obvious?

It can be improved

RouteNode won't throw an error for null.
Best way to user Router5 constructor is with an array: empty or containing RouteNode objects or POJOs.
let router5 = new Router5([/*...*/])


Have 1 common azure-pipelines.yml file for the 4 apps

Let's prefer 1 common azure-pipelines.yml file instead of 4 redundant ones. In the meantime, we are illustrating how generic the Azure build pipeline is.

@sabbour, when you have a minute, ready for your review/merge please. Thanks!


502 bad gateaway... Define path, user/group and port?

hi @hellstad, great script. thanks. Issue is, I can't run the app as it always errors 502. Here's my nginx config for the deployed app:
location / {
    proxy_pass http://<IP_ADDRESS>:3000;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
    proxy_set_header Host $host;
}

As I'm running multiple websites on the same machine, it may be possible it is a configuration issue. Would it be possible to define:

deploy path
user/group
node port
in mplz.json?

Hi @cellulosa, does your machine have SELinux enabled? Nginx might be getting denied. (Check the last part of this gist)
Right now all the deploy path/user/port are predefined. I'm working on a totally new version at the moment which will be more flexible. So unfortunately I will not be working on the current version. PRs are welcome though if you would like to add those features :)

hi @hellstad thank you for your reply. I'm really glad to know you're working on a newer version of meteor-please and I'm looking forward to it.
Anyway no, I do not have SELinux enabled. This is the output of systemctl status app.service:
app[29092]: at Module._compile (module.js:460:26)
app[29092]: at Object.Module._extensions..js (module.js:478:10)
systemd[1]: app.service: main process exited, code=exited, status=1/FAILURE
systemd[1]: Unit app.service entered failed state.
systemd[1]: app.service holdoff time over, scheduling restart.
systemd[1]: Stopping app nodejs daemon...
systemd[1]: Starting app nodejs daemon...
systemd[1]: app.service start request repeated too quickly, refusing to start.
systemd[1]: Failed to start app nodejs daemon.
systemd[1]: Unit app.service entered failed state.

You'll be able to get full error logs from using journalctl -u app instead of systemctl status (and scrolling to the bottom of the file). It looks like maybe some npm module didn't compile properly, but need to see the full error message from before this line
app[29092]: at Module._compile (module.js:460:26)

it says
systemd[1]: Starting app nodejs daemon...
systemd[1]: Started app nodejs daemon.
app[10566]: /var/www/app/bundle/programs/server/boot.js:283
app[10566]: }).run();
app[10566]: ^
app[10566]: Error: $ROOT_URL, if specified, must be an URL
app[10566]: at Error (<anonymous>)
app[10566]: at Meteor.absoluteUrl.options (packages/meteor/url_server.js:8:1)
app[10566]: at /var/www/app/bundle/programs/server/packages/meteor.js:1132:4
app[10566]: at /var/www/app/bundle/programs/server/packages/meteor.js:1274:3
app[10566]: at /var/www/app/bundle/programs/server/boot.js:242:10
app[10566]: at Array.forEach (native)
app[10566]: at Function._.each._.forEach (/var/www/app/bundle/programs/server/nod
app[10566]: at /var/www/app/bundle/programs/server/boot.js:137:5
systemd[1]: app.service: main process exited, code=exited, status=8/n/a
systemd[1]: Unit app.service entered failed state.
systemd[1]: app.service holdoff time over, scheduling restart.
systemd[1]: Stopping app nodejs daemon...
systemd[1]: Starting app nodejs daemon...
systemd[1]: Started app nodejs daemon.

Hmm, from that log it seems it has a problem with the ROOT_URL. Could you try to edit the env.ROOT_URL line in your mplz.json to the domain you're using, like this:
  "env": {
    "ROOT_URL": "http://linkstash.xyz"
  }

Then run mplz setup && mplz deploy again, and see if it helps? I do admit the app generates a less than perfect mplz.json at the moment and I have not supplied any documentation.
Here is a sample of one of my configuration files to compare against yours (use node 0.10.36 for Meteor 1.1.x and lower)
{
  "appName": "linkstash",
  "servers": [
    {
      "host": "<IP_ADDRESS>",
      "username": "root",
      "pem": "~/dev/some-pem"
    }
  ],
  "setupMongo": true,
  "setupNode": true,
  "nodeVersion": "0.10.40",
  "setupNginx": true,
  "sslcert": "",
  "sslkey": "",
  "verbosity": true,
  "appSiteUrl": "linkstash.xyz",
  "app": ".",
  "env": {
    "ROOT_URL": "http://linkstash.xyz"
  }
}

O see. Alright, that worked. Thank you!
Yes I think it would be essential to provide some documentation - even just this sample you supplied.


[SDK Renaming] - Readme Updates

Jira Ticket

ACCSEC-

Github Issue

ISSUE-

Description

Clarify import at Readme file, for the upcoming version 1.0.0

Commit message

Screenshot

Testing

[ ] Added unit tests
[ ] Ran unit tests successfully
[ ] Added documentation for public APIs and/or Wiki

Code coverage
Total coverage: 95.13%

Powered by Slather

  Generated by :no_entry_sign: Danger


Nested if statment looses indent.

Thanks for a really nice tool.
I have noticed an edge case, nested if statements are losing their indention level. Perhaps related to #40
Given
if config["a"] is None:
    include: "module_a_none.smk"

    if config["b"] is None:
        include: "module_b.smk"
    else:
        include: "module_c.smk"

else:
    include: "module_a.smk"

I get
if config["a"] is None:

    include: "module_a_none.smk"

if config["b"] is None:

        include: "module_b.smk"

else:

        include: "module_c.smk"

else:

    include: "module_a.smk"

Notice how the nested if and else are unindented, but the nested includes are at the correct indention level.
# Name                    Version                   Build  Channel
black                     19.10b0                  py38_0    conda-forge
python                    3.8.6           h852b56e_0_cpython    conda-forge
snakefmt                  0.2.3                      py_0    bioconda

This should be fixed in v0.2.4 now @jfear . Let us know if you still have issues.


[WIP] xandikos: init at 0.0.6

Motivation for this change
OwnCloud SUCKS so much for syncing calendar & contact data, fucks up my data every other day, does not allow creating collections from clients (using vdirsyncer) ... so I need an alternative.
Radicale is HELL when trying to set it up,...it just won't work as I request.
So this is another caldav/carddav server which claims to be compatible with my android sync client and vdirsyncer... hopefully it is.

Still waiting to get this issue: https://github.com/jelmer/xandikos/issues/70 resolved, though
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[x] Fits CONTRIBUTING.md.

WIP!

So it looks like dulwich gets build with fastimport, which is not python3 in nixpkgs, but xandikos needs/is python3.
Can someone tell me what to do?

Issue solved. This works now, but I would like to add a service definition for this before it gets merged!

Added a service definition, though I don't know how to test this properly or even implement VM tests for it.
Can someone jump in? :smile:

Done. Still no tests, though.

Can someone suggest how to test this?

https://github.com/pimutils/vdirsyncer seems like an easy way to test this.

For the author, reviewers, and committers: this PR was scanned and appears to add a use of the deprecated types.string, which emits a warning as of #66346. Before merging, please change this to another type, possibly:

types.str for a single string where merging does not make sense, or cannot work
types.lines for multi-line configuration or scripts where merging is possible
types.listOf types.str for a mergeable list of strings


multiple camera access?

I have no problem capturing image from a single webcam, but when i have 2 webcams, and I try to run it, it will complains Error: CAMERA ERROR [VIDIOC_STREAMON] 12 Cannot allocate memory, may I know how can I solve this problem?

Try plugging the camera to another USB port, it's possible that the usb cannot handle the traffic of more than one camera. (I had a similar problem but i didn't remember if it was an ENOSPC or an ENOMEM, yours is ENOMEM).


would you add an open source license and push to Nuget

Hi, there is no good OpenSea lib on nuget and you codes is the best. Would you be interested in adding an open source license (like MIT) and pushing to nuget so it is easier to use?

Hello there!
Yes, I will do that today

https://www.nuget.org/packages/OpenSeaClient/ 🎉


When I check the TensorBoard for a Worker of ParameterServer traning, I can't see any send/recv or push/pull op to the PS? Is this normal?

Hi, developers of tensorflow.
The environment, code are listed below:
TF version: nightly
OS: ubuntu16.04
cuda: 11.0
Worker code:
import multiprocessing
import os
import json
import random
import portpicker
import tensorflow as tf
from tensorflow.keras.layers.experimental import preprocessing

def create_in_process_cluster(num_workers, num_ps):
  """Creates and starts local servers and returns the cluster_resolver."""
  worker_ports = [portpicker.pick_unused_port() for _ in range(num_workers)]
  ps_ports = [portpicker.pick_unused_port() for _ in range(num_ps)]

  cluster_dict = {}
  cluster_dict["worker"] = ["ip:port"]
  if num_ps > 0:
    cluster_dict["ps"] = ["ip:port"]
  print("==========================", cluster_dict["worker"], cluster_dict["ps"])
  cluster_spec = tf.train.ClusterSpec(cluster_dict)

  # Workers need some inter_ops threads to work properly.
  worker_config = tf.compat.v1.ConfigProto()
  if multiprocessing.cpu_count() < num_workers + 1:
    worker_config.inter_op_parallelism_threads = num_workers + 1

  os.environ["TF_CONFIG"] = json.dumps({
    "cluster": {
        "worker": cluster_dict["worker"],
        "ps": cluster_dict["ps"],
    },
    "task": {"type": "worker", "index": 0}
  })
  cluster_resolver = tf.distribute.cluster_resolver.TFConfigClusterResolver()
  return cluster_resolver

# Set the environment variable to allow reporting worker and ps failure to the
# coordinator. This is a workaround and won't be necessary in the future.
os.environ["GRPC_FAIL_FAST"] = "use_caller"

NUM_WORKERS = 1
NUM_PS = 1

cluster_resolver = create_in_process_cluster(NUM_WORKERS, NUM_PS)
os.environ["GRPC_FAIL_FAST"] = "use_caller"
if cluster_resolver.task_type in ("worker", "ps"):
    server = tf.distribute.Server(
        cluster_resolver.cluster_spec(),
        job_name=cluster_resolver.task_type,
        task_index=cluster_resolver.task_id,
        protocol=cluster_resolver.rpc_layer or "grpc",
        start=True)

print("==========================")

strategy = tf.distribute.experimental.ParameterServerStrategy(cluster_resolver)

def dataset_fn(input_context):
  global_batch_size = 64
  batch_size = input_context.get_per_replica_batch_size(global_batch_size)

  x = tf.random.uniform((10, 10))
  y = tf.random.uniform((10,))

  dataset = tf.data.Dataset.from_tensor_slices((x, y)).shuffle(10).repeat()
  dataset = dataset.shard(
      input_context.num_input_pipelines,
      input_context.input_pipeline_id)
  dataset = dataset.batch(batch_size)
  dataset = dataset.prefetch(2)

  return dataset

dc = tf.keras.utils.experimental.DatasetCreator(dataset_fn)

with strategy.scope():
  model = tf.keras.models.Sequential([tf.keras.layers.Dense(10)])

model.compile(tf.keras.optimizers.SGD(), loss='mse', metrics=['accuracy'], steps_per_execution=10)

working_dir = './my_working_dir'
log_dir = os.path.join(working_dir, 'log')
ckpt_filepath = os.path.join(working_dir, 'ckpt')
backup_dir = os.path.join(working_dir, 'backup')
tb_dir = os.path.join(working_dir, 'tb')

callbacks = [
    tf.keras.callbacks.TensorBoard(log_dir=log_dir),
    tf.keras.callbacks.ModelCheckpoint(filepath=ckpt_filepath),
    tf.keras.callbacks.experimental.BackupAndRestore(backup_dir=backup_dir),
    tf.keras.callbacks.TensorBoard(log_dir=tb_dir, profile_batch = 100000000)
]

model.fit(dc, epochs=10, steps_per_epoch=20, callbacks=callbacks)

The graph of tensorboard:

I can't confirm whether this Worker process is interacting with PS to update the parameters in each step.
Are the communication operations not compiled into the graph?

Hi @zpcalan ,Could you please raise the issue in TensorBoard repo.

Hi @mohantym, this seems to be a question for TensorFlow rather than TensorBoard, since TensorBoard just renders whatever is in the op graph it receives, it doesn't control what distribution-related ops are included. Thanks!

Hi @zpcalan , Was getting error in TF 2.6 and TF nightly (2.7.0-dev20210903) , providing gist for reference .

Hello zpcalan@, can you call tf.debugging.set_log_device_placement(True) at the start of the program and see if it provides useful device placement information?

As for the reason why you don't see send/recv ops in the graph, it is because the TensorBoard callback only collects the graph of the "train function", and not the communications.

Hello zpcalan@, assuming you were mainly trying to confirm the interactions between workers and ps, can you call tf.debugging.set_log_device_placement(True) at the start of the program and see if it provides useful device placement information?

Sorry for the late feedback.
I added tf.debugging.set_log_device_placement(True) and the output is below:
 I tensorflow/core/distributed_runtime/rpc/grpc_channel.cc:272] Initialize GrpcChannelCache for job ps -> {0 -> myip:18267}
 I tensorflow/core/distributed_runtime/rpc/grpc_channel.cc:272] Initialize GrpcChannelCache for job worker -> {0 -> myip:21351}
 I tensorflow/core/distributed_runtime/rpc/grpc_server_lib.cc:427] Started server with target: grpc://myip:21351
 I tensorflow/core/distributed_runtime/rpc/grpc_channel.cc:272] Initialize GrpcChannelCache for job ps -> {0 -> myip:18267}
 I tensorflow/core/distributed_runtime/rpc/grpc_channel.cc:272] Initialize GrpcChannelCache for job worker -> {0 -> myip:21351}
 I tensorflow/core/distributed_runtime/rpc/grpc_channel.cc:272] Initialize GrpcChannelCache for job chief -> {0 -> localhost:38320}
 I tensorflow/core/distributed_runtime/rpc/grpc_channel.cc:272] Initialize GrpcChannelCache for job ps -> {0 -> myip:18267}
 I tensorflow/core/distributed_runtime/rpc/grpc_channel.cc:272] Initialize GrpcChannelCache for job worker -> {0 -> myip:21351}
 I tensorflow/core/distributed_runtime/rpc/grpc_channel.cc:272] Initialize GrpcChannelCache for job chief -> {0 -> localhost:38320}
 I tensorflow/core/distributed_runtime/eager/eager_service_impl.cc:272] Creating sync eager service context with rendezvous_id on host node124 /job:worker/replica:0/task:0
 I tensorflow/core/distributed_runtime/rpc/grpc_channel.cc:272] Initialize GrpcChannelCache for job ps -> {0 -> myip:18267}
 I tensorflow/core/distributed_runtime/rpc/grpc_channel.cc:272] Initialize GrpcChannelCache for job worker -> {0 -> myip:21351}
 I tensorflow/core/distributed_runtime/rpc/grpc_channel.cc:272] Initialize GrpcChannelCache for job chief -> {0 -> localhost:38320}
 I tensorflow/core/distributed_runtime/rpc/grpc_server_lib.cc:427] Started server with target: grpc://localhost:38320
 I tensorflow/core/common_runtime/eager/execute.cc:1403] Executing op VarHandleOp on task /job:ps/replica:0/task:0/device:CPU:0
 I tensorflow/core/common_runtime/eager/execute.cc:1403] Executing op AssignVariableOp on task /job:ps/replica:0/task:0/device:CPU:0
 I tensorflow/core/common_runtime/eager/execute.cc:1403] Executing op VarHandleOp on task /job:ps/replica:0/task:0/device:CPU:0
 I tensorflow/core/common_runtime/eager/execute.cc:1403] Executing op AssignVariableOp on task /job:ps/replica:0/task:0/device:CPU:0
 I tensorflow/core/common_runtime/eager/execute.cc:1403] Executing op VarHandleOp on task /job:ps/replica:0/task:0/device:CPU:0
 I tensorflow/core/common_runtime/eager/execute.cc:1403] Executing op AssignVariableOp on task /job:ps/replica:0/task:0/device:CPU:0
 I tensorflow/core/common_runtime/eager/execute.cc:1403] Executing op Fill on task /job:ps/replica:0/task:0/device:CPU:0
 I tensorflow/core/common_runtime/eager/execute.cc:1403] Executing op VarHandleOp on task /job:ps/replica:0/task:0/device:CPU:0
 I tensorflow/core/common_runtime/eager/execute.cc:1403] Executing op AssignVariableOp on task /job:ps/replica:0/task:0/device:CPU:0
 I tensorflow/core/common_runtime/eager/execute.cc:1403] Executing op Fill on task /job:ps/replica:0/task:0/device:CPU:0
 I tensorflow/core/common_runtime/eager/execute.cc:1403] Executing op VarHandleOp on task /job:ps/replica:0/task:0/device:CPU:0
 I tensorflow/core/common_runtime/eager/execute.cc:1403] Executing op AssignVariableOp on task /job:ps/replica:0/task:0/device:CPU:0
 I tensorflow/core/common_runtime/eager/execute.cc:1403] Executing op VarHandleOp on task /job:ps/replica:0/task:0/device:CPU:0
 I tensorflow/core/common_runtime/eager/execute.cc:1403] Executing op AssignVariableOp on task /job:ps/replica:0/task:0/device:CPU:0
 I tensorflow/core/profiler/lib/profiler_session.cc:110] Profiler session initializing.
 I tensorflow/core/profiler/lib/profiler_session.cc:125] Profiler session started.
 I tensorflow/core/profiler/internal/gpu/cupti_tracer.cc:1630] Profiler found 8 GPUs

The reason I want to see the communication ops is that I'm tring to understand the inner implementation of send/recv ops in parameter server startegy.
Is there a way to visualize it? Because if only computation ops are displayed and no optimizer ops, it may confuse users.

@mohantym Maybe this issue could be reopened. : )

From the log you can see that PS is indeed executing some ops related to the variables.

Is there a way to visualize it? Because if only computation ops are displayed and no optimizer ops, it may confuse users.

I'm not aware of a way with eager execution; if you can re-build TF library, you can make sure all the logs in execute.cc or placer.cc in common_runtime directory are logged regardless, which should give you a more detailed view.

As well as process_function_library_runtime.cc

@rchao Thanks for your reply.
Is there a way to open the detailed log you said in common_runtime directory without compiling TF?
Set environment variables or call some TF APIs?

Sounds good! Let me know if you find anything.


Windows Terminal, Command Prompt, OpenConsole processes cannot be suspended

Windows Terminal version (or Windows build number)
10.0.19042.906
Other Software
Command Prompt
Windows Terminal
Open Console
Steps to reproduce

Run Process Explorer and/or resmon.exe /res
Suspend cmd.exe
Suspend WindowsTerminal.exe
Suspend OpenConsole.exe

Expected Behavior
The processes are suspended and show with a grey background in Process Explorer like other software.
For example:

Actual Behavior
cmd.exe and WindowsTerminal.exe cannot be suspended. Additionally OpenConsole.exe shows suspended for 5ms before magically becoming unsuspended.
For example:

This is especially problematic with the built-in resource monitor tool since the console process becomes deadlocked after selecting suspend and the process continues running:

We do not control this. Also, why?

@DHowett
You can suspend other processes perfectly fine except the command prompt so that's a bug that should be looked into??

This is likely due to the job object that is created for WT thanks to it being launched inside a package, which is out of our control and isn’t something we can fix. It’s also just not a supported use case: individually stomping processes on your system to make sure they “behave” isn’t a normal part of system administration...

This is likely due to the job object that is created for WT

I can reproduce the same issue with a lot of other software without job objects... Chrome, Msedge, Visual Studio, Discord, Steam, Skype even Windbg which needs to suspend the thread/process to properly capture thread contexts when debugging the console project.
The whole point of creating this ticket was so you guys could follow up the problem internally with the kernel team since A) It affects the console and debugging the console, so it's a very relevant bug for this project and B) because it's a recent issue with Windows 10 that breaks everything using the SuspendThread function since 2000 (21 years of backwards compatibility).

It’s also just not a supported use case: individually stomping processes on your system to make sure they “behave” isn’t a normal part of system administration

SuspendThread has some very valid uses cases outside of administration especially for debugging. I reported this for the above reasons not because of administration.

It would help us build a case for getting this fixed if we understood your use case, you know? VS and WinDbg don’t seem to have any trouble with Terminal or its subprocesses, and if I understand them properly they’re not using any special APIs for doing so. We’ve been debugging things inside terminal or hosted by OpenConsole (which services console API functions and does not “host” a process in any meaningful way related to its process tree) for the past two years on windows versions 17763 to 21310.
I’m not embellishing this point for no reason. It seems like totally normal debugging tools work just fine for us. I’m worried that we’re looking at an XY Problem.

recent issue

I must have missed in your original report that this was a regression. Sorry about that! 😄
It

@DHowett

I must have missed in your original report that this was a regression

My bad. I should have mentioned earlier that it's a regression and it worked fine prior to Windows 10. For example:
Resmon:

Process Explorer:

Compared to Windows 10:

We’ve been debugging things inside terminal or hosted by OpenConsole for the past two years

It's probably not something that's been noticed/reported until now? Windbg uses GetThreadContext and per MSDN you cannot get a proper valid context for a running thread because of race conditions/other issues which is why suspension is required. If you attach windbg to the command prompt/windows terminal processes then it should suspend the process but doesn't and using some other tools you can see 1 thread never transitions to the Suspended wait state. For example:
The suspend count is 1 but that value is just a counter unrelated to the actual suspension:

The thread hasn't actually suspended and the KWAIT_REASON for the thread permanently remains Executive instead of transitioning to Suspended (the first thread with ntdll.dll!DbgUiRemoteBreakin is windbg and should be ignored)

It seems like totally normal debugging tools work just fine for us

The thread is the console calling ReadFile on a pipe and that thread is probably not something that gets debugged very often and since windbg is showing an incorrect suspension state its probably gone unnoticed?
It's also the same problem causing Process Explorer and the built-in Resmon tool to not show cmd.exe/WindowsTerminal.exe processes' as suspended because that one thread doesn't transition to the Suspended state like previous versions of Windows. Context switches are also supposed to trigger the suspension but queuing a kernel APC which does execute and causes a context switch/increments the counters also doesn't trigger the suspension.

I’m worried that we’re looking at an XY Problem.

I guess that's somewhat related. I'm using cmd.exe and WindowsTerminal.exe as an example of the problem but it's not just those two processes that can be used to demonstrate this issue on Windows 10 - I'm only mentioning cmd.exe and WindowsTerminal.exe since this tracker is only for issues related to the console. I did expect the issue to get closed but sort of expected someone would follow up/report the bug internally with whomever owns that kernel code especially since it can be easily reproduced with cmd.exe and Process Explorer when compared to previous versions of Windows?
These non-suspendable threads end up deadlocked inside IopSynchronousServiceTail which is also the same code apparently responsible for handling the CTRL+C signals for the console and that function also handles synchronous waits on file objects (such as when cmd.exe is calling ReadFile on a pipe handle - the same thread that doesn't get suspended)... So there might have been some changes for console support on Win10 that also accidently broke thread suspension as a side effect - I can't be sure what changed but these threads would successfully suspend on Win7/Win8 but are now permanently running on Win10 despite calling SuspendThread.

It would help us build a case for getting this fixed if we understood your use case

The primary use case is debugging since multiple programs/debuggers are reporting incorrect information. The other primary issue is backwards compatibility since this change on Windows 10 breaks Microsoft's own software (including built-in applications like resmon) and also lots of third party tools released since 2000 (including every version of our own software released over the last 12 years).
Hope this helps
-dmex

The thread hasn't actually suspended and the KWAIT_REASON for the thread permanently remains Executive instead of transitioning to Suspended (the first thread with ntdll.dll!DbgUiRemoteBreakin is windbg and should be ignored)

I think this only applies to a synchronous I/O wait, which is an alertable kernel wait in IopSynchronousServiceTail for the Event of a synchronous-mode file object. The thread should switch to the suspended state as soon as the synchronous I/O wait is completed or canceled.
If I had to guess, I'd say this is related to the implementation of NtCancelSynchronousIoFile (i.e. WinAPI CancelSynchronousIo). I suggest asking about this on a more appropriate forum, such as OSR Developer Community.

These non-suspendable threads end up deadlocked inside IopSynchronousServiceTail which is also the same code apparently responsible for handling the CTRL+C signals

What deadlock? Also, what's the relevance of Ctrl+C? The only connection I see is that, for a conpty session, the Ctrl+C character is read from a pipe, which is a synchronous read. But handling this input by sending a console control event has nothing to do with I/O. The console host simply sends a request to the desktop-session server (csrss.exe in the current session) to request the creation of a control thread in the client.

the thread should switch to the suspended state as soon as the synchronous I/O wait is completed

That's what should occur in theory but in reality it's continuing to schedule and wait indefinitely without ever transitioning to the suspended state.

I'd say this is related to the implementation of NtCancelSynchronousIoFile

The cancellation feature has exited since Vista (2006) but the issue with suspension has only started with recent versions of Win10.

What deadlock?

The suspension APC is scheduled to the thread but it never actually executes regardless of how many days/weeks/months you're waiting for the state change which is the a textbook definition of a deadlock.

handling this input by sending a console control event has nothing to do with I/O

It does and always has.

what's the relevance of Ctrl+C?

The sourcecode for these functions where this deadlock occurs includes specific comments that mention the CTRL+C handling 🤷♂

I suggest asking about this on a more appropriate forum, such as OSR Developer Community.

This isn't a development issue so they won't be much help and they use my project source as a reference so it would basically become a circle jerk with suggestions of switching to linux without anything on Windows ever getting fixed... The reason I created the issue here was so someone like @DHowett would hopefully follow up/raise this issue internally with his co-workers since this issue also affects cmd.exe

I poked around a bit. There are a couple of interesting changes relevant to suspending threads.
The first change should be a transparent optimization. If KiDisableLightWeightSuspend is 0, KiSuspendThread() doesn't call KiSignalThreadForApc() to deliver the suspend APC. If the thread is already waiting, the APC remains in the queue until the wait ends. The thread's Tcb.WaitReason does not change to Suspended (5). However, the SYSTEM_THREAD_INFORMATION for the thread reports that the wait state is Suspended, which effectively is the case for this light-weight suspend. When the thread's suspend count is decremented to 0, the original wait continues. The suspend APC gets delivered when the thread is dispatched, but there's nothing for it to do by then.
The change that's problematic is that, for non-alertable synchronous I/O, system calls such as NtReadFile are now implemented to disable kernel APCs before acquiring the file lock. The alertable kernel wait in IopSynchronousServiceTail() thus only allows delivery of special kernel APCs, such as the I/O completion APC. When the wait ends because the I/O request is completed or canceled (e.g. a console read completes), the file lock is released via IopReleaseFileObjectLock(). This in turn calls KeLeaveCriticalRegion(), which enables kernel APCs (i.e. increments the thread's KernelApcDisable value) and calls KiCheckForKernelApcDelivery(), which delivers the suspend APC. At this point, the thread wait state should become Suspended.
For example, if you suspend cmd.exe, pressing enter in its console or terminal window should complete its ReadConsoleW() call (internally an NtDeviceIoControlFile call) and allow the suspend APC to run. Suspending the host process for a conpty (pseudconsole) session is a bit more complicated. Pressing enter in the console is necessary but not sufficient because the host process may have two threads doing a synchronous read, one for the input pipe and one for the signal pipe. For the latter, resizing the console/terminal window should write to the signal pipe and complete the read in the host. Once both threads become suspended, the conhost.exe or openconsole.exe process should report as suspended in Process Explorer or Resource Monitor.
That said, I do not know why non-alertable synchronous I/O is disabling delivery of normal kernel APCs, such as the suspend APC, while the file object's lock is held.
What I do know is that, starting with Windows Vista, pended synchronous creates were changed to use an alertable kernel-mode wait in order to support cancellation when the requesting thread is terminated. Moreover, it seems that all synchronous I/O waits were changed to use an alertable kernel-mode wait. (Formerly, for an I/O system call from user mode, the wait for non-alertable synchronous I/O was a non-alertabled user-mode wait.) Thread termination in this case relies on KeAlertThread() to interrupt the alertable kernel-mode wait. When the wait in IopSynchronousServiceTail() is alerted, it calls IopCancelAlertedRequest() if the thread is terminating or the file mode is alertable synchronous I/O. Otherwise, since an arbitrary alert shouldn't be allowed to interrupt non-alertable synchronous I/O, it calls IopCheckIrpCancelled() and resumes waiting if the request isn't canceled.

The sourcecode for these functions where this deadlock occurs includes specific comments that mention the CTRL+C handling man_shrugging

The Interix (SUA) subsystem has some way to use APCs to implement POSIX signals. The Windows subsystem (or rather its designers), on the other hand, is opposed to the way Unix signals work, which entails executing handlers asynchronously in user mode on a thread that's not in a alertable wait state. Instead, Windows delivers a console control event, such as CTRL_C_EVENT, by creating a new thread in the target process that begins at the internal CtrlRoutine function.

This isn't a development issue so they won't be much help and they use my project source as a reference so it would basically become a circle jerk with suggestions of switching to linux without anything on Windows ever getting fixed...

The OSR community is for developers of NT device drivers. Like any online forum, I'm sure it has problems with trolls and difficult personalities. But the founding members there, who have been writing NT device drivers for almost 30 years, are better situated to get the attention of the kernel developers at Microsoft. I don't think the terminal/conhost team has the time or inclination to internally report an issue like this and relay feedback to you, but maybe you lucked out here.


Storage Classes Replaced the Disk instead of Expanding

I ensured that the storage class had the allowVolumeExpansion set to true.
I changed the value for the disk from 10GB to 15GB and the PVC reprovisioned totally removing the data.
Instead of expanding, it seems new disks were created meaning data loss for anything that wasnt backed up.
Here is the description of the PVC
│ Name:          export-minio-1
│ Namespace:     product
│ StorageClass:  default
│ Finalizers:    [kubernetes.io/pvc-protection]
│ Capacity:      15Gi
│ Access Modes:  RWO
│ VolumeMode:    Filesystem
│ Used By:       minio-1
│ Events:        <none>

Here is the description of the storage class
│ Name:                  default
│ IsDefaultClass:        Yes
│ Annotations:           storageclass.kubernetes.io/is-default-class=true
│ Provisioner:           disk.csi.azure.com
│ Parameters:            skuname=StandardSSD_LRS
│ AllowVolumeExpansion:  True
│ MountOptions:          <none>
│ ReclaimPolicy:         Delete
│ VolumeBindingMode:     WaitForFirstConsumer                                                                                │
│ Events:                <none>

/transfer kubernetes


Should EC2.InstanceType be a String instead of an enum

New EC2 instances appear fairly regularly but the json model files don't seem to get updated with them fast enough

Maybe a RawRepresentable struct like Region might work

#422


Confusion in dependency version

Hi, you guys really did an excellent work. Since it has been more than a year, may you specify the versions of the packages required? Thanks

Agree.
I have the same issue. Im facing some problems in gradio version changes.
It would be very useful.
Good Job with the project.


Platform changes causing an error.

One needs to modify files of index_data.py and recommendation just a small change of os.sep what it is returning in two of python 2 or 3 in Windows os as I am using Mac it is returning the same in both cases kindly see what it and if someone wants correct it.  I would suggest to make different files for both users as it will be ease for both users to work.

@ulshell
I wish to solve this using platform detection if that too fails we can think about using two files. I am claiming this issue for now.


Missing linebreaks

Hi,
can you add line breaks for 2. Get an access token and 4. Refresh the token?
Thanks

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: f3e20651-5b2b-bd0b-dc92-a4a3643776ef
Version Independent ID: c9e25d10-4298-c95d-d24a-25a45c47e9ac
Content: Authorization code flow - Azure Active Directory B2C
Content Source: articles/active-directory-b2c/authorization-code-flow.md
Service: active-directory
Sub-service: b2c
GitHub Login: @kengaderdus
Microsoft Alias: kengaderdus

@TDroogers
Thanks for your feedback! We will investigate and update as appropriate.

@kengaderdus
could you please review this and update as appropriate.

@TDroogers
I've delegated this to @kengaderdus, a content author, to review and share their valuable insights.

@TDroogers
Thanks for your feedback! Which section of the article are you referring to?

2. Get an access token
grant_type=authorization_code
&client_id=90c0fe63-bcf2-44d5-8fb7-b8bbc0b29dc6
&scope=90c0fe63-bcf2-44d5-8fb7-b8bbc0b29dc6 offline_access
&code=AwABAAAAvPM1KaPlrEqdFSBzjqfTGBCmLdgfSTLEMPGYuNHSUYBrq...
&redirect_uri=urn:ietf:wg:oauth:2.0:oob
&code_verifier=ThisIsntRandomButItNeedsToBe43CharactersLong
4. Refresh the token
grant_type=refresh_token
&client_id=90c0fe63-bcf2-44d5-8fb7-b8bbc0b29dc6
&scope=90c0fe63-bcf2-44d5-8fb7-b8bbc0b29dc6 offline_access
&refresh_token=AwABAAAAvPM1KaPlrEqdFSBzjqfTGBCmLdgfSTLEMPGYuNHSUYBrq...
&redirect_uri=urn:ietf:wg:oauth:2.0:oob
etc. etc.

@TDroogers
Thanks for replying. We'll format. Please note that our changes may take some time before they go live. Meanwhile, we'll close this issue. #please-close


Searching for HPX-Application to generate work on multiple Nodes

Hello, I am looking for an open source HPX-Application, which I can use to generate work on multiple Nodes, in order to analyse the performance of said application.
Any ideas, suggestions and experiences are very welcome,
Leon

Any hpx + cuda + multiple nodes example?

Any hpx + cuda + multiple nodes example?

HPX distributed work and CUDA are orthogonal and unrelated. You should be able to combine these two easily.

For me HPX without any other Libraries would be most fitting.
Can any of the Examples in this GitHub Repository put multiple Nodes to work?

@leonfse I suppose the 1D stencil examples in examples/1d_stencil would be one (simple) option. I think starting from 1d_stencil_5 they are distributed. https://github.com/STEllAR-GROUP/tutorials/tree/master/examples/03_stencil also contains a 2D stencil example, although it may need some tweaking to make it work again with the latest version of HPX.


TensortRT has no effect on ssd_mobilenet_v1_fpn_coco model

When I use the ssd_mobilenet_v1_fpn_coco model to use tensorRT to accelerate,It doesn't work
retinanet mobile no tensorRT
Iteration: 0.430 sec
Iteration: 0.421 sec
Iteration: 0.420 sec
Iteration: 0.427 sec
Iteration: 0.439 sec
Iteration: 0.427 sec
Iteration: 0.411 sec
Iteration: 0.424 sec
Iteration: 0.432 sec
Iteration: 0.429 sec
Iteration: 0.413 sec
Iteration: 0.424 sec
Iteration: 0.424 sec
Iteration: 0.428 sec
Iteration: 0.427 sec
Iteration: 0.431 sec
Iteration: 0.417 sec
Iteration: 0.418 sec
tensorRT
0.505087852478
0.504916906357
0.501970052719
0.505352973938
0.494786024094
0.498456954956
0.504287004471
0.50328707695
0.507141113281
0.499255895615
0.487679004669
0.489063978195
0.492527008057
0.503779172897
0.514405965805
log:
retinanet v1
('config_path', './data/ssd_mobilenet_v1_fpn_shared_box_predictor_640x640_coco14_sync_2018_07_03/pipeline.config')
('checkpoint_path', './data/ssd_mobilenet_v1_fpn_shared_box_predictor_640x640_coco14_sync_2018_07_03/model.ckpt')
2018-09-03 09:24:44.137510: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:865] ARM64 does not support NUMA - returning NUMA node zero
2018-09-03 09:24:44.137784: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1356] Found device 0 with properties:
name: NVIDIA Tegra X2 major: 6 minor: 2 memoryClockRate(GHz): 1.3005
pciBusID: 0000:00:00.0
totalMemory: 7.67GiB freeMemory: 4.45GiB
2018-09-03 09:24:44.137850: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1435] Adding visible gpu devices: 0
2018-09-03 09:24:47.792908: I tensorflow/core/common_runtime/gpu/gpu_device.cc:923] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-09-03 09:24:47.793169: I tensorflow/core/common_runtime/gpu/gpu_device.cc:929]      0
2018-09-03 09:24:47.793277: I tensorflow/core/common_runtime/gpu/gpu_device.cc:942] 0:   N
2018-09-03 09:24:47.793573: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1053] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 2913 MB memory) -> physical GPU (device: 0, name: NVIDIA Tegra X2, pci bus id: 0000:00:00.0, compute capability: 6.2)
Converted 333 variables to const ops.
2018-09-03 09:26:07.919932: I tensorflow/core/grappler/devices.cc:51] Number of eligible GPUs (core count >= 8): 0
2018-09-03 09:26:16.036617: I tensorflow/contrib/tensorrt/convert/convert_graph.cc:383] MULTIPLE tensorrt candidate conversion: 4
2018-09-03 09:26:16.057791: W tensorflow/contrib/tensorrt/convert/convert_graph.cc:418] subgraph conversion error for subgraph_index:0 due to: "Unimplemented: Require 4 dimensional input. Got 0 const6" SKIPPING......( 108 nodes)
2018-09-03 09:26:16.064689: W tensorflow/contrib/tensorrt/convert/convert_graph.cc:418] subgraph conversion error for subgraph_index:1 due to: "Unimplemented: Require 4 dimensional input. Got 0 const6" SKIPPING......( 108 nodes)
2018-09-03 09:26:16.837392: W tensorflow/contrib/tensorrt/convert/convert_graph.cc:418] subgraph conversion error for subgraph_index:2 due to: "Invalid argument: Output node 'const6' is weights not tensor" SKIPPING......( 612 nodes)
2018-09-03 09:26:16.842941: W tensorflow/contrib/tensorrt/convert/convert_graph.cc:418] subgraph conversion error for subgraph_index:3 due to: "Unimplemented: Require 4 dimensional input. Got 1 Postprocessor/BatchMultiClassNonMaxSuppression/MultiClassNonMaxSuppression/zeros_like_47" SKIPPING......( 181 nodes)
['boxes', 'classes', 'scores']
2018-09-03 09:27:47.320719: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1435] Adding visible gpu devices: 0
2018-09-03 09:27:47.320894: I tensorflow/core/common_runtime/gpu/gpu_device.cc:923] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-09-03 09:27:47.320933: I tensorflow/core/common_runtime/gpu/gpu_device.cc:929]      0
2018-09-03 09:27:47.320967: I tensorflow/core/common_runtime/gpu/gpu_device.cc:942] 0:   N
2018-09-03 09:27:47.321106: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1053] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 2913
thanks !

Thanks for sharing this issue.
Which version of TensorFlow, Jetpack are you using, and what parameters do you feed to trt.create_inference_graph?

The  version of TensorFlow is 1.8
The  version of Jetpack is 28
The  parameters  of  feeding to trt.create_inference_graph are listed as follow
trt_graph = trt.create_inference_graph(
input_graph_def=frozen_graph,
outputs=output_names,
max_batch_size=1,
max_workspace_size_bytes=1 << 25,
precision_mode='FP16',
minimum_segment_size=50
)

The version of TensorFlow is 1.8
The version of Jetpack is 28
The parameters of feeding to trt.create_inference_graph are listed as follow
trt_graph = trt.create_inference_graph(
input_graph_def=frozen_graph,
outputs=output_names,
max_batch_size=1,
max_workspace_size_bytes=1 << 25,
precision_mode='FP16',
minimum_segment_size=50
)

does this work for you ? in my experience, i  use this method to speed up Unet model, but it does not worked ,the speed is not faster than with tensorflow, the so i wanna ask your final speed result when  you use this method

@Programmerwyl Got an almost identical result on jetson NANO. Have you got any clue later on this issue ?


FEATURE - GA Home Page: Add ability to show Quick filters in SBAS for Fiori dev space

As a developer using Guided Answers in Fiori dev space in SAP Business Application Studio,
I want to see a list of PRODUCTS and COMPONENT as quick filters in new GA Home Page ,
so that I can click on the any of the suggested filter for faster and accurate search results
Description

The new GA Home Page will show a Quick Filters section. Later we will expand it and add Product and component combination. For now, we can just show list of components

For SAP Fiori dev space, these are the software components which needs to be shortlisted:
CA-UX-IDE
CA-FE-FLP-EU
CA-FE-FLP-DT
CA-FE-FAL
CA-UI2-INT-BE
CA-UI2-INT-FE
CA-UI2-THD

Technical Design
TBA
General User Flow

User launches GA in SBAS Fiori dev space
User is shown the GA Home Page with quick filters which are shortlisted component
User will click on the suggested filter (Currently, it will be all the components)
User is taken to Search Screen where filters applied are shown
User searches for something
Filters will NOT be cleared NOW
Search should be done with the filters applied

Acceptance Criteria
GivenWhenThen format (https://martinfowler.com/bliki/GivenWhenThen.html)
Given I have a dev space of type SAP Fiori open in SBAS
when I open Guided Answers
then the Quick Filters section should show components shortlisted for Fiori dev space
Notes
Tasks

[ ] Unit Tests added
[ ] Test Cases defined

@carwelsh @ritarora done & merged


Enhance and fix unstable test result from test_node_eviction

Split test case into different scenarios.
Removed intermediate state check.
Fix test failure when k8s decided to relocate replicas one time only during anti-affinity node eviction.
Fix test fails when relocated to the wrong node due to a missing node eviction.

Merged, thanks @c3y1huang !


Update README.md

Added the Solidity Challenges Repository, a platform for Solidity developers and security researchers. This repository offers a wide range of challenges, from language-specific tasks to exploit-focused scenarios.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Enhance notification feed item to display more (3) author names

Hi! 👋
The notification's page is a little too brief with displaying the "who" part of a notification, so I thought I would increase it to about 3. The code handles the constant being updated very well, so if it needs to be changed it shouldn't be an issue.
I have tested this by running a local expo frontend just attached to my production account and I looked at my own notifications. My data in production seems to cover all the cases I've considered in my solution.
I think I followed most of the written contributor guidelines, but sorry if I missed something important.
Let me know if I missed anything or if you have any questions! Thanks!

I gotta be honest I find this pretty visually difficult to parse

I feel like the extra tap to expand the avatars is pretty low effort and keeps things visually easier to scan

@pfrazee I am inclined to agree in the case of a large number of authors. But I would say that it probably excels (visually) in cases where there is only ~2 authors in the notification.
But.. my UX chops are limited so I cannot think of a great way to decide when that should be shown versus not. (Only inline if there are 2 or less authors?)
I was also just agreeing with some folks who were discussing that only showing one name was slightly annoying to them: https://bsky.app/profile/danabramov.bsky.social/post/3jznlndi7ot2z
Any suggestions for alternatives? If not I don't mind if this get's improved in some future day. (The attempt was still fun, this is a good codebase for it's early age!)
Thanks for having a look!

Oh okay -- I think for that we just special case the "2 people" situation. That should probably take a different approach than this.
Thank you though!

Definitely different approach. Thanks for taking the time to leave feedback @pfrazee!


[BUG] Unable to create impersonator account

Rancher Server Setup

Rancher version: 2.7.9 (After upgrade from 2.6.13)
Installation option (Docker install/Helm Chart): (Helm

If Helm Chart, Kubernetes Cluster and version (RKE1, RKE2, k3s, EKS, etc): RKE1, Kubernetes 1.24.17

Information about the Cluster

Kubernetes version: 1.14.27
Cluster Type (Local/Downstream): Local and/or downstream ?
Downstream clusters created using Rancher and RKE1 on EC2 instances via node templates

User Information

What is the role of the user logged in? (Admin/Cluster Owner/Cluster Member/Project Owner/Project Member/Custom): Admin

Describe the bug
Since the upgrade from 2.6.13 to 2.7.9 many of our services connected via Rancher, I.e. ArgoCD for deploying helm charts, but also Kubectl and API calls in our custom applications keep getting the following error:
unable to create impersonator account: error setting up impersonation for user u-ohhmqpqaxv: impersonation: error ensuring secret for service account cattle-impersonation-u-ohhmqpqaxv: error ensuring secret for service account [cattle-impersonation-system:cattle-impersonation-u-ohhmqpqaxv]: timed out waiting for the condition

Kubectl makes use of the Kubectl files downloaded from the GUI. ArgoCD makes use of tokens created under an LDAP backed account.
Sometimes things spontaneously work, without things changed. I'm not sure what the problem is. We also see the following errors in the "rancher" pod:
2024/01/07 14:55:15 [INFO] EnsureSecretForServiceAccount: waiting for secret [cattle-impersonation-u-ohhmqpqaxv-token-x8rmt] to be populated with token

The problem seems to appear for both LDAP backed users and Local Users
To Reproduce
Upgrade Rancher and try to connect to any API call using an API token
Result
Expected Result
That API connections to work

Rancher 2.7.9 supports v1.23 at the lowest version of Kubernetes, and 1.14.27 is not suitable.


🚀ctrl+c ctrl+v (added licenc header)🚀

you should have the licencse header heer i thnfk

Thank you! While not important, this is potentially a good idea. I will merge if you remove modifications to README.md, as well as capitalize the 'r' in "Renner".


Crashed when trying to join a world.

After installing the latest version, I can't join a world. It just crashes when I try to join.
crash-2024-02-25_14.22.38-client.txt

I see you are using version 2.0.4
Does the problem presist in 2.0.5 ?

I can Confirm that the problem persists in 2.0.5.
Thank you very much!

fixed in 03ad03a


Jquery UI Autocomplete Bug in Sidr

I'm trying to use Jquery Autocomplete in a Sidr Panel but can't make it work, when i enter letters in the input, i see the ajax request in console.log but now result list is shown under the input after receiving the result
If i put exactly the same code on a normal page, not in sidr, autocomplete is working perfectly.
Any idea where can be the problem ?
( No js error in log anyway )
thanks in advance

I'm having the same problem, unfortunately and the z-index as you said, did not work with me, will this issue be resolved?

I managed to solve using a method of jquery autocomplete "appendTo" and placed to give the append in a div within the Sidr
It was so

autocomplete

$("#pagar_cliente").autocomplete({
  appendTo: "#nome_cliente",
  source: function(request, response){
   $.post("URL", {term:request.term, 'do': 'lstFinPessoas'}, function(data){
      response($.map(data, function(item) {
       return {label: item.valor,value: item.codigo}
    }))
   }, "json");
 }
});

sidr

Digite o nome do  cliente *

     <input id="pagar_cliente" class="campo text required grande  ui-autocomplete-input" validate="required:true " autocomplete="off" role="textbox" aria-autocomplete="list" aria-haspopup="true">
     <input type='hidden' name='cash[codigo_pessoa]' id="cash_codigo_pessoa" />


Feature signaling

This makes the signaling mechanism more flexible. Signals now only consume space on the instance if they actually have connections. You can also disconnect all signals on a per-sender or per-receiver basis. This allows the base class to disconnect all on dispose, which eliminates the need to mentally keep track of connections and the potential to forget to disconnect which would allow a reference to the object to survive the disposal.

Thanks for posting this today!
On Jun 1, 2015 6:32 PM, "S. Chris Colbert"<EMAIL_ADDRESS>wrote:

This makes the signaling mechanism more flexible. Signals now only consume
space on the instance if they actually have connections. You can also
disconnect all signals on a per-sender or per-receiver basis. This allows
the base class to disconnect all on dispose, which eliminates the need to
mentally keep track of connections and the potential to forget to
disconnect which would allow a reference to the object to survive the
disposal.
—
Reply to this email directly or view it on GitHub
https://github.com/phosphorjs/phosphor/pull/31#issuecomment-107736219.


⚠ Bofrost.de has degraded performance

In 22edaf5, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 1092 ms

Resolved: Bofrost.de performance has improved in 195f329.


docs/install: fix arch install instructions

(see commit message)
With thanks to @cbrake who submitted the original fix in #198. That PR
Closes #198.

Uh nice! :tada:


Sticky Navbar

Describe the Feature you want to end
A sticky navbar can be added to enhance the design. It would enable users to have access without having to scroll.
Expected behavior
It will be a fixed element in the page.
I would like to work on this issue under Hacktoberfest 2022.

is this still open? Can you please assign me?

hey is anyone working, can I take this issue ?


change param search char

Subject of the issue
Hello,
I have to share my translations in both front and back app code, it was easy until i need to insert params in my translations. My server translations look like this 'my name is {0}'.
At first i was thinking about create a filter whose parse my translation into the good format, but i wonder if there was a more elegant way (like creating a custom translateInterpolation or easier: configure the delimiters chars)
Thanks!

Hello,
thank you for contacting us.
Short answer: You can built your own custom loader which also can read/transform any data you want.
However, we advise you asking common "How to... ?" questions at community boards like Stack Overflow because you will reach a bigger audience. Also, providing a complete example (i.e. using this plnkr template) might help everybody to find a proper solution faster.
Finally I highly recommend our api documentation and guide available at https://angular-translate.github.io/. Because nothing is perfect, we are welcome any improvements or suggestions found in the docs also provided as a PR.
This issue tracker is mainly for feature and bugs (issues) with the library angular-translate itself. But you're always free to add a feature request later on.
Thanks.

Hi,
thanks for your answer.
I'm sorry, because i needed a fast solution i spoke about how i could do it.
But the feature should be "make the delimiter chars customizables".
Do you want me to create a "proper" new feature?

You are right: You can yourself create a custom interpolation, then you have this under you control.


publish a new version

Hi @leebyron
now that typings(#43) are merged, can you please release a new NPM version that includes it?

Sorry for the wait. I'm clearing out issues and PRs in anticipation of a new release


Please un-pause Viva Insights Daily Briefing Emails!

I love that you're improving the product, but please continue to allow the Daily Briefing emails in the meantime!  I really relied upon those as part of my productivity and workflow.

Thank you for submitting feedback.
I think the best way forward is if you open a service ticket in your tenant so this can get resolved ASAP. Based on the outcome let me know if it is something that can be called out in the docs.
Please follow this link to contact support for business products:https://docs.microsoft.com/office365/admin/contact-support-for-business-products
Please keep us posted here on the resolution so we can feed whatever information you discover into the content.


Gunicorn: mayuth:authbot ... Exited too quickly

Hallo, I just tried to install your bot. I followed your steps. If I reload my supervisor, I get an error. The log file is below:
Traceback (most recent call last):
  File "/home/allianceserver/myauth/bot_conf.py", line 4, in <module>
    from aadiscordbot import launcher
  File "/home/allianceserver/venv/auth/lib/python3.7/site-packages/aadiscordbot/launcher.py", line 8, in <module>
    from .bot import AuthBot
  File "/home/allianceserver/venv/auth/lib/python3.7/site-packages/aadiscordbot/bot.py", line 7, in <module>
    import aiohttp
  File "/home/allianceserver/venv/auth/lib/python3.7/site-packages/aiohttp/__init__.py", line 10, in <module>
    from .protocol import *  # noqa
  File "/home/allianceserver/venv/auth/lib/python3.7/site-packages/aiohttp/protocol.py", line 18, in <module>
    from .helpers import reify
  File "/home/allianceserver/venv/auth/lib/python3.7/site-packages/aiohttp/helpers.py", line 25
    ensure_future = asyncio.async
                                ^
SyntaxError: invalid syntax

I use AA 2.9.2 and python3.7

Ok, I found the mistake myself. I updated the "discord.py" from version 0.17 to 1.7.3. Then reloaded supervisor and the authbot runs fine.

Easiest fix ever. Thanks for the update.


Media Loading Icon Looks Pixelated

The Issue
The loading icon looks pixelated/grainy when spinning. Seems like the background spills out past it at various points in its rotation.
*This screenshot isn't the best. It's more apparent on other videos.
Steps to Reproduce
Play streaming content that is waiting to load.
Expected Behavior
Look more pleasing.
Actual Behavior
Looks broken.
Suggested Solutions
Replace with a nicer icon. Maybe skip using black too. Black is never a good colour for good UX.
System Configuration
App | 0.31.1
Daemon (lbrynet) | 0.34.0
Win10
Screenshots

I suspect this will change when we implement the new video player. Thanks for creating the issue!

Yep. This is also chromes video player, and we're a few version behind so maybe it's fixed upstream? I wonder if you see it when you drag the video into chrome

This was in render media and is replaced by a new spinner in videojs.


How do I add stripe javascript to asset pipeline?

The recommended way to include stripe.js into your web pages (outlined in the readme) is to render it as a script tag.
<%= stripe_javascript_tag %>

This is for security reasons so that stripe can host stripe.js and rollout updates or security patches as needed. You could download stripe.js to your vendor/assets/javascripts directory and include it from there, but that's not explicitly supported by stripe-rails.


Textcolor

Use colormap to determine text color class to apply based on hex colors.

This is based on and includes #29

merged with carousel PR


Error Handling Test Cases

This fixes the issue with error handling. This PR is really more of an FYI, since I am going to push it into master immediately so I don't have to deal with broken tests any longer than nessary. However, if there is an issue with the changes I'll be happy to discuss and alter it if necessary.

:tada: This PR is included in version 0.358.0 :tada:
The release is available on:

npm package (@next dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Refactor kubervirt provider storage

What this PR does / why we need it:
This PR set the default kuebvirt storage target from PVC to Storage in order to leverage multiple CSI drivers in the infra cluster instead if using k8s PVC setting which might lead to different issues, such as access mode, volume binding mode, etc... In addition to that, it can read and set the volume access mode to the data volume spec based on the CDI storage profile.
Which issue(s) this PR fixes:

Fixes #
What type of PR is this?
/kind feature

Special notes for your reviewer:
Tested manually and worked as expected.
Does this PR introduce a user-facing change? Then add your Release Note here:

Support `dataVolumeTemplateSpec.dataVolumeSpec.Storage` and configure DataVolume based onn CDI StorageProfile

Documentation:

None

/approve
/lgtm


Bugfix remove spew

Opening this to make change more visible, will self-merge this hotfix.

          1 Message

:book:
mdspell or proselint not available - prose not linted

  Generated by :no_entry_sign: Danger

Codecov Report

Merging #497 into master will decrease coverage by 0.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #497      +/-   ##
==========================================
- Coverage    69.4%   69.39%   -0.02%
==========================================
  Files         196      196
  Lines       10685    10683       -2
==========================================
- Hits         7416     7413       -3
  Misses       2734     2734
- Partials      535      536       +1

Impacted Files
Coverage Δ

main.go
0% <ø> (ø)
:arrow_up:

util/shelltest/captiveshell.go
59.44% <0%> (-1.4%)
:arrow_down:

util/restful/putbackjson.go
61.11% <0%> (-1.39%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 249ae97...9b670e9. Read the comment docs.


Update sharded cluster to use replica set in background

Q
A

Type
bug

BC Break
no

Fixed issues

Summary
With the new driver and retryable writes enabled by default, we get failure on sharded clusters backed by standalone servers. Since we only run on a single topology, changing the cluster to use replica sets fixes the issue without having to disable retryable writes by default via the connection string.
This will be merged to 2.0, but I think we may want to test additional topologies for 2.0 in the future.

Closing as I can't see what the failure is. I'll be updating the mongo-orchestration config for 1.3 and disable retryWrites for this fix.


The graphs close a program with the error "System.Exception" in Avalonia.Base.dll (SigtermEvent)

Error Description
I upload 3600 points from the database to the graphs. After that, I grab the graphs with the mouse cursor and start moving the graph sharply to the left and right. If you continue doing this for about 1 minute, the application closes with the error "System.Exception" in Avalonia.Base.dll (SigtermEvent).
I also wrote to the graph developer.
I am using:

LiveCharts2 v. 2.0.0-rc2
Avalonia v. 11.0.5
.Net 7

The code is written according to the example from the site:
https://livecharts.dev/docs/avalonia/2.0.0-rc2/samples.general.scrollable
General -> Scrollable
Computer Characteristics

Windows 10 Pro 22H2 19045.2006
Intel i7-11700
32G RAM
NVIDIA GeForce RTX 3060
Samsung SSD 980 1TB

@MrOldOwl please link both issues. thx.

Hello. Today I wrapped the Main method in try-catch and saw that the error is not the one I described earlier.
It is also already discussed at the link: https://github.com/beto-rodriguez/LiveCharts2/issues/1076
I apologize.


Proposal: add tagged template literal support for vscode.l10n

Motivation
Currently, string localization is done by calling a method
vscode.l10n.t('Hello {0}!' name);

This mirrors what we do in core, however, this is a little error-prone. There's no validation in that string to ensure that replacement are done correctly, that all replacement arguments are used, or that enough replacement arguments are provided to satisfy the string. This is what initially inspired me 🙈
Proposal
We propose adding an overload to vscode.l10n.t that allows it to be used with tagged template literals. With tagged literalls, developers use a template literal as they "normally would", but prepend it with the method call
vscode.l10n.t`Hello ${name}!`

The associated definition for t() would gain an overload:
	export namespace l10n {
		/**
		 * Marks a string for localization. If a localized bundle is available for the language specified by
		 * {@link env.language} and the bundle has a localized value for this message, then that localized
		 * value will be returned (with injected {@link args} values for any templated values).
		 *
		 * This signature is for use with tagged template literals. The more verbose object-based
		 * call should be used if comments are required.
		 *
		 * @example
		 * ```
		 * vscode.l10n.t`Hello ${name}!`
		 * ```
		 *
		 * @param message - String message components.
		 * @param args - Replacement components in the string.
		 * @returns localized string with injected arguments.
		 * @see https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Template_literals#tagged_templates
		 */
		export function t(parts: TemplateStringsArray, ...args: Array<string | number | boolean>): string;

Tagged template literals are an ES5 feature, and we already use ES6 features (such as Symbol.iterator), so this does not introduce any new lib/target constraints.
Previously we haven't been big fans of adding "convenience" APIs such as this to the extension host API, but I think this might warrant some deviation since we force that this specific method is used (and not wrapped) in the localization extraction pipeline.
Alternatives
An alternative way to gain the same level of safety would be to use TS' template literal types to enforce argument placement. However, this lacks the ergonomic benefits of template literals, and would introduce previously unseen levels of type acrobatics to vscode.d.ts. Our extension host api is not a venue for Cirque du TypeScript.

A question about NLS comments came up,  one idea is to support tuples like { value, comment } so that each substitution value can be commented but that's still no way to leave a generic comment. Do you have an idea how that could work or is the answer to use the existing API then?

The answer would be to use the existing API for them. There are a few ways we thought about allowing this, but we didn't like any of them...
l10n.t({ comment: 'is a greeting' })`Hello ${name}!`
l10n.t`Hello ${name}! ${l10n.comment('is a greeting')}`
l10n.t`Hello ${name}!`.comment('is a greeting') // extra insane 😉

I'm gonna move this to the backlog for now since we wanna try this in Core first. The core issue is ^^


Fix lint

Checklist

[x] Used a personal fork of the feedstock to propose changes
[ ] Bumped the build number (if the version is unchanged)
[ ] Reset the build number to 0 (if the version changed)
[x] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[ ] Ensured the license file is being packaged.

Fixed the current lint:
conda smithy lint --conda-forge recipe/
## recipe/ has some suggestions:
##   No valid build backend found for Python recipe for package `scvi-tools` using `pip`. Python recipes using `pip` need to explicitly specify a build backend in the `host` section. If your recipe has built with only `pip` in the `host` section in the past, you likely should add `setuptools` to the `host` section of your recipe.
##   `noarch: python` recipes should usually follow the syntax in our [documentation](https://conda-forge.org/docs/maintainer/knowledge_base/#noarch-python) for specifying the Python version.
##    - For the `host` section of the recipe, you should usually use `python {{ python_min }}` for the `python` entry.
##    - For the `run` section of the recipe, you should usually use `python >={{ python_min }}` for the `python` entry.
##    - For the `test.requires` section of the recipe, you should usually use `python {{ python_min }}` for the `python` entry.
##    - If the package requires a newer Python version than the currently supported minimum version on `conda-forge`, you can override the `python_min` variable by adding a Jinja2 `set` statement at the top of your recipe (or using an equivalent `context` variable for v1 recipes).

Added hatchling as Python build backend (for more details see https://github.com/conda-forge/scvi-tools-feedstock/pull/59#issuecomment-2377373103)
Added new python_min syntax to ensure minimum Python version for noarch recipes
Didn't bother bumping the build number. We just uploaded a new binary earlier today from #60. These changes are more for long-term recipe maintenance and don't have any immediate effect on the conda binary produced

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe/meta.yaml) and found it was in an excellent condition.


Question - API / Webservice logging options

What would be the best option to have some (native) logging on API / Webservice (incoming) calls?
We often create a custom logging framework inside NAV to log the received WS calls. For codeunits exposed as webservices, this is fine, since database transactions are allowed inside the functions.
For pages exposed as webservices, we struggled finding the right triggers, since in the OnOpenPageEvent, the filters can not yet be read, so we miss the context of the WS call. Since not all page triggers support database triggers to create a log record, we created some workaround via the OnFindRecord / OnNextRecord (since these triggers know about the Filter / parameter context) to write a log into a SQL table (instead of passing via a NAV record), which would not be supported with extensions. I guess the same constraints would apply for API pages as well?
Using STARTSESSION might be not the best option for performance, using SENDTRACETAG does only provide limited information, using an central Azure function might cause overhead, ... Can we find a solution inside NAV to know which API calls have been received?

I reached out to the owning team and I will update this thread once I hear back.


Correct the plural of index from indexes to indices

Indexes -> Indices
Just a tiny spelling correction.
The plural of the index should be indices.
Reference
https://www.merriam-webster.com/dictionary/indices

https://www.merriam-webster.com/dictionary/indexes
Both indexes & indices look acceptable, then I think indexes is better for variable names because indexes has more same form with singular word index.

That sounds reasonable! I'll close this.


CAMERA EXAMPLE THROWING AN ERROR ON RPI 4

I trained my model on the website for pattern recognition. After training, I exported my model locally on my my computer. I followed the tutorials step by step as mentioned on website and I was successfully able to run standalone camera example on my laptop. Then using scp protocol I transferred the folder to rpi 4. On rpi 4(64 bit) , I am not able to compile the model and i get this following error:

Since you say you transferred the folder, it likely has container compile code files for your laptop architecture (.o files). If you do this you need to clean the project with make clean. I guess in your case it would be something like
$ APP_CUSTOM=1 TARGET_LINUX_AARCH64=1 USE_FULL_TFLITE=1 make clean


Improve blockchain import/export

Improve the status output for the import. At the moment the status output for importing many blocks looks like this:

$ cargo run -- import-blocks --dev ./5k-blocks.bin
    Finished dev [unoptimized + debuginfo] target(s) in 0.82s
     Running `target/debug/substrate import-blocks --dev ./5k-blocks.bin`
2019-02-25 11:19:53 Importing 5000 blocks
2019-02-25 11:19:53 #0
2019-02-25 11:19:53 #1000
2019-02-25 11:19:53 #2000
2019-02-25 11:19:53 #3000
2019-02-25 11:19:53 #4000

It seems as if nearly all blocks were immediately imported, but what actually happened is that they were merely added to the import queue and the program will then hang for a long time whilst processing them all and giving no new output. The output should make clear how many were imported and how many are still processed.

Rate-limit loading from the import file based on backpressure from the import queue, so that the case above is handled better. This was escalated from https://github.com/paritytech/substrate/pull/1807, where @rphmeier mentioned that we do it this way in parity-ethereum. Maybe some inspiration can be taken from there.
Make it clear in the --export-blocks flag documentation that JSON cannot be re-imported.

@cmichi we should see if it's possible to print output using the informant (maybe modifying it so it has an Option<NetworkHandle> or whatever. This uses import notifications internally so it will give the behavior that we want.

The import block part of the issue is fixed.
Remains:

Make it clear in the --export-blocks flag documentation that JSON cannot be re-imported.


[Validate] Create custom evaluation functions

Works with the scaleapi backend PR: https://github.com/scaleapi/scaleapi/pull/40668
Currently provides user with the ability to define a placeholder function, and creating a record in the respective database row.
Try it out:
client = nucleus.NucleusClient(API_KEY)

# create it
my_eval_fn = client.validate.create_placeholder_eval_function('some_name')

# check its recorded by the DB
custom_eval_functions = client.validate.eval_functions.custom_functions

@gatli does it make more sense now? It stays away from the existing CustomEvalFunction logic that existed before.


Monthly Reset

One thing to change with this one - Can we add a prev_month_points column to easily show leaderboards of the last month?

Done!


Fix broken link to Friends and Guests

@justin808 Can you review this?

This change is

Changes Unknown when pulling 42a2476c0708a46c1edf705b4f1c5e1b1a739459 on VagishVela:patch-3 into ** on shakacode:master**.

Thanks! @VagishVela!


GeoJSON linestrings throw malformed error if coordinates empty.

Description
An empty geometry GeoJSON feature throws an error and prevents anything from showing in deck.gl, rather than being silently excluded as in mapbox.gl, QGIS, etc.
This is a weird thing to report as a bug, I know. Why have a linestring with no coordinates? In this case, offsetting lines on the inside of a corner. In PostGIS, if offsetting to the inside of a corner by a distance larger than the distance from both endpoints to the corner, results in an empty geometry.
The “bug” (in my opinion) is really a difference in how deck.gl handles empty geometry compared to how it’s handled elsewhere.
Starting with PostGIS: ST_IsValid() returns true - it considers the empty linestring to be valid geometry, so we can’t just add “WHERE ST_IsValid(geom) IS TRUE” to the server queries.
Next, in a desktop environment, QGIS happily loads the offending view, just silently ignores the empty geometries.
Finally, in mapbox.gl: after wrapping with ST_AsGeoJson() and returning from PostGIS, loading into mapbox.gl causes no problem at all - again, it seems to silently ignore the empty geometry.
So my first experience with deck.gl was rough - calling the same API function as with Mapbox, getting the same GeoJSON, but the map is blank, with a dev console error about “malformed linestring coordinates.”
I tried iterating over the 230 features in the feature collection, wrapping each in a try/catch before updating the deck layers, but wherever the error is thrown, it seemed to go by the try/catch there.
Eventually, by loading slices of the array a bit at a time, narrowed it down to 4 LineStrings with empty geometry.
Personally, I don’t love things silently missing from my maps - we’ve had these edge cases for a few years without realizing it. Maybe throwing an error is better.
OTOH, the workaround is painful - for every layer I grab from the database, I have to explicitly check for empty geometry. And since it’s an edge case, not all projects have these inside corner offsets - so forgetting the explicit check could appear fine until some project has one and suddenly the whole map fails to work.
The big issue IMO is the “gotcha” factor when migrating in to deck.gl.
Either of these would be an improvement:

Silently ignore empty geometries, like Mapbox
OR 2. Add a warning to the Deck.GL documentation about empty geometries preventing anything else from loading.

Environment
I’m using deck.gl in a pure JS / non-React environment.
Just did NPM install today, so whatever the latest version is.
I wouldn’t expect it to be a version-specific thing, but I’ll check the specific version when I get back to desktop, if anyone thinks it’s relevant.

Sorry about the open/close/reopen, the mobile keyboard submitted when I thought it would send me to the details, and posted an empty issue.

@bfreed Thanks for reporting. This error is a result of https://github.com/uber/deck.gl/blob/master/modules/layers/src/geojson-layer/geojson.js#L203, which was put in place after too many users provide invalid GeoJSON Polygons (as a single loop instead of array of loops). I agree that empty coordinates should not result in a hard failure.

This is patched in 7.1.8 - we're now logging a console warning instead of throwing an error.