Set GPU exclusive mode

🚀 Feature
Provide a function that enables GPU exclusive mode.
Motivation

When I try to test different models with different hyperparameters, I want one Trainer to fully occupy some of my GPUs. Now I have two models to train, and I want them trained separately on two GPUs. So, I did something like
trainer = pl.Trainer(
        gpus=1,
        auto_select_gpus=True,
    )  # 1

But when I run two instances of my training script, pl will always select the GPU-0 even if only one training instance can have one GPU run at 100%. Then I have to do something like this
trainer = pl.Trainer(
        gpus="0", # or "1"
        auto_select_gpus=True,
    )

but pl gives me a deprecation warning. I can set a GPU availability scope of course, but it's basically the same as gpus="0". My intention was to have pl to automatically select one idle GPU (instead of a GPU that has enough memory even if it's busy) to train my model. Then I think it's probably pl doesn't set the exclusive mode, so the # 1 block doesn't behave as I expected.
Pitch
I was thinking something like:
trainer = pl.Trainer(
        gpus=1,
        auto_select_gpus=True,
        exclusive=True
    )

then if a training instance is running, another new training instance will be forced to find another GPU.
Alternative
Something like this seems also great
trainer = pl.Trainer(
        gpus=1,
        auto_select_gpus="idle", # or "memory-fit" or False
    )

Additional context
My pytorch-lightning is 1.4.1.
Can you please explain more on the gpus behaviour in 1.5 and later?  Can you also add more explanation or a link when showing the deprecation warning?
The deprecation warning is
LightningDeprecationWarning: Parsing of the Trainer argument gpus='1' (string) will change in the future. In the current version of Lightning, this will select CUDA device with index 1, but from v1.5 it will select gpus [0] (same as gpus=1 (int)).

Can you please explain more on the gpus behaviour in 1.5 and later? Can you also add more explanation or a link when showing the deprecation warning?

Trainer(gpus=n) where n is an integer is selecting the GPUs with indices [0, 1, ..., n-1] (the first n GPUs).
However, in PL < 1.5 when n is a string it doesn't behave like that and instead it was choosing GPU index n. This is not consistent and is therefore changed to be equivalent with the int type. So in PL 1.5 and higher, Trainer(gpus=3) will be the same as Trainer(gpus="3") and will select three GPUs.

@ifsheldon don't you think it's better to let the user set the compute mode on the GPU themselves with nvidia-smi? Since this is affecting the system globally, I think it's safer to let the user or admin set this. It anyway requires root privileges afaik. What do you think?
To clarify, currently, auto_select_gpus will select a GPU if torch can move a tensor to the device.

If setting exclusive mode in pl is too intrusive, then the alternative seems good enough. The current behaviour of auto_select_gpus is same as memory-fit I think, but the problem is even if a GPU is running at 100% but having enough memory, pl will select it despite that another one is totally idle.

The current behaviour of auto_select_gpus is same as memory-fit I think

No, actually not. As I tried to explain above, it will select a GPU if PL can move a tensor to the GPU, i.e., this tests if the GPU is in exclusive mode and running a job.

And this problem will be worst after 1.5 because people cannot select GPUs manually by doing like gpus="1" and then they have to do it via setting CUDA environment variable.

That's not true. Manually selecting GPUs is still possible by providing a list of indices. This has not changed.

this tests if the GPU is in exclusive mode and running a job.

I don't know what you meant by "and" in "i.e., this tests if the GPU is in exclusive mode and running a job". Is it literally a logical and here? I just tried running two instances of training scripts, it turns out that after the first instance is running and GPU 0 is at 100%, then I run the same training script again, pl will run it on GPU 0 again.
The training code is the same as above.
trainer = pl.Trainer(
        gpus=1,
        auto_select_gpus=True,
    )

If the "and" here is logical and, then this bahaviour is expected, although it was not my intention. If you meant that PL checks these two criteria and if one of the two fails, it will not move a tensor to the GPU, then it's a bug now I suppose.

From your screenshot, we can see your GPU is in default mode (middle on the right). It means it allows to run multiple processes per GPU. You can set it to exclusive mode with nvidia-smi -q 3 -i 0  I believe.
Here is the code where PL checks if it can move a tensor:
for i in range(torch.cuda.device_count()):
        if i in exclude_gpus:
            continue
        # Try to allocate on device:
        device = torch.device(f"cuda:{i}")
        try:
            torch.ones(1).to(device)
        except RuntimeError:
            continue
        return i

And it is not even "memory-fit" mode. I think a better solution will be as my proposed alternative. In "memory-fit" mode, PL checks if it can allocate a tensor of the same number of parameters of a model on a GPU. In "idle" mode, PL checks if a GPU is busy right now.

It's a good suggestion, but unfortunately it won't work. The model parameters alone on the device are not determining how much memory will be used during training, as it is the full forward and backward pass plus optimizer states that make the most of the memory consumption. Running two trainings on the same device with tight memory is dangerous. Either of the trainings could fail due to OOM. I do not recommend this approach unless user can reliably know their memory consumption.

Running two trainings on the same device with tight memory is dangerous

Yeah, I agree, but now PL doesn't check anything except the exclusive mode, which is worst.
Alternatively, we can call the current mode as "fit-in" or something and give warning about the potential memory issues. Besides, we should have an "idle" mode, which checks if a GPU is busy, like running at >=80%.


Check inconsistency of backward calculation and show its function name

fix #278

I have two concerns and I do not have clear answers to them.

We check the consistency between data and grad also in variable here. They have some overlaps.
But, these two consistency checks could not be equivalent when we accumulate gradient (e.g. in this case). In the original code, gx need not to be same dtype as data. Only x.grad + gx need to be same dtype. So by this PR, we impose additional constraint to the gradient.

We need to check grad also in Variable because users can set grad directly.

LGTM!


Add note on what to do if debugger gets confused

Hopefully this saves some time to the next bug hunter

:+1: Thanks!


Autotick update

Checklist

[x] Used a personal fork of the feedstock to propose changes
[x] Bumped the build number (if the version is unchanged)
[x] Reset the build number to 0 (if the version changed)
[x] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[x] Ensured the license file is being packaged.

Closes #1, closes #2, closes #3, closes #4 and closes #6.

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

@conda-forge-admin, please restart ci

@conda-forge-admin, please restart ci

@anjos can you add yourself as maintainer? Happy to merge then

@anjos can you add yourself as maintainer? Happy to merge then

Sure, I shall - I thought this had to be a separate PR tho.  Let me push it.


Assembly Name change

Discussed in https://github.com/mbdavid/LiteDB/discussions/2168

Originally posted by bluekuen April  7, 2022
Hello together,
like the title says did my Type name change from DeviceSetMobile to DeviceSet.Mobile my Assembly Name to DeviceSet and now i got these Exceptions from all of my database classes which are stored:
LiteDB.LiteException: 'Type 'DeviceSetMobile.Models.BleDevice, DeviceSetMobile' not found in current domain (_type format is 'Type.FullName, AssemblyName').
Is there an simple and easy way to announce assembly name changes?
I'm thankful for every answer!
Kind regards
bluekuen

Hi, you can use the LiteDB studio to update the type ref in your entries. If you have it already in Prod you can use the statements in a migration function.

Hello, thanks for the answer!
Yes its already in prod and i already tried it this way:
var updateSQL = "UPDATE User SET _type = " +
"REPLACE(_type, 'DeviceSetMobile.Models.BleDevice, DeviceSetMobile', 'DeviceSet.Mobile.Models.BleDevice, DeviceSet') " +
"WHERE _type LIKE 'DeviceSetMobile.Models.BleDevice, DeviceSetMobile'";
_database.Execute(updateSQL);
But it didn't work somehow. Am i doing it wrong?
Thanks in advance!

ok, it looks like you try to set the _type of the user, not the the type of entry of the users device list.
Can you paste a sample text output from LiteDB Studio?

There's just a zero:

I mean a user object

Ok i see. But how am i able to update the Devices inside the User? if i try update User.Devices it would be the list right?
Also User.Devices can't be updated.

make a backup,
try something like:
UPDATE User SET<EMAIL_ADDRESS>= 'DeviceSetMobile.Models.BleDevice, DeviceSetMobile']._type = 'DeviceSet.Mobile.Models.BleDevice, DeviceSet'

thanks so much for your help!
I tried the command you provided but get following error:
Unexpected token $ in position 17.
UPDATE User SET<EMAIL_ADDRESS>-----------------^
Do i need to escape with an escape char?

maybe omit?
UPDATE User SET<EMAIL_ADDRESS>= 'DeviceSetMobile.Models.BleDevice, DeviceSetMobile']._type = 'DeviceSet.Mobile.Models.BleDevice, DeviceSet'

hm, very tricky

maybe you can try convert it with a Untyped collection:
using(var db = new LiteDatabase("mydb.db"))
{
    var col = db.GetCollection("User");

	foreach (var user in col.FindAll())
	{
		foreach (var device in user["Devices"])
		{
			if (device["_type"] == "DeviceSetMobile.Models.BleDevice, DeviceSetMobile")
			{
				device["_type"] = "DeviceSet.Mobile.Models.BleDevice, DeviceSet";
			}
		}
		col.Update(user);
	}
}

Untyped collection also not working?

Untyped collection did work! Thank you very very much! You saved my day!


Introduce Address answer type

What is the context of this PR?
For context see: https://github.com/ONSdigital/eq-questionnaire-runner/pull/241
Introduced a new answer type to support address answers in runner.
This introduces a new data model for storing address answers:
{
  "answer_id": "address",
  "value": {
    "line1": "7 Evelyn Street",
    "town": "Barry",
    "postcode": "CF63 4EN"
  }
}

How to review

Ensure the feature is as expected.

Note:

Support for lookups will be done in a future card.
Support for piping address answers using placeholder will be done in a future card.

Checklist

[ ] New static content marked up for translation
[ ] Newly defined schema content included in eq-translations repo

@JamesGardiner All should now be addressed.

As discussed, we should be testing the summary output for no address answer (optional address), and I've asked Andy about whether address line 1 should be mandatory for optional addresses (if they fill in at least one address field).

Have updated schema/tests for optional version.


Add class in items

It would be nice if were possible to add class to items like already have in boards

I made a modification to use in my projeto, i added in line 258 next comment  //create item
if (itemKanban.class != '' && itemKanban != undefined) {
    nodeItem.classList.add(itemKanban.class);
}

Hello @cmparrela, in a project of mine I set cover image by jQuery. I think css for an item would be handy. I can do a pull request for that. Thank you for your suggestion.

I also made a modification to add items with specific classes in addElement function:
this.addElement = function(boardID, element, className) {
    //...
    if (className)
        nodeItem.classList.add(className);
    //...
}

Hi Guys
Is there any update on this ?
And if it's already implemented, how do I set the class of an item from the start ?
Cheers
Mike

@Mikeproove @cmparrela
Now is possible to add custom classes to an item.
{
                "id"    : "item-id-1",        // id of the item
                "title" : "Item 1"            // title of the item
                "class" : ["myClass",...]     // array of additional classes
}

https://github.com/riktar/jkanban/commit/e3fa10ad69133e4b29fa6c5183c185631b90617c


Add env variables to configure external smtp server other them Postmark

Component(s)
controlplane
Is your feature request related to a problem? Please describe.
Add env variables to configure external smtp server other them Postmark
Describe the solution you'd like
I added a PR https://github.com/wundergraph/cosmo/pull/648
Describe alternatives you've considered
No response
Additional context
No response

Just saw that its a duplicate of https://github.com/wundergraph/cosmo/pull/617

Hi,
The PR has been assigned and will be addressed in due course.
Thank you for your patience!


[Bug]:  Quest 3 lemon loader patch made bonelab become unuseable

All of the following criteria must be met

[ ] Full Latest.log file included. If no file exists then leave this unchecked and state so.

All of the following are optional to answer

[X] Tried reinstalling the game.
[ ] Tried reinstalling LemonLoader.
[X] Tried restarting device.

Describe the issue.
Everytime after patching i press yes uninstall install then i launch the game... dark grey screen for over 10 minutes will not launch anymore please help...
Did you attach your log file?

[ ] Yes, I attached my log file to the text box above.
[ ] No, I could not find a log file at /storage/emulated/0/Android/data/<package name>/files/melonloader/etc/Latest.log

Probably didn't save the game's OBBs which are required for loading. Lemon does not save them automatically.


Hi

Couldn't find a way to message you.
Can you add me on discord?
Sunney#7906

I don't really use Discord. What is it that you need?

Also, how did you even find this? lol

I was making the same tool, but with PyAutoGUI and I felt dumb when I discovered League has a client API. Anyways I have project idea to make this even more cool.
Add feature to automatically select ur champ, ban enemy champ and stuff.
We can work on it if you want

https://gist.github.com/Sunney-X/501f914681b598909903188ab0f34bae
This is what my version of it looks like.

I was thinking about extending it, but I'm not very interested.
First: this going to be somewhat complicated when you start adding all the different champ select types and I'm a little too lazy lol
Second: this wasn't the goal of this small script.
The reason I created it is because me (and others) who have extreme high mmr and wait 20 minutes for blind pick queues pretty often sometimes want to do something during queue (for example pee, if it wasn't obvious yet) and we were either afraid to go as "queue might pop soon better stay for another few minutes" or we would go and then miss the queue (rip all those queues). A script that automatically accepts the queue buys just enough time for us to not worry about missing queue.
And last: lol my Python skills are trash. You probably don't wanna work on it with me. To make it clear, this is the only Python script I ever made. I downloaded Python just for this and I never used it again for anything else.
Feel free to take the code and modify/extend it as you like, but I probably won't be helping much.

you dumbass you can just play tft 7 times ffs, you just wasted your time.


Support promises on top of callbacks

Communication with child process is built on top of postMessage now, and that's why it doesn't support returning value. And we have to use that old-style callbacks to return value from function.
It would be much better to wrap that functions into promises to allow returning values as well as promises and all such things.
Example of current plugin code:
application.setInterface({
  getSomeValue: function (callback) {
     callback(123);
  },
  doAsyncJob: function (callback) {
     setTimeout(callback, 1000);
  }
});

What it could look like:
application.setInterface({
  getSomeValue: function (callback) {
     return 123; //return simple value from promise is ok
  },
  doAsyncJob: function () {
    return new Promise(function(resolve) {
     setTimeout(resolve, 1000);
    });
  }
});

And then you can call plugin code from your App so easy:
plugin.remote.doAsyncJob()
   .then(doSomethingElse);

Hello Andrey, thanks for your suggestion. This will not be implemented, because Promises is just a syntacsic sugar over the callbacks - you still provide a callback to the Promise constructor which as result makes the code more complex without a reason.
It is technically possible to handle the returned value of an exported method and send the result over a message, but since messages are handled asynchronously, it is only possible to handle the result in some sort of callback on the oppsite side (no matter if this is an ordinary callback, or sugared with a Promise pattern). Therefore in order to keep the code simplier and consistent, returned value is ignored, and callbacks are suggested to be used instead. This make the exported function behave in the way as if it would be used directly.
Also I don't like Promises ;-)

@asvd Okay and thank you for the honest reply.


Fixes trading as, return by date formatting, details link, and #99

What

Remove the edit address details link (issue #103)
Displays the Trading As name where supplied (issue #104)
Formats the Return By date correctly
Confirms that the period_str is being used by the template as referenced in #99

How to test

Check out the branch and run the app
Confirm that all dates are formatted correctly
Confirm that the trading name appears
Confirm that the edit address has gone
Using dev mode, create a new token and confirm that the value you entered for period_str is displayed at the top right of the page.

Who can test
Anyone except @weapdiv-david

If Node.JS would stop breaking, this change would fix issues #99, #103, and #104

This is a problem with phantomjs specifically, we need to find a way around it. Some people just ship a binary with the app, it's not ideal but maybe it would work.

Hmm maybe this is it: https://github.com/Medium/phantomjs/issues/517


Order error handling and delete_order

Order error handling was built to handle over-placing orders (more than 10 in one minute), but not to handle deletion. Placing and deleting orders counts against the rate limit and so placing 9 orders and trying to cancel 2 will lead to a rate limit.

After speaking with devs, delete order will be removed from the rate limit.


Important: UI Grid broken in Safari

Latest version has broken many components on Safari. Many css rules started with -webkit has disappear from the css files. You can check it on: http://semantic-ui.com/collections/grid.html

Semantic uses autoprefixer plugin to parse CSS and add vendor prefixes to CSS rules. You can find supported browsers in semantic's tasks.js file. Current value is:
/* What Browsers to Prefix */
prefix: {
  browsers: [
    'last 2 versions',
    '> 1%',
    'opera 12.1',
    'bb 10',
    'android 4'
  ]
}

Autoprefixer uses Browserslist to specify browsers. Maybe you are using an old version of safari not supported.
You can use http://browserl.ist/ to see what browser are really supported in the list.

I'm using Safari 8.0.4

Semantic UI 2.2.6 says about browser support:

Last 2 Versions FF, Chrome, Safari Mac
IE 11+
Android 4.4+, Chrome for Android 44+
iOS Safari 7+
Microsoft Edge 12 & 13

Accordingly I use this browser list for autoprefixer, it has been tested with Safari:
'> 1%',
'last 2 versions',
'Firefox ESR',
'Safari >= 7',
'iOS >= 7',
'ie >= 11',
'Edge >= 12',
'Android >= 4'


context make_or_load_nwbfile to raise an error when nwbfile needs to be created but metadata is None

Related to the issue #936

Dandi seems to be asking us to upgrade?

Dandi seems to be asking us to upgrade?

They did a release today and it might have put their side out of sync. We'll keep an eye on it but it shouldn't have anything to do with our side of things


fix: toml rendering

Description
Fix: #567
Checklist

[x] Conventional commits were used
[x] People were tagged for review

Ah! Thanks Diwaker!
I actually saw the bug on the doc, found a fix, then opened a PR, then saw that there was an issue. But I didn't check the opened PRs 🙈


Improve the speed of ls/lint CLI invocation

Pandas is great, but it's big and slow with a huge callstack.
Remove pandas from the ls and lint commands, by rewriting _print_list.

Irrelevant now we've bet on pandas in 4992f11


Port to the 2018 edition and cargo-semver cleanups

This commit ports rust-semverver to the 2018 edition and tidies cargo-semver a bit (more comments, some renames for clarity, etc.)

I did the tidying of cargo-semver and rust 2018 in a different branch while trying to use cargo metadata instead of a cargo fork, but that did not pan out, so I moved them here.
The problem with cargo metadata is that it does not provide any information whatsoever about the paths of build artifacts, so we can't use it to try to find rlib files at all.

The features::regions test has started breaking with the latest nightly release (the nightly before works fine).

Maybe we can modify the cargo metadata subcommand to emit the relevant info and extend the cargo metadata crate to expose those fields

@oli-obk @ehuss suggested using the json output of cargo build for determining the paths of build artifacts like rlibs.

I've looked at the cargo build feature that outputs JSON -- and it seems to be feasible to use for our purposes. I'll hack a version together that finds the rlibs using that and see whether this gives us a working version that doesn't rely on a patched cargo (which we cannot publish anyway).


feat(chromium): roll to Chromium 91.0.4469.0 (r869685)

Roll to Chromium 91.0.4469.0 (r869685).
Update CONTRIBUTING.md.

why used Canary chromium version

Doesn't look like this contains patch(s) for CVE-2021-21224 right?

https://chromium-review.googlesource.com/c/v8/v8/+/2820971
https://chromium-review.googlesource.com/c/v8/v8/+/2826114

http://crrev.com/869685 returns a commit from Apr 06 , while patches where submitted Apr 12 / 14

The release was cut a day before promoting 91 to Beta. It's still pointing to the latest build available for all 4 platforms (linux, mac, win32, win64).
You are right, the mentioned commits are not included in this version of Chrome.
Please justify the need of cutting another minor version including above commits.

Well there are multiple use cases for Puppeteer, some of which might be running on public servers which are now susceptible to this attack.
Was trying to figure out which minor version from 91.x.x contains those fixes via omahaproxy but I got lost in the process.. 🤦♂

Thanks for the heads-up! I will cut the new minor version once the corresponding build is available.

Heads-up: there is still no newer builds available. 869685 is the latest.


Show diff option

Adding an option to allow to enable or disable diffs when running Puppet.
I run mcollective in a secure environment, and allowing diffs to output passwords to logs and reports is not an option.
This defaults to true, but enables us to turn it off.
I'm not sure how to spec this, so if anyone has any ideas for testing please let me know!

ping @igalic or @ffrank - any ideas on how to add tests for this, or do you think it's good to go?

I'm not sure I like this approach. Can we get away with just documenting how the user can achieve this using the snippet below?
Datacat<| |> { show_diff => false }

same with File, btw…

Agreed, there's better ways to do this. Closing.


RTDB query gets should not send server requests for listened-to data

b/177263312

@schmidt-sebastian friendly ping on this, disregard if you were waiting for android to settle before reviewing this one.


Add providers for Germany (nationwide) and each German state

Since the states in Germany differ in their holidays I had to split them up in different providers but inherited from the original class for nationwide holidays.

Hey,
Looks great! But please prepare unit tests for all new providers and add phpdoc to methods without comments.
Thanks
Michal

Are the tests sufficient @michalmanko ?

This looks great, @michalmanko : any chance this could be merged?


remove inaccurate docs

Our docs suggest using this babel plugin to remove console.log statements in production. Unfortunately, it does not actually work when you run it on a new React Native project, and the root cause is that the plugin does not handle all cases correctly. See discussion in https://github.com/facebook/react-native/issues/10412 . For now, we should just stop recommending that people use this plugin, because it doesn't work.

Did this somehow make its way back into the docs?
Was facing some trouble getting this to work and then ended up here after a few searches.


React step 5: https://carbon-tutorial-michaelsgardner.mybluemix.net/#/

Closes #
{{short description}}
Changelog
New

{{new thing}}

Changed

{{change thing}}

Removed

{{removed thing}}

Bluemix URL: https://carbon-tutorial-michaelsgardner.mybluemix.net/#/


[Documentation] How do you use it?

Hello. The readme looks really comprehensive, it looks like this tool has loads of features, which is great... But after the "INSTALLATION" section, it just throws you in at the deep end.
How do I just, like, download a video? It's weird the readme doesn't give an example of how to use it after installing.

https://github.com/yt-dlp/yt-dlp#usage-and-options

It is a command-line program. There is only one way to use it - through the command line! If you dont how to use a command line, google it. And if u dont wanna learn, maybe this program isnt for u

Thanks, I know how to use a command line. Just trying to help you have a better readme. I think an examples are always nice, but up to you.

I think an examples are always nice, but up to you.

I don't know how I can give examples for basic usage. It it literally just pass the URL to the program. Some of the more advanced options such as format selection, output template etc already have examples. If you have any actual suggestions for what to put in the readme to improve it, I would like to hear it


Feat: Enforce mandatory fuel code validation for schedules - 1429

This PR implements logic to enforce mandatory fuel code selection in schedules when "Fuel code - section 19 (b) (i)" is chosen.
Closes #1429

Looks great! Tested and works great!. Just wondering if we need those imports?

Thanks for the code review. We need these imports for backend debugging purposes. It looks like they were accidentally removed. They enable remote debugging in development if needed. Please refer to line 111 of this file for more details.


Documentation translation to pt-BR

Hi!
I want to translate the documentation into Brazilian Portuguese.
How do I do that?

In the community we've already discussed (forum thread) how to internationalize websites in Hugo and created a tutorial for it.
The easiest way would be to create a subfolder that just contains the translated markdown files. But you would still have elements like the menu in English because the links a defined in the config file.
Therefore we would need to create seperate site as described in the links above. And as you can see in the templates, some strings are even hardcoded (but could also be moved to the configs).
But in general it would be interesting to know if the docs will be multilingual in the future.

Contributions are welcome. I see one problem with multilanguagual documentation: We make updates/corrections to the doc all the time. Having language versions known to few means they will not get updated, and it is better to have correct info in English than wrong and outdated in your own.
@spf13 is the boss, and probably has something to say on this.

@calheira I appreciate your interest for translating the docs. I know that not all people are fluet in English and would prefer translated docs but, as @bep mentions, the biggest problem is to keep the docs up to date. That's why I mostly prefer English resources, even if it's not my native language.
This doesn't mean that you shouldn't try it.

I'll fully support any translation efforts provided that we have a structure in place to keep them up to date. I know we have many japanese users who would also benefit from native language docs.

Got it.
How about doing as in [translated Express.js documentation] (http://expressjs.com/es/)?
They put a warning:
"This document might be outdated relative to the documentation in English. For the latest updates, please refer the documentation in English."

Having translations for the docs would be great, especially with the i18n feature. But this issue looks rather stall and should be closed if there is no interest anymore.


Update base image to v3.0.1

It would appear the commit hook to quay.io is broken for the parent repo.


Show loading on index page

closes https://github.com/aeternity/aepp-raendom/issues/53

@stanislav-slavov I think this should be https://github.com/aeternity/aepp-raendom/issues/53

I'll take over work on this :)

closing this for now


IdentityApp sample

In the configureServices function,
this line:

services.AddCors |> ignore

should be:

services.AddCors() |> ignore

Otherwise, you get an error about satisfying ICorsService....

Thx! Fixed it now.


Apply scrollback reflow patch ?

Hey, thanks for your work ! Love the custom dmenu stuff. Would you mind applying the scrollback reflow patch ?

Apply the following patch on top of the original scrollback patch (=>0.8.5) to allow column and row reflow.

Very useful when you resize windows or switch to fullscreen.
Take care

How is this any different from the current functionality?

Okay. Yeah I'll try to add it, but looks like there are a bunch of little incompatibilities with other patches I or someone else will have to figure our first. I'll get to it if I can.

Yep, many incompatibilities indeed. At first I was willing to submit a PR but I gave up when I saw the amount of lines in the rejected diff. Ended up doing my own fork of st, which I'm pretty sure saved me some time. Even for you I wonder if it wouldn't be faster to reapply all your patches from scratch using a clean base of st 9.0. Many of the fixes you have here on this repo are probably also fixed upstream and some others are now available as patches (such as glyph_wide_support).
Anyway, good luck to anyone reading this and willing to put in the effort

The main issue is probably getting this, boxdraw and ligatures all to work in one place. I'll try to repatch from scratch, but I think it will have to grapple with this same problem.

I feel like the boxdraw patch is not worth it if you're already using a "good" font. I use Fantasque Sans Mono for my terminal, and it looks pretty much the same with and without boxdraw. At least the difference is not noticeable enough to me to add boxdraw. I added NotoSansMono as comparison, it doesn't render well indeed. It's up to you of course but for me I choose to rely on Fantasque Sans Mono instead of messing up my code even more using boxdraw. Maybe you should consider getting rid of it.
NotoSansMono font on my fork of st, without boxdraw (pretty ugly I agree) :

NotoSansMono font on your fork of st, with boxdraw :

Fantasque Sans Mono on my fork of st, without boxdraw :

Fantasque Sans Mono on your fork of st, with boxdraw :

Yeah it's not that bad. I had added boxdraw from public outcry, but if it's either or, I might go with this instead due to the wider functionality.

The original redraw patch that was mentioned was suggested in the st-flexipatch repository but never added because it is so massive and conflicts with a lot of patches.
There is a much smaller patch that achieves nearly the same functionality. See this patch. I have this in my own build and it works great!
Only functionality this version seems to miss is reflow for the top, but text does seem to redraw for example when you're changing mfact in dwm.
This version written by nimaipatel does not include scrollback but it can be applied on top ofl scrollback without issues!

@LukeSmithxyz if you have time and interested in repatching st using version 0.9 as base you can see my st build it has these patches applied:

it has the patch suggested in this issue and glyph wide support from my PR and the ligature patch that i used is from your original commit.


Sensors report as Unavailable, and error log shows "Cannot connect..."

G'day,
The integration was previously working without incident.
Now all the sensors are reporting unavailable.  I had tried to update to 1.12, that failed.  I have now updated to 1.15, apparently successfully, but sensors are still unavailable.  The last line in the error log reads:
"aiohttp.client_exceptions.ClientConnectorError: Cannot connect to host api.weather.bom.gov.au:443 ssl:default [Try again]"
I don't know where to look for more details.
--
Regards,
Ken
BoM_ErrorLog.txt

This sounds like when the integration started that there was no internet connectivity. Possibly a duplicate of #113

Thanks for your comment.
I'd agree that a lack on connectivity could result in the symptom I'm seeing, but I'd expect such a lack of connectivity to affect some of the other ~20 integrations, a number of which are collecting cloud sourced data.  (Although some of those are behaving oddly/intermittently, AirVisual, Accuweather, NSW Fuel prices.)
I've just noticed release 1.1.7 and updated & restarted, the sensors are still unavailable.
purcell-lab mentioned in issue #113 that "If I manually restart the component after the Internet is up everything is fine."  How do you "restart the component"?

You can Reload from the 3dots box, but I suspect this should be resolved with the resolution of #113

Assuming fixed.


Failed to build on FreeBSD

Hello,
I will start with that I succeeded to build it on one FreeBSD 10.2, the second one, while I expected to be the same, had issues when building the uv-common.c file (note that I build in fact today's clone of Julia).
The compilation errors were related to undefined cpu_set_t and CPU_SET (and alike macros). Hence, after few hours banging my head, I hacked my way out - I just inserted near the top of the file the following lines:
#include <sys/param.h>
#include <sys/cpuset.h>
typedef cpuset_t cpu_set_t;

This is more for your information and reference for persons which could stuck like me.

fixed by https://github.com/JuliaLang/libuv/pull/43 I believe, leave a comment and we'll reopen if not


Package request: PDFium

Project description
PDFium library for working with PDFs. Several projects (e.g. libreoffice) seem to use pre-built versions of it, but it should be built from source instead.
This repo seems to build PDFium with patches.
Metadata

homepage URL: https://github.com/bblanchon/pdfium-binaries
source URL: https://pdfium.googlesource.com/pdfium/
license: unknown
platforms: unix

I also need PDFium.

Bump?


The 21H2 build number is 20348.288 not 20348.11.

[Enter feedback here]
Hi ASHCI team,
The 21H2 build released on 10/19/2021 is 20348.288, not 20348.11.
Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 22ce0662-cc07-34bc-1d9c-de7fd075bdee
Version Independent ID: fc01822a-5d78-b4e0-eb1a-ccd8d9c7bdf0
Content: Azure Stack HCI release information - Azure Stack HCI
Content Source: azure-stack/hci/release-information.md
Service: azure-stack
Sub-service: azure-stack-hci
GitHub Login: @khdownie
Microsoft Alias: v-kedow

@katyhuang72727 Thanks for your feedback.
Assigning to content author @khdownie   for review.


Wrap RichTextCoordinator.syncContextWithTextView() in DispatchQueue

Xcode 14 produces runtime warnings for this function, as it updates the UI from a background thread.
This PR wraps the function in `DispatchQueue.main.async`. An alternative would be to use `@MainActor`.
This PR should get tested before merging!

Sorry, I got confused with the warning. I thought it was about using the background thread, while in fact this looks like an issue with the SwiftUI SDK in Xcode 14.0.

Hi!  I first started looking into adding async support, but since it causes other problems I decided to wait until Xcode 14.1 to see if it's solved there.


fix: Updating page sagas for fixing the scroll position issue on page

Description
Updating setupPageSaga and setupPublishedPageSaga for fixing the scroll position issue on page
PR fixes following issue(s)
Fixes #30481
Type of change

Bug fix (non-breaking change which fixes an issue)

Testing
How Has This Been Tested?

[x] Manual
[ ] JUnit
[ ] Jest
[ ] Cypress

Checklist:
Dev activity

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes
[ ] PR is being merged under a feature flag

QA activity:

[ ] Speedbreak features have been covered
[ ] Test plan covers all impacted features and areas of interest
[ ] Test plan has been peer reviewed by project stakeholders and other QA members
[ ] Manually tested functionality on DP
[ ] We had an implementation alignment call with stakeholders post QA Round 2
[ ] Cypress test cases have been added and approved by SDET/manual QA
[ ] Added Test Plan Approved label after Cypress tests were reviewed
[ ] Added Test Plan Approved label after JUnit tests were reviewed

Summary by CodeRabbit

Refactor

Updated action types for page initialization in the editor.
Renamed action types for setting up published pages.

/build-deploy-preview skip-tests=true

/ok-to-test /ok-to-test<EMAIL_ADDRESS>

/ok-to-test /ok-to-test<EMAIL_ADDRESS>

/build-deploy-preview skip-tests=true


Improved conditionals

Wherever conditionals are applied, they should be more flexible than the current "single variable with single fixed value".
They should have the following properties:

Not present unless required.
Allow specific values, greater/less than comparisons, ranges of values, sets of values.
Allow logic to be reversed (not operator)
Multiple conditionals should be allowed with boolean operators between them.

I think this would require conditionals to be modeled as one or more sub-resources rather than just simple properties on the graph nodes.
Consider allowing the conditionals to be edited in a dialog instead of being on the nodes themselves. Just indicate that there are conditionals set, or maybe even summarise the conditionals in a label if there are not too many.

I have started building a dialog for this.

This went through multiple iterations, but was ultimately implemented as "expressions".


fix bug responsive for admin pages and audit result page

https://github.com/user-attachments/assets/c9d9cd64-9fb5-4b42-ae43-d04c6b868264


Open Telegram links within Ferdium

Pre-flight Checklist
Please ensure you've completed all of the following.

[x] I have read the Contributing Guidelines for this project.
[x] I agree to follow the Code of Conduct that this project adheres to.

Description of Change

I modified Telegram’s recipe so that links which contain t.me or web.telegram.org open within Ferdium no matter what the setting for all links is and despite their target="_blank".
Fixes https://github.com/ferdium/ferdium-app/issues/1275

From my testing it seems to work anyway.


plonter - when clicked on component that in edit mode - the polyline and points that construct the polygon are shown

Intended outcome:
when clicking on entity that in edit mode and activating the plonter, it should show the entities.
Actual outcome:
when clicking on entity that in edit mode om the point, it shows the polyline and point that construct the polygon.
How to reproduce the issue:
create two polygons on on the other, add plonter, put one of the polygons in edit mode,click on one of the points that on the editor-mode-polygon - see the plonter shows the unwanted entities
Version

<EMAIL_ADDRESS>

Stale issue... closing


Question: browser objects

any ideas how to not get this error? HTMLElement is apart of the window object but if i use the import statement webpack can't resolve it. (originally i was thinking webpack would resolve it like path in node or something similar)

https://github.com/feross/standard#how-do-i-hide-a-certain-warning
/* global HTMLElement */

But honestly,  just use window.HTMLElement and avoid the ambiguity.


Include scopes in JWT payload

Hi there! This is my first contribution to the iNat repository, so please let me know if there's anything I've missed.
The context for this change is that I'm working on an app (iNaturalist Bingo) that users can sign in to through iNaturalist. The app doesn't need write access to the user's iNaturalist account—it only needs the user's ID. At present, the only way to get a user ID on a cross-origin domain through the OAuth flow seems to be to request a JWT token and fetch /v1/users/me from the API server. This requires the write scope, because of these two lines in users_controller.rb:
https://github.com/inaturalist/inaturalist/blob/fc8218f329abbb5462015e723a2756031d8834b1/app/controllers/users_controller.rb#L6-L7
i.e., an access_token with only the login scope cannot actually fetch any details about the logged-in user.
My proposal here is that we extend the JWT to include details about the granted scopes, which the API server can then read and apply.
Thoughts? Have I missed some way to implement this "Sign in with iNaturalist"-type flow more simply?

At present, the only way to get a user ID on a cross-origin domain through the OAuth flow seems to be to request a JWT token and fetch /v1/users/me from the API server.

That's not strictly correct: you can use https://api.inaturalist.org/v1/search?q=nornagon&sources=users to find users (and user IDs) associated with a username, though there are going to be some edge cases where a username is a substring of many other usernames and our less-than-perfect search ranking won't return it in the first page of results.
Also, though undocumented and thus not really a part of the API, you can use https://api.inaturalist.org/v1/users/nornagon or https://www.inaturalist.org/users/nornagon.json to retrieve a user ID given a username.
On the other hand, if your client requires your user to sign in with iNat because you need to authenticate them (i.e. ensure that only the user with that username is using your client), you can request an access token with the login scope and then GET www.inaturalist.org/users/edit using that token and you'll get a response with the user ID.
To the larger issues of whether the concept of OAuth access token scopes should extend to our JSON Web Tokens, I guess I can imagine uses where that might be desirable or necessary, but since none are on the table at the moment I don't think that's something we want to build or support at the moment. Doing so would require us not only to write the bake the scopes into the JWT and duplicate some of Doorkeeper's OAuth scope recognition in iNaturalistAPI, but we'd also have to do the same in the Rails app, which will also accept JWT as a valid form of auth. Not a trivial amount of work, and it adds further complexity to our already-complex auth system that we then have to maintain in perpetuity.
Zooming back even further, please read https://github.com/inaturalist/inaturalist/blob/main/CONTRIBUTING.md. I realize this is a super technical issue that might seem weird for the Forum, but that's still the best place to propose new functionality. Maybe a Github issue would be ok for stuff like this, but in general we use Github to coordinate work that we on staff have decided to do. Proposals for other ideas should start in the Forum where they can be considered and discussed before spending time doing the the work.

Hi Ken-ichi! Thanks for the detailed response. I did file a feature request in the forum, here: https://forum.inaturalist.org/t/allow-api-authentication-without-write-access/29091
My goal with this change is to allow fetching the user's details—which I have none of to begin with, not even the username—through a "Sign in with iNaturalist" button. The desired experience is:

The user visits my app, & is unauthenticated.
The user clicks "Sign in with iNaturalist" or similar.
The user authenticates my app through the OAuth interstitial on inaturalist.org, and is then redirected back to my app.
My app can verify that the user is now logged in, & fetch basic user info (e.g. user name, user id).

I tried to GET http://www.inaturalist.org/users/edit but unfortunately this endpoint is not CORS-enabled.

Hm, you're right about the CORS issue, but why do you need the user to authenticate at all? Is there some harm if I see your bingo page? If not you could just have people enter a username.

There's no particular harm right now, but I think it's a better experience to just click a button and be connected than to type in your username. Also, in future I can imagine wanting to rely more strongly on the authentication mechanism, e.g. to save preferences or coordinate a scoreboard. It seems handy to support!
CORS-enabling something like /users/edit might be a simpler way to go.


Get the depth/level of the tree and add a new event

api.tree.getHierarchy()
Get a Hierarchy object. Hierarchy object contains functions to retrieve hierarchical structure from loaded model.
children(row), descendants(row) and ancestors(row) returns a list of Row model object.
parent(row) returns a Row model object.

Would be nice if it could retrieve the level of the row. Why?
I need to get the level of row and then add a new event with the above/bellow parent. Pretty much like the angular-tree itself.

The level is the length of list returned by ancestors(row).

How about the siblings?

children(parent(row)) give siblings.
Le 26 nov. 2015 20:15, "xskid"<EMAIL_ADDRESS>a écrit :

How about the siblings?
—
Reply to this email directly or view it on GitHub
https://github.com/angular-gantt/angular-gantt/issues/548#issuecomment-159978968
.


[Feature]: Caching of SHOW WAREHOUSES to significantly increase performance

Use Cases or Problem Statement
We observed and confirmed by engineering that the performance of "SHOW WAREHOUSES" and "SHOW WAREHOUSES LIKE 'WAREHOUSE_NAME'" is identical. Since we have a lot of warehouses (~1000) the command ("SHOW WAREHOUSES LIKE 'WAREHOUSE_NAME'") takes approximately 10 seconds to complete; again for each of the 1000 warehouses.
As a result, the build step takes a very long time and is responsible for more than 80% of the total running time.
Engineering proposed to run the SHOW WAREHOUSES once and query the result cache for subsequent calls.
This would greatly improve the running times.
Category
category:resource
Object type(s)
No response
Proposal
Engineering proposed to run the SHOW WAREHOUSES once and query the result cache for subsequent calls.
How much impact is this issue causing?
Medium
Additional Information
No response
Would you like to implement a fix?

[ ] Yeah, I'll take it 😎

Hi @rd-thomas-uhren :wave:
This is related to #3118. As announced on our roadmap, we plan to target the performance of running similar/subsequent queries. Caching is one of the potential solutions to improve the performance. But the options currently on the table are: caching, batching requests, and/or not reading the state initially at all. They all have their pros, cons, and challenges.
Unfortunately, we can't improve the performance of a single SHOW WAREHOUSES query for now. However, if you split your deployment into smaller ones, the number of such queries will be reduced, leading to a potential time improvement of the plan.


Remember positioning

Is your feature request related to a problem? Please describe.
It would be great if F3D remembers the positioning when changing models when using the arrow keys. They are always reset which means the user needs to constantly reposition the viewport camera
Describe the solution you'd like
Remember the viewport camera position and direction

That would be actually pretty easy to do thanks to @snoyer work in #788 .
We could add a modifier (Ctrl) to enable the feature when changing file. Wdyt @Meakk @snoyer ?

Adding a modifier would be a good enough quick fix for now.
However I believe this should also/instead be configurable because I feel like it's a behavior that you'd want to choose by session rather that by individual model.
For example you'd want it on when looking at "related" models (CAD files representing parts of the same objects, assets from same game, ...) but have it off when cycling through random collections of models where the scale/centering may be all over the place.
I don't really see deciding "do I want to keep the view or not?" every-time I press left/right.

Indeed @snoyer , configurable interactions are indeed planned for the 3.0 :)

I do not mind if it can intelligently position the camera to encompass the model, the current situation is a bit anti user. Any solution is going to be better than what we have. Thanks for considering it.

I do not mind if it can intelligently position the camera to encompass the model, the current situation is a bit anti user. Any solution is going to be better than what we have.

I'm confused, could you explain how it is anti-user ? The camera position can be controlled from the command line and F3D behave as the user expects imo.

https://github.com/f3d-app/f3d/assets/8519469/fe751cf2-7181-46ac-91b4-4a69441491c0
See the video, the way it works now is quite frustrating, I need to constantly adjust position.

I do not mind if it can intelligently position the camera to encompass the model, the current situation is a bit anti user. Any solution is going to be better than what we have.

I'm confused, could you explain how it is anti-user ? The camera position can be controlled from the command line and F3D behave as the user expects imo.

Could you share a file ?  Your idea of keeping position is valid, but default camera should be much better than it is in your video.

I showed you what the default camera is with the app, I am not sure what the issue is. Here are couple file
tmp.zip

Ok, It's caused by this issue: https://github.com/f3d-app/f3d/issues/57
@Meakk, you may want to look at above video for a real life example of the issue.

Using --geometry-only as a workaround could work

Im afraid it would not work because this is a .fbx, no reader implemented yet.

Indeed

It would be also nice if there is a way to lock the camera for the session. For instance, I have a pack that I just want to look at from the front view. I can set the current view to front by pressing 1, but that is reset when I jump to the next model model. Naturally it also does not remembed the position when I get back to the previous mode that I used the front view with.

That sounds a bit too smart for F3D, it would mean writing some settings to disk, Im not sure we want that. You can set this up yourself though by using the right settings.

related to #856


Multi-GPU Configuration with dali Signal (11)s on triton 21.12 during ensemble processing

Hi!
I'm trying to use dali for preprocessing images to send to yolo. I have a two GPU system, that after a few minutes running dali, segfaults and dies. The Dali step is run in an triton ensemble model that sends data to a yolo model compiled to tensorrt. The stacktrace seems to indicate the problem is within dali.
Repro:
The preprocessing code is the following:
import nvidia.dali as dali

@dali.pipeline_def(batch_size=64, num_threads=4, device_id=0)
def pipe():
    images = dali.fn.external_source(
        device="gpu", name="DALI_INPUT_0", dtype=dali.types.UINT8)
    images = dali.fn.color_space_conversion(
        images, image_type=dali.types.BGR, output_type=dali.types.RGB, device='gpu')
    images = dali.fn.resize(images, mode="not_larger",
                            resize_x=640, resize_y=384, device='gpu')
    images = dali.fn.crop(images, crop_w=640, crop_h=384, crop_pos_x=0, crop_pos_y=0,
                          fill_values=114, out_of_bounds_policy="pad", device='gpu')
    images = dali.fn.transpose(images, perm=[2, 0, 1], device='gpu')
    images = dali.fn.cast(images, dtype=dali.types.FLOAT, device='gpu')
    return images

pipe().serialize(filename="1/model.dali")

and config.pbtxt
name: "preprocessbgr"
backend: "dali"
max_batch_size: 64
input [
{
    name: "DALI_INPUT_0"
    data_type: TYPE_UINT8
    dims: [ -1, -1, 3 ]
}
]

output [
{
    name: "OUTPUT_0"
    data_type: TYPE_FP32
    dims: [ 3, 384, 640 ]
}
]
dynamic_batching { }

There are about 100fps (1 batch size with 32 concurrency) of images being sent to triton, causing it to throw this error after several rounds of processing. The error reproduces after a few minutes:
Signal (11) received.
 0# 0x00005572DF1FBBD9 in tritonserver
 1# 0x00007F6BFF552210 in /usr/lib/x86_64-linux-gnu/libc.so.6
 2# 0x00007F6BF5602D40 in /usr/local/cuda/compat/lib.real/libcuda.so.1
 3# 0x00007F6BF57383E3 in /usr/local/cuda/compat/lib.real/libcuda.so.1
 4# 0x00007F6BF585EB02 in /usr/local/cuda/compat/lib.real/libcuda.so.1
 5# 0x00007F6BF55BF2E3 in /usr/local/cuda/compat/lib.real/libcuda.so.1
 6# 0x00007F6BF55BFAC4 in /usr/local/cuda/compat/lib.real/libcuda.so.1
 7# 0x00007F6BF55C1BD5 in /usr/local/cuda/compat/lib.real/libcuda.so.1
 8# 0x00007F6BF562FAAE in /usr/local/cuda/compat/lib.real/libcuda.so.1
 9# 0x00007F6BC80CF0D9 in /opt/tritonserver/backends/dali/libtriton_dali.so
10# 0x00007F6BC809EFED in /opt/tritonserver/backends/dali/libtriton_dali.so
11# 0x00007F6BC80F3B65 in /opt/tritonserver/backends/dali/libtriton_dali.so
12# 0x00007F6BC8098130 in /opt/tritonserver/backends/dali/libtriton_dali.so
13# dali::ThreadPool::ThreadMain(int, int, bool) in /opt/tritonserver/backends/dali/dali/libdali.so
14# 0x00007F6ABDAE526F in /opt/tritonserver/backends/dali/dali/libdali.so
15# 0x00007F6BFFDBE609 in /usr/lib/x86_64-linux-gnu/libpthread.so.0
16# clone in /usr/lib/x86_64-linux-gnu/libc.so.6

Also, interestingly enough, running this with
instance_group [
    {
        count: 1
        kind: KIND_GPU
        gpus: [ 0 ]
    }
]

Causes Signal (11) to show up faster (seconds) rather than minutes during ensembling over two GPUs.
Other things I've tried
I've tried multiple changes, such as

calling .gpu() and setting the external source to cpu, which causes the signal 11 to show up later.
single GPU instance makes dali play nicely and does not crash.

Theories

There is some kind of weirdness with the dali backend confusing gpu tensors across gpus perhaps?
Since the issue takes a few minutes to occur without instance groups, it is likely that cross scheduling devices (preprocessing in gpu:0 and then tensorrt on gpu:1) is causing issues and segfaults.

Versions
NVIDIA Release 21.12 (build 30441439)
Any thoughts? Appreciate this in advance.

Hi @natel9178 !
Thanks for thorough description of the problem. If I understand correctly, you are running 32 parallel models here. Additionally, I see, that there are these in the stacktrace:
13# dali::ThreadPool::ThreadMain(int, int, bool) in /opt/tritonserver/backends/dali/dali/libdali.so
15# 0x00007F6BFFDBE609 in /usr/lib/x86_64-linux-gnu/libpthread.so.0

This would suggest a problem with CPU threads used. Could you try to reduce num_threads=1 in DALI pipeline definition and see, if this help? Generally, CPU operations in DALI used thread-per-sample mapping. So if you have batch_size=1 there won't be any difference, still only one thread will be used.
If this does not help, could you support us with a core dump or a repro, that we can run on our side?


Not getting a packet with empty pose landmarks for a frame without a detection

On Android pose tracking demo (bazel version 0.8.9):
When a detection is not found in input video's stream (human is outside of camera's frame), no output packet is received.
Expected behavior: receive a packet with landmark list of size 0.

Hi @tomerlevi444 ,
Could you provide steps to reproduce this issue.

I followed the exact instructions here: https://google.github.io/mediapipe/getting_started/android.html
Installed via:
bazel build -c opt --config=android_arm64 mediapipe/examples/android/src/java/com/google/mediapipe/apps/handtrackinggpu:handtrackinggpu
and:
adb install bazel-bin/mediapipe/examples/android/src/java/com/google/mediapipe/apps/handtrackinggpu/handtrackinggpu.apk
Made sure I was working against master branch (with no local changes).

When there is no detection in frame, no packet data is written to the log (MainActivity.java:44).
Further checks in debug mode have also shown that the processor's callback is not being triggered in this case.

Hi @tomerlevi444 ,
Possibly the model produce "empty packet" instead of vector of zero detections.
(please try ObserveTimestamps option, in FrameProcessor.  option is only available for AddMultiStreamCallback.  user would need to alter code locally to use AddMultiStreamCallback, and select the ObserveTimetampBounds parameter.  https://github.com/google/mediapipe/blob/master/mediapipe/java/com/google/mediapipe/components/FrameProcessor.java#L213)


emscripten

issue description
using emscripten 1.36.4 (which i packaged myself but which is consitent with 1.35.4 in regards to this error) i can' compile xml.js on nixos 16.03. compiling xml.js on ubuntu with emscripten 1.36.4 works however.
question: what could cause this NixOS injection? i've tried to minimize PATH and other environment variables but the problem probably lies deeper in the way NixOS handles things.
help
please advice where i could search next.
steps to reproduce
emconfigure ../libxml2/configure --with-http=no --with-ftp=no --with-python=no --with-threads=no --with-debug
is supposed to set environment variables as CC/CXX which seems to work very well. however, when ./configure is executed it silently sneaks NixOS specific values into the toolchain:
[pid 30735] execve("/nix/store/n4zpfi4zzw10s2g91v1swkwq41v1afzz-coreutils-8.25/bin/cat", ["cat",
"/nix/store/1yc1mxy0qa4dkybvckxdn58i87wmzc36-gcc-wrapper-5.3.0/nix-support/libc-ldflags-before"],
[/* 124 vars */] <unfinished ...>

build/config.log:
configure:3582: $? = 0
configure:3571: /nix/store/0llznkhcyyndfhwqah01i1zcdhqxrx0d-emscripten-1.36.4/share/emscripten/emcc -v >&5
emcc (Emscripten gcc/clang-like replacement + linker emulating GNU ld) 1.36.4
clang version 3.9.0  (emscripten 1.36.4 : 1.36.4)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /nix/store/phd4xnam50rp46c1xncjnwn1wpy7zas8-emscripten-fastcomp-1.36.4
configure:3582: $? = 0
configure:3571: /nix/store/0llznkhcyyndfhwqah01i1zcdhqxrx0d-emscripten-1.36.4/share/emscripten/emcc -V >&5
ERROR:root:no input files
note that input files without a known suffix are ignored, make sure your input files end with one of: ('.c', '.C', '.i', '.cpp', '.cxx', '.cc', '.c++', '.CPP', '.CXX', '.CC', '.C++', '.ii', '.m', '.mi', '.mm', '.mii', '/dev/null', '.bc', '.o', '.obj', '.lo', '.dylib', '.so', '.a', '.ll', '.h', '.hxx', '.hpp', '.hh', '.H', '.HXX', '.HPP', '.HH')
configure:3582: $? = 1
configure:3571: /nix/store/0llznkhcyyndfhwqah01i1zcdhqxrx0d-emscripten-1.36.4/share/emscripten/emcc -qversion >&5
ERROR:root:no input files
note that input files without a known suffix are ignored, make sure your input files end with one of: ('.c', '.C', '.i', '.cpp', '.cxx', '.cc', '.c++', '.CPP', '.CXX', '.CC', '.C++', '.ii', '.m', '.mi', '.mm', '.mii', '/dev/null', '.bc', '.o', '.obj', '.lo', '.dylib', '.so', '.a', '.ll', '.h', '.hxx', '.hpp', '.hh', '.H', '.HXX', '.HPP', '.HH')
configure:3582: $? = 1
configure:3602: checking whether the C compiler works
configure:3624: /nix/store/0llznkhcyyndfhwqah01i1zcdhqxrx0d-emscripten-1.36.4/share/emscripten/emcc    conftest.c  >&5
configure:3628: $? = 0
configure:3676: result: yes
configure:3679: checking for C compiler default output file name
configure:3681: result: a.out
configure:3687: checking for suffix of executables
configure:3694: /nix/store/0llznkhcyyndfhwqah01i1zcdhqxrx0d-emscripten-1.36.4/share/emscripten/emcc -o conftest    conftest.c  >&5
configure:3698: $? = 0
configure:3720: result:
configure:3742: checking whether we are cross compiling
configure:3750: /nix/store/0llznkhcyyndfhwqah01i1zcdhqxrx0d-emscripten-1.36.4/share/emscripten/emcc -o conftest    conftest.c  >&5
/nix/store/60a1aqx579bjff3c90cw0l1zwyj2qxhg-binutils-2.26-dev/bin/ld: cannot find crt1.o: No such file or directory
/nix/store/60a1aqx579bjff3c90cw0l1zwyj2qxhg-binutils-2.26-dev/bin/ld: cannot find crti.o: No such file or directory
/nix/store/60a1aqx579bjff3c90cw0l1zwyj2qxhg-binutils-2.26-dev/bin/ld: cannot find crtbegin.o: No such file or directory
/nix/store/60a1aqx579bjff3c90cw0l1zwyj2qxhg-binutils-2.26-dev/bin/ld: cannot find -lgcc
clang: error: linker command failed with exit code 1 (use -v to see invocation)
configure:3754: $? = 1
configure:3761: ./conftest
../libxml2/configure: line 3763: ./conftest: No such file or directory
configure:3765: $? = 127
configure:3772: error: in `/home/joachim/Desktop/projects/nlnet/xml.js/build':
configure:3774: error: cannot run C compiled programs.
If you meant to cross compile, use `--host'.
See `config.log' for more details

a more complete log can be found here:

https://gist.github.com/qknight/cc64f6965b02971af290ad20b8215784
https://gist.github.com/qknight/5d5b4fed3c10b1358d8476dbae335d6e

strace
this strace shows that now binutils-2.26-dev/bin/ld is used for linking. normally that should just be emcc as it wraps the linking.
`
strace -s 234 -f emconfigure ../libxml2/configure --with-http=no --with-ftp=no --with-python=no --with-threads=no --with-debug
...
[pid 30725] execve("/nix/store/60a1aqx579bjff3c90cw0l1zwyj2qxhg-binutils-2.26-dev/bin/ld", ["/nix/store/60a1aqx579bjff3c90cw0l1zwyj2qxhg-binutils-2.26-dev/bin/ld", "-dynamic-linker", "/nix/store/x4p1cj8hi8n3xw60hpd62bfd20fi2arl-glibc-2.23/lib/ld-linux-x86-64.so.2", "--eh-frame-hdr", "-m", "elf_x86_64", "-dynamic-linker", "/lib64/ld-linux-x86-64.so.2", "-o", "conftest", "crt1.o", "crti.o", "crtbegin.o", "/run/user/1000/conftest-d7e90f.o", "-lgcc", "--as-needed", "-lgcc_s", "--no-as-needed", "-lc", "-lgcc", "--as-needed", "-lgcc_s", "--no-as-needed", "crtend.o", "crtn.o", "-rpath", "/nix/store/rc6gvgn52z44v1x8sdmmyjc2lili492a-shell/lib64", "-rpath", "/nix/store/rc6gvgn52z44v1x8sdmmyjc2lili492a-shell/lib", "-L/nix/store/97aj51dl9jvn2qyjapi7m2v2gai9mp27-nodejs-4.3.1/lib", "-L/nix/store/g47gdl1h8mzixl9566dnjlwgwvgpyvkr-libtool-2.4.6-lib/lib", "-L/nix/store/x4p1cj8hi8n3xw60hpd62bfd20fi2arl-glibc-2.23/lib", "-L/nix/store/m2ir96lr7a9kmljl5whmndfwgxf291za-gcc-5.3.0-lib/lib", "-rpath", "/nix/store/x4p1cj8hi8n3xw60hpd62bfd20fi2arl-glibc-2.23/lib", "-rpath", "/nix/store/m2ir96lr7a9kmljl5whmndfwgxf291za-gcc-5.3.0-lib/lib"], [/* 125 vars */]) = 0
`
the important parte of the strace with debugging enabled:

https://lastlog.de/misc/strace-emscripten.txt

@bosu @matthewbauer @magnetophon maybe you could have a look?

joelmo and me are having a rather big discussion on IRC right now about this problem with a possible solution of adding:
emcc as ld and also adding try linking ld to ld in clang-wrapper
and things like that

Looking in emcc it looks like it "fakes" it for the conftest because you can't natively run javascript output. You need to trick configure into thinking EMCC is a real compiler. Try setting the "CONFIGURE_CC" to which gcc. If that fixes it we can mod the default.nix to set it by default.
https://github.com/kripken/emscripten/blame/07b87426f898d6e9c677db291d9088c839197291/emcc.py#L187

@matthewbauer i've tried CONFIGURE_CC=which gcc but this needs more experiments.
back on ubuntu with emscripten-1.36.4 i see that /usr/bin/ld from binutils is used:
[pid 13245] execve("/usr/bin/ld", ["/usr/bin/ld", "-z", "relro", "--hash-style=gnu", "--build-id", "--eh-frame-hdr", "-m", "elf_x86_64", "-dynamic-linker", "/lib64/ld-linux-x86-64.so.2", "-o", "conftest", "/usr/lib/gcc/x86_64-linux-gnu/5.2.1/../../../x86_64-linux-gnu/crt1.o", "/usr/lib/gcc/x86_64-linux-gnu/5.2.1/../../../x86_64-linux-gnu/crti.o", "/usr/lib/gcc/x86_64-linux-gnu/5.2.1/crtbegin.o", "-L/usr/lib/gcc/x86_64-linux-gnu/5.2.1", "-L/usr/lib/gcc/x86_64-linux-gnu/5.2.1/../../../x86_64-linux-gnu", "-L/lib/x86_64-linux-gnu", "-L/lib/../lib64", "-L/usr/lib/x86_64-linux-gnu", "-L/usr/lib/gcc/x86_64-linux-gnu/5.2.1/../../..", "-L/home/joachim/Downloads/emsdk_portable/clang/fastcomp/build_master_64/bin/../lib", "-L/lib", "-L/usr/lib", "/tmp/conftest-efd354.o", "-lgcc", "--as-needed", "-lgcc_s", "--no-as-needed", "-lc", "-lgcc", "--as-needed", "-lgcc_s", "--no-as-needed", "/usr/lib/gcc/x86_64-linux-gnu/5.2.1/crtend.o", "/usr/lib/gcc/x86_64-linux-gnu/5.2.1/../../../x86_64-linux-gnu/crtn.o"], [/* 92 vars */] <unfinished ...>

@matthewbauer funny as it sometimes, is you pointed me exactly to the right place to find the solution! thanks very much. took me 3 days to figure a way to build this software on NixOS without manual fiddling.
solution
export EMCONFIGURE_JS=2
before
emconfigure ../libxml2/configure --with-http=no --with-ftp=no --with-python=no --with-threads=no
the xml.js unit test shows that the code runs out of the box! yay
EMCONFIGURE_JS documentation
https://github.com/kripken/emscripten/blame/07b87426f898d6e9c677db291d9088c839197291/emcc.py#L192

  # Whether we fake configure tests using clang - the local, native compiler - or not. if not we generate JS and use node with a shebang
Beither approach is perfect, you can try both, but may need to edit configure scripts in some cases
By default we configure in js, which can break on local filesystem access, etc., but is otherwise accurate so we
disable this if we think we have to. A value of '2' here will force JS checks in all cases. In summary:
0 - use native compilation for configure checks
1 - use js when we think it will work
2 - always use js for configure checks
use_js = int(os.environ.get('EMCONFIGURE_JS') or 1)

yet i have to find out if this solution really makes me happy. more interesting is, can this be applied to other projects as nix-instantiate, which was used for the tour of nix at https://nixcloud.io/tour, also. if so i would really consider releasing the source code but until now it used to be a very huge mess to get things working and byond anything i would want to document.

@qknight Awesome! Do you think we should set EMCONFIGURE=2 by default in Nix? I think emscripten is just having trouble finding whatever binutils it needs.
A while back I was looking into making a "crossSystem" for JavaScript - that is an stdenv based for JavaScript that would shim in the relevant emscripten stuff. It's difficult to do though because cross compilation is kind of broken right now.

@matthewbauer i will have to evaluate EMCONFIGURE=2 much more. but option 0 and 1 are probably there, for a reason. next goal is to create a toolchain using emscripten to build the software i'm currently working on automatically (or using nix-shell to hack on it).

@matthewbauer
i've been playing with stdenv.mkDerivation but what used to work interactively inside a nix-shell seems to fail in mkDerivation. any idea why?
json-c = stdenv.mkDerivation rec {
  name = "json-c";

  buildInputs = [ emscripten autoconf automake libtool pkgconfig gnumake ];

  src = ../json-c;

  configurePhase = ''
    export EMCONFIGURE_JS=2
    ./autogen.sh
    emconfigure ./configure
  '';

  buildPhase = ''
    emmake make
  '';

  installationPhase = ''
    ...
  '';

};

maybe because of the ~/.emscripten_cache which is missing then?

@matthewbauer
@domenkozar
what i do:
nix-shell, then it build the dependencies using emscripten and afterwards i can use make to hack on our project (here at nixcloud.io).
you asked me if export EMCONFIGURE_JS=2 is a good thing and should be a default? after using it with these 4 targets, it is the only option which makes it work past the 'check if the compiler produces executable code' section. i also figured that you have to set a $HOME variable to make emscripten work.
default.nix
with import <nixpkgs> { };

let

libz = stdenv.mkDerivation {
  name="libz";

  buildInputs = [ emscripten autoconf automake libtool pkgconfig gnumake ];

   src = fetchurl {
     url = "http://zlib.net/zlib-1.2.8.tar.gz";
     sha256 = "039agw5rqvqny92cpkrfn243x2gd4xn13hs3xi6isk55d2vqqr9n";
   };

  configurePhase = ''
    export EMCONFIGURE_JS=2

    # Some tests require writing at $HOME
    HOME=$TMPDIR

    emconfigure ./configure --prefix=$out
  '';

  # FIXME unit tests

  buildPhase = ''
    emmake make
  '';
};

json-c = stdenv.mkDerivation rec {
  name = "json-c";

  buildInputs = [ emscripten autoconf automake libtool pkgconfig gnumake ];

  version = "json-c-0.12-20140410";

  src = fetchgit {
    url = "https://github.com/json-c/json-c";
    rev = "refs/tags/${version}";
    sha256 = "1cj9qvsc8djgs0x9g3xb7szgjpmyqkw4gr3pf26qal35lv2wb1ig";
  };

  configurePhase = ''
    export EMCONFIGURE_JS=2

    # Some tests require writing at $HOME
    HOME=$TMPDIR

    ./autogen.sh
    emconfigure ./configure --prefix=$out
  '';

  buildPhase = ''
    emmake make
  '';

# FIXME unit tests

};

xml-js = stdenv.mkDerivation rec {
  name = "xml-js";

  buildInputs = [ emscripten autoconf automake libtool pkgconfig gnumake ];

  src = ../xml.js/libxml2;

  configurePhase = ''
    export EMCONFIGURE_JS=2

    # Some tests require writing at $HOME
    HOME=$TMPDIR
    autoreconf -if -Wall

    emconfigure ../libxml2/configure --with-http=no --with-ftp=no --with-python=no --with-threads=no --prefix=$out #--with-debug
  '';

  buildPhase = ''
    emmake make
  '';

  # FIXME unit tests

};

# tested to work with emscripten 1.35.4
fastXmlLint = stdenv.mkDerivation rec {
  name = "fastXmlLint";

  buildInputs = [ json-c libz xml-js emscripten gnumake pkgconfig ];

  phases= [ "unpackPhase" "patchPhase" "configurePhase" "buildPhase" "checkPhase" "installPhase" ] ;

  src = ./.;

  configurePhase = ''
    pkg-config --list-all
    pkg-config json-c --libs

    export EMCONFIGURE_JS=2

    # Some tests require writing at $HOME
    HOME=$TMPDIR
  '';

  # wrapping make using 'emmake' would remove the required exports for pkg-config to function properly
  buildPhase = ''
    make
  '';

  # FIXME unit tests
  checkPhase = "";

  # this doesn't make much sense yet
  installPhase = ''
    mkdir $out/
    cp -R *.js* $out/
    cp -R *.xhtml $out/
    cp -R *.html $out/
    cp -R codemirror-5.12 $out/
    cp -R html5-rng $out/
    cp -R libreoffice-relax-ng-example $out/
  '';
};

in

stdenv.mkDerivation rec {

  name = "nlnet-${version}";
  version = "0.0.1";

  shellHook = ''
    export HISTFILE=".zsh_history"
    alias make=colormake
    alias c="while true; do inotifywait * -e modify --quiet > /dev/null; clear; make; done"
    alias s="python -m SimpleHTTPServer"
    alias cl="while true; do inotifywait -e modify --quiet ../* > /dev/null; clear; make| head -n 30; done"
  '';

  buildInputs = [ json-c libz xml-js ] ++ [ colormake nodejs emscripten autoconf automake libtool pkgconfig gnumake strace ltrace python openjdk ];

}

Makefile

LIBXML20_CFLAGS += `pkg-config libxml-2.0 --cflags`
LIBXML20_LDFLAGS += `pkg-config libxml-2.0 --libs`

JSONC_CFLAGS += `pkg-config json-c --cflags`
JSONC_LDFLAGS += `pkg-config json-c --libs`

ZLIB_CFLAGS += `pkg-config zlib --cflags`
ZLIB_LDFLAGS += `pkg-config zlib --libs`

all: fastXmlLint.c
        emcc -O3 -Wall -pedantic -Wextra $(JSONC_CFLAGS) $(ZLIB_CFLAGS) $(LIBXML20_CFLAGS) \
                -c fastXmlLint.c\
                -s AGGRESSIVE_VARIABLE_ELIMINATION=1 \
                --llvm-lto 3 \
                -o fastXmlLint.o

        # https://github.com/kripken/emscripten/issues/2872
        # so -s EMULATE_FUNCTION_POINTER_CASTS=1
        emcc -O3 -Wall -pedantic -Wextra -std=gnu1x -i fastXmlLint.o  $(JSONC_LDFLAGS) $(ZLIB_LDFLAGS) $(LIBXML20_LDFLAGS)\
                -s EXPORTED_FUNCTIONS="['_validate', '_set_schema', '_my_echo', '_set_schema_file', \
                        '_set_relaxNG_schema', '_set_relaxNG_schema_file', '_validate_NG']" \
                -s ASSERTIONS=2 --emit-symbol-map  \
                -s EMULATE_FUNCTION_POINTER_CASTS=1 \
                --memory-init-file 0 \
                --pre-js pre.js \
                --memory-init-file 1 \
                -s TOTAL_MEMORY=234217728 \
                --llvm-lto 3 \
                --closure 0 \
                -o fastXmlLint.js


Update typings

Refs: https://github.com/metarhia/impress/pull/1648
Refs: https://github.com/metarhia/impress/pull/1655
Refs: https://github.com/metarhia/impress/pull/1649

Landed in https://github.com/metarhia/impress/commit/811af757a4a2b9bcdc49e10e6d6dbe4b97195951


Add "miter_limit" draw parameter for "lines" styles

"miter_limit" and "outline:miter_limit" parameters are now parsed and applied to geometry built with PolylineStyle. Miter length was already being calculated in Builders::buildPolyLine so the changes are mostly simple. The one non-obvious change is that buildPolyLine can no longer take a size hint function. Now that extra triangles may be added to replace miter joins, it is no longer possible to predict the number of vertices and indices that will be used for a given line. The performance impact of this change should be negligible - the automatic growth of std::vector may actually be more efficient than hinting sizes.
Resolves #500

@karimnaaji can you take a look? Thanks :)

Yep!

LGTM. @karimnaaji you can merge when you done reviewing this. Thanks


browser-esbuild: An unhandled exception occurred: EPERM: operation not permitted, copyfile

Command
build
Is this a regression?

[X] Yes, this behavior used to work in the previous version

The previous version in which this bug was not present was
No response
Description
Hi. browser-esbuild gives up with this error:
An unhandled exception occurred: EPERM: operation not permitted, copyfile 'C:\Users\123\WebstormProjects\123\src\manifest.json' -> 'C:\Users\123\WebstormProjects\123\dist\123'
See "C:\Users\ikpoc\AppData\Local\Temp\ng-G3vwG2\angular-errors.log" for further details.

Error log is the same. Tried running with admin privileges with same result.
Minimal Reproduction
ng build
Exception or Error
No response
Your Environment
Angular CLI: 17.0.0-next.8
Node: 18.18.1
Package Manager: npm 9.8.1
OS: win32 x64

Angular: 17.0.0-next.8
... animations, cli, common, compiler, compiler-cli, core, forms
... localize, platform-browser, platform-browser-dynamic
... platform-server, router, service-worker, ssr

Package                         Version
---------------------------------------------------------
@angular-devkit/architect       0.1700.0-next.8
@angular-devkit/build-angular   17.0.0-next.8
@angular-devkit/core            17.0.0-next.8
@angular-devkit/schematics      17.0.0-next.8
@angular/cdk                    17.0.0-next.6
@angular/material               17.0.0-next.6
@schematics/angular             16.2.5
rxjs                            7.8.1
typescript                      5.2.2
zone.js                         0.14.0

Anything else relevant?
No response

Update: created fresh project with ng new, same error:
An unhandled exception occurred: EPERM: operation not permitted, copyfile 'C:\Users\123\WebstormProjects\test-project\src\favicon.ico' -> 'C:\Users\123\WebstormProjects\test-project\dist\test-project'


Integrated the automated version bump code to the cirq plugin

Can I disable the Codefactor issue? Or should I include a module level doctoring?

[sc-10712]


Need a better way to write a snippet, better than json

Problem
json is terrible at:

reuse common code snippets
write multiline string

Suggestion
support yaml or js as snippets format

/duplicate of #25661


Change folder of tasks.json

Describe the bug
For some technical reasons i want to change the folder where the tasks.json is placed so that i can update it from a mount for example but that i don;t have to mount the whole project folder.
To Reproduce
Steps to reproduce the behavior:

Go to tasks.json
Change "input_path": "./tasks.json" to for example "input_path": "./upload/tasks.json"
start the labelstudio
You will see in tasks tab that the input path is still ./Labeller/tasks.json

Also changing the source arguments of the config json dont have effect. The thing i tried for the config json was:
{ input_path: "./upload/tasks.json",
 "source": {
    "name": "Tasks",
    "type": "tasks-json",
    "path": "./upload/tasks.json"
  },

Expected behavior
I would expect it to change the input path for the tasks.json to change to the upload folder, i have tried absolute paths also.
Screenshots
If applicable, add screenshots to help explain your problem.
Environment (please complete the following information):

OS: macOS 10.14.6 and Linux VMs
Label Studio Version 0.8.2

@bram2506 why might not you use a symlink to tasks.json? Probably we won't add this feature, because right now we are working on switching to SQLite instead of files DB. But your PR with this feature is highly appreciated.

@bram2506 why might not you use a symlink to tasks.json? Probably we won't add this feature, because right now we are working on switching to SQLite instead of files DB. But your PR with this feature is highly appreciated.

@makseq Okay thanks for the explanation. Feels to me then that this issue is redundant when the switch is made to SQLite. And to be honest I don't know how to change it correctly in the code myself.

@makseq Okay thanks for the explanation. Feels to me then that this issue is redundant when the switch is made to SQLite. And to be honest I don't know how to change it correctly in the code myself.


Chat feature: enable chat in the venue request form

When the comment stage is enabled and the chat interface is enabled for all the selected participants that are members of the reviewing committee: SACs, ACs and reviewers.

Chat invitation is created for each submission with the same start and end date of the official comments
Chat reaction tag invitation is created for each submission in order to react to the chat replies
Submission invitation is updated to two tabs are not shown: one for general discussion and another for the chat discussion
Email notifications are sent, 1 email every 5 new comments or 1 email every 4 hours
PCs can disable the chat functionality and keep the official comments open.

@zbialecki I pushed a change using items, I kept the line to use enum in case you need it.

@melisabok I think the tests need to be changed too here: comittee --> committee

ok, I will change it

I tested disabling and enabling the chat via the comment stage and it's working.

Email notifications are sent, 1 email every 5 new comments or 1 email every 4 hours

What happens if there are no new comments after 4 hours? Is an email still sent?
I also think that it's not necessary to send 1 email every 5 comments. I think that sending 1 email every 4 hours if there are any new messages is enough.

What happens if there are no new comments after 4 hours? Is an email still sent?

no emails are sent if there are no new comments since the last_notified_comment_id.

I also think that it's not necessary to send 1 email every 5 comments. I think that sending 1 email every 4 hours if there are any new messages is enough.

I'm worried that could be too many messages in 4 hours and the participants won't get any notifications.


Token with scope to CreateChannel

I am trying to create a bot that creates channels via a /command. Is this possible? I was looking at the documentation and both custom and app bots don't have that access to that scope.
What are my options? Has anybody had any luck with this?
The permissions are here: https://api.slack.com/bot-users

Looks like only user tokens can create channels. Is there anyway to get a user token without having to do an oauth flow?
https://api.slack.com/methods/channels.create

Not sure if you have figured it out already, but when you create a Slack App, you can give the app the channels:write permission under OAuth & Permissions >> Select Permission Scopes, which will let them create channels. Then you just need to use the apps token rather than the bots token for the channel creation requests.


Use order not degree to calculate a buffer size in ecdsatest

Otherwise this can result in an incorrect calculation of the maximum
encoded integer length, meaning an insufficient buffer size is allocated.
Thanks to Billy Brumley for helping to track this down.
Fixes #8209

Also see #8232 for a discussion on this.
@bbbrumley

@mattcaswell Looks good :) Glad to help!

Pushed.


[UNDERTOW-2097] DMI_RANDOM_USED_ONLY_ONCE error reported by spotbugs

Let's try to reuse instance of SecureRandom class when possible. This
change is attempt to fix error identified by SpotBugs plugin and
described in [1].
[1] https://spotbugs.readthedocs.io/en/stable/bugDescriptions.html#dmi-random-object-created-and-used-only-once-dmi-random-used-only-once
https://issues.redhat.com/browse/UNDERTOW-2097

just rebased against master branch

Failed tests on JDK17 seem to be the very same as are in other PRs so I suppose they are not tied with my changes here.


Omit or mask sensitive data in logs

When attempting to update the SCS config-server with a new Git repository that requires HTTP Basic authentication, the JSON sent includes the Git repo's credentials. These are logged in plain text by o.s.c.a.s.WorkflowServiceInstanceService in the broker's logs.
Either the request JSON shouldn't be logged at all or it should be scrubbed to remove or mask properties with names like "password".
Here's a (manually-scrubbed with ********) sample of what was logged:
o.s.c.a.s.WorkflowServiceInstanceService : Updating service instance ServiceBrokerRequest{platformInstanceId='null', apiInfoLocation='api.cf.indigo.springapps.io/v2/info', originatingIdentity=Context{platform='cloudfoundry', properties={user_id=7ec8e984-3ab7-4c78-8d7d-e3b36e2d8f2b}}}AsyncServiceInstanceRequest{asyncAccepted=true}AsyncParameterizedServiceInstanceRequest{parameters={git={label=master, password=********, uri=https://gitlab.com/habuma/test-config, username=********}}, context=Context{platform='cloudfoundry', properties={spaceGuid=e21bc29a-d346-484a-990d-85fd12302895, organizationGuid=64481248-61ac-4843-9d77-780754858a64}}}UpdateServiceInstanceRequest{serviceDefinitionId='279d22f6-7d8c-4113-9fcc-e319113aaa6a', planId='b43d9058-c163-42d3-a863-b55480c7b341', previousValues=PreviousValues{planId='b43d9058-c163-42d3-a863-b55480c7b341'}, serviceInstanceId='df20a301-7698-4f87-b05b-c5451f10097c'}

The object being printed in this log example comes from Spring Cloud Open Service Broker. That project logs requests and responses in a similar manner, but at a debug logging level where App Broker is currently using a the info logging level. Using debug level for these tracing logs in App Broker would make the projects consistent and provide useful info for troubleshooting without exposing potentially sensitive info by default.


ERROR: Cannot find model files at ...\lib\site-packages\subaligner\models\training\weights

Command used:
&"C:\Users\Shid\AppData\Local\Programs\Python\Python37\Scripts\subaligner.exe" -m dual -v "aa aa.mkv" -s "aa aa-1.srt" -o "aa aa.srt"

Output:
2022-03-19 21:43:58.992740: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'cudart64_110.dll'; dlerror: cudart64_110.dll not found
2022-03-19 21:43:58.992950: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
ERROR: Cannot find model files at c:\users\shid\appdata\local\programs\python\python37\lib\site-packages\subaligner\models\training\weights

  File "c:\users\shid\appdata\local\programs\python\python37\lib\site-packages\subaligner\__main__.py", line 247, in main
    exit_segfail=exit_segfail,
  File "c:\users\shid\appdata\local\programs\python\python37\lib\site-packages\subaligner\predictor.py", line 120, in predict_dual_pass
    video_file_path, subtitle_file_path, weights_file_path
  File "c:\users\shid\appdata\local\programs\python\python37\lib\site-packages\subaligner\predictor.py", line 731, in __predict
    network = self.__initialise_network(os.path.dirname(weights_file_path), self.__LOGGER)
  File "c:\users\shid\appdata\local\programs\python\python37\lib\site-packages\subaligner\predictor.py", line 488, in __initialise_network

In 0.2.0 a similar error seems to occur:
2022-03-19 22:28:13.843741: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'cudart64_110.dll'; dlerror: cudart64_110.dll not found
2022-03-19 22:28:13.843904: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
2022-03-19 22:28:16.555773: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'nvcuda.dll'; dlerror: nvcuda.dll not found
2022-03-19 22:28:16.555975: W tensorflow/stream_executor/cuda/cuda_driver.cc:326] failed call to cuInit: UNKNOWN ERROR (303)
2022-03-19 22:28:16.562222: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:169] retrieving CUDA diagnostic information for host: BJCG020218
2022-03-19 22:28:16.562546: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:176] hostname: BJCG020218
2022-03-19 22:28:16.583596: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
ERROR: [WinError 2] The system cannot find the file specified

  File "c:\users\shid\appdata\local\programs\python\python37\lib\site-packages\subaligner\__main__.py", line 218, in main
    weights_dir=os.path.join(FLAGS.training_output_directory, "models/training/weights")
  File "c:\users\shid\appdata\local\programs\python\python37\lib\site-packages\subaligner\predictor.py", line 80, in predict_single_pass
    video_file_path, subtitle_file_path, weights_file_path
  File "c:\users\shid\appdata\local\programs\python\python37\lib\site-packages\subaligner\predictor.py", line 593, in __predict
    video_file_path, True, 16000
  File "c:\users\shid\appdata\local\programs\python\python37\lib\site-packages\subaligner\media_helper.py", line 90, in extract_audio
    bufsize=1,
  File "c:\users\shid\appdata\local\programs\python\python37\lib\subprocess.py", line 800, in __init__
    restore_signals, start_new_session)
  File "c:\users\shid\appdata\local\programs\python\python37\lib\subprocess.py", line 1207, in _execute_child
    startupinfo)

If I take model.hdf5 from the model folder and place it in weights then it comes up with this; in 0.2.1 it comes up with the exact same error without moving it:
2022-03-19 21:53:47.468418: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'cudart64_110.dll'; dlerror: cudart64_110.dll not found
2022-03-19 21:53:47.468589: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
ERROR: list index out of range

  File "c:\users\shid\appdata\local\programs\python\python37\lib\site-packages\subaligner\__main__.py", line 247, in main
    exit_segfail=exit_segfail,
  File "c:\users\shid\appdata\local\programs\python\python37\lib\site-packages\subaligner\predictor.py", line 120, in predict_dual_pass
    video_file_path, subtitle_file_path, weights_file_path
  File "c:\users\shid\appdata\local\programs\python\python37\lib\site-packages\subaligner\predictor.py", line 731, in __predict
    network = self.__initialise_network(os.path.dirname(weights_file_path), self.__LOGGER)
  File "c:\users\shid\appdata\local\programs\python\python37\lib\site-packages\subaligner\predictor.py", line 496, in __initialise_network
    hyperparams_path = "{}/{}".format(config_dir, hyperparams_files[0])

Python version: 3.7.9
Subaligner version: 0.2.3

Hi @Shidouuu. Thanks for trying it out on Windows, which however has not been supported. For now to run subtitle synchronisation on Windows, please use Docker Desktop and more details can be found in here.


Gitlab API authentication failed. Please check your gitlab's token. HTTP status code 401.

When I use the the Markdown Preview in Browser, Sublime throws the error message indicated above.

You need to register a personal access token to Gitlab.com to be able to use the Markdown to HTML facilities.
To do so, go to Gitlab.com, connect to your account, go to your profile, click on Access Tokens panel, create a new token with at least read_api rights & copy the newly created token.
Then, in sublime, go to Preferences -> Package Settings -> Markdown Preview -> Settings & in the right editor, add:
	"gitlab_personal_token":  "<YOUR_TOKEN>"

Then everything should work fine.


COM Out arguments in DXGI/D3D11 are suddenly unmanaged

Actual behavior
COM Out arguments in methods in the Graphics.DXGI and .D3D11 namespaces are now I..._unmanaged** (instead of out I... in 0.3.49beta). Look at e.g. all the Create* functions in ID3D11Device.
Note that this doesn't apply to top level functions such as D3D11CreateDevice. Those still work as expected.
Expected behavior
I'd really like to have the managed out arguments back if possible.
Repro steps

NativeMethods.txt content:

Windows.Win32.Graphics.Direct3D11
Windows.Win32.Graphics.DXGI
D3DCompile

NativeMethods.json content (if present):

{
  "$schema": "https://aka.ms/CsWin32.schema.json",
  "useSafeHandles": false,
  "allowMarshaling": true
}

Context
CsWin32 version: 0.3.106
Target Framework: net8.0

I suspect this change is a fallout of improving support in other areas where we discovered certain interop rules had to be honored, and this was an area where we weren't fully following the rules, causing failures in at least some cases.
I'll leave this active till we have a chance to confirm.

having the same issue with D3D11 after updating from 0.3.49-beta to 0.3.106

Same for me. It's actually a showstopper for upgrading nuget packages.


fix: Escape libSQL vector store inserts by using object syntax with placeholders

Summary
This PR fixes issue #7040, which causes SQL syntax errors due to newline characters, quotes, or similar characters not being properly escaped.
The proposed solution uses @libsql/client's built-in object syntax with placeholders to use the internal mechanism for escaping values and therefore fixes the mentioned problem.
Related Issues
Fixes #7040

Hey @florivdg, thanks for this! I was actually unable to test this, and it seems this made it in without an integration test.
Are you part of the team/are you running this in production?

Are you part of the team/are you running this in production?

Actually not, I am building a proof of concept for my company and this immediately came up while developing.

I keep getting:
LibsqlError: SQL_INPUT_ERROR: SQL input error: no such column: -0.013785726,-0.009188198...

Any chance you could add an integration test?

Hey @florivdg, seems the get documents and delete queries were non-functional. Updated docs as well.
Can you try this and make sure it fits your usecase?

Hey @jacoblee93, thanks for your work! I confirm that adding documents and performing the similarity search works as expected now.

Ok great, thanks!


ProduceAsync not throwing exceptions on error

Description
I use this code _producer.ProduceAsync(_errorTopic, originalMessage.Key, originalMessage.Value).Wait() to produce a message and I expected that if the method completes (after the Wait) without an exception the message is published. But in fact it can happen that the Message object returned by ProduceAsync has an Error on it - in my case it was "Not enough in-sync replicas". So to get the behaviour I want I actually have to do
var result = _producer.ProduceAsync(_errorTopic, originalMessage.Key, originalMessage.Value).Result;                     if (result.Error.HasError)                     {                         throw new KafkaException(result.Error);                     }
This is not what I expect in C#. I realize this is quite a breaking change to change it now. So I'm starting this to initiate a discussion.
Checklist
Please provide the following information:

[x] Confluent.Kafka nuget version: 0.11.3
[x] Apache Kafka version: 0.11
[x] Client configuration:
"request.required.acks", -1
[x] Operating system: windows/linux - affects both
[x] Provide logs (with "debug" : "..." as necessary in configuration)
[x] Provide broker log excerpts: Not relevant
[x] Critical issue NO

we will be making breaking changes elsewhere when moving to 1.0 (which will coincide with introducing EoS functionality). I think you're right here (but haven't thought through all the details).

@roederja2 I am seeing the Task.Result paradigm used frequently with asynchronous programming these days. Otherwise you are forced into either a synchronous wait or an async wait surrounded by a try/catch which has ramifications so as to determine the status of an async task. Returning a result provides information about the status of the task which gives options.
This is taken from the Azure Cloud Storage tutorials:
TableOperation retrieveOperation = TableOperation.Retrieve<customerentity>("Harp", "Walter");
TableResult result = await table.ExecuteAsync(retrieveOperation);

Here is another snippet via .NET Azure tutorials:
Task<bool> tableStatus = CreateTable(storageAccount, tableName);
tableStatus.Wait();
if (!tableStatus.Result) { Console.WriteLine("table creation error");}

If you return a Task you can also look at the Exception property if you don't want to use try/catch. Plus, as it stands the ProduceAsync methods does throw exceptions in some cases, but not others depending on when they happen inside the library - this is inconsistent and also not what is expected in C#. Checking return codes is C style and very much discouraged in languages that support Exceptions.

I understand what you are saying.  Here is Microsoft .NET's statement on Task-based Asynchronous Patterns (TAP) and the link to the statement below:

Task-based Asynchronous Pattern (TAP)   03/30/2017
The Task-based Asynchronous Pattern (TAP) is based on the System.Threading.Tasks.Task and System.Threading.Tasks.Task types in the System.Threading.Tasks namespace, which are used to represent arbitrary asynchronous operations. TAP is the recommended asynchronous design pattern for new development.

https://docs.microsoft.com/en-us/dotnet/standard/asynchronous-programming-patterns/task-based-asynchronous-pattern-tap
It seems to be the paradigm going forward for .NET.  I found that there was plenty of very well written documentation from Microsoft .NET on the subject which made my transition to TAP quicker.

This change is in the 1.0-experimental branch. Also, any method that returns more than one result, each of which could have an associated error (e.g. Consumer.Pause) will throw an exception if any of the results is in error (with the full result available as a property on the exception). You're right, expecting users to check for errors is inconvenient and error-prone in C#.


Assets/Arcadia/configuration.edn does not exist

I am getting the following error upon import:
System.Exception: Default Arcadia configuration file missing. Assets/Arcadia/configuration.edn does not exist
  at arcadia$internal$config$default_config__3592.invokeStatic () [0x00082] in <f96487d39d5f4bbbb5c01bd75e07d11a>:0
  at arcadia$internal$config$default_config__3592.invoke () [0x00000] in <f96487d39d5f4bbbb5c01bd75e07d11a>:0
  at arcadia$internal$config$merged_configs__3616.invokeStatic () [0x00019] in <f96487d39d5f4bbbb5c01bd75e07d11a>:0
  at arcadia$internal$config$merged_configs__3616.invoke () [0x00000] in <f96487d39d5f4bbbb5c01bd75e07d11a>:0
  at arcadia$internal$config$update_BANG___3622.invokeStatic () [0x0000a] in <f96487d39d5f4bbbb5c01bd75e07d11a>:0
  at arcadia$internal$config$update_BANG___3622.invoke () [0x00000] in <f96487d39d5f4bbbb5c01bd75e07d11a>:0
  at clojure.lang.Var.invoke () [0x00006] in <c6c70eec5ec44c4283b925eef0e247cb>:0
  at Arcadia.Initialization.LoadConfig () [0x0001a] in /home/mac/Dropbox/projects/tmp/aatest/Assets/Arcadia-1.0.0-beta/Editor/Initialization.cs:80
  at Arcadia.Initialization.Initialize () [0x0000b] in /home/mac/Dropbox/projects/tmp/aatest/Assets/Arcadia-1.0.0-beta/Editor/Initialization.cs:50
  at Arcadia.Initialization..cctor () [0x00063] in /home/mac/Dropbox/projects/tmp/aatest/Assets/Arcadia-1.0.0-beta/Editor/Initialization.cs:23
UnityEditor.EditorAssemblies:ProcessInitializeOnLoadAttributes()

I am on Ubuntu 18.10 using:

Version 2018.2.7f1 (dad990bf2728) Personal
Mon, 10 Sep 2018 10:49:10 GMT
Branch: 2018.2/linux/editor

Try renaming the directory from Arcadia-1.0.0-beta to Arcadia.  That worked for me (2017.4.17f1).

Renaming works. Thanks!

Yes, the folder everything lives in must be called Arcadia. This is something we can relax in the future, but for now @faried's advice is the correct.