Fix #856

Coverage increased (+0.3%) to 49.523% when pulling 7976f13de74243f97fe02bade49d6c819667c896 on ywk253100:161010_pagination into e9a3bb989cb06c586cbd4c84e2bb2784f3ed2f9d on vmware:master.

Coverage increased (+0.3%) to 49.523% when pulling 769c9c63f12fadcd0bd622ae5cd50a333cb8b44d on ywk253100:161010_pagination into e9a3bb989cb06c586cbd4c84e2bb2784f3ed2f9d on vmware:master.


Cannot call Object.defineProperty with a frozen property descriptor when zone.js is loaded

We have the following code in our application library:
var NULL_WRITABLE = Object.freeze({
  value: null,
  writable: true
});

...
Object.defineProperty(this, "stroke", NULL_WRITABLE);
Object.defineProperty(this, "strokeOpacity", NULL_WRITABLE);
Object.defineProperty(this, "strokeWidth", NULL_WRITABLE);
...

We freeze the object, because we use it in different places and don't want it to be modified (by accident or otherwise, by our code or by other JavaScript libraries).
This works fine until zone.js (we are using v0.8.14) is loaded in the application, then you get the following stack trace:
Uncaught TypeError: Cannot add property configurable, object is not extensible
    at rewriteDescriptor (zone.js:2202)
    at Function.Object.defineProperty (zone.js:2167)
    at Object.e [as defineProperty] (Lang.js:1)
    at new Stroke (Stroke.js:12)
    at Object.<anonymous> (_defaults.js:1)
    at Object.../../../../luciad/ogc/se/painter/symbolizer/_defaults.js (_defaults.js:1)
    at __webpack_require__ (bootstrap 128e3854f0217f61564f:54)
    at Object.../../../../luciad/ogc/se/painter/SEPainter.js (SEPainter.js:1)
    at __webpack_require__ (bootstrap 128e3854f0217f61564f:54)
    at Object.../../../../luciad/ogc/se/painter/PainterBuilderVisitor.js (PainterBuilderVisitor.js:1)

Which makes sense because zone.js is trying to alter the object that we froze, which is exactly what we were trying to avoid.
I would prefer that zone.js does not alter input parameters in this particular case and operates on a clone when it tries to rewrite the descriptor?

@tommahieu, I will try to fix this one.

Thank you, @JiaLiPassion  :)


input type='number' value is not cleared on property reset

Hi,
please take a look at http://jsfiddle.net/pktYz/
Press the CLICK ME button

expected: second input field to be empty
result: second input field still contains the old value

Tested with AngularJS v0.10.4, Firefox 7, Chrome 15, IE 8

Actually, that works as intended. :-( The issue is that html5 browsers do
not let you set the input to blank if you specify number. So short of
changing the input type to text, there is no way to clear the content.
-- Misko
2011/10/25 Philipp Müller <
<EMAIL_ADDRESS>
Hi,
please take a look at http://jsfiddle.net/pktYz/
Press the CLICK ME button

expected: second input field to be empty
result: second input field still contains the old value

Tested with AngularJS v0.10.4, Firefox 7, Chrome 15, IE 8
--
Reply to this email directly or view it on GitHub:
https://github.com/angular/angular.js/issues/623

That's not true. You can set input@type=number to blank either via UI or programmatically. Unless I'm missing something,  this bug is valid.

The case for 'undefined' value is missing in numericRegexpInputType(). I created a patch that's works for me: https://gist.github.com/1320239

Sorry we dropped the ball, and thanks for reminding us. Here is the fix
which should make it to the next release:
https://github.com/angular/angular.js/pull/656
BTW, may we request that you send us a pull request with tests next time.
That would make the process much faster.
-- Misko
2011/11/15 Philipp Müller <
<EMAIL_ADDRESS>

The bug still exists in v0.10.5. Could you please have a look at my patch?

Reply to this email directly or view it on GitHub:
https://github.com/angular/angular.js/issues/623#issuecomment-2748399

Thanks! I'll try to send a pull request + tests the next time...


smack: handle binary files with deploy-files

smack can maintain deploy-files.bbclass itself. then it shouldn't have inherit issue if this layer is referenced by other project.

There was already a PR very similar to this one (https://github.com/01org/meta-intel-iot-security/pull/27/files) that tried to use copybin.
I do not mind merging this change and dropping the other, as long as it is correct. Please see comments in the files changed.

remove comments.
deploy-files is a sstate safe version of copybin, It means deploy-files will cache these files to sstate. This is useful for CI, CI use sstate to reduce build time.

Looks good to me. +1

Who will merge this?

I'm about to test it, then I will merge.

Tested with the following in local.conf:
INHERIT += "testimage"
TEST_SUITES="ssh smack"
And ran:
bitbake -c testimage iot-os-image
Worked fine -> merging.


fix Revert "fix fix test case failure of insert null value into not null column (#20963)"

Revert "fix fix test case failure of insert null value into not null column (#20963)"
This reverts commit 55a6649da962fb170ddb40fea8ef26bdc552a51a.
Mannual Revert "fix in strict mode, return error for insert if datatype convert fails (#20378)"
This mannual reverts commit 1b94b6368f5e871c9a0fe53dd7c64409079a4c9d
Proposed changes
Issue Number: close #xxx

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall

run buildall

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 47.76 seconds
stream load tsv:          551 seconds loaded<PHONE_NUMBER>9 Bytes, about 129 MB/s
stream load json:         19 seconds loaded<PHONE_NUMBER> Bytes, about 118 MB/s
stream load orc:          65 seconds loaded<PHONE_NUMBER> Bytes, about 16 MB/s
stream load parquet:          31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select:          29.5 seconds inserted 10000000 Rows, about 338K ops/s
storage size:<PHONE_NUMBER>4 Bytes

run buildall

run buildall

run buildall

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 47.27 seconds
stream load tsv:          535 seconds loaded<PHONE_NUMBER>9 Bytes, about 133 MB/s
stream load json:         20 seconds loaded<PHONE_NUMBER> Bytes, about 112 MB/s
stream load orc:          65 seconds loaded<PHONE_NUMBER> Bytes, about 16 MB/s
stream load parquet:          31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select:          29.5 seconds inserted 10000000 Rows, about 338K ops/s
storage size:<PHONE_NUMBER>5 Bytes

run buildall

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.86 seconds
stream load tsv:          534 seconds loaded<PHONE_NUMBER>9 Bytes, about 133 MB/s
stream load json:         20 seconds loaded<PHONE_NUMBER> Bytes, about 112 MB/s
stream load orc:          65 seconds loaded<PHONE_NUMBER> Bytes, about 16 MB/s
stream load parquet:          31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select:          29.1 seconds inserted 10000000 Rows, about 343K ops/s
storage size:<PHONE_NUMBER>5 Bytes


chore(deps): update common-dev-assets digest to 36f1bcd

This PR contains the following updates:

Package
Update
Change

common-dev-assets
digest
d583c56 -> c26ebe5

Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Renovate will not automatically rebase this PR, because other commits have been found.
👻 Immortal: This PR will be recreated if closed unmerged. Get config help if that's undesired.

[ ] If you want to rebase/retry this PR, click this checkbox. ⚠ Warning: custom changes will be lost.

This PR has been generated by Renovate Bot.

:tada: This PR is included in version 1.0.1 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Update Dockerfile

Update to latest version of mysql. This will (at this time) provide 5.7.16 which will allow use of the json field type.

@brianvice Thank you for your contribution but please refer to #364 and #366

@brianvice @philtrep lool I just made that 5.6 today, I was thinking of defining the versions in every dockerfile and then manually updating them after testing each release! Because this is not the first time users find bugs with the latest updates, and we cannot easily reproduce these errors since we all have our own containers..
Basically we have 2 options:
Option 1: keep everything to the latest and wait for bugs to occure and fix them.
Option 2: manually upgrade each file version and make sure all users have working versions.
I know each has it's own disadvantages, I personally don't mind any one. What do you guys think?

@Mahmoudz @philtrep
Hey guys, thanks for getting back to me on this PR. I saw the update to 5.6. My reason for having 5.7 is that it allows for the full function of laravel's database migration features. With 5.6, if someone tries to use the json field type they'll get an SQL syntax error when performing their migrations.
While it's easy enough to just swap out the version number in the dockerfile and rebuild the container, I think the second option you mention is a good one. As in the php container, we could define multiple dockerfiles and then give the user the option of which image to use. But for completeness and in order to provide up to date packages I don't see a problem with this option.

@Mahmoudz @brianvice
I think manually selecting the version is the best idea. Brian's idea of having multiple Dockerfile is fine by me BUT we should try to limit the amount of Dockerfile we have per image type so as not to have to maintain a dozen different Dockerfile for each image type.
The way I see it, we could focus on keeping the core Laravel containers like php-fpmand mysql up to date with the latest versions and keep the secondary/less used containers either on LTS or latest major release.
For example, we could support mysql 5.6 and 5.7 but for mongo only 3.2. That way we maximize ease of use for most users while keeping issues to a minimum.

@brianvice @philtrep thanks for your replies. Phill I totally agree with you, w'll define the versions for the default most basic containers and let the rest use the :latest.


Proposed quote support (#27)

Attempt to address issue #27.
There were some version issues. Using QuoteRaw/QuoteTyped didn't seem to be an option so #nowarn "44" (obsolete warning) and Patterns.Quote is used. The needed Expr type (Expr or Expr<_>) is determined using inp.Type

THis looks good to me, are there any reasons you know of that this isn't ready?

Nope, seems fine to me.

Cool, will merge

@kevmal See also #29 - let me know if you'd like to help with getting this repo updated thanks


[stable/postgresql] Fix chart when persistence is enabled

Signed-off-by: tompizmor<EMAIL_ADDRESS>
What this PR does / why we need it:
This PR uses an initContainer when persistence and securityContext are enabled to ensure that the postgresql data directory permissions are 0700.
Which issue this PR fixes
(optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged)

fixes https://github.com/bitnami/bitnami-docker-postgresql/issues/91

Special notes for your reviewer:
None
Checklist
[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

[X] DCO signed
[X] Chart Version bumped
[X] Variables are documented in the README.md

/assign @carrodher

/ok-to-test
/lgtm


[fix] allow for replacement values with equal

I just ran into this problem. I need to find/replace with a value containing "=".

Hey, This is pretty much needed thing, can any one merge this with master?

Guys, I just faced the same issue. Could someone appove this?

Same here :(

Please approve the PR

I have the same issue. Can you merge this PR?


Overrided toString and added toByteArray method for ObjectID similar to org.bson.ObjectId

Existing ObjectId implementation is outdated and should be updated to follow Mongo 3.0 format.
That means ObjectId should be stored as array of bytes. As a first step I propose to implement toByteArray method from original org.bson.ObjectId

Thanks for the PR. Is this code backward-compatible? Or can it possibly break existing applications using bson4jackson and still connecting to MongoDB 2.x?

This one is fully backward-compatible. It just overrides toString method and adds one new - toByteArray() that is similar to existing ByteOrderUtils, but more straightforward. toString override is very helpful in many ways. For example in logging. And should not break anything.
After this PR is merged I wanted to add next one, with single private field inside OrderId - private final byte[] bytes. But I can't see the right way to make it backward-compatible

Thanks for the pull request again. However, I don't see the benefits of this change yet (apart from the new toString method). It actually does the same as the previous code, but a little bit slower. We now have to create four more short-lived byte arrays whenever an ObjectId should be written. This is unnecessary IMHO.
I appreciate your contribution but I'd rather not merge it yet. I'd like to see this commit in a larger context. If you want to implement support for MongoDB 3 feel free to submit a new pull request. I'd be more than happy to have a look at it. Thanks.


How to get total count of entities by type or by criteria

Is there an existing issue for this?

[X] I have searched the existing issues

Current Behavior
Using Egeria API I could not found the methods to get total count of entities by type or by criteria like in findEntities method.
Expected Behavior
I am able to get total count of entities by type or by criteria used in findEntities method.
Steps To Reproduce
No response
Environment
- Egeria: 3.9
- OS: Linux
- Java: 16

Any Further Information?
No response

@sergey-soloviov We have considered this in the past and have decided hat Egeria would not do these count orientated queries. Theses queries could be very costly, they could mean issuing paged federated queries across multiple repositories, to return a number. Egeria rest calls are more geared to issuing calls page by page producing data as required.

@sergey-soloviov having said that, you can code a loop that keeps updating the startFrom to step through the pages.  Keep the loop going until you get null

Also there is work currently around building indexes of egeria content, which in future could be queried to gather similar information.
I wonder also if this kind of information, gathered from the index, could be kept as metrics -- not an accurate number, but some indication of size which might help with capacity planning & reporting.

I guess a count means a count of the content in assets in the zones I can see, that are not deleted and at the appropriate effectivity date, without mementos and they may or may not contain duplicates depending on the calls used.


Two symbols missing in simulator setup

Hello,
SecKeychainGetTypeID and SecAccessGetTypeID are missing at link time when I target the iphone6 simulator. Strangely I don't have the problem when I target an actual iphone6. Do we have another cfg() mixup somewhere ?
undef: _SecKeychainGetTypeID
undef: _SecAccessGetTypeID
Undefined symbols for architecture x86_64:
  "_SecKeychainGetTypeID", referenced from:
      _$LT$security_framework..keychain..SecKeychain$u20$as$u20$core_foundation..base..TCFType$LT$$BP$mut$u20$security_framework_sys..base..OpaqueSecKeychainRef$GT$$GT$::type_id::h0b49430b489586d5 in SnipsSda(security_framework-d4c2fbca6aed6c3a.0.o)
  "_SecAccessGetTypeID", referenced from:
      _$LT$security_framework..access..SecAccess$u20$as$u20$core_foundation..base..TCFType$LT$$BP$mut$u20$security_framework_sys..base..OpaqueSecAccessRef$GT$$GT$::type_id::h5bfb1124c8dad652 in SnipsSda(security_framework-d4c2fbca6aed6c3a.0.o)
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Some findings...
As far as i can tell, the symbol _SecKeychainGetTypeID is declared (for both platforms) in security-framework-sys/src/keychain.rs, and used from security-framework/src/keychain.rs, on both platforms. I can not see it implemented anywhere. This is consistent with the output of nm.
My best guess so far is, we do miss the definition of the functions on both platforms, but the two linkers behave subtly differently.

So my understanding is that access and keychain are just not here on ios. I have cfg() filtered them, it's relatively straightforward. The one place where it does not look so good, imho, is the keychains() method in the item search.


Removed ContextSize from most examples

Removed ContextSize from most examples.
If it's not set it's retrieved from the model, which is usually what you want. Leaving it in the examples is just unnecessary complexity.

I think that the context size is an important parameter for GPU memory usage. If it is too high, then you may not be able to upload all layers to the GPU or you may not be able to optimize for speed. You can also have some models with huge context size nowadays which may cause trouble. Maybe better to make some tests and see what happens with memory usage based on different context size.

To be clear I'm not removing the ability to set the context size if you want to!
This is just removing the redundant setting of the context size in all the examples. Since you can load any model you like into the examples that context size was almost always wrong and an extra bit of unnecessary complexity in sample code which should be as simple as possible.

I think this is a good idea.

You can remove it from SpeechChat.cs as well:

As I have expected when running an example with a recent model the example crashes with:
ggml_backend_cuda_buffer_type_alloc_buffer: allocating 49152.00 MiB on device 0: cudaMalloc failed: out of memory
Fatal error. System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.

This is a very small 3GB model, but the huge context size the model can handle crashes the model.
It is not a good idea to remove the context size from the examples because when people are trying out the library they will just leave because they will think that it is garbage. The context size should be set to the value the example needs and a comment may be placed to explain it.

50GB of context space! What's the model?
I think you're right though, we'll probably need to add context size back in but with a comment on it saying that it's not necessary and that leaving it unset will use the default value from the model.
We could also do with handling that error better, but I don't think that's possible without changes to llama.cpp (making the APIs fallible).

Phi-3 128K context size (but small model), recently came out. This will be the trend, increasing context size...

I had a feeling it might be Phi-3 😆

Saying that it is not necessary to set is not a good idea either

The reason I removed this originally was because in most cases you don't want to set it, and yet almost every snippet of code anyone ever pastes does include it. Unfortunately that's the power of example code, whatever you put there will be what everyone uses even if it's not quite right! In general you should only be setting ContextSize when you have a specific reason to (e.g. default model context is huge, as in your case).
e.g. a few recent examples:

this question on Discord
https://github.com/SciSharp/LLamaSharp/issues/686
https://github.com/SciSharp/LLamaSharp/issues/622
https://github.com/SciSharp/LLamaSharp/issues/606#issuecomment-2002558470

Models with long context-length will be more and more popular, but I don't think there're many users using  it with only CPU. I can't imagine how slow it will be! The key point is still the GPU memory instead of CPU memory. To solve the problems from the root, there are two ideal approaches.

Automatically set the ContextSize and GpuLayerCount based on the information of model and machine.
Catch the error when the memory is not enough.

Unfortunately, there're problems for both of them.

Though we could calculate the approximation size of the memory needed the model will use based on model info, ContextSize and GpuLayerCount, it's hard to get the free GPU memory at multiple platforms. AFAIK there's no such .NET library for it.
llama.cpp just doesn't support it, though their core team had already be aware of it. https://github.com/ggerganov/llama.cpp/issues/4315

The setting is actually important for both examples and applying to production environment. For examples, I think it's okay to take the first approach above. We could let user input a number which indicates the GPU memory in GiB. Then we could try to automatically set the ContextSize and GpuLayerCount, using a conservative strategy to avoid OOM. Though users are still required to input a value themselves in this way, it's much more easy for users to understand. For production application, however, I haven't had any good idea about it yet. :(

Yeah it's a bit of a usability pain-point for llama.cpp that it doesn't have some way to determine the best split of memory and context size etc. I've seen a lot of discussions over on the main repo about automatically trying to do it, and every time they decide it's too complex to tackle.

Instead, we should return an error when an allocation fails. Let's fix that first, and then we can think about a memory estimation API.

I really hope this happens! llama.cpp has way too many APIs that just crash the entire application with an assertion failure. At least if you could detect an allocation failure you could just have a loop shrinking context size until it fits!

Back to the issue at hand - how about adding context size back into all of the examples something like this:
var parameters = new ModelParams(Program.modelPath)
{
    // Set the maximum amount of tokens the model can hold in memory. If this is set too high it will cause
    // llama.cpp to crash with an out-of-memory error. If you do not set this parameter the default context
    // size of the model will be used.
    ContextSize = 1024,

    // Set how many layers are moved to GPU memory. Setting this parameter too high will cause llama.cpp
    // to crash with an out-of-memory error.
    GpuLayerCount = 80
};

That way these are no longer just magical numbers hardcoded into the example, but they teach the user what they mean and how to use them to solve problems (too few tokens, too much memory).

I've seen a lot of discussions over on the main repo about automatically trying to do it, and every time they decide it's too complex to tackle.

If we don't pursue maximizing the GPU memory utilization but somewhat lower number such as 80%, things will be easier. However, unless we find a way to get the free GPU memory size, it could be only a toy in our examples. 😞

adding context size back into all of the examples something like this

I agree. We should also tell the users what will happen if their conversation with LLM exceeds the context length.
var parameters = new ModelParams(Program.modelPath)
{
    // Set the maximum amount of tokens the model can hold in memory. If this is set too high it will cause
    // llama.cpp to crash with an out-of-memory error. If your conversation with the LLM exceeds the context length,
    // some early memory in the model will be dropped. If you do not set this parameter the default context
    // size of the model will be used.
    ContextSize = 1024,

    // Set how many layers are moved to GPU memory. Setting this parameter too high will cause llama.cpp
    // to crash with an out-of-memory error.
    GpuLayerCount = 80
};

First of all Context size must be set for all applications based on the needs of the application! Some applications need only 100 tokens (e.g., instruction based short answer), but some other need 8000 tokens (e.g., chat), etc. This, in order to optimize memory use and in order to optimize speed. The context size must thus be set by the user case by case.

GPU will always have priority because it is just several times faster than CPU. There are use cases for CPU only application, for example, on small devices (smartphone).

It is possible to have an automatic load balancing of the (GPU) memory use. I am doing it. llama.cpp could do it, but we can also do it. You need to take into account several variables, free GPU memory at the moment you need to do inference, the available GPU's with their available memory (total memory of all devices), the speed of the GPU's (for optimizing the loading - first use the best GPU), etc... I am doing already all of this, thus it is possible.

Estimating the needed GPU memory per model is possible (we just need to overestimate it). The size of the model is already a possible estimate with an extra buffer added, but this can still be more precise. Since we have the option to distribute the layers to GPU's and CPU's it does not matter if the estimate is not exact. Knowing the free GPU memory at all times is possible on Windows, I do not know about other platforms, but with the cuda toolkit it should be possible everywhere.

Until we do not have automatic load balancing in the library (as an option) we need to explain the users that setting the context size and all GPU parameters is very important (for the above reasons). We should also add to the example all other parameters needed for this and not only GpuLayerCount (split mode...)! An example must be comprehensive to help new users to understand how to use the library.

GPU will always have priority because it is just several times faster than CPU. There are use cases for CPU only application, for example, on small devices (smartphone).

Yes, but for long context (> 16K) I'm afraid few users will use it with only CPU. Smartphone has been much more powerful on  computation than 10 years ago. With OpenCL and Vulkan it could benefit from GPU computation, so smartphone != cpu-only.

It is possible to have an automatic load balancing of the (GPU) memory use. I am doing it.

Estimating the needed GPU memory per model is possible (we just need to overestimate it).

It will be great if we could have these features! I'm looking forward to your work!

What I meant Rinne is that I am doing load balancing of the GPU in my project. I will not start a new thread/work about that in LLamaSharp because you and Martin are working on the scheduling and batching already which at the end when merged should include GPU load balancing also. Since GPU load balancing is relatively simple compared to scheduling and batching I am sure that you will be able to include it. If you need any info, then please ask.
Short summary of GPU load balancing:

check the free GPU memory (all GPU's)
check total available GPU memory (all GPU's)
estimate the needed GPU memory for the model inference (model+context). Estimation can be simply done by using the model size on disk + an estimate for context use (just test a model with a few context sizes and you will see how much, for example, each 1K context size uses) + small buffer.
decide if GPU can be used
decide if full offload is possible without splitting layers between GPU's and CPU
decide how many layers can be offloaded to all available GPU's (this can be simply calculated from the above information)
set LLamaSharp GPU parameters according to the above

@zsogitbe One of the biggest problems for me is the lack of a library to do the following things you mentioned.

check the free GPU memory (all GPU's)
check total available GPU memory (all GPU's)

Is there any cross-platform library to achieve them?

I don't think it's something that would be covered by scheduling batched work - instead I think what you're describing would happen first (model loading). Batching works within the constraints of a pre-established KV cache laid down by the loading process. That means you can work on it indepenently of any other work going on.
I think you'd want to be automatically determining values for these parameters in the IModelParams:

MainGpu
SplitMode
GpuLayerCount
TensorSplits

And perhaps also these context params:

ContextSize
TypeK
TypeV
NoKqvOffload

If you do manage to come up with a way to do this it would be great. The llama.cpp project have consistently written it off as too complex a problem to solve, but it would be a huge usability enhancement if it could be done!
Personally I'd approach this by not even automating the process yet. Just design an algorithm to decide on those 4 model values and plug the numbers in by hand to see how well it works. That's the quickest way to validate the idea.

@zsogitbe One of the biggest problems for me is the lack of a library to do the following things you mentioned.

check the free GPU memory (all GPU's)
check total available GPU memory (all GPU's)

Is there any cross-platform library to achieve them?

@AsakusaRinne, I am sorry, but I do not work multiplatform. My solution is Windows only, and I told Martin the way how to do this on Windows.
If you want to do it multiplatform, then you will need to have a different solution. I advice you to look at the https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator project. You will need the C++ library and then create a C# bridge to get the info. Since you are probably also a game developer as Martin I guess that you know Vulkan. I don't know Vulkan.

I think you'd want to be automatically determining values for these parameters in the IModelParams:
MainGpu
SplitMode
GpuLayerCount
TensorSplits

@martindevans, yes this is exactly how I do it on Windows and how I tried to explain it above!


Can't download arangodb for ubuntu

I'm trying to download ArangoDB 3.x for ubuntu, but I'm getting:
Forbidden

You don't have permission to access /repositories/arangodb31/xUbuntu_16.04/amd64/arangodb3-3.1.8-1_amd64.deb on this server.

Apache/2.4.7 (Ubuntu) Server at www.arangodb.com Port 443

@venomlust there was a network error on our file service provider. It is fixed now.


Correct markdown headers in unit-testing.md

@seth10 Can you resolve conflicts? Would be happy to merge your formatting changes.

Closing PR. Changes have been merged.

Looks like @ghchinoy beat me to it in de17f3e!


Sync correct version on rubygems

When you publish on rubygems, please ensure code parity with github versions. Thanks!

Hi there, not sure what you mean but if it's a missing v3.0.0 it's deleted now.
Thanks.


Made wagtail-embedvideos compatible with current Wagtail version (1.9)

With this changes wagtail-embedvideo is NOT compatible with versions <= 1.6!

There is only one migration now;
Changed the structure of "AbstractEmbedVideo", because "TagSearchable" is not used anymore;

👍

This update doesn't apply all new features for documents and images, we have development this features, poorly we know, in our branch. You close it.

Hi @SalahAdDin,
Right now our main objective was to update wagtail_embedvideos to be compatible with the latest version of Wagtail (1.9).
Soon we will expand this package to use every new feature that was developed for documents and images.

Can you use my fork? I already did all of this, you can take as base and over it make other changes.


NodeJS: fix bug where hyphenated package names cause an error.

Resolves #1290

Codecov Report

Merging #1466 into master will increase coverage by <.01%.
The diff coverage is 87.5%.

@@             Coverage Diff              @@
##             master    #1466      +/-   ##
============================================
+ Coverage     87.17%   87.17%   +<.01%
  Complexity     4041     4041
============================================
  Files           387      387
  Lines         16252    16253       +1
  Branches       1774     1774
============================================
+ Hits          14167    14168       +1
  Misses         1620     1620
  Partials        465      465

Impacted Files
Coverage Δ
Complexity Δ

...pi/codegen/viewmodel/testing/MockCombinedView.java
100% <ø> (ø)
17 <0> (ø)
:arrow_down:

...api/codegen/viewmodel/OptionalArrayMethodView.java
100% <ø> (ø)
9 <0> (ø)
:arrow_down:

...rmer/nodejs/NodeJSGapicSurfaceTestTransformer.java
100% <100%> (ø)
24 <0> (ø)
:arrow_down:

...codegen/transformer/nodejs/NodeJSSurfaceNamer.java
90.44% <100%> (ø)
74 <1> (ø)
:arrow_down:

...n/transformer/DynamicLangApiMethodTransformer.java
99.31% <100%> (ø)
30 <0> (ø)
:arrow_down:

...transformer/nodejs/NodeJSPackageMetadataNamer.java
81.81% <80%> (ø)
4 <0> (ø)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 57cd7c9...9704759. Read the comment docs.


Multiple arguments / activities?

At the moment the only viable way I see is to support one active and one inactive argument, and leave it to ComponentArrays to combine several arguments if necessary. It could be done by passing a tuple (x, c) instead of the input x. @wsmoses do you think that might get us 90% of the way to where we need to be?
See the following discussions:

#307
https://github.com/LuxDL/Lux.jl/issues/685
https://github.com/SciML/OptimizationBase.jl/pull/54
https://github.com/tpapp/LogDensityProblemsAD.jl/pull/29

That would be an improvement but we need to make sure that the decomposition of the arguments doesn't result in a materialized intermediate variable

Can you give an example and explain why that's a bad thing?

On movile so link chasing is hard but the logprovlemsAD has a thread where
I described why closures were bad. The intermediate variable — whether
tuple struct or closure — will have the same impact if not expanded
On Fri, Jun 7, 2024 at 12:40 AM Guillaume Dalle @.***>
wrote:

Can you give an example and explain why that's a bad thing?
—
Reply to this email directly, view it on GitHub
https://github.com/gdalle/DifferentiationInterface.jl/issues/311#issuecomment-2153513788,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAJTUXEDEIHL5QQ54Z6Z2SDZGDQMNAVCNFSM6AAAAABI5QNRGKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNJTGUYTGNZYHA
.
You are receiving this because you were mentioned.Message ID:
@.***>

Yeah I added that link above, I guess I'm just wondering what you mean by intermediate variable in this setting. I don't see why grouping multiple arguments into one or unpacking them would require a closure

and leave it to ComponentArrays to combine several arguments if necessary

Not sure, this would work for my use case. Typically, I have a function logp(data, model, parameters). data is constant and usually a vector of named tuples of whatever. model is usually a structure that contains configuration and state variables. The state variables need to be shadowed, as they change, but I don't care about their derivatives. The only thing I care about are the partial derivatives of logp with respect to parameters. Maybe it would make sense to follow Lux style and separate the model into two parts: constant configuration and non-contant state, which would result in logp(data, config, state, parameters). Would it really make sense to put data, model or data, config, state into a ComponentArray? How much would it limit the way data, config and state can be written?
To be clear: I don't care, if DifferentiationInterface covers my use case or not. I'm fine with using Enzyme directly. I was just exploring this direction and thought I report back my experience :slightly_smiling_face: .

Thanks for your feedback! I imagine the following people may also have opinions:

@avikpal for Lux
@oxinabox for Zygote/Diffractor
@willtebbutt for Tapir
@adrhill forever <3

As for your specific use case, we might have to make a distinction between "active / inactive" and "shadowed (mutated) / not shadowed"?

No strongs views on my end -- activity analysis isn't presently really a thing in Tapir.jl, so happy to take the lead from others.

Also note that the reason I'm mentioning ComponentArrays is because the original plan was never for DI to support an arbitrary number of arguments: that's what we agreed AbstractDifferentation would try to achieve based on DI (ping @mohamed82008). Hence I feel like disguising multiple arguments into one (x) or two (x and c for constant) is the best we can hope for in the short term.

Forcing multiple arguments into a single struct — even two structs (one active one constant) will still result in the performance issues I outlined in the log density link.

Which is why I'm suggesting to force all the active arguments together into X, and all the inactive arguments together into c

Yes but I’m saying that will still have performance oenalties — I’d say you were condensing two active args into one

Copying my write up from before here — most of the arguments apply equally to non differentiated code. Thus the “wrap multiple args together forcing indirection” be it through a closure struct or whatever else will cause perf difficulties of the undifferentiated and differentiated code and possibly prevent differentiation.
Essentially replace the use of the word closures below with struct or whatever else (though box will only come up in a closure iirc).
Closures can present several issues both for Enzyme's ability to AD something, the performance of differentiated programs, and even the performance of programs not being differniated.
I'll write up a few concerns below, in no particular order:

Type stability. Since you mention this already I won't dive more into.

Boxing. If a variable is modified in a closure, it may end up being boxed by julia. In essence Julia will force it to be captured by reference. Even if the function is otherwise type stable, a box is essentially a type unstable any type, even it otherwise would be type stable. Furthermore an unnecessary box will add a level of indirection which hurts performance (see below)

julia> function h(x,y)
           function g(z)
               x[y] *= 2; y+=1; return z*x[y]
           end
           return g
       end
h (generic function with 1 method)

julia> h([1,2,3],1)
(::var"#g#3"{Vector{Int64}}) (generic function with 1 method)

julia> fieldtypes(typeof(h([1,2,3],1)))
(Vector{Int64}, Core.Box)

julia> fn = h([1,2,3],1)
(::var"#g#3"{Vector{Int64}}) (generic function with 1 method)

julia> fn(1)
2

julia> fn(1)
3

Indirection / Aliasing: Indirection can cause overhead. For example consider an array of ints vs an array of Ref{Int}  (notably both being type stable). The first is much faster than the other.

julia> function sumvals(x)
           res = 0
           @simd for i in 1:length(x)
             res += @inbounds x[i]
           end
           return res
       end^C

julia> function sumvals_ref(x)
           res = 0
           @simd for i in 1:length(x)
             res += @inbounds x[i][]
           end
           return res
       end
sumvals_ref (generic function with 1 method)

julia> array = collect(1:10000);

julia> rarray = collect(map(Ref, 1:10000));

julia> @btime sumvals(array)
  595.877 ns (1 allocation: 16 bytes)
50005000

julia> @btime sumvals_ref(rarray)
  3.534 μs (1 allocation: 16 bytes)
50005000

There are many reasons for this. First of all there's one less load instruction per loop in the top (the first loop needs to load the data from the array, the second loads a ref from the array, then the float from the ref pointer). Secondly, this data is layed out consecutively. This allows vector instructions to apply on the first, whereas it is impossible in the second (vector loads/stores require consecutive data). Secondly you get much better cache performance. In the second case you can get a cache miss per iteration. In the first case you get a cache hit only once per cache size. (since a single cache hit loads a consecutive cache size number of elements all at once).
Beyond the extra information, closures can obscure aliasing information. Aliasing is the ability to say that two pointers point to different memory. This is critical for optimization. Consider the following.
@noinline function inner(y)
   @show y
end

function twouse(x, y)
  x1 =  x * 2
  inner(y)
  x2 = x * 3
  return x1 + x2
end

julia> @code_llvm twouse(2, 3)
;  @ REPL[30]:1 within `twouse`
define i64 @julia_twouse_778(i64 signext %0, i64 signext %1) #0 {
top:
;  @ REPL[30]:3 within `twouse`
  %2 = call i64 @j_inner_780(i64 signext %1)
;  @ REPL[30]:5 within `twouse`
; ┌ @ int.jl:87 within `+`
   %3 = mul i64 %0, 5
; └
  ret i64 %3
}

Here the two uses of x get optimized into one, fantastic. However if now we have it go through a pointer (via a Ref)
@noinline function inner_ref(y)
   @show y[]
end

function twouse_ref(x, y)
  x1 =  x[] * 2
  inner(y)
  x2 = x[] * 3
  return x1 + x2
end

julia> @code_llvm twouse_ref(Ref(2), Ref(3))
;  @ REPL[33]:1 within `twouse_ref`
define i64 @julia_twouse_ref_790({}* noundef nonnull align 8 dereferenceable(8) %0, {}* noundef nonnull align 8 dereferenceable(8) %1) #0 {
top:
;  @ REPL[33]:2 within `twouse_ref`
; ┌ @ refvalue.jl:59 within `getindex`
; │┌ @ Base.jl:37 within `getproperty`
    %2 = bitcast {}* %0 to i64*
    %3 = load i64, i64* %2, align 8
; └└
; ┌ @ int.jl:88 within `*`
   %4 = shl i64 %3, 1
; └
;  @ REPL[33]:3 within `twouse_ref`
  %5 = call nonnull {}* @j_inner_792({}* nonnull %1)
;  @ REPL[33]:4 within `twouse_ref`
; ┌ @ refvalue.jl:59 within `getindex`
; │┌ @ Base.jl:37 within `getproperty`
    %6 = load i64, i64* %2, align 8
; └└
; ┌ @ int.jl:88 within `*`
   %7 = mul i64 %6, 3
; └
;  @ REPL[33]:5 within `twouse_ref`
; ┌ @ int.jl:87 within `+`
   %8 = add i64 %7, %4
; └
  ret i64 %8
}

You'll note that now the two loads of x[] are there explicitly and not combined into one. This is because the call to inner could possibly write to the same memory of x, making it possible that it changed the value of x inside the function (even though here we know it doesn't).  Getting rid of the inner function call indeed does let a single load replace both uses.
julia> function twouse_ref2(x, y)
         x1 =  x[] * 2
         x2 = x[] * 3
         return x1 + x2
       end
twouse_ref2 (generic function with 1 method)

julia> @code_llvm twouse_ref2(Ref(2), Ref(3))
;  @ REPL[35]:1 within `twouse_ref2`
define i64 @julia_twouse_ref2_793({}* noundef nonnull align 8 dereferenceable(8) %0, {}* noundef nonnull align 8 dereferenceable(8) %1) #0 {
top:
;  @ REPL[35]:2 within `twouse_ref2`
; ┌ @ refvalue.jl:59 within `getindex`
; │┌ @ Base.jl:37 within `getproperty`
    %2 = bitcast {}* %0 to i64*
    %3 = load i64, i64* %2, align 8
; └└
;  @ REPL[35]:4 within `twouse_ref2`
; ┌ @ int.jl:87 within `+`
   %4 = mul i64 %3, 5
; └
  ret i64 %4
}

Introducing a closure means that there is significant worse aliasing information since each of the captured variables is behind a pointer, and any aliasing information (e.g. variables x and y don't overlap) is now lost.
This makes the regular code slow, and can also make the genereated AD code much slower. As the various enzyme papers have shown running optimization before AD makes a compound impact on the generated code. In this case worse aliasing information may mean that Enzyme has to cache a lot more data, dramatically slowing things down and possibly preventing previously differntiable code from being able to be run at all (if available memory is insufficient).

Mixed activity. Just like how closures can introduce type instability, they can introduce activity instability. In this case the question is whether a variable is differentiable or not. If one stores a constant vector into a struct alongside other duplicated vectors that will create a mixed activity error (since there is no derivative vector for that which can be defined consistently -- in the worst case leading to wrong results if done incorrectly, so instead here we error). Notably the user can enable runtime activity to work around this like described in our docs, but this will come with a performance penalty (again again provide some combination of previously working code no longer working and slowdown).

Unnecessary differentiation. Multiple arguments allows you to mark some data as non-differentiable, which feeds an analysis (activity analysis) which determines the minimum number of instructions/values to differentiate. Combining a bunch of variables together (e.g. in a struct/closure), and in particular no longer specify some as being non-differentiated, can prevent the analysis from proving some code doesn't need to be differentiated. If that code isn't presently supported (e.g. due to a type instability in the function being differentiated), you'll now error

Thanks for the clarification, I didn't realize that some of the issues with closures also apply to indirection.
Do you have suggestions for a middle ground that would improve performance on enzyme without being as permissive as enzyme itself? I really am open to hearing your ideas cause mine are running out.
The trouble is that everything I allow for the enzyme backend I also have to support and test for the other 12 (and for combinations thereof in the case of second order). Except that for most other backends, this level of generality will be an outright nuisance. For instance, ForwardDiff and ReverseDiff only support single argument functions.

At the moment Enzyme has lots of momentum, so it makes sense for me to go a little out of my way to improve support for you. But three years ago it was Zygote everywhere, and maybe three years from now it will be Tapir everywhere, who knows. Either project may run out of funding or volunteer manpower. In case changing backends becomes necessary, people will be happy to have an interface that is designed to be a good compromise, instead of fully customized to Enzyme's very specific needs. That's why I'm kinda hesitant to go in all the way on activities and shadows and stuff.

In case changing backends becomes necessary, people will be happy to have an interface that is designed to be a good compromise

It's worth reiterating that this goes both ways. For me, part of the motivation for DI and the DifferentiationInterfaceTest.jl tests and benchmarks has always been to make the transition from Zygote to Enzyme on my own projects as simple as possible. And in my eyes, swapping Zygote.gradient(f, x) for gradient(f, AutoZygote(), x) and finally gradient(f, AutoEnzyme(), x) achieves this beautifully.
To add more background to this comment:
The original idea of DI was to implement the "intersection" of functionality between AD backends, while AbstractDiff would build on DI and extend it to the "union" of functionality. Guillaume has managed to squeeze a lot of backend-specific functionality into a unified interface by introducing the extras argument and operator preparation, but by supporting multiple arguments and Enzyme-specific functionality, we might end up exporting an "interface" that isn't fully compatible with all backends.

Fundamentally it's going to be a question of trade offs.
Either you choose to have less/simpler code in DifferentiationInterface and consequently worse support and/or performance for various AD package backends. Obviously this makes it easier to get things up and running, and perhaps an easier entry point for users. However, the trade off you make here is that it'll be harder for bigger / more workload-critical users to adopt the package. For example, if a codebase already successfully makes use of a given AD package, its going to be much more difficult to have them swap to DI if the use of DI itself (in contrast to direct use of the AD package) prevents code from being differentiated, or makes the differentiated code slower. I'm struggling to find the right link, but I recall that closures + Zygote was a significantly reason it didn't take off. If any interface were causing such a "cannot differentiate and/or slower perf issue" for any backend that doesn't exist on the original tool, I would likely recommend the user directly use a tool and file a bug on the interface in the meantime.
On the other hand, you could have more/specialized code in DI. Obviously this means more code/complexity here, in exchange for more potential users.
Either way, happy to try to support you.
Side note, all the code examples I gave above weren't even differentiated -- so at least the performance penalties I mention would be true of most julia code regardless of additional AD.

Is there a milestone for the interface for non-differentiated inputs?

Not at the moment unfortunately. My current priority is to make the single-input part good enough that it can make its way into most of the SciML ecosystem (in particular Optimization and DifferentialEquations). Once I get that right, I'll turn to multiple arguments, but it is an insane amount of code to write and I'm just one dude ^^
If you want to help speed things up, the most useful contribution would be suggestions on how I could achieve this without 1) completely transforming the existing API and 2) turning into an Enzyme front end. What's hard is that a solid 2/3 of backends don't support this at all, so I need to make it work for them with a closure while allowing top performance for more sophisticated ones like Tapir and Enzyme.

Also note that the reason I'm mentioning ComponentArrays is because the original plan was never for DI to support an arbitrary number of arguments: that's what we agreed AbstractDifferentation would try to achieve based on DI (ping @mohamed82008). Hence I feel like disguising multiple arguments into one (x) or two (x and c for constant) is the best we can hope for in the short term.

I don't think aiming for supporting multiple arguments in DI.jl for backends that don't natively support multiple arguments will be too useful. It might be better to document backend limitations and ask users to define their own closures. If DI.jl is to implement multiple argument support, we might as well deprecate AD.jl and go with DI.jl for everything. I personally have no issue with that if DI eventually supported everything AD currently supports. I would have preferred AD.jl to be the one package but development here is clearly much more active than in AD so maybe that's better for the ecosystem as a whole. Alternatively, we could keep the arrangement we agreed on but you contribute the multiple argument support to AD.jl. I think everyone here has write access to AD.jl and can contribute there instead. It's really up to you where you want to contribute this code. If we decide to make DI.jl the new AD.jl, then perhaps we should also figure out a way to give some credit to the AD.jl contributors and original designers.

@mohamed82008 I don't think I have write access. I left comments on the AD.jl wip Enzyme PR a while ago, I'd be happy to do some minor fixups but don't have permission to xD

@mohamed82008 I don't think I have write access.

Fixed :)

Thanks for your answer Mohamed 🙏 A lot has happened in DI over the last couple of months, and it's already being used by big ecosystems like SciML and JuliaSmoothOptimizers. As a result, I honestly think it would be best for the ecosystem to put all of this stuff in one place, and aim for multiple argument support in DI as well (either for a few backends or for all with a closure fallback, not sure yet).
Putting myself in your shoes, I understand how frustrating that can be, and we definitely need to give credit where credit is due. DI could not exist without AbstractDifferentiation, which was literally the blueprint I based my code on. How about we put both the AbstractDifferentiation paper and the future DI paper in the CITATION.bib here?

How about we put both the AbstractDifferentiation paper and the future DI paper in the CITATION.bib here?

That might be a reasonable way to give credit to the paper's authors but not all of the contributors, e.g David. I suppose there is no perfect solution. But the CITATION.bib suggestion + a README comment would be the best we can do. I am ok with that.

That might be a reasonable way to give credit to the paper's authors but not all of the contributors, e.g David.

Yeah I agree that is a problem, but how do you even cite contributors that are not authors, except by citing the repo itself? Which we can also recommend btw

What do you think of the wording here?
https://github.com/gdalle/DifferentiationInterface.jl?tab=readme-ov-file#citation

What do you think of the wording here?
https://github.com/gdalle/DifferentiationInterface.jl?tab=readme-ov-file#citation

Looks good. Thanks. I think the reference to AD.jl as "the inspiration" sufficiently gives credit to the AD.jl contributors and the citation sufficiently gives credit to the AD.jl designers and paper authors. I don't think we can do much better than that. Asking users to cite the AD.jl repo as well may be a bit much but if David prefers that, it would be nice to do as well. I am happy with the current status.

The gist is that DI is a math-inspired interface: functions only have one input and one output, everything else is either a parameter of the function $f_p$ or a cache

I don't fully understand what you mean by "care about initial values"?

 When you provide a parameter, you do care

This is me trying to sum up the discussion and propose a new API.

I like this API and I believe that it is near optimal given the constraint that DI should be the "intersection of all AD packages" (a job which it does really well right now). Beyond enabling the use of Enzyme, your API would also facilitate some additional performance optimization; i.e. to work around the slow closure bug. I don't think that there is a much more flexible API hat all packages can support. This also makes it easier to support future packages quickly in DI without them having to develop a large set of features before joining the DI family.
The only alternative I see is to "wait for Enzyme to be able to handle closures reliably" and stick to single-arg API; the simplicity of which is certainly appealing. If "one day" :tm: the slow closure bug is gone and Enzyme is able to handle closures more reliably, I would certainly prefer the single-arg + closure route.
As for naming bike shedding: I feel like users with ML background would expect the function to be differentiable in params. I think constant or context would be clearer.

Something we might want to consider as well when discussing future API changes are "auxiliary" outputs à la JAX:

has_aux – Optional, bool. Indicates whether fun returns a pair where the first element is considered the output of the mathematical function to be differentiated and the second element is auxiliary data. Default False.

As of the newly-released v0.6, users can specify an arbitrary number of "context" arguments which are not differentiated. At the moment, there is only one context type named Constant, but I'll add a second one named Cache soon enough.
The syntax looks as follows:
gradient(f, backend, x, Constant(c1), Constant(c2))

There still cannot be more than a single active (differentiated) argument x, but I think this goes most of the way towards what is needed in the ecosystem (especially since many backends support non-array inputs in practice). Closing this issue in favor of more focused ones.


[Sonarr] ...

Extension
https://www.raycast.com/Aayush9029/sonarr
Description
Receiving "Unexpected end of JSON input" error
Steps To Reproduce
Set auth to HTTP or HTTPS (both give same error when testing)
Input API key (regenerated it multiple times for testing)
Verified port (changed for testing as well)

Current Behaviour
No response
Expected Behaviour
It should show the current shows based upon selection in extension

@raycastbot assign @j3lte

@raycastbot close this issue


naming of "special files"?

Is there a more precise term than special files for these special files?
The original name was "Readme-files", but I think this is insufficient to finding them, and gives currently only results for README and Changelog.

Agreed, this was the first repository that came up on my google search results.


Initial build image

Dockerizing application with alpine 3

Thanks @trants! The merge will be carried out after review is complete,


Switch github source to https

Switch github source to https, to prevent bundler warnings.
Preserved same revisions of gems installed from github.
Same pattern is used in Rails 5.1 generated Gemfiles.

looks good but could you explain why https:// better than git://?

Bundler issues a warning when using git:// protocol, since it's not encrypted.
https://github.com/bundler/bundler/blob/master/lib/bundler/source_list.rb#L147


What the frick were those sprites supposed to be

What the frick were those sprites supposed to be

/shrug


运行bash scripts/generate.sh --input-source interactive --sequential-initialization报错

你好，我的环境是6*A6000(48G),运行上面命令报错如下：
raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:
/home/edcuser/models/glm/GLM-130B-main/generate.py FAILED
Failures:
[1]:
time      : 2023-03-14_11:22:52
host      : localhost.localdomain
rank      : 7 (local_rank: 7)
exitcode  : 1 (pid: 160925)
error_file: <N/A>
traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
Root Cause (first observed failure):
[0]:
time      : 2023-03-14_11:22:52
host      : localhost.localdomain
rank      : 6 (local_rank: 6)
exitcode  : 1 (pid: 160924)
error_file: <N/A>
traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
请问这个问题怎么解决？

我也出现了同样的问题，请问是如何解决的？

我在4张4090gpu上，也出现了类似的问题，请问是如何解决的？

看报错信息前面提示的哪行代码报错，根据报错行提示修改就好了

WARNING:torch.distributed.run:

Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.

WARNING: No training data specified
WARNING: No training data specified
WARNING: No training data specified
using world size: 4 and model-parallel size: 4

padded vocab (size: 150528) with 0 dummy tokens (new size: 150528)
WARNING: No training data specified
initializing model parallel with size 4
Set tokenizer as a icetk-glm-130B tokenizer! Now you can get_tokenizer() everywhere.
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: -7) local_rank: 0 (pid: 4660) of binary: /home/sukibean/bin/python
Traceback (most recent call last):
File "/home/sukibean/bin/torchrun", line 8, in
sys.exit(main())
File "/home/sukibean/.local/lib/python3.8/site-packages/torch/distributed/elastic/multiprocessing/errors/init.py", line 346, in wrapper
return f(*args, **kwargs)
File "/home/sukibean/.local/lib/python3.8/site-packages/torch/distributed/run.py", line 794, in main
run(args)
File "/home/sukibean/.local/lib/python3.8/site-packages/torch/distributed/run.py", line 785, in run
elastic_launch(
File "/home/sukibean/.local/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 134, in call
return launch_agent(self._config, self._entrypoint, list(args))
File "/home/sukibean/.local/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 250, in launch_agent
raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:
============================================================
/home/sukibean/work/src/python/GLM-130B-main/generate.py FAILED

Failures:
[1]:
time      : 2023-06-08_02:34:04
host      : 3046826faa76
rank      : 1 (local_rank: 1)
exitcode  : -7 (pid: 4661)
error_file: <N/A>
traceback : Signal 7 (SIGBUS) received by PID 4661
[2]:
time      : 2023-06-08_02:34:04
host      : 3046826faa76
rank      : 2 (local_rank: 2)
exitcode  : -7 (pid: 4662)
error_file: <N/A>
traceback : Signal 7 (SIGBUS) received by PID 4662
[3]:
time      : 2023-06-08_02:34:04
host      : 3046826faa76
rank      : 3 (local_rank: 3)
exitcode  : -7 (pid: 4663)
error_file: <N/A>
traceback : Signal 7 (SIGBUS) received by PID 4663
Root Cause (first observed failure):
[0]:
time      : 2023-06-08_02:34:04
host      : 3046826faa76
rank      : 0 (local_rank: 0)
exitcode  : -7 (pid: 4660)
error_file: <N/A>
traceback : Signal 7 (SIGBUS) received by PID 4660

WARNING:torch.distributed.run:
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
WARNING: No training data specified WARNING: No training data specified WARNING: No training data specified using world size: 4 and model-parallel size: 4

padded vocab (size: 150528) with 0 dummy tokens (new size: 150528)
WARNING: No training data specified
initializing model parallel with size 4
Set tokenizer as a icetk-glm-130B tokenizer! Now you can get_tokenizer() everywhere.
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: -7) local_rank: 0 (pid: 4660) of binary: /home/sukibean/bin/python
Traceback (most recent call last):
File "/home/sukibean/bin/torchrun", line 8, in
sys.exit(main())
File "/home/sukibean/.local/lib/python3.8/site-packages/torch/distributed/elastic/multiprocessing/errors/init.py", line 346, in wrapper
return f(*args, **kwargs)
File "/home/sukibean/.local/lib/python3.8/site-packages/torch/distributed/run.py", line 794, in main
run(args)
File "/home/sukibean/.local/lib/python3.8/site-packages/torch/distributed/run.py", line 785, in run
elastic_launch(
File "/home/sukibean/.local/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 134, in call
return launch_agent(self._config, self._entrypoint, list(args))
File "/home/sukibean/.local/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 250, in launch_agent
raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:
/home/sukibean/work/src/python/GLM-130B-main/generate.py FAILED

Failures:
[1]:
time : 2023-06-08_02:34:04
host : 3046826faa76
rank : 1 (local_rank: 1)
exitcode : -7 (pid: 4661)
error_file: <N/A>
traceback : Signal 7 (SIGBUS) received by PID 4661
[2]:
time : 2023-06-08_02:34:04
host : 3046826faa76
rank : 2 (local_rank: 2)
exitcode : -7 (pid: 4662)
error_file: <N/A>
traceback : Signal 7 (SIGBUS) received by PID 4662
[3]:
time : 2023-06-08_02:34:04
host : 3046826faa76
rank : 3 (local_rank: 3)
exitcode : -7 (pid: 4663)
error_file: <N/A>
traceback : Signal 7 (SIGBUS) received by PID 4663
Root Cause (first observed failure):
[0]:
time : 2023-06-08_02:34:04
host : 3046826faa76
rank : 0 (local_rank: 0)
exitcode : -7 (pid: 4660)
error_file: <N/A>
traceback : Signal 7 (SIGBUS) received by PID 4660

generate.py报错，但是没有明确的信息

OMP_NUM_THREADS

请问最后解决了吗？遇到了一样的问题


"Remote login enabled" rollback fails (requires Full Disk Access)

systemsetup -setremotelogin on requires full disk access and fails. Need a workflow to acquire rights by the helper (or just create education for UI actions)

As per the Apple policy, it's impossible to have a workflow to acquire rights beside having the use granting access to the helper. Fortunately, once the helper tries to access the protected system files it's getting prepopulated in the Security / Full Disk Access security settings, which make it relatively easy to do. For now, pending a better way, this is indicated as localized comments in the CLI preview of the remediations.


ci(cdn_publish.sh): fix the version tag expression

Description
Fixes the expression used to capture git version tag

:tada: This PR is included in version 2.2.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Why doesn't the returned Lat Lon not find street view?

In kludgy.sh, I tried switching from ID to lat/long.
Just doesn't seem to work. I was hoping that would allow me to use any returned pano, vs just Google ones (see #4).
For example, if I remove the check for "Google" here:
https://github.com/mhulse/kludgy/blob/fa39840b0566a9b25b39e9fd8141657ccabb4cd8/kludgy.js#L50
And if you remove the first two args below, and replace with "lat,lon" (see docs for that lib):
https://github.com/mhulse/kludgy/blob/fa39840b0566a9b25b39e9fd8141657ccabb4cd8/kludgy.sh#L71-L78
Unfortunately, pano not found when doing this.
I suspect it's due to user panos. I need to test by keeping the "Google" check and seeing if it just doesn't like lat/lon in general. If so, then I might suspect it's due to precision in the numbers?

The real issue was due to user panos. Official Google panos worked, but user panos did not due to different URLs used for the tile images.
This has been fixed/solved in v2 codebase.


Add tf-init-booster tool

Add lint to tf-init-booster tool

Hello @shuaibiyy , any chance to merge this suggestion?

Hi @hayorov, very cool tool. Thanks for adding.


Remember scroll position in editor outline

When returning to the entry outline inside the editor, currently the scroll position of the list is at the top. It would be nice to remember the last scroll position such that the last edited page is easy to find.

Dom element in sidebar with overflow auto:
https://github.com/codevise/pageflow/blob/master/app/views/pageflow/entries/edit.html.erb#L16
Singleton top level view which could in turn create a view which controls the scroll position:
https://github.com/codevise/pageflow/blob/master/app/assets/javascripts/pageflow/editor/views/editor_view.js
PageItemView could trigger an ensurevisible event if its page is the active page.

Fixed in #405


Fix automodule

Pull Request Test Coverage Report for Build 25

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 83.271%

Totals

Change from base Build 23:
0.0%

Covered Lines:
224

Relevant Lines:
269

💛  - Coveralls


python3Packages.torch.tests: MPS tests for darwin

Things done
Depends on #351778. Add new test to pytorch for Darwin platform. It tests CPU without torch.compile decorator, and MPS if MPS device is present.
Modify pytorch tests so it also tests MPS during CPU tests on Darwin. I'm not sure is this the right way to do this but at least this is a good starting point for discussion. I was also unsure do the tests work for other platforms. MPS is built for macOS by default if #351778 gets merged in its current state, so it does not need specific torchWithMps build.

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[X] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[X] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Also got bitten by nixfmt check, fixing that made the diff quite hard to read

a lot of the diff appears to be just formatting changes. let's try to keep the git blame clean

I re force-pushed the whole thing and now the diff only shows the lines I changed, however I now get complaint from the nix formatting checks.

a lot of the diff appears to be just formatting changes. let's try to keep the git blame clean

I re force-pushed the whole thing and now the diff only shows the lines I changed, however I now get complaint from the nix formatting checks.

Made another commit on top to clearly show which part is formatting and which is not

Actually are you doing merge commits nowadays? Should I make separate PR for the .git-blame-ignore-revs in case the commit hash gets mangled

Actually are you doing merge commits nowadays

Yes we do, I think it's ok to keep it in 1 PR

I changed this now so there's a separate test for Darwin. I think in the end it is cleaner this way


Minimum circle radius

Is it possible to prevent a user from creating a circle smaller then X meters?

You could try putting a custom filter on the radius field
<ng-map center="37.09024, -95.712891" zoom="4" mayTypeId="TERRAIN">     <shape name="circle" ng-repeat="circle in vm.circles" no-watcher="true"       stroke-color="#FF0000"       stroke-opacity="0.8"       stroke-weight="2"       fill-color="#FF0000"       fill-opacity="0.35"       center="{{circle.position}}"       radius="{{circle.radius | minRadius : 20)}}">     </shape>   </ng-map>
and something like:
app.filter('minRadius', function(){     return function(input, min) {       return input < min ? min : input;     };   });
Caveat: I haven't tested this at all, and I'm not sure if NgMaps will update the radius if you do it like this.

That's a great suggestion, but sadly this does not work.
The value in circle.radius is fine, but the UI still lets me re-size it to
smaller than the minRadius.
Op za 13 feb. 2016 om 18:45 schreef Josh<EMAIL_ADDRESS>
You could try putting a custom filter on the radius field

and something like:
app.filter('minRadius', function(){
return function(input, min) {
return input < min ? min : input;
};
});
Caveat: I haven't tested this at all, and I'm not sure if NgMaps will
update the radius if you do it like this.
—
Reply to this email directly or view it on GitHub
https://github.com/allenhwkim/angularjs-google-maps/issues/512#issuecomment-183712532
.

You have to do some trickery with the radius change events then. Can you
create a plunkr with some example code?
On Sun, 14 Feb 2016, 09:26 Hans van den Elsen<EMAIL_ADDRESS>wrote:

That's a great suggestion, but sadly this does not work.
The value in circle.radius is fine, but the UI still lets me re-size it to
smaller than the minRadius.
Op za 13 feb. 2016 om 18:45 schreef Josh<EMAIL_ADDRESS>
You could try putting a custom filter on the radius field

and something like:
app.filter('minRadius', function(){
return function(input, min) {
return input < min ? min : input;
};
});
Caveat: I haven't tested this at all, and I'm not sure if NgMaps will
update the radius if you do it like this.
—
Reply to this email directly or view it on GitHub
<
https://github.com/allenhwkim/angularjs-google-maps/issues/512#issuecomment-183712532
.

—
Reply to this email directly or view it on GitHub
https://github.com/allenhwkim/angularjs-google-maps/issues/512#issuecomment-183860350
.

--
https://echoes.xyz

You can probably use the plunkr on this page.
https://ngmap.github.io/#/!shape.html
Op zo 14 feb. 2016 12:04 schreef Josh<EMAIL_ADDRESS>
You have to do some trickery with the radius change events then. Can you
create a plunkr with some example code?
On Sun, 14 Feb 2016, 09:26 Hans van den Elsen<EMAIL_ADDRESS>wrote:

That's a great suggestion, but sadly this does not work.
The value in circle.radius is fine, but the UI still lets me re-size it
to
smaller than the minRadius.
Op za 13 feb. 2016 om 18:45 schreef Josh<EMAIL_ADDRESS>
You could try putting a custom filter on the radius field

and something like:
app.filter('minRadius', function(){
return function(input, min) {
return input < min ? min : input;
};
});
Caveat: I haven't tested this at all, and I'm not sure if NgMaps will
update the radius if you do it like this.
—
Reply to this email directly or view it on GitHub
<

https://github.com/allenhwkim/angularjs-google-maps/issues/512#issuecomment-183712532

.

—
Reply to this email directly or view it on GitHub
<
https://github.com/allenhwkim/angularjs-google-maps/issues/512#issuecomment-183860350
.

--
https://echoes.xyz
—
Reply to this email directly or view it on GitHub
https://github.com/allenhwkim/angularjs-google-maps/issues/512#issuecomment-183874378
.

There is radius_changed event. You can make use of it.
https://developers.google.com/maps/documentation/javascript/3.exp/reference#Circle
Example;
http://plnkr.co/edit/FBJrp3RKLGZuzr1YCY8F?p=preview

In your plunkr, I can still change the radius to smaller than 4000. So not closed.

angular.module('ngMap').controller("MyCtrl", function($scope){
  this.radiusChanged = function(event) {
    if (this.radius < 4000) {
      this.setRadius(4000);
    }
  }
})


DownloadButton: Fix formatting Windows and Apple

It looks off compared to the Linux side that looks cleaner when you download releases on pcsx2.net

If this code:

9bcfbd0 (#257)
produces this:

163ca13 (#257)

If df76858 (#257)

with 32898b3 (#257)

With 77212b0 (#257)

a224964 (#257)

6898014 (#257)

Seems fine now

With: 076d15d (#257)


Customer registration event in OAuthUserProvider

Refactored some parts of the class as well

ping @claretcrab Can you test it please?

ok

@mmoreram tested, works as expected :D


debug.onDidChangeActiveDebugSession not always fired when active session implicitly changes after old active session terminated

Does this issue occur when all extensions are disabled?: Yes

VS Code Version: 1.94.1 (Universal)
OS Version: Sequoia 15.0.1

Version: 1.94.1 (Universal)
Commit: e10f2369d0d9614a452462f2e01cdc4aa9486296
Date: 2024-10-05T05:44:32.189Z
Electron: 30.5.1
ElectronBuildId: 10262041
Chromium: 124.0.6367.243
Node.js: 20.16.0
V8: <IP_ADDRESS>-electron.0
OS: Darwin arm64 24.0.0
Steps to Reproduce:

Subscribe to debug.onDidTerminateDebugSession and debug.onDidChangeActiveDebugSession
Start two debug sessions.
Click on the second one in the call stack view to make it active.
Press terminate on the second session via the popup button in the call stack view

Correctly receive the terminate event but not always a active session changed event.
When the second session is killed, the UI correctly reverts to show the state of the first session - so it is now de facto the active one. E.g. variables and buttons update. The combo box in the toolbar updates to reflect the only remaining session (but doesn't disappear, despite there being only one session to show).
It causes us a problem as we have our own custom debug type and a UI panel that we update to reflect the active debug session, showing a diagram relevant to the active session (if one of ours). When the second session is terminated and the first one becomes active by default, we should update the UI as well, but we don't get the event. And because the vscode UI has de facto changed to have the first one active (but without firing an event), there isn't anything the user can do that will trigger that event to fire.
I'm running our extension from vscode in the extension host.

Hi, yes, just seen it in 1.94.2. It didn't occur every time so was hard to be sure. It seems to be an issue more after a 'cold start' with minimal interaction thereafter to trying the sequence above. If I started up with a different workspace then switched then it didn't occur.
Some of the debugger interactions cause a flurry of active editor and focus changes. Dunno if that could be a factor.


[part 3][wallet] remove redundant / hacky logic for resetting queries across network selections

Description
This is a follow-up PR to clean up unnecessary logic in Sui Wallet for resetting our query cache upon changing networks via the network selector. We still need to re-render the component tree using a key prop since our api instance isn't reactive, but we can at least remove logic for manually clearing the query cache.
Part 1 (fixing all of the query dependencies in our apps): https://github.com/MystenLabs/sui/pull/11889
Part 2 (updating Sui Explorer to no longer rely on hacky logic): https://github.com/MystenLabs/sui/pull/11598
Part 3 (updating Sui Wallet to no longer rely on hacky logic): https://github.com/MystenLabs/sui/pull/11890
Test Plan

Verified queries are persisted properly in IDB
Verified the bug in https://github.com/MystenLabs/sui/pull/7891 doesn't happen
Verified the network switches properly and queries call to the correct nodes

If your changes are not user-facing and not a breaking change, you can skip the following section. Otherwise, please indicate what changed, and then add to the Release Notes section as highlighted during the release process.
Type of Change (Check all that apply)

[X] user-visible impact
[ ] breaking change for a client SDKs
[ ] breaking change for FNs (FN binary must upgrade)
[ ] breaking change for validators or node operators (must upgrade binaries)
[ ] breaking change for on-chain data layout
[ ] necessitate either a data wipe or data migration

Release notes
N/A

Not going to go with this approach


Fixes Xcode 8 Beta 3 warnings / errors

This PR aims to remove warnings and error introduced by the newest Xcode 8 Beta 3.

Fixes compiler warning for explicit let in conditional unwrapping.
Compiler seem to get confused on which _newState method to call, when really there's only one implementation. Differentiating the calls with SelectedState argument or State seem to do the trick, for now.

This PR will be followed by a swiftlint PR that silences a few warnings that currently surface on that branch.

After discussing with @agentk I will be closing this PR as master is Swift 3 and will re-submit a PR in regards to the fixes for SE-0099.


couldn't find a Compose file for locale "en_NG"

Describe the bug
I get this error after running go build on godcr:
xkbcommon: ERROR: couldn't find a Compose file for locale "en_NG"
xkbcommon: ERROR: couldn't find a Compose file for locale "en_NG"
Expected behavior
Running go build should start the UI frontend
Actual behavior
After running go build execution continues but no view is displayed
Platform
GOARCH="amd64"
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/sam/go"
GOROOT="/opt/goSDK/go1.14.3"
GOTOOLDIR="/opt/goSDK/go1.14.3/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
Relevant screenshots or logs

Changing the current locale of my OS from "en_NG" to "en_US" for both language and keyboard solved the issue.
The error was from xkbcommon library libxkbcommon; no default support for "en_NG" locale
xkbcommon is a library for the handling of keyboard descriptions, including loading them from disk, parsing them and handling their state. It's mainly meant for client toolkits, window systems, and other system applications.


Let user specify custom runtime

I think somehow we should give user an option to specify custom runtime pre-packaging required libraries for their function.
$ kubeless runtime create --name runtime_name --image docker_image

@tuna I have a solution for this.
In the case of python we can keep list of dependencies in a requirements.txt file . For nodejs we can keep  package.json file.
kubeless cli need to have an optional --dependencies=<file>. The requirements file gets added to the function configmap.
Then we can use an init container, to load all dependencies and share them with the runtime "app" container.
In python, the init container does a pip install -r requirements.txt. In nodejs, the init container does npm install. The files/libs are shared via a volume, and an environment var is set to configure the non standard path where the libraries live (e.g PYTHONPATH).
Something like this:
apiVersion: v1
kind: Pod
metadata:
  name: init-demo
  annotations:
    pod.beta.kubernetes.io/init-containers: '[
        {
            "name": "install",
            "image": "python:2.7-alpine",
            "command": ["pip", "install", "--prefix=/pythonpath", "-r", "/requirements/requirements.txt"],
            "volumeMounts": [
                {
                    "name": "pythonpath",
                    "mountPath": "/pythonpath"
                },
                {
                    "name": "requirements",
                    "mountPath": "/requirements"
                }
            ]
        }
    ]'
spec:
  containers:
  - name: python
    image: python:2.7-alpine
    env:
      - name: PYTHONPATH
        value: /opt/kubeless/pythonpath/lib/python2.7/site-packages
    command:
      - sleep
      - "3600"
    volumeMounts:
    - name: pythonpath
      mountPath: /opt/kubeless/pythonpath
  dnsPolicy: Default
  volumes:
  - name: pythonpath
    emptyDir: {}
  - name: requirements
    configMap:
      name: requirements

We just need to handle the logic of when there is a requirements file provided and when it is not.
Do you think, you can add this to the controller quickly next week ?

@sebgoa Yes added it to the wish list this week.

I will close this issue in case of having support for nodejs and python. We can create other issues specifying for other languages.
Regarding to Go, Java and other languages require compilation, there was a discussion between me @migmartri @anguslees and @mmikulicic at the hackathon session. I will document it.


Line graph with react-event hooks

[x] An actual line graph
[x] Line graph now defaults to support label
[x] Add support for custom react event hooks
[x] Verify support for custom svg inclusion
[x] Fix webpack for demo code
[ ] Update documentation

@jesserathbun


[Feature Request] Let us change the name of the messages table

messages is a very generic table name and a lot of setups might already be using a messages tables.. such as is the case for me. It would have been nicer if you had called it agent_messages to be in line with agent_memories and to reduce the risk of a table name conflict.
It would be nice if you made this change.. but if you would rather not.. then it would be nice to have it be configurable via the config file.
Many thanks!

Hey this is a good call I will update this weekend and take care of it.


Subscript default arguments

https://bugs.swift.org/browse/SR-6118

@swift-ci Please smoke test

@swift-ci Please test source compatibility

I'm pretty sure this needs a proposal, especially since it has potentially weird implications for exclusivity.

@jrose-apple It was part of the generic subscripts proposal but never implemented; however that predates exclusivity, IIRC. What are the weird implications for exclusivity? When emitting a subscript, SILGen evaluates the entire argument list (including default args) once, then passes it to the getter and setter as needed. I didn't actually have to change anything in subscript argument emission because John redid it recently to use the same mechanism is call argument emission.

Oh, sorry, I forgot that subscripts already can't have inout indices. Withdrawn. And the citation of the generic subscripts proposal does make this okay.
Can you add tests for

inlinability checking
parseable interface printing

to make sure this won't regress those features?

@swift-ci Please smoke test

@swift-ci Please smoke test

@swift-ci Please test source compatibility


{ errorType: 'UnauthorizedException', message: 'Permission denied' } when calling mutation from GitHub action

How did you install the Amplify CLI?
No response
If applicable, what version of Node.js are you using?
16
Amplify CLI Version
12.10.1
What operating system are you using?
linux
Did you make any manual changes to the cloud resources managed by Amplify? Please describe the changes made.
No manual changes made
Describe the bug
I have a mutation query in my schema:
registerTenant(name: String, emailAddress: String, tenantName: String): Tenant @function(name: "plateletAddNewTenant-${env}") @auth(rules: [{allow: private, provider: iam}])

that I want to be able to call from a GitHub action. So far I've been able to set up permissions that allow the IAM account to call that mutation, but I'm getting this error:
{ errorType: 'UnauthorizedException', message: 'Permission denied' }
This is the policy I'm using for AppSync:
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "appsync:GraphQL"
            ],
            "Resource": [
                "arn:aws:appsync:*:<id>:apis/*/types/Query/fields/getTenantByTenantName",
                "arn:aws:appsync:*:<id>:apis/*/types/Mutation/resolvers/registerTenant"
            ]
        }
    ]
}

It seems like it can call the getTenantByTenantName query fine, but gets that error on the registerTenant call.
in amplify/backend/api/platelet/custom-roles.json I have the following (with my proper id):
{
    "adminRoleNames": [
        "arn:aws:iam::<id>:user/test-test.bloodbikes.cloud"
    ]
}

I use this on another API I have and get no issue when making calls to it using IAM.
I tested running the action with a full admin access policy attached to the IAM user but it was still denied, so it seems like the API is returning that permission denied error rather than it being a problem with the policy.
It's strange that no error is returned when using getTenantByTenantName, which is just a queryField added to an index, but the registerTenant which is a custom function throws that error.
type Tenant
@auth(rules: [
{allow: private, operations: [read]},
{allow: private, provider: iam, operations: [read]},
])
@model {
  id: ID!
  name: String! @index(name: "byTenantName", queryField: "getTenantByTenantName")
  referenceIdentifier: String!
  admin: User! @hasOne
}

Expected behavior
I expect the query to complete without a permission denied error.
Reproduction steps

Add a custom mutation to type Mutation with lambda function.
Create access keys and set IAM policy to allow running that mutation.
Set up GitHub action that calls the mutation.

This is the code I use in the GitHub action:
import * as core from "@actions/core";
import { Sha256 } from "@aws-crypto/sha256-js";
import { SignatureV4 } from "@aws-sdk/signature-v4";
import { HttpRequest } from "@aws-sdk/protocol-http";
import fetch, { Request } from "node-fetch";
import {
  RegisterTenantMutationVariables,
  GetTenantByTenantNameQueryVariables,
} from "./API";
import { registerTenant } from "./graphql/mutations";
import { getTenantByTenantName } from "./graphql/queries";

const request = async (queryDetails: { variables: any; query: string }) => {
  const apiURL = core.getInput("tenantApiUrl");
  const region = core.getInput("awsRegion");
  const accessKeyId = core.getInput("awsAccessKeyId");
  const secretAccessKey = core.getInput("awsSecretAccessKey");
  console.log("apiURL", apiURL);
  console.log("region", region);
  console.log("queryDetails", queryDetails);

  const endpoint = new URL(apiURL);
  const credentials = {
    accessKeyId,
    secretAccessKey,
  };
  const signer = new SignatureV4({
    credentials,
    region,
    service: "appsync",
    sha256: Sha256,
  });

  const requestToBeSigned = new HttpRequest({
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      host: endpoint.host,
    },
    hostname: endpoint.host,
    body: JSON.stringify(queryDetails),
    path: endpoint.pathname,
  });

  const signed = await signer.sign(requestToBeSigned);
  const request = new Request(endpoint, signed);
  return await fetch(request);
};

const errorCheck = (body: any) => {
  if (body?.errors) {
    console.error(body?.errors);
    throw new Error(body?.errors[0].message);
  }
};

export const registerTenantQuery = async (
  input: RegisterTenantMutationVariables
) => {
  const response = await request({
    variables: input,
    query: registerTenant,
  });
  const body = await response.json();
  errorCheck(body);
  return body?.data?.registerTenant;
};

export const getTenantByName = async (
  variables: GetTenantByTenantNameQueryVariables
) => {
  const response = await request({
    variables,
    query: getTenantByTenantName,
  });
  const body = await response.json();
  errorCheck(body);

  console.log("aaaa: ", body?.data?.getTenantByTenantName?.items);
  return body?.data?.getTenantByTenantName?.items[0] || null;
};

Project Identifier
f97b80f7ee5ceef2a72f107f35e6d453
Log output

# Put your logs below this line

Additional information
No response
Before submitting, please confirm:

[X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.
[X] I have removed any sensitive information from my code snippets and submission.

Why are you defining a separate policy when you define the Auth rules with the Auth directive @auth in the mutation declaration?

@biller-aivy the policy is for my IAM user that I use with GitHub actions. I want to be able to call that query from that user.
If I don't add any policy, I get a different kind of not authorized error.

Hi @duckbytes,
I see you're declaring your IAM policy resource with resolvers in the registerTenant clause:
"arn:aws:appsync:*:<id>:apis/*/types/Mutation/resolvers/registerTenant"

I believe that should be fields:
"arn:aws:appsync:*:<id>:apis/*/types/Mutation/fields/registerTenant"

Please see AWS AppSync documentation for supported resource types.
Hope this helps.

hi @palpatim
I gave it a try with this policy:
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "appsync:GraphQL"
            ],
            "Resource": [
                "arn:aws:appsync:*:<id>:apis/*/types/Query/fields/getTenantByTenantName/*",
                "arn:aws:appsync:*:<id>:apis/*/types/Mutation/fields/registerTenant/*",
                "arn:aws:appsync:*:<id>:apis/*/types/Query/fields/getTenantByTenantName",
                "arn:aws:appsync:*:<id>:apis/*/types/Mutation/fields/registerTenant"
            ]
        }
    ]
}

but I get the same permission denied error:
message: 'Not Authorized to access getTenantByTenantName on type Query'
The error is actually on the query and not the registerTenant mutation. I haven't been able to try if registerTenant works yet.

Hi @duckbytes, you've mentioned earlier that:

It's strange that no error is returned when using getTenantByTenantName, which is just a queryField added to an index, but the registerTenant which is a custom function throws that error.

And from the latest error message you've shared i.e Not Authorized to access getTenantByTenantName on type Query this seems to now be broken after adding those additional policies?
I don't see anything in those policies that could've caused this. Can you clarify which of the query or mutation is not working?

Hey 👋 , This issue is being closed due to inactivity. If you are still experiencing the same problem and need further assistance, please feel free to leave a comment. This will enable us to reopen the issue and provide you with the necessary support.


Window.new_html_sheet returns HtmlSheet not Sheet

Wouldn't make sense if it didn't.

>>> type(window.new_html_sheet('foo', 'bar'))
<class 'sublime.HtmlSheet'>

Confirmed. Thanks!


Add ETL performance tracing or telemetry

or both - since we've been getting reports of UI thread delays (VS gets frozen for a few seconds) in setwd() and at the end of long calculation (data transfer or log writing?)

Cleaning up backlog


A UI bug on iOS devices: iOS native window is shown after focusing on a multiple choices select element

Steps to reproduce:

Open the following page from an iPhone:
https://select2.org/appearance#labels

Focus on the multiple select element field.

You'll see that the 2nd iOS native window saying "No Options" is temporarily shown (just for a second):
https://github.com/select2/select2/assets/32933641/90085668-1509-4eb9-a843-ba547942b496
I've tested on 2 iPhone devices with different iOS versions. The bug persists.

Hey! any luck on this one?

@kevin-brown or @jayaddison, please, take a look at this.

Unfortunately this one is going to be hard to debug and correct since I don't have ready access to iOS devices anymore and this doesn't appear in my emulator.

@vitalii-work is the relevent select control loading data from AJAX, or are the options baked-in to the webpage content?

Self-quoting:

@vitalii-work is the relevent select control loading data from AJAX, or are the options baked-in to the webpage content?

I can answer that myself, because the question is about the select2 documentation.
The answer is: it's not AJAX, but it's also not pure HTML.
From 'most static' to 'most dynamic', the options we generally have are:

[ ] Embed options into the HTML.
[x] Dynamically add options using JavaScript and data within the page.
[ ] Dynamically add options using JavaScript from a remote resource (either same-origin or otherwise).

In this case the geoselection options are configured from a static set of states in this file: https://github.com/select2/select2/blob/595494a72fee67b0a61c64701cbb72e3121f97b9/docs/themes/site/templates/partials/js/source-states.html.twig#L70
Why is all that relevant?
We can't inspect the source code of iOS itself - that's closed.  But something I would infer is that the No options prompt shows up because at the time-of-calculation, it finds the select element has.. no options listed.
When does that time-of-calculation occur?  Perhaps it is during the parsing of the raw HTML itself?  (that would, in some ways, be quite fast and efficient - preparing as much of the UI/dialogs as possible based on known-static data)
What are you saying???
My guess would be that to avoid this in a production application, you should, if possible, use HTML to declare the <option> values statically.  I can't initially think of any drawbacks to that as compared to using JavaScript to dynamically insert options based on.. a static initial list of values, declared as HTML.

(I could be mistaken about much of the above; it's best-guess based on some mental models I have about how iOS might work, and why that dialog might appear in a dialog that is displaying the given web content)

It might be possible to confirm/reject the theory that this occurs when <option>s are created by JavaScript (either local or AJAX) by comparing the behaviour of this multi-select select2 example, also in the documentation, where the options are declared using HTML: https://select2.org/tagging#tagging-with-multi-value-select-boxes


windows build lccgen fail

solve problem with add legacy_stdio_definitions.lib to linker
fix please

and
#define stdin  (__acrt_iob_func(0))
#define stdout (__acrt_iob_func(1))
#define stderr (__acrt_iob_func(2))

FILE _iob[] = { *stdin, *stdout, *stderr };
extern "C" FILE * __cdecl __iob_func(void) { return _iob; }

to main.cpp

windows 10 1909 x64

Problem on OpenSSL 1.0.2u version.
Add support to older versions?

Thanks for reporting. I'm completing a feature, as soon as i finish the current work i'll take a look at it.


Liquid duplication

If there is no Fluid Storage  (or room) available in the system the liquid imported via Import Bus starts filling tank connected via Storage Bus infinitely. Amount of liquid in the source tank retains.
If there is a Fluid Storage in the system with free room or already having this particular liquid for every bucket imported from tank one bucket goes into Fluid Storage and one bucket goes into tank through Storage Bus doubling total amount of imported liquid.
EC: 2.2.53b82/2.2.52b80
Scheme: [Any Tank]>(Import Bus)---(ME Network with ME Drive)---(Storage Bus)|[Any Tank]

I managed to accidentally run into this too on a server I play on. This should be fixed quickly as duplication bug are annoying especially when you don't want to exploit them and it still happens...

So, what probably happens in the bus is the following:
BUS tries to import fluid to ME.
BUS checks if the ME fluid storage is full. If this is true, it does not remove fluid from the inventory.
This is going to leave the fluid in a tank when the storage is filled. That happens commonly when using an export and an import bus at the same time and the storage is full beforehand.

I'm noticing similar bug when using drums on storage buses and importing with an interface.
The interface wont empty and the 11th bucket will cause it to fill the drums indefinitely.
If i put a fluid-disk in a fluid drive the fluid goes to the drive correctly, when i lower priority on the drive-bay, to force to fluid to to to a drum on a storage-bus it receives dubble amount fluid.

I will try to fix this Bug the next days

Fixed in rv2


refactor: removed events capability

Removing events capability for now since we haven't figured out how to combine pubsub and events. There is only one commit that's relevent to this PR:
https://github.com/deislabs/spiderlightning/pull/243/commits/8bf816f711c890615753bc837aa905e4c511e4fc

I was under the impression we wanted to keep this in until using pubsub as event handling was more solidified. Do we want to maybe keep this as a draft until then?

My intention is to remove code for events for now. Later if we decide on how we deal with pubsub and events, we can always bring back the code if we want.


Update README.md

Update README.md

Update links to examples
Update Getting Started and Running Examples
Fix grammar and typo.

Signed-off-by: Mete Ozay<EMAIL_ADDRESS>

:memo: TAOS-CI Version: 1.4.20191203. Thank you for submitting PR #741. Please a submit 1commit/1PR (one commit per one PR) policy to get comments quickly from reviewers. Your PR must pass all verificiation processes of cibot before starting a review process from reviewers. If you are new member to join this project, please read manuals in documentation folder and wiki page. In order to monitor a progress status of your PR in more detail, visit http://nnsuite.mooo.com/.

INFO: You can read if there are misspelled characters at our misspelling check report. Please read http://nnsuite.mooo.com/nntrainer/ci/repo-workers/pr-checker/741-202011111943100.56902289390564-05dce0e23db36a4a6d426e78625ae64476e0e59d/report/misspelling_result.txt.

INFO: You can read if there are misspelled characters at our misspelling check report. Please read http://nnsuite.mooo.com/nntrainer/ci/repo-workers/pr-checker/741-202011111944350.108717918396-8cc4de14a44e8e4a84efb0a1110ffaaadac7cfec/report/misspelling_result.txt.