Metadata Attribute research: KEY

This issue aims to document thoughts about the KEY attribute from the Root and Delegations classes.
The goal is to understand how we use that attribute, what might go wrong with it and how we can validate it.
When thinking about how we use KEY worth reading what is implemented in pr #1408 and pr #1436 implementing threshold validation.
We want to answer/address the following 6 questions/points based on my comment here:

How do we use it?
What might go wrong?
Rethink how we store it?
If, after the changes in step 3 there are concerns, then add a validation function
Make the necessary changes for points 1 - 4, think and propose a way to integrate it into the class/classes (decorators, descriptors, etc.)
create a pr for all additional changes on top of the validation functions

PS: The 7-th point is covered by documenting this issue.

Should I add thoughts about keyid considering in #1436 we are adding it to the Key class?

How do we use it?

During Root and Delegations initialization.
When passing a Key argument in Root.add_key().

What might go wrong?

If somebody passes a Key without the required fields it will just fail during initialization when Key.from_dict() is called. So, no problem here.
If somebody passes invalid keytype, scheme or keyval -> those attributes are  validated through securesystemslib.schemas.
Currently, in the MetadataBundle we are using keys when calling verify_with_threshold and there we actually call securesystemslib.keys.format_metadata_to_key() which checks the formats of all of the three keyval, keytype and keytype.
The situation is the same we call any function from securesystemslib.keys accepting key_dict as an argument.
Duplicating keys https://github.com/theupdateframework/tuf/issues/1429 for context-> verifying that keys are uniques can be done during verification https://github.com/theupdateframework/tuf/pull/1436 or in Root and Targets.
Do we have to worry if somebody passes a wrong Key class instance when calling Root.add_key()? Can this be used to validate wrong metadata format? -> If wrong key is passed, then when they are using during verification (https://github.com/theupdateframework/tuf/pull/1436) the key itself will be verified.

Rethink how we store it?

This depends if we want to add any type of validation. It's possible that the securesystemslib validation is enough in this case.
Do we want to validate keys uniqueness in Root and Targets?

If, after the changes in step 3 there are concerns, then add a validation function

None for now

Make the necessary changes for points 1 - 4, think and propose a way to integrate it into the class/classes (decorators, descriptors, etc.)

None for now

The question is do we trust securesystemslib for validation?
Right now, each of the times we do operations with keys we end up calling a securesystemslib function which makes argument validation.
The problem is that tomorrow this behavior could change and we can end up with no validation at all.
I agree that we should add some validation in __init__ time.

I think in issue #1429  we agreed we would want to throw an exception if we have duplicate keys. This check should be done in Root and Targets.
Do we want to validate the values of keytype, scheme and keyval?
a) In securesystemslib keytype is one of rsa, ed25519, ecdsa, ecdsa-sha2-nistp256 or ecdsa-sha2-nistp384.
do we want to hardcode those values somewhere in securesystemslib and use them?
b) schema in the spec is defined as a string, maybe we can at least check that?
c) keyval["public"] is defined as a string in PEM format. Can we do any checks there?

PS: @jku  do you think we should discuss keyid here considering you are adding it to the Key class in #1436?

Now that keyid is officially part of the Key class we should think about it as well.

My recollection is that the real validity checks in SSLib are hidden inside verify_signature() and other functions like that, so we can't really use them. We could run things like sslib_formats.KEYTYPE_SCHEMA.check_match(keytype) (which would at least check for the possible values) but my understanding is that there's no way to check that e.g. the schema is valid for the keytype without more code. But do have a look, if you find a way to make SSLib do the validation that would be good.

keyval["public"] is defined as a string in PEM format

I don't think this is true for all keytypes. I believe we should consider this just a string (content is internal implementation detail of the specific key).
So maybe

use sslib schema validation  iff it brings actual value (keytype looks like one?)
otherwise I guess we do want to check if the variable type is str? I'm not fond of type checking but I guess it's better than pushing random objects as arguments to library calls we make...

As in most other cases, I think we're primarily interested in the deserialization case... Making sure the validation happens on init() should make this useful for new created keys as well (we don't have code that does that yet but that's what I imagine at least).

I experimented with a solution testing all of the attributes reusing the securesystemslib schemas.
Here what I got:
https://github.com/MVrachev/tuf/commit/e7cdab018153a8fe81a0ec691a3a00f181fba4b9

Yeah I think that reinforces my opinion:

the moment we start writing constants like "rsa" or "ecdsa-sha2-nistp256" in TUF code we're doing something wrong, we shouldn't have to care about details like that (as we should not even know all valid values).
In fact I wonder if we want to use even KEYTYPE_SCHEMA at deserialization time: nothing in the spec promises that securesystemslib can recognise the key type that a specific key uses -- maybe we only want to fail if that key is actually used for something. I'm thinking of a situation where some key uses a new type/scheme that our older client does not know about but we don't actually end up using that key (and the keys we do use are of type that we do understand)  -- this is maybe a contrived example but I think according to spec we should not actually assume we know about the keytypes/schemes

As I understand it, the source of knowledge about supported keys and schemas is securesystemslib.
The best we can do is make sure securesystemslib continues to be the source of truth for supported keytypes and schemas.
What we can do is probably provide a utility function in securesystemslib which for a given keytype calls the
correct schema check.
That way adding/removing new keys in securesystemslib won't require any new code in TUF.

I've not responded since I wasn't quite sure why this felt wrong, now I think I've got it: Metadata validity and our implementations ability to sign or verify signatures are two different things. For a client or a even a repository implementation it might be reasonable to consider both instant failures but I do not think that applies to Metadata API.
In other words, I think it is wrong for Metadata API to fail de/serialization of data just because the data contains an algorithm that our securesystemslib does not recognize: New/unknown algorithms do not make the metadata invalid in any way: they just mean we can't currently verify the signatures or hashes in it.

We had a discussion with @jku and agreed that we shouldn't focus on checking if keytype, scheme, keyval, or keyid have valid semantics, but rather check that all of those are with the expected types and we can compare them the way
we expect.
The reasons are that:

we could have a metadata file with one or a couple of keys with unsupported or invalid values for their attributes, but that doesn't mean we have to stop instantiating the Key class.
When verifying with threshold with keys it's allowed to have multiple invalid keys until the required threshold is achieved.
it's better to throw an exception during verification that certain keys have unsupported algorithms instead of doing that when nobody is expecting - during initialization.


Fix HttpUtil.isKeepAlive to behave correctly when Connection is a comma separated list

Motivation:
According to the specification, the Connection header's syntax is:
The Connection header field's value has the following grammar:

     Connection        = 1#connection-option
     connection-option = token

Connection options are case-insensitive.

https://tools.ietf.org/html/rfc7230#section-6.1
This means that Connection's value can have at least one element or
a comma separated list with elements
When calculating whether the connection can remain open,
HttpUtil.isKeepAlive(HttpMessage) should take this into account.
Modifications:

Check for close and keep-alive in a comma separated list
Add unit test

Result:
HttpUtil.isKeepAlive(HttpMessage) works correctly when Connection: Upgrade, close
Related to https://github.com/reactor/reactor-netty/issues/632

@netty-bot test this please

@slandelle Modified the code to become single line

LGTM

@violetagg thanks a lot!


Fixed file/alloc limitation in GetDirectoryFiles()

MAX_DIRECTORY_FILES macro removed.
ClearDirectoryFiles() tweaked to reflect changes.

@lib-omar Thank you very much for this improvement! :D


Aspire plugin fails to open docker ports

I have an AppHost that runs great under the regular launch config but fails under the Aspire Plugin.
When launching via the Rider Aspire plugin I always see a blocking error that I never see when using the default AppHost https launch config:
fail: StackExchange.Redis.ConnectionMultiplexer[0]
StackExchange.Redis.RedisConnectionException
UnableToConnect on localhost:55845/Subscription

My project is an Orlean Silo, which I'm constructing via:
var builder = DistributedApplication.CreateBuilder(args);

var redis = builder.AddRedis("orleans-redis");
var orleans = builder.AddOrleans("orleans-cluster")
    .WithClustering(redis);

var silo =
    builder.AddProject<Projects.Apps_Silo>("silo")
        .WithReference(orleans)
        .WithReplicas(3);

builder.Build().Run();

In Docker Desktop, when launching via the default AppHost https launch config I always see a running container with assigned ports:

And when launching via the Rider Aspire plugin I SOMETIMES see a running container with assigned ports like above and I SOMETIMES see a running container with no assigned ports like below. Assigned ports or not, I always get the blocking exception listed above:

fixed in 0.6.1


Use rasterio's transform instead of homemade coordinates

[x] Closes https://github.com/pydata/xarray/issues/1686
[x] Tests added / passed
[x] Passes git diff upstream/master **/*py | flake8 --diff
[x] Fully documented, including whats-new.rst for all changes and api.rst for new API

OK, I think this is ready for review (@shoyer or @jhamman)

@maaleske would you mind taking a look at this to see if it looks right to you?

I added a parse_coordinates=True kwarg and added a check for 2D coordinates.
I proposed an implementation for the 2D case, but it feels a bit clunky. A second option would be to not parse 2D coords and raise a Warning in case of parse_coordinates=True and 2D coordinates.
(I don't think that 2D coords happen very often).
Thoughts?

Would it be possible to put the coordinate generation into a separate function? It would be nice to pass in the affine, width, height, and is_2d as args and get back the spatial coordinates.

@snowman2 I added an example in the code on how to do this - can be useful for downstream libraries
@maaleske @jhamman this is ready for review / opinions. This PR still closes https://github.com/pydata/xarray/issues/1686 in a backwards compatible manner and adds a special case for irregular coordinates. I don't think that 2D coordinates happen very often (never encountered one in my work), but xarray should handle them now. The PR also makes some minor other modifications.

@fmaussion Seems okay to me. The rasterio version check is a bit iffy but I can't think of a way to do it better.
I do agree with @snowman2 though, having the coordinate generation separately exposed would be nice (and easier to write unit tests for). Something like
def xy_pixel_coordinates(nx, ny, transform=None)
    transform = Affine.identity if not transform else transform

    # xarray coordinates are pixel centered
    x, _ = (np.arange(nx)+0.5, np.zeros(nx)+0.5) * transform
    _, y = (np.zeros(ny)+0.5, np.arange(ny)+0.5) * transform
    return x,y

would probably suffice, though I don't know which module this would fall under. I'd imagine it might be useful for backends besides rasterio, though.


blip-2 build error

I was following the guide for 'blip-2' to build, but encountered the following error in the section below.
trtllm-build --checkpoint_dir=./opt/2.7B/trt_ckpt/fp16/1-gpu/
--max_batch_size 8
--use_gpt_attention_plugin float16
--use_gemm_plugin float16
--max_input_len 924
--max_output_len 100
--max_beam_width 5
--output_dir ../blip2/trt_engine/blip-2-opt-2.7b/fp16/1-gpu
--max_prompt_embedding_table_size 256 # 256 = 32 (query_token number) * 8 (max_batch_size)
[12/19/2023-04:23:59] [TRT-LLM] [W] Found pynvml==11.4.1. Please use pynvml>=11.5.0 to get accurate memory usage
[12/19/2023-04:24:04] [TRT] [I] [MemUsageChange] Init CUDA: CPU +1, GPU +0, now: CPU 141, GPU 987 (MiB)
[12/19/2023-04:24:06] [TRT] [I] [MemUsageChange] Init builder kernel library: CPU +1974, GPU +348, now: CPU 2251, GPU 1335 (MiB)
Traceback (most recent call last):
  File "/usr/local/bin/trtllm-build", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.10/dist-packages/tensorrt_llm/commands/build.py", line 217, in main
    build_and_save(source, build_config, args.output_dir, workers,
  File "/usr/local/lib/python3.10/dist-packages/tensorrt_llm/commands/build.py", line 154, in build_and_save
    build_and_save_shard(rank, rank % workers, ckpt_dir, build_config,
  File "/usr/local/lib/python3.10/dist-packages/tensorrt_llm/commands/build.py", line 130, in build_and_save_shard
    engine = build(build_config,
  File "/usr/local/lib/python3.10/dist-packages/tensorrt_llm/builder.py", line 621, in build
    return build_shard_model(model, build_config)
  File "/usr/local/lib/python3.10/dist-packages/tensorrt_llm/builder.py", line 570, in build_shard_model
    model(**inputs)
  File "/usr/local/lib/python3.10/dist-packages/tensorrt_llm/module.py", line 40, in __call__
    return self.forward(*args, **kwargs)
TypeError: DecoderModelForCausalLM.forward() got an unexpected keyword argument 'tasks'

I also tried it on the 'rel' branch, but the issue persisted. Could you help me?

@KKwanhee  we have a fix internally and will be synced to github main branch soon.

@QiJune Thanks for your work. I have a question, does trtiton_server support blip2? Could you help me？

@jessiewiswjc Blip2 is not supported in Triton backend yet.

@KKwanhee The issue should have been fixed on the latest main branch, please kindly check. Thanks very much.
Closing. Please let us know if you have questions.

@jessiewiswjc Blip2 is not supported in TensorRT-LLM Triton backend yet.

@kaiyux Thanks for yout replay. Is there a plan for this(trtion for multi-modal)?


Merge pull request #2 from Christiana-AE/main

Pull request for changes made in local repository

Pulling new updates to index HTML and CSS files


searching for nothing... displays error rather than 0 results

Describe the bug
if the user clicks the search box and then return it seems to run a search and return the following message 'Exception while fetching data (/search) : 'query' parameter cannot be null or empty'
To Reproduce
Steps to reproduce the behavior:

Go to Search bar
Hit Return
See error

Expected behavior
Id expect it to either show all results or 0 results..
Screenshots

Desktop (please complete the following information):

OS:MacOs
Browser Safari
Version 14

Additional context
Add any other context about the problem here.

Thanks for reporting @dougallaninthecloud  - this is definitely a bug!
Let us take care of this. cc. @gabe-lyons @topwebtek7

Fixed via #2556

Nice to see both the fix and tests updated thanks :)


filterInstance is not falling back to the default no-op filter source when browser refreshes

Steps to reproduce:
After you get a HttpfilterInstance and you send a short circuit your own response back for url A,
For the next request url B, the same short circuit response is being sent.
I observed that, for the request url B (on same browser tab), the HttpFilters filterInstance was still pointing to the previous HttpFilters filterInstance meant for Url A
Let me know if you need further info.

I'm not able to reproduce this. How are you configuring your HttpFiltersSource? Is it returning the same instance for each request? It'd also help to see the code of your HttpFilters implementation as well.

public HttpFilters filterRequest(HttpRequest originalRequest, 			ChannelHandlerContext ctx) {   if(originalRequest.getUri().contains("macmillandictionary.com/dictionary/british/green_1")){     return new HttpFiltersAdapter(originalRequest, ctx,null){       public HttpResponse clientToProxyRequest(HttpObject httpObject) {         String answer = "byebye"; 	ByteBuf buffer = null; 	try { 	        buffer = Unpooled.wrappedBuffer(answer.getBytes("UTF-8")); 	     } catch (UnsupportedEncodingException e) {                 e.printStackTrace(); 	      } 	 HttpResponse response = new DefaultFullHttpResponse(HttpVersion.HTTP_1_1, HttpResponseStatus.OK, buffer); 	 HttpHeaders.setContentLength(response, buffer.readableBytes()); 	 HttpHeaders.setHeader(response, HttpHeaders.Names.CONTENT_TYPE, "text/html"); 	    	        return response; 	 }      };   }		   return null; }
Steps to reproduce:

After using this as my filtersource, I went to the site
The response i gave came.
Upon changing the url from browser to another, it started giving the same response.
Upon debugging found that in ClientToProxyConnection->doReadHTTPInitial method
currentfilters instance was not being replaced with the default one, instead it continues to use the     provided filterssource instance.

So to fix this i added else in
if (filterInstance != null) {
currentFilters = filterInstance;
}else{
currentFilters = new HttpFiltersAdapter(null);
}
Let me know for any further info.

Yep, I can see how this would be a problem when the browser/client is reusing a connection to the proxy.
Your solution looks good to me; can you submit a PR?

Since this is fixed now, I'll close it.


start fail in spring cloud gateway with elasticsearch

Describe the bug
start fail in spring cloud gateway with elasticsearch
ExceptionStackTrace:
Caused by: java.lang.IllegalStateException: null
at java.util.Optional.orElseThrow(Optional.java:290)
at org.springframework.cloud.kubernetes.discovery.KubernetesDiscoveryClient.findEndpointPort(KubernetesDiscoveryClient.java:189)
at org.springframework.cloud.kubernetes.discovery.KubernetesDiscoveryClient.getInstances(KubernetesDiscoveryClient.java:157)
at org.springframework.cloud.client.discovery.composite.CompositeDiscoveryClient.getInstances(CompositeDiscoveryClient.java:53)
at reactor.core.publisher.FluxMapFuseable$MapFuseableConditionalSubscriber.tryOnNext(FluxMapFuseable.java:301)
at reactor.core.publisher.FluxIterable$IterableSubscriptionConditional.slowPath(FluxIterable.java:515)
Reason:
When I install elasticsearch by cloud-on-k8s(https://github.com/elastic/cloud-on-k8s), it auto create a service myes-es-default without Port, but,  must have Port in spring-cloud-kubernetes

Things have changed since this was reported, can you try again with Hoxton.SR6?

Hello! I ran into this as well and wanted to talk about my findings in case root cause has not been assessed!
Elasticsearch endpoint does not specify a port, which as far as k8s is concerned is OK.
logging                  elastic-webhook-server               <IP_ADDRESS>:9443
logging                  elasticsearch-es-default              <IP_ADDRESS>
logging                  elasticsearch-es-http                   <IP_ADDRESS>:9200
logging                  elasticsearch-es-transport           <IP_ADDRESS>:9300

Calling public List getInstances(String serviceId), means that you'll eventually call private EndpointPort findEndpointPort(EndpointSubset s).
Looking at the code you'll see:
https://github.com/spring-cloud/spring-cloud-kubernetes/blob/e27dbbc66d8f4dfacc8385b8918aaf8165b0f596/spring-cloud-kubernetes-discovery/src/main/java/org/springframework/cloud/kubernetes/discovery/KubernetesDiscoveryClient.java#L202-L221
Thus, you'll get an IllegalStateException, when no exception should be thrown. The port should either not be set, or be put as a default.
Does this align with your thinking @ryanjbaxter?

@joshrosso that makes sense, can you create a PR for the 1.1.x branch?

@ryanjbaxter yes

Awesome thanks!


Contents of the byte array requests

Hi,
Three questions
Question 1.  I'm confused, why do some of the requests have an & character?
full_request=[b'\x08\x00\x00\x00&\x08\x00\x97\x00\x01\x00\x00']  //this while looping for readings
Question 2. Or % and \n characters for that matter?
Init
full_request[b'\x08\x00\x00\x00\x07\x00\x00\x80\x01\xff\x12\xff'] //straight byte array
localtime
full_request[b'\x0c\x00\x00\x00\x04\n\x00\x81\x17\x01\x18\x00\x1d$\x0b\x00'] //this has a \n
devicetime
full_request[b'\x0c\x00\x00\x00%\x08\x00\x82\x04\x05\x00\x00\x00\x00\x00\x00'] \this has a %
version
full_request[b'\x04\x00\x00\x00\n\x00\x00\x83']   //This one has a \n
Question 3. Are the write request byte arrays the same for the BLE characteristics?
Thanks.

Hi
When you print(byte_var) in python it prints ASCII characters as-is:
In [1]: b'\x41\x30\x31\x00\x01'
Out[1]: b'A01\x00\x01'

You can use .hex() to output every char as hex:
In [2]: b'\x41\x30\x31\x00\x01'.hex()
Out[2]: '4130310001'

I don't understand question 3, can you describe it in more detail?

Thanks for the .hex() hint!  I'm a c++, c#, java so I'm translating python.
Now my debug looks much better.
full_request=[080000002608009300010000]   which returns:
2024-01-23T14:35:30.246307 RealTimeData(dt=datetime.datetime(2024, 1, 23, 14, 35, 30, 246307), count_rate=4.8125, count_rate_err=5.6, dose_rate=6.796455181756755e-06, dose_rate_err=20.3, flags=8256, real_time_flags=0)
So Question 3.
I'm tasked to code an Android java Bluetooth app to request the spectra, readings, n42, any data I can get.
Question 3 is asking do I writeCharacteristic  with the same byte array on Bluetooth like USB?
For instance, in bluetooth.py function execute the line self.p.writeCharacteristic(self.write_fd, rp) will rp be the same request bytes?  full_request=[080000002608009300010000].   (sequence number notwithstanding as it increments)
    def execute(self, req) -> BytesBuffer:
        for pos in range(0, len(req), 18):
            rp = req[pos : min(pos + 18, len(req))]
            self.p.writeCharacteristic(self.write_fd, rp)

        while self._response is None:
            self.p.waitForNotifications(2.0)

        br = BytesBuffer(self._response)
        self._response = None
        return br

Incidentally, is there a way to get the latest spectrum.XML file?  I'll have to periodically get that too.

will rp be the same request bytes

yes, def execute(self, req) -> BytesBuffer is method of the interface and both bluetooth.py and usb.py implement it

latest spectrum.XML

I don't know about spectrum.xml, looks like it's not part of the protocol, but something at a higher level

In dozens of ble apps I've never seen notify_fd+1 self.p.writeCharacteristic(notify_fd + 1, b'\x01\x00')
Why the +1?
In c# you get the characteristic and call SetNotification(Characteristic, true);
        service = self.p.getServiceByUUID('e63215e5-7003-49d8-96b0-b024798fb901')
        self.write_fd = service.getCharacteristics('e63215e6-7003-49d8-96b0-b024798fb901')[0].getHandle()
        notify_fd = service.getCharacteristics('e63215e7-7003-49d8-96b0-b024798fb901')[0].getHandle()
        self.p.writeCharacteristic(notify_fd + 1, b'\x01\x00')

on C# i just call uuid without +1:
private const string RC_BLE_Notify = "e63215e7-7003-49d8-96b0-b024798fb901";
GattCharacteristicsResult charNotifyResult = await service.GetCharacteristicsForUuidAsync(Guid.Parse(RC_BLE_Notify));

Thanks.  I just got it and made my first read from java.  The +1 must point to a descriptor of the characteristic.
In my java code I setup for notifications like this:
boolean connectCharacteristics3(BluetoothGatt gatt)
        {
            BluetoothGattService gs            = gatt.getService(UUID_RadiaCodeCustomService);
            BluetoothGattCharacteristic gc     = gs.getCharacteristic(UUID_RadiaCodeCustomServiceCharacteristicdd1N);
            for (BluetoothGattDescriptor descriptor : gc.getDescriptors())
            {
                //descriptor.setValue( BluetoothGattDescriptor.ENABLE_INDICATION_VALUE);  //this way or
                descriptor.setValue(new byte[]{0x01, 0x00}); //this way
                gatt.writeDescriptor(descriptor);
                SystemClock.sleep(10);
            }
            gatt.setCharacteristicNotification   (gc,true);
            SystemClock.sleep(10);
            gatt.readCharacteristic(gc);

            onSerialConnect();
            connected = true;
            return true;
        }

And make the request like this:
byte [] writeMe = hexStringToByteArray("080000002608008800010000");// a reading from the usb vsc test that worked
sc.setValue (writeMe);
boolean tf = gatt.writeCharacteristic(sc); //where sc=UUID.fromString("e63215e6-7003-49d8-96b0-b024798fb901");

Which triggers the onCharacteristicChanged (notify) function with the returned byte data.
Next I have to figure out and translate the python request commands I need to make, such as:
byte [] writeMe = hexStringToByteArray("080000002608008800010000");   Which I believe is the request for current values.
Then port the decoders from python to java.  But I'm getting data so I'm ok.

The Init request fr=[080000000700008001ff12ff] returns the following data:
init=[070000800501dada2020001001010000]
But I have not been able to find where this is decoded, there I don't see an entry in the VS enumeration.
Is this init returned data important enough to worry about?
Does the init have to be called?
What does the init request do?


ApiParam Integer range is shown as double

Using spring mvc the following annotation
@ApiParam(name = "size", example = "100",
                     allowableValues = "range[0,1000]")
@RequestParam int size

will generate the yaml file with double values:
        type: "integer"
        maximum: 1000.0
        minimum: 0.0
        format: "int32"

It should be
        type: "integer"
        maximum: 1000
        minimum: 0
        format: "int32"

+1


[Bug]: Footer should remain at bottom

Description
The footer should be positioned at the bottom of the page on every page and remain persistent.
Attach Screenshots (if applicable)
Currently, the footer doesn't stick to the bottom on all devices. This problem is not visible on mobile devices.

It should stick to the bottom like this:

Which browser are you using when experiencing the issue?
Brave
Checklist

[X] I have checked for existing issues related to this problem.
[X] I have read the Contributing Guidelines.
[ ] I am willing to work on this issue (optional).

Can you assign this issue to me?

sure, why not?

Done PR #32 .


x/tokenfactory: cli

Closes: #XXX
What is the purpose of the change
*Add a description of the overall background and high level changes that this PR introduces *
(E.g.: This pull request improves documation of area A by adding ....
Brief change log
(for example:)

The metadata is stored in the blob store on job creation time as a persistent artifact
Deployments RPC transmits only the blob storage reference
Daemons retrieve the RPC data from the blob cache

Testing and Verifying
(Please pick one of the following options)
This change is a trivial rework / code cleanup without any test coverage.
(or)
This change is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(example:)

Added unit test that validates ...
Added integration tests for end-to-end deployment with ...
Extended integration test for ...
Manually verified the change by ...

Documentation and Release Note

Does this pull request introduce a new feature or user-facing behavior changes? (yes / no)
Is a relevant changelog entry added to the Unreleased section in CHANGELOG.md? (yes / no)
How is the feature or change documented? (not applicable   /   specification (x/<module>/spec/)  /  Osmosis docs repo   /   not documented)

instead of cw20, or in addition to?

Tokenfactory is intended to be a replacement for cw20.  Instead of using cw20 tokens, CosmWasm contracts on Osmosis can mint/burn native sdk coins

Codecov Report

Merging #1376 (20dc9a4) into sunny/tokenfactory-logic (34decb7) will not change coverage.
The diff coverage is 0.00%.

@@                    Coverage Diff                    @@
##           sunny/tokenfactory-logic    #1376   +/-   ##
=========================================================
  Coverage                     19.52%   19.52%
=========================================================
  Files                           226      226
  Lines                         30945    30945
=========================================================
  Hits                           6041     6041
  Misses                        23799    23799
  Partials                       1105     1105

Impacted Files
Coverage Δ

x/tokenfactory/module.go
59.45% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 34decb7...20dc9a4. Read the comment docs.

Please merge #1435 first

Hopefully telescope fixes this


MAINT start on modular report, encapsulating bokeh header scripts in js-files

Outsourcing the js-scripts in the header. Outsourcing div-tags proved to be somewhat of a deal, so there will be a follow-up PR

Thats the plan. So far I'm hung up on the div-tags: they are rendered really weird and it seems to be more complicated in html to include docs than, e.g., in latex. I'll merge this and work on div-tags.


Improvement for FIDO2 Rest API

Describe the improvement
Currently, we have only the device registration part. A customer has requested the authentication part also in the existing API. Hence decided to add as an improvement for the existing Rest API and deliver.

This issue is being closed due to extended inactivity. Please feel free to reopen it if further attention is needed. Thank you for helping us keep the issue list relevant and focused!


Make use of the model, datasets and strategy to classify sentences as urgent not urgent

Hello, I am wondering if I can use this code to classify sentences as urgent / not urgent.
Also if I could use even these datasets and model to accomplish this classification as urgent not urgent

It would probably work if you fine-tune on a few example (like 32) where the format for the labels are urgent and not urgent and you have different input formats.


Allow editing a filter inline

If we can display a filter's value on the filter list page, I don't see why we can't add an inline edit for that same filter.

User visits the filter list page and clicks the pencil next to a filter to edit it.

The user can edit the value of the filter, and then click the checkmark to finish editing. The filter list returns to its original state.

Potential downsides:

Inability to edit filter's name (current mockup)
If we give the ability to edit the name and value, having two text fields open up for each filter might be confusing to the user

Mockup:

I :heart: this idea!


WebdriverIO e2e test setup

Marquee has a complex webview and many interactions with VSCode. To ensure everything works before releasing new versions of our extension, let's run some e2e tests using the WebdriverIO service.

While I like to add more e2e tests I will start trying to get these ones merged for now and add more PRs and this one gets too big.


Incorrect indexed transactions

Describe the bug
Transaction was successfully executed, but Arbitrum's transaction service shows that the proposal was cancelled. Instead, there's a new transaction that is shown as executed by the service, although it wasn't. This transaction also shows an incorrect owner address in one of the confirmations.
To Reproduce
Steps to reproduce the behavior:

Do GET on https://safe-transaction-arbitrum.safe.global/api/v1/multisig-transactions/161EA0BDED794C84AFA87878DE4E8B45C3950989717E07AAEED4A41311AC41AD/

The result shows that this transaction wasn't executed, but here you can see the successful transaction with the same safeTxHash:
https://arbiscan.io/tx/0x06c36d4ebde377a96e0f5873ef8fa7e964b6d8afae860598257777954ef330bf#eventlog

Do GET  on https://safe-transaction-arbitrum.safe.global/api/v1/multisig-transactions/0x5161f6d78a10a24b0ec43a8fa15df44830b537f42476653363f3ce6b59b901a2/
This safeTxHash is shown as executed, although it wasn't, and it includes an incorrect owner on its first confirmation.

Expected behavior
There should be only one executed safeTxHash, which is 161EA0BDED794C84AFA87878DE4E8B45C3950989717E07AAEED4A41311AC41AD, executed in transaction 0x06c36d4ebde377a96e0f5873ef8fa7e964b6d8afae860598257777954ef330bf on Arbitrum.
Environment (please complete the following information):

production
Arbitrum
iOS
chrome

Additional context
Here are the respective transactions as shown in the Safe App:

https://app.safe.global/transactions/tx?safe=arb1:0x58Fab6d931B5a7C38B2E428D6cFEa7F071a1a225&id=multisig_0x58Fab6d931B5a7C38B2E428D6cFEa7F071a1a225_0x161ea0bded794c84afa87878de4e8b45c3950989717e07aaeed4a41311ac41ad
https://app.safe.global/transactions/tx?safe=arb1:0x58Fab6d931B5a7C38B2E428D6cFEa7F071a1a225&id=multisig_0x58Fab6d931B5a7C38B2E428D6cFEa7F071a1a225_0x5161f6d78a10a24b0ec43a8fa15df44830b537f42476653363f3ce6b59b901a2

Thanks for the detailed report. This should be fixed with the last version of the service we deployed yesterday. Please let us know if the bug still persists


rubyPackages.pry-nav: init at 1.0.0

Adds more features to pry gem for debugging
Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[x] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.05 Release Notes (or backporting 21.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

@marsam Can you help me with this PR? It looks alright to me, should not be an issue.

Thank you for your contribution, and sorry for the delay.
pkgs/top-level/ruby-packages.nix is automatically generated, you need to follow this steps https://nixos.org/manual/nixpkgs/stable/#adding-a-gem-to-the-default-gemset to add a gem.

Thanks for the reply. I will close this PR.


Directory (Container) Commands

Directory commands
Although ldp uses a
rm [dirname]
The rm commands removes either a directory or a file.

-r | --recursive No option.  The rm command will remove any container in system recursively.  This option has no effect.
-f | --force Also removes the fcr://tombstone.

This is interesting, I don't know what happens to the tombstones of the other files in a recursive deletion.
>cd /fcrepo/rest/user/quinn
>mkdir foo/bar/baz
>rm -rf foo

pwd
Echos the current working directory for the cli.
cd dir
This will change the CLIs tool notion of the CWD.  Whether this succeeds depends on the --template we are operating under.  If template=ldp, then this will
--force  Do no checking of the acl:type on the container.
cd $HOME
mkdir foo
cp file://./bar.jpg foo/bar
cd foo/bar  # Fails ./foo/bar is not an ldp:Container
cd -f foo/bar # This is OK.  Good luck!

```mkdir -p dir``
>mkdir foo/bar/baz
>ls bar

> rm -rf foo

>mkdir -p foo/bar/baz
>la bar

This is not yet complete So don't comment.


Fixed linters in point cloud codec branch

@john-maidbot
the only one that I now that it's doing this automatically is ament_uncrustify. You can run
ament_uncrustify --reformat <paths>

The others one I'm fixing them one by one

I think we can close this one https://github.com/john-maidbot/point_cloud_transport/pull/3#issuecomment-1648772880


Server Error: Resource of type 'node--blog_post' not found

Keep getting error Server Error Error: Resource of type 'node--blog_post' not found

Blog post node configuration in Drupal 9

Appreciate any help...

@chandantbs Can you confirm if you see node--blog_post when you visit /jsonapi on your Drupal site? Thank you

@shadcn Thanks for reply, I see there are some issue on this url '/jsonapi' there are php errors. I'm looking into this.


Possibility to update regex callback configurations

#74 is related to this one

#128 is related to this one


Change the title number of multiple duplicate windows e.g. Falkon <2> to Falkon (2)

To make visual comfort.

Agree. Do you know where this change would need to be made? KWin?

Aye. It's part of KWin itself.

What might be better would be if the numbers didn't show at all...
Whatever move you make will break someone's workflow. (I believe there's something in xkcd about emacs and spacebars)

Please do report this upstream to the KWin developers. Thanks!


Can't get "TORTOISE_ORM" from module

My workdir list:
.
├── Dockerfile
├── __init__.py
├── aerich.ini
├── docker-compose.yml
├── main.py
├── migrations
├── models.py
├── requirements.txt
└── settings.py

aerich.ini
[aerich]
tortoise_orm = settings.TORTOISE_ORM
location = ./migrations

settings.py
TORTOISE_ORM = dict()

I had got an error when executed aerich init-db in work folder root.
Error: Can't get "TORTOISE_ORM" from module "<module 'settings' from './settings.py'>"

TORTOISE_ORM can`t be empy

Hi,nice to you resolve it.


[Feature Request] belongsToMany in HybridRelations

Add belongsToMany in HybridRelations

Feature request tracked in PHPORM-116

Hello, are there any updates on this thread because for my project using Laravel filament I need this.
Or are there any workaround ?


Control the system (actually set battery charge level dynamically)

Hello together,
just a thing, that came to my mind:
Is there a possibility to change the charge level of the battery beside setting it in the RCT app?
My goal would be to control the charge level dynamically from Homeassistant.
I believe this is an enhancement and maybe somebody give any hint how to manage this.
Greetings,
Heiko

I would also appreciate this feature, since the internal SoC feature of the RCT devices is not as flexible as I would want it to be. It would be sufficient to change the "non-island" Target SoC, which is available in the app through the installer menu.

+1 to this, even it would mean that the devs have to implement the write function (which they do not want to, for obvious reasons). But having just battery soc manipulation as write would benefit me a lot as well

This would be a huge benefit. Specially if you want to charge an EV and "hold" the SoC of the battery, instead of having to wait for the battery to be fully charged.

@philoxio, this exactly my case of application!

I agree, being able to control the SoC Target and maybe max SoC would be awesome.

#!/usr/bin/env python3

## power_mng.soc_strategy		0: SOC-Ziel = SOC, 1: Konstant, 2: Extern, 3: Mittlere Batteriespannung, 4: Intern, 5: Zeitplan (default: 4)
## power_mng.soc_target_set		Force SOC target (default: 0.5) --> (1: Konstant)
## power_mng.battery_power_extern	Battery target power (positive = discharge) --> (2. Extern)

## power_mng.soc_min			Min SOC target (default 0.07) --> ggf. als Ersatzstrom-Reserve
## power_mng.soc_charge_power		Charging to power_mng.soc_min with given W (default: 100)
## power_mng.soc_charge			Trigger for charing to soc_min (default: 0.05)

import sys
import socket
import select
from rctclient.frame import make_frame, ReceiveFrame
from rctclient.registry import REGISTRY
from rctclient.types import Command
from rctclient.utils import decode_value, encode_value

def show_help():
    print("Usage:")
    print("  test.py get <parameter> --host=<ip_address_or_hostname>")
    print("  test.py set <parameter> <value> --host=<ip_address_or_hostname>")
    print("\nValid Parameters:")
    print("  power_mng.soc_strategy - SOC charging strategy")
    print("    Valid Values:")
    print("      0: SOC target = SOC (State of Charge)")
    print("      1: Konstant (Constant)")
    print("      2: Extern (External)")
    print("      3: Mittlere Batteriespannung (Average Battery Voltage)")
    print("      4: Intern (Internal)")
    print("      5: Zeitplan (Schedule)")
    print("    Default Value: 4 (Internal)")
    print("  power_mng.soc_target_set - Force SOC target")
    print("    Valid Range: 0.00 to 1.00, with at most two decimal places")
    print("    Default Value: 0.50")
    print("  power_mng.battery_power_extern - Battery target power")
    print("    Valid Range: -6000 to 6000")
    print("      Positive values indicate discharge, negative values indicate charge")
    print("    Default Value: 0")
    print("  power_mng.soc_min - Min SOC target")
    print("    Valid Range: 0.00 to 1.00, with at most two decimal places")
    print("    Default Value: 0.07")
    print("  power_mng.soc_charge_power - Charging power to reach SOC target")
    print("    Default Value: 100")
    print("  power_mng.soc_charge - Trigger for charging to SOC_min")
    print("    Default Value: 0.05")

def set_value(parameter, value, host):
    valid_parameters = [
        "power_mng.soc_strategy",
        "power_mng.soc_target_set",
        "power_mng.battery_power_extern",
        "power_mng.soc_min",
        "power_mng.soc_charge_power",
        "power_mng.soc_charge"
    ]

    if parameter not in valid_parameters:
        print(f"Error: Invalid parameter '{parameter}'.")
        show_help()
        sys.exit(1)

    if parameter == "power_mng.soc_strategy" and value not in ["0", "1", "2", "3", "4", "5"]:
        print(f"Error: Invalid value '{value}' for parameter '{parameter}'.")
        show_help()
        sys.exit(1)
    elif parameter == "power_mng.soc_strategy":
        try:
            value = int(value)
        except ValueError:
            print(f"Error: Invalid value '{value}' for parameter '{parameter}'.")
            show_help()
            sys.exit(1)
    elif parameter == "power_mng.soc_target_set":
        try:
            value = float(value)
            if not (0.00 <= value <= 1.00) or len(str(value).split(".")[1]) > 2:
                raise ValueError
        except ValueError:
            print(f"Error: Invalid value '{value}' for parameter '{parameter}'.")
            show_help()
            sys.exit(1)
    elif parameter == "power_mng.battery_power_extern":
        try:
            value = float(value)
            if not (-6000 <= value <= 6000):
                raise ValueError
        except ValueError:
            print(f"Error: Invalid value '{value}' for parameter '{parameter}'.")
            show_help()
            sys.exit(1)
    elif parameter == "power_mng.soc_min":
        try:
            value = float(value)
            if not (0.00 <= value <= 1.00) or len(str(value).split(".")[1]) > 2:
                raise ValueError
        except ValueError:
            print(f"Error: Invalid value '{value}' for parameter '{parameter}'.")
            show_help()
            sys.exit(1)
    elif parameter in ["power_mng.soc_charge_power", "power_mng.soc_charge"]:
        try:
            value = float(value)
        except ValueError:
            print(f"Error: Invalid value '{value}' for parameter '{parameter}'.")
            show_help()
            sys.exit(1)

    object_name = parameter
    command = Command.WRITE
    host_port = (host, 8899)

    object_info = REGISTRY.get_by_name(object_name)
    encoded_value = encode_value(data_type=object_info.request_data_type, value=value)
    send_frame = make_frame(command=command, id=object_info.object_id, payload=encoded_value)

    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    sock.connect(host_port)
    sock.send(send_frame)
    sock.close()

    print(f"Setting value {value} for parameter {parameter} on host {host}")

def get_value(parameter, host):
    valid_parameters = [
        "power_mng.soc_strategy",
        "power_mng.soc_target_set",
        "power_mng.battery_power_extern",
        "power_mng.soc_min",
        "power_mng.soc_charge_power",
        "power_mng.soc_charge"
    ]

    if parameter not in valid_parameters:
        print(f"Error: Invalid parameter '{parameter}'.")
        show_help()
        sys.exit(1)

    object_name = parameter
    command = Command.READ
    host_port = (host, 8899)

    object_info = REGISTRY.get_by_name(object_name)
    send_frame = make_frame(command=command, id=object_info.object_id)

    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    sock.connect(host_port)
    sock.send(send_frame)

    # Receive the response
    response_frame = ReceiveFrame()
    while True:
        ready_read, _, _ = select.select([sock], [], [], 2.0)
        if sock in ready_read:
            # receive content of the input buffer
            buf = sock.recv(256)
            # if there is content, let the frame consume it
            if len(buf) > 0:
                response_frame.consume(buf)
                # if the frame is complete, we're done
                if response_frame.complete():
                    break
            else:
                # the socket was closed by the device, exit
                sys.exit(1)

    # Decode the response value
    decoded_value = decode_value(object_info.response_data_type, response_frame.data)

    sock.close()

    #print(f"The decoded value for '{parameter}' on host {host} is: {decoded_value}")
    print(f"{decoded_value}")

# Main script logic
if __name__ == "__main__":
    if len(sys.argv) < 4 or sys.argv[1] in ("-h", "--help"):
        show_help()
        sys.exit(1)

    subcommand = sys.argv[1]
    host = None

    # Durchlaufe die Argumente, um die Host-Adresse zu extrahieren
    for arg in sys.argv[2:]:
        if arg.startswith("--host="):
            host = arg[len("--host="):]
            break

    if not host:
        print("Error: Host parameter is missing.")
        show_help()
        sys.exit(1)

    if subcommand == "set":
        if len(sys.argv) != 5:
            print("Error: Please provide a parameter, a value, and a host to set.")
            show_help()
            sys.exit(1)
        set_value(sys.argv[2], sys.argv[3], host)
    elif subcommand == "get":
        if len(sys.argv) != 4:
            print("Error: Please provide a parameter and a host to get.")
            show_help()
            sys.exit(1)
        get_value(sys.argv[2], host)
    else:
        print(f"Error: Unknown command '{subcommand}'.")
        show_help()
        sys.exit(1)

Thanks a lot for this proposal!
I've tried your script to GET values from my system and it works.
But I'm confused a little bit, because I've limited maximum SOC to 80% (with strategy "constant") at the moment and I guess, the proper method is power_mng.soc_target_set to get this.
But I was wrong. python test.py get power_mng.soc_target_set --host=<IP_ADDRESS> is reporting 1.0 instead of expected 0.8.
Maybe you advice some help!
Greetings, Heiko

Unfortunately there is no documentation. We have to find out for ourselves how the settings work.
Special RCT Infos
Starting on Page 32
Please share your insights!

The first one is helpful, I think.
@do-gooder, did you test to use power_mng.soc_max instead of power_mng.soc_target_set to dynamically limit the charge?
This register is listed in rctclient documentation and also #8 and #12 in thread "RCT Power Storage - SOC Zielauswahl "Extern" nutzen" you provided.

My script is more than months old. But I don't use it due to lack of time.

Once we have figured out how this all works, we could write a wrapper that provides the functions such as charging from the grid for Tibber, changing the SOC, etc. via an HTTP API. This is what RCT Power should have done.

Hi, meanwhile I've tested to change 'power_mng.soc_max' by adding this to your script in the same way 'power_mng.soc_min' was already available.
Result: Works as expected!
Inverter is is immediately accepting new maximum SOC level and further charges or stops, depending on the actual SOC.
I agree, this could be the basis for any other solution. Just changing the maximum SOC seems to be of limited and acceptable risk for me at the moment. We should remember, that writing settings is not recommended by the rctclient documentation ...
When I can find time during my vacation, I will propose a solution for adjusting maximum SOC from input number of Homeassistant.
Greetings, Heiko

Ok, please use rct_writesupport for your updates!


Cleanup on SIGINT & uncaughtException

Would it be worth supporting cleanup on non-normal process exit, such that:
/** Removes the PID file on normal process exit. */
Pid.prototype.removeOnExit = function() {
    // do something when app is closing
    process.on('exit', this.remove.bind(this));
    // catches ctrl+c event
    process.on('SIGINT', this.remove.bind(this));
    // catches uncaught exceptions
    process.on('uncaughtException', this.remove.bind(this));
};

When doing development I find that the process file remains when I hit ctrl-c.

This is by design since setting handlers for these events override the default node behavior which is to print a stack trace and exit the process. I didn't want this library to interfere with the default behavior since it would be surprising. The way around it is to specify handlers for uncaught exceptions and signals that call process.exit or remove the pid file explicitly before exiting. I'd be open to suggestions in order to integrate some scaffolding for people who want the behavior you describe by default, but it seems unnecessary.


Is this only meant for Discord Bots?

Hey! I just used your module in my WhatsApp Bot, it worked but the problem was that all of the users share the same balance, cooldown and everything. Can you please look into it?

hello, how do you define userID?

userID must be unique for every user e.g you can use there phone number as there ID!

All of them are different.

Thanks for the fast reply.

Thanks for the fast reply.

you need change the function first from settings.user.id to settings.user  and use sender

and for settings.user to give there id just do
user: { id: "some id" }
and the reason for fast reply is... https://discord.gg/qtZpw7S7

and for settings.user to give there id just do user: { id: "some id" }
and the reason for fast reply is... https://discord.gg/qtZpw7S7

Thanks! I did what you told me to and it worked.

good, glad it worked.


Include Limit Columns

Hello,
I just started using the ardalis Specification. Is it possible to limit the columns when using include?
Query.Include(x => x.SomeItems.Select(s => new item( Id = s.Id, Col1 = s.Col1)));
Is something like that possible?
Thanks

Hmm, I don't remember. I know basic Select works, basic Include works, and I think Include Where works. I'm not sure I've tried Include with Selection. Have you tried it? Did it work for you?

I have tried, but the extension is not recognized.. so I assumed its not possible or I was not doing it correctly. I did a search of the issues to see if others had tried something similar, but did not find anything. If working with the DbContext directly, I am able limit the query to only the columns needed.
var entity = _appDbContext.ServerAccess
                .Where(x => x.Id == request.Id)
                .Select(x => new ServerAccess() {
                    Id = x.Id,
                    AccessMethod = x.AccessMethod,
                    Description = x.Description,
                    InventoryItems = _appDbContext.SqlInventory
                                    .Where(p => p.AccessId == x.Id)
                                    .Select(s => new SqlInventory()
                                    {
                                        InstanceName = s.InstanceName
                                    }
                                    ).ToList()
                })
                .FirstOrDefault();

I will keep reading your docs and the videos you have posted online.
Thanks

Just following up. I have not been able to limit the columns returned from include using a specification.
I have the specification defined called ServerAccessSpecById
public class ServerAccessSpecById: Specification<ServerAccess, ServerAccess>
    {
        public ServerAccessSpecById(int id)
        {
            Query.Select(x => new ServerAccess()
            {
                Id = x.Id,
                AccessMethod = x.AccessMethod,
                Description = x.Description
            }) ;

            Query.Where(x => x.Id == id);

            Query.Include(x => x.InventoryItems.Select(x => new SqlInventory(){ InstanceName = x.InstanceName }));
        }
    }

When the query executes, I get the error below.
InvalidOperationException: The expression 'x.InventoryItems.AsQueryable().Select(x => new SqlInventory() {InstanceName = x.InstanceName})' is invalid inside an 'Include' operation, since it does not represent a property access: 't => t.MyProperty'. To target navigations declared on derived types, use casting ('t => ((Derived)t).MyProperty') or the 'as' operator ('t => (t as Derived).MyProperty'). Collection navigation access can be filtered by composing Where, OrderBy(Descending), ThenBy(Descending), Skip or Take operations.

@goforebroke
Your specification is not the same as query using dbSet directly.
More specifically:
Query.Select(x => new ServerAccess()
            {
                Id = x.Id,
                AccessMethod = x.AccessMethod,
                Description = x.Description
            }) ;

is not setting InventoryItems so they're not part of the expression thus efcore will ignore Include IF it could be compiled but if Include has Select it couldn't.
As i can see this project does not support such queries because efcore relies on captured dbSet property, see expression visualization
dbContext.Stores!
        .AsNoTrackingWithIdentityResolution()
        .Where(x => x.Id == StoreSeed.VALID_STORE_ID)
        .Select(x => new Store
        {
          Id = x.Id,
          City = x.City,
          Products = dbContext.Products!
            .Where(z => z.StoreId == x.Id)
            .Select(z => new Product { Id = z.Id, Name = z.Name, })
            .ToList(),
        })

[Microsoft.EntityFrameworkCore.Query.QueryRootExpression]
    .AsNoTrackingWithIdentityResolution<Store>()
    .Where<Store>(x => x.Id == 1)
    .Select<Store, Store>(
        x => new Store
        {
            Id = x.Id,
            City = x.City,
            Products = RepositoryOfT_GetBySpec.dbContext.Products // << Expression.Property(
                                                       // Expression.Constant(dbContext), productsPropertyInfo)
                .Where<Product>(z => z.StoreId == x.Id)
                .Select<Product, Product>(z => new Product
                {
                    Id = z.Id,
                    Name = z.Name
                })
                .ToList<Product>()
        })

Test
public class StoreByIdIncludeProductsWithSelectSpec : Specification<Store, Store>
{
  public StoreByIdIncludeProductsWithSelectSpec(int id)
  {
    Query.Where(x => x.Id == id);
    Query.Select(x => new Store
    {
      Id = x.Id,
      City = x.City,
      Products = ((IQueryable<Product>)null!) // not working, efcore needs dbSet property
        .Where(z => z.StoreId == x.Id)
        .Select(z => new Product { Id = z.Id, Name = z.Name, })
        .ToList(),
    });
  }
}

[Fact]
public virtual async Task ReturnStoreWithProductsAndCustomMaterialization()
{
  // using spec
  _ = await Assert.ThrowsAsync<InvalidOperationException>(() => storeRepository.GetBySpecAsync(new StoreByIdIncludeProductsWithSelectSpec(StoreSeed.VALID_STORE_ID)));

  // using ef core
  // left join on products
  _ = await dbContext.Stores!
    .AsNoTrackingWithIdentityResolution()
    .Where(x => x.Id == StoreSeed.VALID_STORE_ID)
    .Select(x => new Store
    {
      Id = x.Id,
      City = x.City,
      Products = dbContext.Products!
        .Where(z => z.StoreId == x.Id)
        .Select(z => new Product { Id = z.Id, Name = z.Name, })
        .ToList(),
    })
    .SingleAsync();

  // ignores products
  _ = await dbContext.Stores!
    .AsNoTrackingWithIdentityResolution()
    .Where(x => x.Id == StoreSeed.VALID_STORE_ID)
    .Include(x => x.Products)
    .Select(x => new Store { Id = x.Id, City = x.City, })
    .SingleAsync();

  var exception = await Assert.ThrowsAsync<InvalidOperationException>(() => dbContext.Stores!
    .AsNoTrackingWithIdentityResolution()
    .Where(x => x.Id == StoreSeed.VALID_STORE_ID)
    .Include(x => x.Products.Select(z => new Product { Id = z.Id, Name = z.Name, }))
    .Select(x => new Store { Id = x.Id, City = x.City, })
    .SingleAsync());
  Assert.Equal(
    "The expression 'x.Products.AsQueryable().Select(z => new Product() {Id = z.Id, Name = z.Name})' is invalid inside an 'Include' operation,"
    + " since it does not represent a property access: 't => t.MyProperty'. To target navigations declared on derived types,"
    + " use casting ('t => ((Derived)t).MyProperty') or the 'as' operator ('t => (t as Derived).MyProperty')."
    + " Collection navigation access can be filtered by composing Where, OrderBy(Descending), ThenBy(Descending), Skip or Take operations."
    + " For more information on including related data, see http://go.microsoft.com/fwlink/?LinkID=746393.",
    exception.Message);
}

@devbased
Thank you for the explanation and the example...much appreciated... Closing issue/question


net/http/transport.go: Is it possible to reuse gzip reader from pool

https://github.com/golang/go/blob/089e37a931d30d4055c7468facb602c4cfa9b537/src/net/http/transport.go#L2858
Is it possible to reuse this gzip reader from a sync.Pool to reduce the memory allocation cause by flate.NewReader

gzip.Reader does have a Reset method, but we would also need some way to know when the gzip.Reader is safe to reuse.
Hmm. (*http.gzipReader).Read already has some locking in Read. Perhaps we could do something like:
func (gz *gzipReader) Read(p []byte) (n int, err error) {
	gz.body.mu.Lock()
	if gz.body.closed {
		err = errReadOnClosedResBody
	} else if gz.zr == nil {
		if gz.zerr == nil {
			zr := gzipPool.Get().(*gzip.Reader)
			gz.zerr = zr.Reset(gz.body)
			if gz.zerr == nil {
				gz.zr = zr
			} else {
				gzipPool.Put(zr)
			}
		}
		err = gz.zerr
	}
	gz.body.mu.Unlock()

	if err != nil {
		return 0, err
	}
	return gz.zr.Read(p)
}

func (gz *gzipReader) Close() error {
	err := gz.body.Close()

	gz.body.mu.Lock()
	if gz.zr != nil {
		gz.zr.Reset(emptyReader{})  // drop reference to gz.body
		gzipPool.Put(gz.zr)
		gz.zr = nil
	}
	gz.body.mu.Unlock()

	return err
}

var (
	gzipPool = sync.Pool{
		New: func() any { return new(gzip.Reader) },
	}
	_ flate.Reader = emptyReader{}
)

type emptyReader struct{}

func (emptyReader) Read([]byte) (int, error) { return 0, io.EOF }
func (emptyReader) ReadByte() (byte, error) { return 0, io.EOF }

Of course, such a change would also need a supporting benchmark. @Sunyue, want to send it for Go 1.22?
(CC @neild)

Seems like something we could do; I'm happy to review a CL if someone wants to send me one.

Created https://github.com/golang/go/pull/61390


Request to merge implementation of fromPsr7 method

Ok, arm twisted. Would love to stop using my own fork of the Guzzle library and have found a legitimate use-case for this method. Check it out and merge, if satisfied. Otherwise, let me know what else needs to change. (refs PR #158 and issue #159)

Thank you for this PR.
Can you elaborate on the use cases for this?

@Nyholm This might horrify you guys, given our little back-and-forth about ServerRequest-as-value, but I've been experimenting with the concept of encapsulating layered communications protocol validations in the request object itself, among other things, and also with delegating requests to different submodules using more specialized Request objects (e.g., ExampleApiRequest vs TesterApiRequest, which might encapsulate two different authentication mechanisms appropriate for the "Example" api and "Tester" api, respectively).
Specifically, I've been working with JsonApi to handle API requests. If you don't know about it, it's both a relational data specification and a data transport protocol built on top of HTTP. That means that there's at least some logic that can be considered part of the request itself, and I'm experimenting with what it might be like to actually put it there. (For example, the actual parts of the path are meaningful and consistent; the declared Accept and Content-Type headers and their values are part of the protocol itself.)
I originally wanted to extend ServerRequest to capture this.
Now, though, I'm playing more with converting requests from one type to another easily to handle hierarchical applications. In this case I might start out with a general ApiRequest that has a getApiName method (extracting meaning from the parts of the path) that allows me to delegate to more specific apis. I would then use fromPsr7 (or, as @sagikazarmark suggested, and I agree, fromRequest) to convert from the more general ApiRequest to the more specific ExampleApiRequest or TesterApiRequest according to what my app knows about the result of the getApiName() method, then delegate that request to those other sections of my hierarchical app. Then of course if those levels delegate further, they can convert the Request to still different versions.
I don't know. Like I said, it's an experiment. It's been fun and promising so far, though, so I'm keen on continuing it.

I took a look at those conversations and I would like to bring up some other points backing @Nyholm 's side: I like to think about PSR-7 objects as a direct representation of HTTP messages which they are but we often forget about this detail.
When you take a look at them from this point of view it's quite obvious that the layer where you operate on PSR-7 objects is part of your transport layer.
For me it's also obvious that you don't want your transport layer mixed with higher level API layers. Take a look at this example: let's say you have an RPC API over HTTP using JSON as the serialization layer. You have these specialized request objects in your code.
Now let's say you want to release a 2.0 of your RPC API, but you have rewritten the backend using gRPC and protocol buffers. You probably have to rewrite everything from scratch since PSR-7 stuff leaks into your higher level APIs of your code.
If you strictly separate the transport layer and your API in your code, you can quite easily replace the transport layer with a new one based on whatever new technology you used.
Of course I don't know every detail of what you are doing, but normally strict separation means you transform your PSR-7 requests to something else which does not leak PSR-7 into that context anymore, so it sounds like you don't have that kind of separation in your code.

I'm still opposed to this change, what you are suggesting as a use case is an obvious break of SRP IMHO (see the explanation above) and supporting that as a library is one of the worst things that maintainers can do.
Also, I don't see why this tiny function couldn't just be a function somewhere in your code. Libraries are supposed to be focused on a narrow set of features and not bloated by user specific use cases (unless that's widely used enough to support which I don't think is the case now).
So I decided to reject this feature for now.


upd(bitwarden-deb): 1.31.3 -> 1.32.0

The maintainer is already requested to update the pacscript: https://github.com/pacstall/pacstall-programs/pull/849#pullrequestreview-912877225
Closing this.


[PL-32478] Update API rate limit documentation

Harness Developer Pull Request
Thanks for helping us make the Developer Hub better. The PR will be looked at
by the CODEOWNERS.
What Type of PR is This?

[ ] Issue
[ ] Feature
[ ] Maintenance/Chore

If tied to an Issue, list the Issue(s) here:

Issue(s)

House Keeping
Some items to keep track of. Screen shots of changes are optional but would help the maintainers review quicker.

[ ] Tested Locally
[ ] Optional Screen Shoot.

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://656e63878ceaa70085aafedb--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://656f3d7eaae099095619296a--harness-developer.netlify.app


How can I set it up to use markdown?

Hi!
Seems like the documentation has not been updated since you changed it to using html instead of markdown. In one of the issue someone wrote that it is possible to set it back to using markdown. How can I achieve this?
I'd like to let users to submit content through sirtrevor.
Additionally, the heading element inserts a p tag around the contents. Don't know if this is intentional. ( Chrome Version 40.0.2214.111 (64-bit) - xubuntu )

Hi,
Yes, currently the documentation is a bit behind. My current recommendation, if you want to use Markdown is using latest stable version (v0.4.3) of SirTrevor - current v0.5 betas are rather unstable (wrapping heading in <p> elements that you noticed was one of the problems).
When we release v0.5 there will be an option that you could enable to save blocks back in Markdown, however it's going to be deprecated in next versions (rationale behind our move from Markdown to HTML is available in this comment)

Thanks for the answer!
Yeah, seems sensible. Only problem now is that XSS type attacks become much more possible. That's why I just wanted to use markdown but it is true that rendering it is not very easy.

I get your point.
You have to remember that standard Markdown lets you inline HTML as well, so you have to filter them from the input. Behaviour for HTML as data format isn't that much different, you just have to sanitize SirTrevor generated content (probably using whitelist).

If I didn't answer all your questions or you want to know more, reopen the issue.

Please add some additional documentation to mitigate XSS attacks.
As I understand it, a client uses sir trevor to generate some json which contains html, and sends that json to a server.
The server must sanitize the untrusted client supplied json, and if the server does not, that json might contain an XSS which could be rendered by sir trevor (stored xss vuln).
The responsibility is on the server to sanitize the json, please confirm, and provide any relevant references.

You are correct.
The responsibility is on the server, as it is with any user entered
information, to validate / sanitize the content.
The normal process we adhere to is to analyse the json array on submission
and sanitize any fields that need doing so before it gets to the database.
Hope that answers your question.
Raffi
On Tue, 6 Jun 2017 at 18:23, Orie Steele<EMAIL_ADDRESS>wrote:

Please add some additional documentation to mitigate XSS attacks.
As I understand it, a client uses sir trevor to generate some json which
contains html, and sends that json to a server.
The server must sanitize the untrusted client supplied json, and if the
server does not, that json might contain an XSS which could be rendered by
sir trevor (stored xss vuln).
The responsibility is on the server to sanitize the json, please confirm,
and provide any relevant references.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/madebymany/sir-trevor-js/issues/293#issuecomment-306557565,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABvmqIm5y78OhcSDcPET7e3eMrnfQyPks5sBYsFgaJpZM4DemfS
.


Index on "index_oauth_access_tokens_on_token" fails on Postgres

When a user creates an oauth token, the following error is being generated by Postgres:
ERROR: index row size [NUMBER] exceeds maximum [NUMBER] for index "index_oauth_access_tokens_on_token" HINT: Values larger than [NUMBER]/[NUMBER] of a buffer page cannot be indexed. Consider a function index of an MD5 hash of the value, or use full text indexing.

The error seems to occur due to the btree index used on token, as btree indexes can't handle data with more that 2713 bytes.
This doesn't seem to occur using doorkeeper without doorkeeper-jwt.
I've resolved the issue by replacing the hashing function of the index with an md5 function, which I've detailed on the doorkeeper wiki.
There should probably be a reference to the issue over here, for the benefit of future doorkeeper-jwt users. I'm happy for you to either reference the doorkeeper wiki, or to copy the solution - up to you.

Sorry for the delay in responding to this - no excuse for it.
Thanks for figuring this out and writing up a solution. What do you think would be the best way to reference the issue? What would make the most sense to you if you came across it?

I'd likely find this through the error itself. Certainly that's what led me to the solution.


add links around twitter, GitHub, homepage

Also there should be a check around each of them, if the user supplied it (like you do with twitter in the header manually).
Maybe just use a templating engine like moustache?
fixes #4

Man, getting help 2 hours after releasing the beta. You are awesome.
Would it be a hassle investigating the complexity of adding mustach/handlebars/etc instead?
If you don't have the time, I am more than glad to merge this.

I'll take a look which seems most suitable

According to https://garann.github.io/template-chooser/, I'd use https://github.com/janl/mustache.js/, since it's Mustache, thus more widely known. Handlebars seems to add more things, but that also makes it complicated. In this case we'd use:
{#twitter}

twitter: {{twitter}}

{/twitter}

As seen here
Asking the render should be like this:
Mustache.render('get this from the index file', {data});

Anyway, this pull request can be merged before deciding on a templating system, since it'll stay the same.
P.S. another options is liquid-node, if you prefer that syntax (like Jekyll)


Multiple matrices in linear combination

Here is a suggestion for #73 . Vectors instead of Tuples. Still WIP.
Modifications needed:

[ ] Change graph_ps to use multiple add_lincombs
[ ] Code generation Julia
[ ] Code generation Matlab
[ ] Code generation BLAS (maybe convert to a bi-graph again)
[ ] graph compression

The "old" tests now pass.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (6d103c0) 93.72% compared to head (1a1ab74) 93.29%.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##             main      #75      +/-   ##
==========================================
- Coverage   93.72%   93.29%   -0.44%
==========================================
  Files          30       30
  Lines        2646     2610      -36
==========================================
- Hits         2480     2435      -45
- Misses        166      175       +9

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

This seems pretty ready.  BLAS-tests done separately. Coverage okay.  I will merge this in a couple of days unless there are other comments.


AdamsMoulton increase t+=dt twice

I'm looking at the AdamsMoultonSolver::Step function, and it has this fragment:
   if (s >= smax-1)
   {
      f->SetTime(t);
      for (int i = 1; i < smax; i++)
      {
         x.Add(a[i]*dt, k[idx[i]]);
      }
      f->ImplicitSolve(a[0]*dt, x, k[idx[0]]);
      x.Add(a[0]*dt, k[idx[0]]);
   }
   else
   {
      RKsolver->Step(x,t,dt);
      f->Mult(x,k[idx[0]]);
   }
   t += dt;

The first if section is most time steps, and the else to initialize solutions for the multistep method with appropriate HO accuracy. But by calling RKsolver->Step(x,t,dt), this should already increase t+=dt, and then after the if-statement, this is called again. As far as I understand, this is increasing dt twice for the initialization time steps, and t += dt; shoudl be moved inside of the if statement.

It does look like a bug. @IdoAkkerman, any thoughts?
On the other hand, the unit tests (https://github.com/mfem/mfem/blob/master/tests/unit/linalg/test_ode.cpp) seem to work fine.

One other comment: I believe these multistep methods assume a fixed dt over multiple time steps. So something like the common MFEM syntax inside a time-stepping loop
      double dt_real = min(dt, tf - t);
      ode->Step(u, t, dt_real);

would break this if dt doesn't perfectly divide tf (i.e., if you change dt), and you could lose accuracy on your final step.

ping: @IdoAkkerman

One other thing I noticed which I am confused about here is the use of implicit solve. Take trapezoid rule,
y_{n+1} = y_n + 0.5*dt*( f(y_n,t_n) + f(y_{n+1},t_{n+1}) )    (1)
Here, the implicit solve takes the form
y_{n+1} - 0.5*dt*f( y_{n+1}, t_{n+1} ) = b      (2)
But the TimeDependentOperator ImplicitSolve function solves k = f(x + dt*k, t). So in AdamsMoulton, when we call
f->ImplicitSolve(a[0]*dt, x, k[idx[0]]);
we are solving
k = f(x + a[0]*dt*k)    (3)
where we have defined x as a sum over previous k, and then finish with the additive update x += a[0]*dt*k.
Is this a different way to write Adams Moulton methods than (1)? It's not clear to me how (3) and (2) are equivalent for a general nonlinear function. But, if there is a way to write multistep methods like this, it may be beneficial for a BDF implementation using the same functions as RK (https://github.com/mfem/mfem/pull/2474).

ping: @IdoAkkerman

Sorry.. I still have to learn how to use github and the notification system.

@bensworth I think you are right on the first 2 issues, 3 issue I do not know.

There is likely a doubling up of dt initially, I 'll check.
I am not  and AM/AB specialist, so my understanding is the integrator works for constant time steps. Perhaps  a check needs to be added.
I need to check you question regarding nonlinear functions. I do remember the time stepper is a bit odd as you solve for the rate, not the solution. This has to do with a uniform handling of implicit/explicit integrator, but makes everything slightly different then usual.

I think the idea is as follows.
Take trapezoid rule,
y_{n+1} = y_n + 0.5*dt*( f(y_n,t_n) + f(y_{n+1},t_{n+1}) ) (1)
Define
k[0] = f(y_{n+1},t_{n+1}) (2a)
k[1] = f(y_{n},t_{n}) (2b)
This translates (1) directly into
y_{n+1} = y_n  + 0.5*dt*k[1] + 0.5*dt*k[0] (3)
In the loop
 for (int i = 1; i < smax; i++)
{
      x.Add(a[i]*dt, k[idx[i]]);
}

we compute
x = y_n  + 0.5*dt*k[1]  (4)
Using this x eq (3) can be written as
y_{n+1} = x + 0.5*dt*k[0] (5)
Using this in eq (2a) can be written as
k[0] = f(x + 0.5*dt*k[0],t_{n+1}) (6)
Which is what is solve in
f->ImplicitSolve(a[0]*dt, x, k[idx[0]]);
I hope that explains it? And does not have any errors...
I think it is just a reshuffling of terms, it does not rely on the function being linear.

In short: we do not solve for y_{n+1}. Instead we solve for k[0], which requires some rewriting.
This rewriting does not invoke linearity of the underlying problem.

Great, that makes sense, I just hadn't seen the multistep methods written in that way. Will move discussion to the PR https://github.com/mfem/mfem/pull/3466.

Thanks @IdoAkkerman


User control of how enums are interpreted

Describe the bug
When an enum has only values of 2^n it is treated as a combination of flags and or'ed values are interpreted correctly.  However, if there's a value that is also a combination of 2 or more flags things get weird.
To Reproduce
Eg, if I create a simple 1 byte enum with 8 elements 1,2,4,..,128 and apply it to a variable whose value is 3, everything okay.  However, if I add a 9th element with value 0x?1 the enum combo doesn't get output, just the number 3.  However, if I add the 9th element as value 0x?2, the combo gets printed.  If I add a number either 9,10,11,12,13,14 or 15, the combo is printed.  If I add either 5 or 7, I get a weird ~ combo printed.
Expected behavior
The correct or'ed list of flags to be output.
Environment (please complete the following information):

OS: Windows 10.
Java Version: 16.0.1
Ghidra Version: 10.X
Ghidra Origin: locally built

sounds sort of similar to what I was seeing in #3810

sounds sort of similar to what I was seeing in #3810

@mumbel Can you confirm this is still broken?  I know we have done some work in this area.

@dragonmacher   This appears to be there still as of faf73e8ecb26320a998da6af1a0dcb81c427657f
enum bitsenum {
BIT0 = 1,
BIT1 = 2,
BIT2 = 4,
BIT3 = 8,
BIT4 = 0x10,
BIT5= 0x20,
BIT6 = 0x40,
BIT7 = 0x80
}; // sizeof(enum bitsenum) == 1

extern void foo(enum bitsenum);

void test(void)
{
    foo(3);
}

in the decompiler this will show as foo(BIT0|BIT1)
if you were to add to the enum  WRONG=0x11 and save, the decompiler will show 3... remove WRONG and its back to BIT0|BIT1


Fix sync traversals of containers passed as r-value

Proposed Changes

This fixes the reference correctness of traverse_pack when traversing a container passed as rvalue

@Naios could you please fix the remaining inspect check errors (https://9740-4455628-gh.circle-artifacts.com/0/tmp/circle-artifacts.pTMGwoc/hpx_inspect_report.html)? We can't move ahead with merging otherwise. Thanks!

@Naios you can ignore the AppVeyor problem (this is unrelated to your work), but could you please look into the errors reported by our pycicle CI system?

@hkaiser I took a look into the error, this is a greater bug with which affects the whole GCC (6.x) family.
I could reproduce it on one of my remote machines using GCC 6.3.
Currently I don't have the resources to reproduce it (currently I'm missing time to do it and also I don't have an environment for debugging GCC properly).


ConditionalWidget known bug and issue

even ConditionalWidget works like a ternary operator but Flutter code doesn't perform like it.
When providing onFalse and onTrue widget, both widgets have already been initialized, so you will get an error on some exception like provide null imageUrl to CachedNetworkImage even though we do not add them to widget tree yet.
Workaround: Replace ConditionalWidget with Ternary operator instead

Solution: change onTrue and onFalse type from Widget to a function that returns a widget


Change LICENSE to AGPLv3

Closes osism/openstack-image-manager#663

@fkr @garloff

Relicensing to AGPLv3 would solve the issue in #663 - at least if I understand this document correctly:

The ordinary GNU General Public License and the GNU Affero General Public License are two different copyleft licenses, so they are naturally incompatible. We have set up a special kind of explicit compatibility between them: you can include source code under the GNU GPL version 3 together with other source code under the GNU Affero GPL in a single combined program. This is permitted because both of those licenses explicitly say so, and the effect is that the GNU AGPL applies to the combined program.

However the usual IANAL applies.

We should have a process for how we do this with SCS projects. Which team can we best bring this to?

I'm not really sure, wether I can assess wether the relicensing can take place, since I'm not sure wether all (substantial) contributors have waived their rights to do this or need to be asked.

I did some more research on this and while the IANAL still applies, my opinion is as follows:
The license change from a permissive (ASL-2) license to AGPLv3 is possible without involving the other (substantial contributors) as long as the new license still complies with the minimal requirements of the (permissive) license.
As to @berendt's question:

We should have a process for how we do this with SCS projects. Which team can we best bring this to?

PB is the best place atm.

Let's discuss & document this in the next PB.


az dls fs access set-entry errors out in powershell, bash and shell

Originally reported in DevOps, but they don't think its their problem.
https://developercommunity.visualstudio.com/content/problem/1090879/az-dls-fs-access-set-entry-errors-out-in-powershel.html
While running manually in Azure web Shell, command succeeds and returns no errors.
While running the same command in Azure DevOps release pipeline with fail on standard error turned off:
Powershell succeeds in adding access but returns error and fails entire deployment.
Bash fails to add access, returns error and fails deployment
Shell fails to add access, returns error but doesn't fail deployment
All other CLI commands in the script run successfully, its just this one that fails and only in deployment pipeline.
I'm running out of ideas, please help I can't move forward with this issue.
Powershell:
az dls fs access set-entry --account datalakeName --acl-spec group:"test-datalake-store-dev":-w --path /
Bash:
az dls fs access set-entry --account %1 --acl-spec group:"%2":-w --path /
Shell:
az dls fs access set-entry --account $1 --acl-spec group:"$2":-w --path /
I was able to bypass this issue by using straight AZ Powershell, but Cli command keeps failing.
$SecurityGroupId = (Get-AzADGroup -SearchString $DataLakeSecurityGroupName).Id
Set-AzDataLakeStoreItemAclEntry -AccountName "$DataLakeName" -Path / -AceType Group -Id $SecurityGroupId -Permissions All
I did try using --only-show-errors at the end of the command, without any success in pipeline.
In Azure portal I open the Azure Cloud Shell with Powershell mode and run this, then verify and user or group was added with access correctly.
PS /home/username> az dls fs access set-entry --account datalakestorename --acl-spec user:"username":-w- --path /
Command group 'dls' is in preview. It may be changed/removed in a future release.
PS /home/username>
cli in DevOps  Error output :
2020-06-24T13:48:04.6936772Z D:\a\r1\a>REM Normal code goes here 2020-06-24T13:48:04.6937157Z 2020-06-24T13:48:04.6937859Z D:\a\r1\a>ECHO setting reporting datalakeName access policy for test-datalake-store-dev 2020-06-24T13:48:04.6938439Z setting reporting datalakeName access policy for test-datalake-store-dev 2020-06-24T13:48:04.6938792Z 2020-06-24T13:48:04.6941676Z D:\a\r1\a>az dls fs access set-entry --account datalakeName --acl-spec group:"test-datalake-store-dev":-w --path / 2020-06-24T13:48:06.1569780Z Command group 'dls' is in preview. It may be changed/removed in a future release. 2020-06-24T13:48:07.5705928Z The command failed with an unexpected error. Here is the traceback: 2020-06-24T13:48:07.5706266Z 2020-06-24T13:48:07.5724150Z Data-lake REST exception: MODIFYACLENTRIES, . 2020-06-24T13:48:07.5724548Z Traceback (most recent call last): 2020-06-24T13:48:07.5724939Z File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-install-kvuguic\knack\knack\cli.py", line 215, in invoke 2020-06-24T13:48:07.5725559Z File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure\cli\core\commands_init.py", line 654, in execute 2020-06-24T13:48:07.5726177Z File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure\cli\core\commands_init_.py", line 718, in run_jobs_serially 2020-06-24T13:48:07.5726758Z File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure\cli\core\commands_init.py", line 711, in run_job 2020-06-24T13:48:07.5727281Z File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-install-kvuguic\six\six.py", line 703, in reraise 2020-06-24T13:48:07.5727795Z File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure\cli\core\commands_init.py", line 688, in run_job 2020-06-24T13:48:07.5728369Z File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure\cli\core\commands_init.py", line 325, in call 2020-06-24T13:48:07.5728946Z File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure\cli\core_init.py", line 545, in default_command_handler 2020-06-24T13:48:07.5729566Z File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure\cli\command_modules\dls\custom.py", line 304, in set_adls_item_acl_entry 2020-06-24T13:48:07.5730191Z File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-install-_kvuguic\azure-datalake-store\azure\datalake\store\core.py", line 509, in modify_acl_entries 2020-06-24T13:48:07.5730816Z File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-install-_kvuguic\azure-datalake-store\azure\datalake\store\core.py", line 459, in _acl_call 2020-06-24T13:48:07.5731409Z File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-install-_kvuguic\azure-datalake-store\azure\datalake\store\lib.py", line 454, in call 2020-06-24T13:48:07.5732009Z File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-install-_kvuguic\azure-datalake-store\azure\datalake\store\lib.py", line 349, in log_response_and_raise 2020-06-24T13:48:07.5732599Z azure.datalake.store.exceptions.DatalakeRESTException: Data-lake REST exception: MODIFYACLENTRIES, . 2020-06-24T13:48:07.5732895Z 2020-06-24T13:48:07.5733107Z To open an issue, please run: 'az feedback' 2020-06-24T13:48:07.7208548Z ##[error]Script failed with exit code: 1 2020-06-24T13:48:07.7448489Z [command]C:\windows\system32\cmd.exe /D /S /C ""C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\wbin\az.cmd" account clear" 2020-06-24T13:48:09.4233766Z ##[section]Finishing: data lake store access

add to S173

I found another issue, same problem, command works in web terminal but it fails in DevOps release pipeline
I'm not sure if your coming up fix will repair this too or not, but figure you may want to know this is widespread issue.
command:
$dlaExists =  az dla account list --subscription $SubscriptionId --query "[?contains(name, '$DataLakeAnalyticsName')].[name]" --output tsv
error output:
2020-07-14T13:09:25.7274118Z [command]C:\windows\System32\WindowsPowerShell\v1.0\powershell.exe -NoLogo -NoProfile -NonInteractive -ExecutionPolicy Unrestricted -Command ". 'd:\a_temp\azureclitaskscript1594732156968.ps1'"
2020-07-14T13:09:33.8864291Z az : Command group 'dla' is in preview. It may be changed/removed in a future release.
2020-07-14T13:09:33.8872287Z At D:\a\r1\a\foldername\projectname\foldername\DataLake.ps1:65 char:15
2020-07-14T13:09:33.8873344Z + ... laExists =  az dla account list --subscription '12345678-1234-1234-12 ...
2020-07-14T13:09:33.8874158Z +                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2020-07-14T13:09:33.8875354Z     + CategoryInfo          : NotSpecified: (Command group '...future release.:String) [], RemoteException
2020-07-14T13:09:33.8876222Z     + FullyQualifiedErrorId : NativeCommandError
2020-07-14T13:09:33.8876759Z
2020-07-14T13:09:33.9244178Z ##[error]Script failed with exit code: 1

@akharit could you also take a look? I can reproduce with current azure cli.
The command failed with an unexpected error. Here is the traceback:

Data-lake REST exception: MODIFYACLENTRIES, .
Traceback (most recent call last):
  File "E:\Microsoft\AzureCLI\azure-cli\env\lib\site-packages\knack\cli.py", line 215, in invoke
    cmd_result = self.invocation.execute(args)
  File "e:\microsoft\azurecli\azure-cli\src\azure-cli-core\azure\cli\core\commands\__init__.py", line 654, in execute
    raise ex
  File "e:\microsoft\azurecli\azure-cli\src\azure-cli-core\azure\cli\core\commands\__init__.py", line 718, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
  File "e:\microsoft\azurecli\azure-cli\src\azure-cli-core\azure\cli\core\commands\__init__.py", line 711, in _run_job
    six.reraise(*sys.exc_info())
  File "E:\Microsoft\AzureCLI\azure-cli\env\lib\site-packages\six.py", line 703, in reraise
    raise value
  File "e:\microsoft\azurecli\azure-cli\src\azure-cli-core\azure\cli\core\commands\__init__.py", line 688, in _run_job
    result = cmd_copy(params)
  File "e:\microsoft\azurecli\azure-cli\src\azure-cli-core\azure\cli\core\commands\__init__.py", line 325, in __call__
    return self.handler(*args, **kwargs)
  File "e:\microsoft\azurecli\azure-cli\src\azure-cli-core\azure\cli\core\__init__.py", line 776, in default_command_handler
    return op(**command_args)
  File "e:\microsoft\azurecli\azure-cli\src\azure-cli\azure\cli\command_modules\dls\custom.py", line 304, in set_adls_item_acl_entry
    client.modify_acl_entries(path, acl_spec)
  File "e:\microsoft\azurecli\azure-cli\env\lib\site-packages\azure\datalake\store\core.py", line 509, in modify_acl_entries
    self._acl_call('MODIFYACLENTRIES', path, acl_spec, invalidate_cache=True)
  File "e:\microsoft\azurecli\azure-cli\env\lib\site-packages\azure\datalake\store\core.py", line 459, in _acl_call
    to_return = self.azure.call(action, posix_path, **parms)
  File "e:\microsoft\azurecli\azure-cli\env\lib\site-packages\azure\datalake\store\lib.py", line 454, in call
    self.log_response_and_raise(response, err, level=exception_log_level)
  File "e:\microsoft\azurecli\azure-cli\env\lib\site-packages\azure\datalake\store\lib.py", line 349, in log_response_and_raise
    raise exception
azure.datalake.store.exceptions.DatalakeRESTException: Data-lake REST exception: MODIFYACLENTRIES, .

@Juliehzl  Sorry somehow missed this in my email.
Can you tell me how you reproduced this?

@Juliehzl Sorry somehow missed this in my email.
Can you tell me how you reproduced this?

Hi!
I too have this issue!
You can reproduce it with the CLI :
$ az dls fs access set-entry --account testdlsaccount --acl-spec user:${A_USER_ID}:r-w --path "/"
Command group 'dls' is in preview. It may be changed/removed in a future release. CLIInternalError: The command failed with an unexpected error. Here is the traceback: Data-lake REST exception: MODIFYACLENTRIES, . Traceback (most recent call last):   File "/usr/local/lib/python3.6/site-packages/knack/cli.py", line 215, in invoke     cmd_result = self.invocation.execute(args)   File "/usr/local/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 654, in execute     raise ex   File "/usr/local/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 718, in _run_jobs_serially     results.append(self._run_job(expanded_arg, cmd_copy))   File "/usr/local/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 711, in _run_job     six.reraise(*sys.exc_info())   File "/usr/local/lib/python3.6/site-packages/six.py", line 703, in reraise     raise value   File "/usr/local/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 688, in _run_job     result = cmd_copy(params)   File "/usr/local/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 325, in __call__     return self.handler(*args, **kwargs)   File "/usr/local/lib/python3.6/site-packages/azure/cli/core/__init__.py", line 784, in default_command_handler     return op(**command_args)   File "/usr/local/lib/python3.6/site-packages/azure/cli/command_modules/dls/custom.py", line 304, in set_adls_item_acl_entry     client.modify_acl_entries(path, acl_spec)   File "/usr/local/lib/python3.6/site-packages/azure/datalake/store/core.py", line 509, in modify_acl_entries     self._acl_call('MODIFYACLENTRIES', path, acl_spec, invalidate_cache=True)   File "/usr/local/lib/python3.6/site-packages/azure/datalake/store/core.py", line 459, in _acl_call     to_return = self.azure.call(action, posix_path, **parms)   File "/usr/local/lib/python3.6/site-packages/azure/datalake/store/lib.py", line 454, in call     self.log_response_and_raise(response, err, level=exception_log_level)   File "/usr/local/lib/python3.6/site-packages/azure/datalake/store/lib.py", line 349, in log_response_and_raise     raise exception azure.datalake.store.exceptions.DatalakeRESTException: Data-lake REST exception: MODIFYACLENTRIES, . To open an issue, please run: 'az feedback'
Hope this helps!


Question on clamp weights

In the current implementation, the parameters of batchnorm layers will also be clamped. Is this a desired behavior?

Discussed in #18


[data-export] Add Opt-in Localization for Date-Time Formatting

Is your feature request related to a problem? Please describe.
The Data Export functionality currently works as expected. However, this request focuses on a feature enhancement to support localized date-time formats. Currently, my colleagues need to manually convert date-time values to the stakeholder's preferred format (such as en-US, en-GB, etc.) when creating ad-hoc data reports. Enabling users to configure localization settings before export would save time for administrators, power users, and developers.
Describe the solution you'd like
Introduce an opt-in localization feature within the Data Export settings (Options -> Data Export). Users should be able to choose a preferred locale, or select from two special options:

'' (empty / default): No format conversion, display date-time as provided by Salesforce.
local: Use the browser's navigator.language setting.

To promote maintainability and consistency, this feature should rely on the "Intl.DateTimeFormat" API, and users should be encouraged to refer to standards organizations for proposed date-time format changes rather than project-tailored solutions. The user's localization preferences should persist across sessions via local storage configurations.
Additional context
This feature would reduce manual effort and improve the usability of exported reports. Business users often rely on Data Export reports for decision-making, and having dates displayed in the correct local format would enhance clarity and efficiency.
This functionality should integrate with Feature 527 to support localized time-zone conversions, but should remain independent to allow users to retain UTC/GMT times if preferred.
Here are mockups of the export configuration screen showing the localization options:

Default (no localization):

Local (using browser's locale, e.g., en-US):

Specified locale: "en-IN":

Specified locale: "de-DE":

Hi @dmoruzzi,
I recently implemented the local switcher from the popup, I think that using the standard FORMAT SOQL function + this local switcher would allow users to quickly change the date format.
What do you think about it ?


NIFI-5451 Added test resources for 128 bit encryption.

Fixed unit test to perform properly without JCE unlimited strength policy installed.
Thank you for submitting a contribution to Apache NiFi.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
For all changes:

[x] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?

[x] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.

[x] Has your PR been rebased against the latest commit within the target branch (typically master)?

[x] Is your initial contribution a single, squashed commit?

For code changes:

[x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder?
[x] Have you written or updated unit tests to verify your changes?
[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?
[ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?
[ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?

For documentation related changes:

[ ] Have you ensured that format looks appropriate for the output in which it is rendered?

Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.

@mosermw I think this should resolve the issue. I ran this with both JCE limited / unlimited settings on my machine. Please review.

Reviewed and the issue is resolved, +1 will merge.


Update use-container-storage-with-elastic-san.md

Added new ACStor roles for Elastic SAN storage pool type

Hi @jiarongjoyce - This pull request was opened in the public repo. PMs should work in the private repo, per the Microsoft Docs contributor guide. We can keep this PR open for review and merge, but would you make future content updates in the private repo? Thank you!

@khdownie
Can you review the proposed changes?
When the changes are ready for publication, add a #sign-off comment to signal that the PR is ready for the review team to merge.
#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team

#please-close


Fraud prevention

https://www.gdpreu.org/the-regulation/key-concepts/legitimate-interest/
http://www.privacy-regulation.eu/nl/grond-47-EU-AVG.htm

Added to list of links in repository


Session keys not mangled in performance files

Session keys in django are generated by taking 32 random chars in string.ascii_lowercase + string.digits (see django.contrib.sessions.backends.base), however this pattern is not mangled by regexes in django_perf_rec.cache.CacheOps.VARIABLE_RES̀.
Should we add /\b[0-9a-z]{32}\b/ to the list ? That seems acceptable to me as 32-letter column/table/variable without underscores or uppercase letters should not be very frequent.

Sounds good!

Fixed by #33


updated ouput

sist output:
OS
Microsoft Windows 7 Enterprise
Windows_NT
x64
node
npm -v: 3.7.5
node --version: v5.7.0
Time created: Fri Mar 04 2016 16:07:02 GMT-0500 (Eastern Standard Time)
Description
Command: sist node
Always adds OS. The OS Name is only for windows but the other two should work for mac, linux, and SunOS

You can edit your original issue with the pencil button in the top right. Closing, and copying this text over.


Use equality check to filter ENUMs

Fix for #19
It's based on the work done in https://github.com/SoftwareBrothers/admin-bro-sequelizejs/pull/30