[WIP] Mobile associations

Description
Created a generic mobile-associations component for adding/excluding clients & services  (mobile-resources) from each other

To do

[x] Remove deprovisioned services from service instances list
[x] checks ng-disable & isServiceInstanceReady
[ ] Replace modal with dropdown

looks good, I would suggest to follow UI feedback from here https://github.com/openshift/origin-web-console/pull/2903 and replace modal with dropdown list on add button.

@dimitraz Can we close this one for https://github.com/aerogear/origin-web-console/pull/20 ?


Intellisese for tsconfig.json module is incorrect

VSCode Version: 1.11.2
OS Version: Windows 10

Steps to Reproduce:
With "module": "ES6", in my tsconfig.json I get a green squiggle and tooltop says ES6 is not an accepted value.
According to https://www.typescriptlang.org/docs/handbook/compiler-options.html it is
If you change ES6 to es6 the error disappears?
Note that target accepts both cases.

Tracked upstream by https://github.com/SchemaStore/schemastore/issues/79
Here's the actual scheme: https://github.com/SchemaStore/schemastore/blob/e0251c339130529dfb8ea132365111e89a71ddd0/src/schemas/json/tsconfig.json#L212
It's seems that the case sensitive enum config overrules the pattern one that should allow case insensitivity

Thanks. BTW the list of options in the tooltip never included es6 - perhaps another bug, perhaps not?


feat(darkmode): use a button for the theme toggle

Use a button instead of a checkbox input for the theme toggle. This puts the theme toggle in the tab order, gives it a correct accessible label, and ensures it can be operated with space or enter keys.
As for the why: Building accessible toggle buttons is a pretty good explanation of why using a button is preferable. That implementation uses aria-pressed but since this button doesn't have a real "on" or "off" state I think it's better to omit that and just update the accessible label (by labeling the SVGs and using display:none).

this will probably break css caches for people again 😭 but ty! really appreciate


2019-05-02: Add, update and delete nested objects in Elasticsearch documents

Feel free to write any comments you have for the post Add, update and delete nested objects in Elasticsearch documents.

Great post, I didn't realize you could do this with nested objects. Thanks!

@THEjoezack Thank you, I also didn't know about this functionality until recently. :+1:

@iridakos  How to add a field to an existing nested object. What would the PUT query look like?

@mohittyagi6677 Hi,
Suppose we created the following index:
PUT my_index
{
  "mappings": {
    "_doc": {
      "properties": {
        "users": {
          "type": "nested",
          "properties": {
            "firstname": {
              "type": "text"
            }
          }
        }
      }
    }
  }
}

If we wanted to add another property to the users nested object, we would execute the following:
PUT my_index/_mapping/_doc
{
  "properties": {
    "users": {
      "type": "nested",
      "properties": {
        "lastname": {
          "type": "text"
        }
      }
    }
  }
}

@iridakos
Thank you!
What can you say about performance?
'To update, add, or remove a nested object, we have to reindex the whole document'
https://www.elastic.co/guide/en/elasticsearch/guide/current/nested-objects.html#nested-objects

I want to knew why this specific code you've given is not working on me, its not updating all of my fields inside the nested object..
"script": {
"source": "def targets = ctx._source.cats.findAll(cat -> cat.breed == params.current_breed); for(cat in targets) { cat.breed = params.breed }",
"params": {
"current_breed": "European",
"breed": "European Shorthair"
}
}
I rework it as this:
"script": {
"source": "def targets = ctx._source.eng.findAll(); for(cat in targets) { cat.breed = params.new_mm }",
"params": {
"new_mm": "1"
}
},
"query": {
"match": {
"cli_id": "696668"
}
}

Hello, great information ! Based on that artile, I am trying to add nested tags in existing mappings.
I want to add tag "keyw" and "weight" inside the "content". My existing mappings are:
{
  "microlearning" : {
    "mappings" : {
      "dynamic_templates" : [
        {
          "raw_as_text" : {
            "path_match" : "meta.raw.*",
            "mapping" : {
              "fields" : {
                "keyword" : {
                  "ignore_above" : 256,
                  "type" : "keyword"
                }
              },
              "type" : "text"
            }
          }
        }
      ],
      "properties" : {
        "attachment" : {
          "type" : "binary"
        },
        "attributes" : {
          "properties" : {
            "group" : {
              "type" : "keyword"
            },
            "owner" : {
              "type" : "keyword"
            }
          }
        },
        "content" : {
          "type" : "text"
        },
...

and I am using the following code to add the needed nested ones:
PUT /microlearning
{
	"mappings": {
		"properties": {
			"content": {
				"tags": {
					"type": "nested",
					"properties": {
						"keyw": {
							"type": "keyword"
						},
						"weight": {
							"type": "float"
						}
					}
				}
			}
		}
	}
}

However the result is erroneous, like:
{
  "error": {
    "root_cause": [
      {
        "type": "resource_already_exists_exception",
        "reason": "index [microlearning/NRUB38smQim_EsUpb_LJzQ] already exists",
        "index_uuid": "NRUB38smQim_EsUpb_LJzQ",
        "index": "microlearning"
      }
    ],
    "type": "resource_already_exists_exception",
    "reason": "index [microlearning/NRUB38smQim_EsUpb_LJzQ] already exists",
    "index_uuid": "NRUB38smQim_EsUpb_LJzQ",
    "index": "microlearning"
  },
  "status": 400
}

What am I missing here?

@iridakos
Thank you!
What can you say about performance?
'To update, add, or remove a nested object, we have to reindex the whole document'
https://www.elastic.co/guide/en/elasticsearch/guide/current/nested-objects.html#nested-objects

That is correct, the whole document has to be reindexed.

I want to knew why this specific code you've given is not working on me, its not updating all of my fields inside the nested object..
"script": {
"source": "def targets = ctx._source.cats.findAll(cat -> cat.breed == params.current_breed); for(cat in targets) { cat.breed = params.breed }",
"params": {
"current_breed": "European",
"breed": "European Shorthair"
}
}
I rework it as this:
"script": {
"source": "def targets = ctx._source.eng.findAll(); for(cat in targets) { cat.soc_mm_score = params.new_mm }",
"params": {
"new_mm": "1"
}
},
"query": {
"match": {
"cli_id": "696668"
}
}

Hi @benjieperez,
A little hard to understand without the index information. Are you using the _update on a specific document as described in the post? I am confused with the query > match > cli part of your query.

Hi @edrimon,

Hello, great information ! Based on that artile, I am trying to add nested tags in existing mappings.
I want to add tag "keyw" and "weight" inside the "content". My existing mappings are:
{
  "microlearning" : {
    "mappings" : {
      "dynamic_templates" : [
        {
          "raw_as_text" : {
            "path_match" : "meta.raw.*",
            "mapping" : {
              "fields" : {
                "keyword" : {
                  "ignore_above" : 256,
                  "type" : "keyword"
                }
              },
              "type" : "text"
            }
          }
        }
      ],
      "properties" : {
        "attachment" : {
          "type" : "binary"
        },
        "attributes" : {
          "properties" : {
            "group" : {
              "type" : "keyword"
            },
            "owner" : {
              "type" : "keyword"
            }
          }
        },
        "content" : {
          "type" : "text"
        },
...

and I am using the following code to add the needed nested ones:
PUT /microlearning

^ This line is to create an indexing and not to update it.
You should be using: PUT /microlearning/_mapping/_doc

{
	"mappings": {
		"properties": {
			"content": {
				"tags": {
					"type": "nested",
					"properties": {
						"keyw": {
							"type": "keyword"
						},
						"weight": {
							"type": "float"
						}
					}
				}
			}
		}
	}
}

However the result is erroneous, like:
{
  "error": {
    "root_cause": [
      {
        "type": "resource_already_exists_exception",
        "reason": "index [microlearning/NRUB38smQim_EsUpb_LJzQ] already exists",
        "index_uuid": "NRUB38smQim_EsUpb_LJzQ",
        "index": "microlearning"
      }
    ],
    "type": "resource_already_exists_exception",
    "reason": "index [microlearning/NRUB38smQim_EsUpb_LJzQ] already exists",
    "index_uuid": "NRUB38smQim_EsUpb_LJzQ",
    "index": "microlearning"
  },
  "status": 400
}

What am I missing here?

That's why you get an error that the index with this name already exists.

BR, Makis

But even if you did it the proper way, there's another problem here.
In the original index, the content has been defined as text. You can't add nested properties to a text property.
mapper [content] of different type, current_type [text], merged_type [ObjectMapper]

Cheers!

Hi, are you sure? I was checking this example below which is using nested tags to text properties..
https://conan.is/blogging/weighted-elasticsearch.html

In your first snippet, content is declared as text:
"content" : {
  "type" : "text"
},

When you try to update the mappings, you use:
"properties": {
  "content": {
    "tags": {
	"type": "nested",
	"properties": {
	  "keyw": {
	    "type": "keyword"
	  },
	 "weight": {
	   "type": "float"
	 }
      }
    }
  }
}

in other words, you are trying to add a property tags inside the previously defined property content which is of type text.
Any chance you copy/pasted the code erroneously?
Did you mean:
"properties": {
    "tags": {
	"type": "nested",
	"properties": {
	  "keyw": {
	    "type": "keyword"
	  },
	 "weight": {
	   "type": "float"
	 }
      }
    }
}

Yes, now I see. It is really too interesting.. thanks..!
Makis

I did like you said and it worked, the nested fields are added to the mappings. But now I want to populate those fields of an existing document, for example to add a new keyw with value "X" and weight 0.8 and also Y,0.2.
I cannot find a clear way after googling to perform it? Some say use scripting , some other to try directly but it is not clear.. How could I just update the current document and populate for example two pairs of the nested fields keyw,weight there?

Hi, Thanks for the well explained article. I have a scenario, where each object within the nested document with different values. Taking your cats example, let's say I have updated the mapping to have another property "age" of type "short". Now I want to set the age of every cat cat.name:"Irida" to age 2, cat.name:"Phoebe" to age 4, etc. how will I do that. I am trying by iterating over the cats and params list, yet have not figured it out. Thanks

Excellent post! Used your update example for work. Thank you!

Thank you very much for this awesome post, I have been searching for something like this for months.
I wanted to point an issue I am having. I have an User index with uuid, name and email fields that are set on document creation. When user creates a comment, I want to execute the following script sending the comment data within params:
ctx._source.comments.add(params.subentityItem)

If user has previous comments (therefore "comments" field is already present on the document) it works properly, but if the document doesn't have the property it crashes with error
Cannot invoke "Object.getClass()" because "receiver" is null

I have tried some ways to create "comments" field dynamically (source)
  "query": {
   "bool": {
    "exists": {
      "field": "comments"
    }
   }
  }

but it gives the error
[UpdateRequest] unknown field [query]

Is there any way to create that field dynamically when trying to modify it? I could add the field with empty value during the document creation, but the idea is to have it only when needed
Thank you very much

Hi @brianbooth
thank you for the tip, I'll give it a try!

Hello, I found this link very useful. Is there a way of finding the index of a particular item in the cats collection and use a set method replace an item?


Problem with timestamp cast to uint

Casting timeSpan.TotalMilliseconds to uint will truncate the timestamp. As secs and nsecs are casted to uint later anyway, removing the cast seems to be the way to go.

Thank you @Reinbert !


feat(tutorial): complete step 2 - rochester

Closes #
{{short description}}
Changelog
New

{{new thing}}

Changed

{{change thing}}

Removed

{{removed thing}}

Looks good.  I've marked your lab 2 complete.


bug: cortex.cpp Installer IPersistFile::Save failed; code 0x80070005. Access is Denied

Cortex version
0.5.0-55
Describe the Bug

Windows Installer for cortex.cpp has error, but continues to install
UX for downloading llama.cpp dependency needs some work

Steps to Reproduce
No response
Screenshots / Logs
Error that showed up:

After that, cortex.cpp indicated that Cortex was added to path:
CUDA download in progress

As a user, it was confusing as no progress was shown

What is your OS?

[ ] MacOS
[X] Windows
[ ] Linux

What engine are you running?

[X] cortex.llamacpp (default)
[ ] cortex.tensorrt-llm (Nvidia GPUs)
[ ] cortex.onnx (NPUs, DirectML)

Thanks, @dan-homebrew. Since Cortex is not a GUI/Desktop application, I will disable the options Create a desktop icon and Create a Quick Launch icon. I believe this will fix the error.
Regarding the download percentage for the llamacpp engine, the current source code does not support showing the download progress.I'll look for a way to add it to the installer dialog instead of showing the console log.

@dan-homebrew, can you check if this installer fixes your issue?


2.x Fix failing test

Ticket: #1617
Issue
There’s at least one failing test that appears when testing against newer versions of WordPress.
For example, the test TestTimberPost::testPostWithVideo() will fail, because with newer versions of WordPress, the embed will look different. Instead of

the result will include a title attribute

We can’t just include that title attribute in the expected string, because the expected string is different for different WordPress versions.
Solution
Resolve these tests.
For the TestTimberPost::testPostWithVideo() test, a regular expression check will be used.
Impact
Fixes failing Travis tests.
Usage Changes
None.
Considerations
Nope.
Testing
Yep.

Travis looks good. It looks like this has been the only failing test.
@jarednova Can you confirm and merge?


Send RTMP messages to receiver_pid

Hello! Upon upgrading to v0.26 I've lost access to the RTMP messages which were previously available to the Validator.
I saw that Membrane.RTMP.MessageHandler already has an unused receiver_pid in it's state, so I used that for sending messages to the receiver.
To set the receiver_pid users can return a three-tuple from the handle_new_client callback like this:
{Membrane.RTMP.Source.ClientHandlerImpl, %{}, receiver_pid}

RTMP messages that get sent are simply forwareded Membrane.RTMP.Messages structs, not wrapped in any 'scoping' tuple. The OnDataFrame message is particularly important to me, as I am using it to calculate frame dimensions for transcoding with membrane_ffmpeg_swscale_plugin.

Hello @lastcanal !
That's a good observation, that it used to be possible to do it that way, but it'a not possible anymore.
But could you provide some more context why do you need that "OnDataFrame" message (BTW - didn't you mean OnMetaData message? I think all the necessary information (like resolution etc.) should be available in the stream format.
Another thing is that I guess adding a new callback in the ClientHandlerBehaviour (like handle_rtmp_message) which would be called with each message received after RTMP handshake for a given client, aligns to the current architecture of the RTMP server in greater degree.
On the other hand, it would mean that for your case you would need to implement a custom behaviour.
WDYT about such an approach?

Hello! Sorry, I meant SetDataFrame, although OnMetaData should work too: I really only need width, height, and framerate for scaling video tracks while keeping the aspect ratio intact.
I took your suggestion and tried to use the stream format, but unfortunately framerate is always set to nil... Perhaps this is a bug? Here is the stream format for my OBS RTMP stream intercepted by a Tee.Parallel:
%Membrane.H264{
  width: 1280,
  height: 720,
  profile: :high,
  alignment: :au,
  nalu_in_metadata?: true,
  framerate: nil,
  stream_structure: {:avc3, <<1, 100, 0, 32, 255, 224, 0>>}
}

Sending messages through the client handler is a great idea and i managed to get that working. I've pushed a new branch #103 with the new handle_rtmp_message callback. Thanks!

@lastcanal - sure, I understand. Indeed, the framerate is missing, because the H264 stream format is created by the FLV Demuxer (a part of the SourceBin) and FLV demuxer is apparently unable to read that information from FLV, and at the same time metadata sent within RTMP messages is lost.
There are is a couple of possible approaches to that problem, I've described them here: https://github.com/membraneframework/membrane_core/issues/903


Feature request: Show photos from Mapillary or OpenStreetCam

In OsmAnd you can select a POI, click "details" and see nearby photos (if you are online). These photos are fetched from Mapillary.
If you click on a photo you can also see the whole route.
Google Maps also has a similar feature (street view).
Is this something you are planning to add to Qwant Maps? It is very useful when travelling, and I can imagine it's something people would look for when considering switching from Google Maps.
Anyway, thanks a lot for making Qwant Maps! I'm really happy with it.

Image from OsmAnd (sorry it's in Japanese):


Fix bdcs-api docker volume mount issue.

Codecov Report

Merging #202 into master will increase coverage by 35.9%.
The diff coverage is n/a.

@@            Coverage Diff            @@
##           master    #202      +/-   ##
=========================================
+ Coverage    7.49%   43.4%   +35.9%
=========================================
  Files          69      70       +1
  Lines        1961    2341     +380
  Branches      319     319
=========================================
+ Hits          147    1016     +869
+ Misses       1502    1112     -390
+ Partials      312     213      -99

Impacted Files
Coverage Δ

routes.json
72.22% <0%> (ø)

components/ListView/ComponentDetailsView.js
1.21% <0%> (+1.21%)
:arrow_up:

components/ListView/ComponentSummaryList.js
5.88% <0%> (+5.88%)
:arrow_up:

components/Layout/Layout.js
100% <0%> (+10%)
:arrow_up:

core/utils.js
32.25% <0%> (+19.35%)
:arrow_up:

components/Toolbar/Toolbar.js
27.27% <0%> (+27.27%)
:arrow_up:

components/Link/Link.js
33.33% <0%> (+27.77%)
:arrow_up:

components/Tabs/pf-tab.component.js
29.16% <0%> (+29.16%)
:arrow_up:

core/reducers/sort.js
30% <0%> (+30%)
:arrow_up:

components/ListView/ListItemComponents.js
30.55% <0%> (+30.55%)
:arrow_up:

... and 44 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5946446...be5e27a. Read the comment docs.


How to dump muGraph?

Sorry to ask a naive question, but I would like to dump the optimatized muGraph searched by mirage, how can I do that?

自问自答，it seems I can run python python/mirage/visualizer.py to get muGraph！Beautiful!

Yes, you can use the visualizer to draw the muGraphs. Mirage also supports API to print the generated CUDA program for each muGraph.


6.6.3 caused an issue with preview just failing to load if you change parcels in scene.json(have example)

See the image, but that is all it takes to break the scene so it doesn't load.  Confirmed with doug from nftworld.io
When the scene loads, it loads the structure fine, but when loaded with the 2 parcel list it fails to even load the previewer, (the model is within the bounds of the 2 parcels that fail).  4 also doesn't work. Only the 6 parcel list works, which was previously deployed a few months ago, with a different 2x3 parcel model.

fixed in https://github.com/decentraland/explorer/pull/1900

fixed in https://github.com/decentraland/explorer/pull/1900


Dimension mismatch error when running mxnetR

I am trying to build a Feed Forward neural network with MXNetR. My input is a data frame with 6380 rows and 180 columns. My training and testing outputs are one-dimensional vectors with 319 elements each.
I run the model with the batch size set to 1 and the number of neurons at the output layer set to 319. So for each batch, I expected to get a vector with 319 elements. I aim to minimize my loss function, which is the correlation between my predicted output vector and actual output vector.
Below is my code:
    # Generate testing data
    train.x = matrix(data = rexp(200, rate = 10), nrow = 120, ncol = 6380)
    test.x = matrix(data = rexp(200, rate = 10), nrow = 60, ncol = 6380)
    train.y = matrix(data = rexp(200, rate = 10), nrow = 120, ncol = 319)
    test.y = matrix(data = rexp(200, rate = 10), nrow = 60, ncol = 319)

    # Reshape testing data
   train.array <-train.x
   dim(train.array) <-c(20,319,1,ncol(train.x))
   test.array<-test.x
   dim(test.array) <-c (20,319,1,ncol(test.x))

    # Define the input data
    data <- mx.symbol.Variable("data")

    # Define the first fully connected layer
    fc1 <- mx.symbol.FullyConnected(data, num_hidden = 100)
    act.fun <- mx.symbol.Activation(fc1, act_type = "relu") # create a hidden layer with Rectified Linear Unit as its activation function.
    output <<- mx.symbol.FullyConnected(act.fun, num_hidden = 319)

    # Customize loss function
    label <- mx.symbol.Variable("label")
    lro <- mx.symbol.MakeLoss(mx.symbol.Correlation(mx.symbol.reshape(output
    ,shape = c(1,319)),mx.symbol.reshape(label, shape = c(1,319,1,1))))

    model <- mx.model.FeedForward.create(symbol=lro, X=train.x, y=train.y,
                                     num.round=5000, array.batch.size=1, optimizer = "adam",
                                     learning.rate = 0.0003, eval.metric = mx.metric.rmse,
                                     epoch.end.callback = mx.callback.log.train.metric(20, logger))

And here is the error when I run the code above:
[Error in symbol$infer.shape(list(...)) :
Error in operator correlation6: [14:36:37] src/operator/./correlation-inl.h:177: Check failed:
 dshape2.ndim() == 4U (0 vs. 4) data should be a 4D tensor

Stack trace returned 10 entries:
[bt] (0) /usr/lib64/R/library/mxnet/libs/libmxnet.so(_ZN4dmlc15LogMessageFatalD1Ev+0x29) [0x7f64602b28b9]
[bt] (1) /usr/lib64/R/library/mxnet/libs/libmxnet.so(_ZNK5mxnet2op15CorrelationProp10InferShapeEPSt6vectorIN4nnvm6TShapeESaIS4_EES7_S7_+0x33d) [0x7f6460f082bd]
[bt] (2) /usr/lib64/R/library/mxnet/libs/libmxnet.so(+0xd461f9) [0x7f6460d841f9]
[bt] (3) /usr/lib64/R/library/mxnet/libs/libmxnet.so(+0x116630f) [0x7f64611a430f]
[bt] (4) /usr/lib64/R/library/mxnet/libs/libmxnet.so(+0x1167bb2) [0x7f64611a5bb2]
[bt] (5) /usr/lib64/R/library/mxnet/libs/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorISsSaISsEE+0x501) [0x7f64611c1481]
[bt] (6) /usr/lib64/R/library/mxnet/libs/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKSs+0x8e) [0x7f64610f9f2e]
[bt] (7) /usr/lib64/R/library/mxnet/libs/libmxnet.s

At the moment, I am clueless about how I should fix this error. I have been searching for a way to reshape my data sets so that they are 4D tensors but could not find any. I do not look for an explicit solution for my problem, but any suggestions on how I should tackle this error would be greatly appreciated.

This was working for me one I added the missing "c"  for the shape:
lro <- mx.symbol.MakeLoss(mx.symbol.Correlation(mx.symbol.reshape(output,shape = c(1,319)),label))

You also to add X and y and where you get the error.
Make sure you have a reproducible code.

Hard to say without the and X example.
It might also be due to batch.size=1.

@offerm: I updated the code to make sure that it includes a test data set. Will try to experiment with different batch sizes.

I simplified the code so it will be easier to understand the situation (consider to change your example above):
Generate testing data
train.x = matrix(data = rexp(1206380, rate = 10), nrow = 120, ncol = 6380)
train.y = matrix(data = rexp(120319, rate = 10), nrow = 120, ncol = 319)
Define the input data
data <- mx.symbol.Variable("data")
Define the first fully connected layer
fc1 <- mx.symbol.FullyConnected(data, num_hidden = 319)
Customize loss function
label <- mx.symbol.Variable("label")
lro <- mx.symbol.MakeLoss(mx.symbol.Correlation(fc1 ,label))
model <- mx.model.FeedForward.create(symbol=lro,
X=t(train.x),
y=t(train.x),
num.round=5000, array.batch.size=1, optimizer = "adam",
learning.rate = 0.0003,
#eval.metric = mx.metric.rmse,
array.layout = "colmajor",
epoch.end.callback = mx.callback.log.train.metric(20, logger))
With this, you get your original error:
[16:13:04] /Users/admin/github/mxnet/dmlc-core/include/dmlc/logging.h:304: [16:13:04] src/operator/./correlation-inl.h:176: Check failed: dshape1.ndim() == 4U (2 vs. 4) data should be a 4D tensor
The error is coming directly from the correlation c function that checks the shape for the arguments. I still try to understand the C code there.
I tried to simplified even more by using executor code as below:
require(mxnet)
A = mx.symbol.Variable('A')
B = mx.symbol.Variable('B')
#C = mx.symbol.Correlation(A,B)
C = mx.symbol.sort(A+B)
a = mx.nd.zeros(c(4), mx.cpu())
b = mx.nd.array(as.array(c(7, 2,4,5)), mx.cpu())
exec = mxnet:::mx.symbol.bind(
symbol=C,
ctx=mx.cpu(),
arg.arrays = list(A=a, B=b),
aux.arrays = list(),
grad.reqs = list("null", "null"))
calculate outputs
mx.exec.forward(exec)
out = as.array(exec$outputs[[1]])
print(out)
It is working as expected until you change the mx.symbol.sort to mx.symbol.correlation (A,B).
C = mx.symbol.Correlation(A,B)
When you do and run it, it crashes.....
Will try to give it some more time over the weekend.

@offerm: Thank you. I will spend time looking at the problem myself as well.

Also check test_operator.py line 1721 (unittest_correlation). Looks like it does expect 4U tensor.

So  I managed to use the correlation with this code:
require(mxnet)
A = mx.symbol.Variable('A')
B = mx.symbol.Variable('B')
C = mx.symbol.Correlation(A,B,kernel_size = 1,max_displacement = 1,stride1 = 1,stride2 = 1
,pad_size = 4,is_multiply = FALSE)
#C = mx.symbol.sort(A+B)
a = mx.nd.reshape(mx.nd.zeros(c(4), mx.cpu()),shape=c(1,1,1,4))
b = mx.nd.reshape(mx.nd.array(as.array(c(7, 2,4,5)), mx.cpu()),shape=c(1,1,1,4))
exec = mxnet:::mx.symbol.bind(
symbol=C,
ctx=mx.cpu(),
arg.arrays = list(A=a, B=b),
aux.arrays = list(),
grad.reqs = list("null", "null"))
calculate outputs
mx.exec.forward(exec)
out = as.array(exec$outputs[[1]])
print(out)
However, as you can see from the result this is not a simple correlation of two arrays.

@nnguyen24 You need to make the two symbols in mx.symbol.Correlation have exactly the same shape:
library(mxnet)

train.x = matrix(data = rexp(120 * 319, rate = 10), nrow = 120, ncol = 319)

# Define the input data
data <- mx.symbol.Variable("data")
# Define the first fully connected layer
fc1 <- mx.symbol.FullyConnected(data, num_hidden = 100)
act.fun <- mx.symbol.Activation(fc1, act_type = "relu") # create a hidden layer with Rectified Linear Unit as its activation function.
output <- mx.symbol.FullyConnected(act.fun, num_hidden = 319)
# Customize loss function
label <- mx.symbol.Variable("label")
lro <- mx.symbol.MakeLoss(mx.symbol.Correlation(
  mx.symbol.reshape(output, shape = c(100, 319, 1, 1)),
  mx.symbol.reshape(label, shape = c(100, 319, 1, 1))))

model <- mx.model.FeedForward.create(symbol = lro, X = t(train.x),
                                     y = t(train.x), num.round = 5,
                                     array.batch.size = 100,
                                     optimizer = "sgd",
                                     learning.rate = 0.0003,
                                     array.layout = "colmajor",
                                     epoch.end.callback = mx.callback.log.train.metric(20, logger))


can not reproduce the the result of roberta large on dataste sst-2

my transformers version == 4.28.1
i run the the following command
TASK=SST-2 K=16 SEED=42 BS=64 LR=1e-6 EPS=1e-3 MODEL=roberta-large bash mezo.sh
get the test acc 70.2 which in your paper is 89.6 (1.2)
i did not modify any scripts or files, should i ?

Can you try the full grid lr = {1e−7, 1e−6, 1e−5} (Table 15 from our paper)?

I can't reimplement the result either. However after degrading our Python, PyTorch and Transformers packages the results are good. Weird.

I can't reimplement the result either. However after degrading my Python, PyTorch and Transformers packages the results are good. Weird.


Replace PersistentConnectionBase with ansible.utils' version

SUMMARY

PersistentConnectionBase is intended to work around the need for self._sub_plugin entirely.
This PR updates the base plugin to the current version and removes the unnecessary NetworkConnectionBase class.
ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME

logix
ADDITIONAL INFORMATION

@maxamillion I believe you are best for the lower level Ansible API interactions this appears to fix. Failing checks appear to be a missing changelog.rst which can be easily remedied


msg_container padding?

Hi, I've been trying to workout how msg_container works, at this point I've established a connection and I've sent a ping to the server, this is the response to that. I broke down the contents of said response and it looks as follows:
    [02000000: int 2 length]-
    [01XXXXXXXXXXf45f: long message_id]-
    [01000000: 1 int seqno]
    [1c000000: int bodylength]

    [[0809c29e: new_session_created int message_type]-
    [00000000XXXXf45f: long first_msg_id]-
    [XXXXXXXXXXXXXXXX: long unique_id]
    [XXXXXXXXXXXXXXXX: long server_salt]
    XXXXXXXXXXXX- 6 bytes of padding?]

    [[02000000: 2 int seqno]
    [14000000: 20 int bodylength]
    [c5737734: pong int message_type]-
    [00000000XXXXf45f: long message_id]-
    [dd63000000000000: long ping_id]]

Assuming those 6 bytes I've pointed out are indeed padding I'm not sure what algorithm determines how much padding gets added and I can't seem to find anything explicitly discussing it in https://core.telegram.org/mtproto/service_messages#containers

There is no way an object can have padding of 6 bytes. Every object length is divisible by 4 if protocol isn't obfuscated. Moreover TL serialization doesn't have any implicit paddings with exception for stored strings. In place denoted by "6 bytes of padding" there should be "long message_id" of the next message.

There is no way an object can have padding of 6 bytes. Every object length is divisible by 4 if protocol isn't obfuscated. Moreover TL serialization doesn't have any implicit paddings with exception for stored strings. In place denoted by "6 bytes of padding" there should be "long message_id" of the next message.

Thanks, turns out my problem was that I'd read some data in and skipped too far along the byte array.

Thanks, turns out my problem was that I'd read some data in and skipped too far along the byte array.


CLI request for community gallery list

Is your feature request related to a problem? Please describe.

For azure compute gallery, from https://github.com/Azure/azure-cli/issues/19882, we have GET community gallery/image/version and LIST community image/version. The LIST community galleries call is missing there because we do not have API level support for it.
Describe the solution you'd like

Currently on portal we plan to reply on ARG query for user to explore the LIST call, so no gallery service API won't be called. For CLI, we can schedule the same flow. CLI can just query ARG community galleries resources and display the result to user.
Describe alternatives you've considered

Additional context

Sample ARG request url:
POST https://centraluseuap.management.azure.com/providers/Microsoft.ResourceGraph/resources?api-version=2021-06-01-preview
request body:
{ "query": "CommunityGalleryResources |where * contains 'communitygalleries'  | where * contains 'westcentralus' "}

compute

Replaced by #22222


Release 0.1.27

Notable changes:

Updated src/Dockerfile to use the repo root directory as the context and use the /generated/gha/dagster-cloud.pex as the input to installing dependencies. Updated the release.py script and tests for this adjustment.

Update references from dagster-cloud-action:0.1.26 -> dagster-cloud-action:0.1.27. This was automatic by running release.py.

Promoted to v0.1 just now.


Error in order command

Hello
I use the below commands to create an order:
order = Order(symbol="DOGE-USDT", side="BUY", position_side="LONG", type="MARKET", quantity=100)
Trade.create_order(order)
but I faced with below Error:  (the error in attached

)
ClientError                               Traceback (most recent call last)
[c:\Users\Administrator\Desktop\BingX](file:///C:/Users/Administrator/Desktop/BingX) Perpetual Commands.ipynb Cell 21 line 2
1 order = Order(symbol="DOGE-USDT", side=Side.BUY, position_side=PositionSide.LONG, quantity=100.0)
----> 2 Trade.create_order(order)
[c:\Users\Administrator\Desktop\BingX](file:///C:/Users/Administrator/Desktop/BingX) Perpetual Commands.ipynb Cell 21 line 2
19 endpoint = "/openApi/swap/v2/trade/order"
20 payload = order.to_dict()
---> 22 response = self.__http_manager.post(endpoint, payload)
23 return response.json()["data"]
File [c:\Users\Administrator\AppData\Local\Programs\Python\Python312\Lib\site-packages\bingX_http_manager.py:103](file:///C:/Users/Administrator/AppData/Local/Programs/Python/Python312/Lib/site-packages/bingX/_http_manager.py:103), in _HTTPManager.post(self, endpoint, payload, headers)
[93](file:///C:/Users/Administrator/AppData/Local/Programs/Python/Python312/Lib/site-packages/bingX/_http_manager.py:93) def post(self, endpoint: str, payload: dict[str, Any] = {}, headers: dict[str, Any] = {}) -> requests.Response:
[94](file:///C:/Users/Administrator/AppData/Local/Programs/Python/Python312/Lib/site-packages/bingX/_http_manager.py:94)     """
[95](file:///C:/Users/Administrator/AppData/Local/Programs/Python/Python312/Lib/site-packages/bingX/_http_manager.py:95)     It makes a POST request to the given endpoint with the given payload and headers
[96](file:///C:/Users/Administrator/AppData/Local/Programs/Python/Python312/Lib/site-packages/bingX/_http_manager.py:96)
(...)
[100](file:///C:/Users/Administrator/AppData/Local/Programs/Python/Python312/Lib/site-packages/bingX/_http_manager.py:100)     :return: A response object
[101](file:///C:/Users/Administrator/AppData/Local/Programs/Python/Python312/Lib/site-packages/bingX/_http_manager.py:101)     """
--> [103](file:///C:/Users/Administrator/AppData/Local/Programs/Python/Python312/Lib/site-packages/bingX/_http_manager.py:103)     return self._request("POST", endpoint, payload, headers)
File [c:\Users\Administrator\AppData\Local\Programs\Python\Python312\Lib\site-packages\bingX_http_manager.py:78](file:///C:/Users/Administrator/AppData/Local/Programs/Python/Python312/Lib/site-packages/bingX/_http_manager.py:78), in _HTTPManager._request(self, method, endpoint, payload, headers)
[75](file:///C:/Users/Administrator/AppData/Local/Programs/Python/Python312/Lib/site-packages/bingX/_http_manager.py:75) else:
[76](file:///C:/Users/Administrator/AppData/Local/Programs/Python/Python312/Lib/site-packages/bingX/_http_manager.py:76)     #TODO: Check if this is the correct way to check for errors
[77](file:///C:/Users/Administrator/AppData/Local/Programs/Python/Python312/Lib/site-packages/bingX/_http_manager.py:77)     if req_json.get("code") is not None and req_json.get("code") != 0:
---> [78](file:///C:/Users/Administrator/AppData/Local/Programs/Python/Python312/Lib/site-packages/bingX/_http_manager.py:78)         raise ClientError(req_json.get("code"), req_json.get("msg"))
[79](file:///C:/Users/Administrator/AppData/Local/Programs/Python/Python312/Lib/site-packages/bingX/_http_manager.py:79)     return req
ClientError: Invalid parameters, err:Key: 'PlaceOrderRequest.ReqOrderData.PositionSide' Error:Field validation for 'PositionSide' failed on the 'required' tag

it's solved


make observation image have different values for empty and unseen cells (#35)

add ('unseen', 0) to OBJECT_TO_IDX dictionary;  all other values are
shifted up by 1 to make space for the new value.

update Grid.encode to take an optional vis_mask argument, and
return an image which now distinguishes between unseen and empty cells
(values respectively 0 and 1).

update Grid.decode to handle the above changes;  values 0 and 1 in
input array are both mapped to None in output grid.

make Grid.decode a static method;  it was already used as a static
method but was missing the appropriate decorator.

update "observation encode/decode roundtrip" test in run_tests.py;
the visualization mask necessary to perform the test is found by
manually checking the first channel of the image for value 0.

@maximecb please verify that this is what you were going for and that there are no other issues;  I've only tested with a few manual runs and with the run_tests.py script, but as far as I could tell that is somewhat limited.

PR looks very good.
It's true that there are not a lot of tests, but most deep learning code that I see out there has no tests at all (lol).
I guess it would be kind of cool to write an automated test that puts the agent at a known orientation in a little map and checks that the observation is as expected (eg: can see an object in front of the agent, can't see an object behind a wall). I understand if this is too much work though.


Make the report submit nonce a global counter or make it optional

When 2 tx for different query ID end up in the same block or with very small time apart the second one will still pass causing the reporter to not get any rewards and loose money for the TX.

Is it possible/worthwhile to recognize the situation and attempt to cancel the losing submission?

not enough time for this. I think making it global and optional will also solve the linked issue


Orienteer Guidebook

Being tracked here: https://www.gitbook.com/book/orienteer/orienteer/details  and here https://github.com/OrienteerDW/Orienteer-Guidebook


[Feature-996][*] Release v0.6.7 (#997)

Co-authored-by: wenmo<EMAIL_ADDRESS>Purpose of the pull request

Brief change log

Verify this pull request

This pull request is code cleanup without any test coverage.
(or)
This pull request is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:

update


Move enrollment scenarios into main README

Move enrollment scenarios into main README and delete scenarios page

Closing for now


Avoid test interwining by disabling parallel execution

Some tests or resources are not completely isolated and hang the sbt test command in a powerful enough machine.
This PR disables parallel execution of tests so they are executed sequentially.

Confirmed! @tonicebrian  has signed the Individual Contributor License Agreement. Thanks so much

Incorporated into #3965 , thanks a lot! :+1:


HTML not encoded in the listing of replacements

Thank you for the great extension!
HTML is not encoded in the list of replacements that have been set up, making it difficult to see them properly.
e.g. replacing

([^<]+)

with
$1
only shows
([^<]+) --> $1

I think that this is likely to be destructive to people who want to replace strings that show up in tags -- the page can get mangled pretty quickly. I'd be totally open to a PR to enable HTML replacement for certain rules if you feel so moved!


Worker endpoints are unset in StartWorkerRequest.

User reported that worker endpoints are unset when attempting to start a worker in a worker pool using Kubernetes. https://stackoverflow.com/questions/66498209/how-to-configure-beam-python-sdk-with-spark-in-a-kubernetes-environment
Starting worker with command ['/opt/apache/beam/boot', '--id=1-1', '--logging_endpoint=', '--artifact_endpoint=', '--provision_endpoint=', '--control_endpoint=']
These endpoints are expected to be set to use localhost and some random allocated port. The server is started here: https://github.com/apache/beam/blob/b101064f18e71adb81c8fe6ba7beae53e9b13b08/runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/control/DefaultJobBundleFactory.java#L672-L697
And the StartWorkerRequest is constructed here: https://github.com/apache/beam/blob/master/runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/environment/ExternalEnvironmentFactory.java
Imported from Jira BEAM-11957. Original Jira may contain additional context.
Reported by: ibzib.

Has this issue been fixed? These issues can be easily identified if devs do some integration testing


List of supported metric namespaces in AzureMonitorQueryBuilder needs to be updated

The list of supported metric namespaces in AzureMonitorQueryBuilder needs to be updated to match the following doc:
https://docs.microsoft.com/en-us/azure/monitoring-and-diagnostics/monitoring-supported-metrics
For example, Microsoft.Insights/autoscalesettings is not in the list of supported metric namespaces.
Additionally I wonder if we should whitelist specific Azure resource types and not entire Azure resource provider namespace.  For example, 'Microsoft.Compute' is listed in the supported namespace but should it instead be added as 'Microsoft.Compute/virtualmachines' and 'Microsoft.Compute/virtualmachinescalesets'?  If you add 'Microsoft.Compute', it brings in other resource types under that namespace in the drop down but they won't have any metrics.

Closed by 5dd6d8cbdf46b0304b9d2d1bbf35f7eefe86abc1
Added the following namespaces to the whitelist:

Microsoft.DataFactory/datafactories
Microsoft.DataFactory/factories
Microsoft.KeyVaults/vaults
Microsoft.LocationBasedServices/accounts
Microsoft.PowerBIDedicated/capacities
Microsoft.Relay/namespaces


Tracemode utilization crashes in production build of Charm++

Original issue: https://charm.cs.illinois.edu/redmine/issues/1273

Charm++ programs compiled in production mode with tracemode utilization yield in a segmentation fault. The error occurs for larger examples like wave2d or jacobi2d(32000X16000).
Further analysis of the bug led to the following observations :

The error occurs in the sumDetailCompressedReduction reducer in trace-utilization.C
Specifically in the mergeCompressedBin method minEp is not correctly updated due to junk values being read from srcBufferArray
The program does not crash in non-production mode possibly because the contents of srcBufferArray are zeroed out(during malloc)

Example crash log :
Completed 1160 iterations
Completed 1180 iterations
Completed 1200 iterations
------------- Processor 0 Exiting: Caught Signal ------------
Reason: segmentation violation
Suggestion: Try running with '++debug', or linking with '-memory paranoid' (memory paranoid requires '+netpoll' at runtime).
[0] Stack Traceback:
  [0:0]   [0x577183]
  [0:1] +0x36cb0  [0x7ffff7006cb0]
  [0:2] _ZN16compressedBuffer5writeIsEEvT_i+0x2c  [0x5ad406]
  [0:3] _ZN16compressedBuffer4pushIsEEiT_+0x35  [0x5ace09]
  [0:4] _Z18mergeCompressedBinP16compressedBufferiPiiRS_+0x262  [0x5aa4d7]
  [0:5] _Z28sumDetailCompressedReductioniPP14CkReductionMsg+0x220  [0x5a99aa]
  [0:6] _ZN14CkReductionMgr14reduceMessagesEv+0x2d1  [0x51d7b9]
  [0:7] _ZN14CkReductionMgr15finishReductionEv+0x162  [0x51cecc]
  [0:8] _ZN14CkReductionMgr7RecvMsgEP14CkReductionMsg+0xc3  [0x51d39b]
  [0:9] _ZN22CkIndex_CkReductionMgr28_call_RecvMsg_CkReductionMsgEPvS0_+0x2b  [0x526e13]
  [0:10] CkDeliverMessageFree+0x39  [0x4e3ca7]
  [0:11]   [0x4e3d9f]
  [0:12]   [0x4e3ebf]
  [0:13]   [0x4e53ad]
  [0:14]   [0x4e5438]
  [0:15] _Z15_processHandlerPvP11CkCoreState+0x126  [0x4e58ef]
  [0:16] CmiHandleMessage+0x4d  [0x57da31]
  [0:17] CsdScheduleForever+0xad  [0x57dcb2]
  [0:18] CsdScheduler+0x16  [0x57dbe3]
  [0:19]   [0x576c17]
  [0:20] ConverseInit+0x2e0  [0x576b21]
  [0:21] main+0x3f  [0x4d5048]
  [0:22] __libc_start_main+0xf5  [0x7ffff6ff1f45]
  [0:23]   [0x4c4b09]
Charmrun> error on request socket to node 0 'localhost'--

Original date: 2017-02-14 18:33:59

These were fixed as far as I remember, but I think Karthik had the relevant information (I sat down with him at his desk to fix this and he was supposed to make the logs afterward, I think).  I'll check back to see if these were really done.

Original date: 2017-10-03 16:55:56

Karthik can you test this again?

Original date: 2017-10-03 17:54:26

I tested this again on my lab machine (netlrts-linux-x86_64-smp) for wave2d and jacobi2d. I didn't run into any crashes.

Original date: 2017-10-04 18:29:21

Was likely fixed a while ago, but never updated.  Since it's not reproducible, I'll close it for now.


Followup on tfjs-node error

Tensorflow's tfjs-node version 4.2.0 released on Jan 3, 2023 caused the scheduled GH actions that were generating Stampy's FAQ encodings to fail. We've reverted to version 4.1.0 to avoid the error but need to:

Create standalone code based on questions-encodings.mjs to reproduce the error then use the bug report template to file a GitHub Issue tag type:bug along with failure log
Follow up when fixed then update gen-encodings-deploy.yml to remove specification for version @4.1.0 when calling npm install tfjs-node

/home/runner/work/stampy-ui/stampy-ui/node_modules/@tensorflow/tfjs/node_modules/@tensorflow/tfjs-core/dist/tf-core.node.js:2978
    return env().platform.isTypedArray(a);
                          ^
TypeError: env(...).platform.isTypedArray is not a function
    at isTypedArray (/home/runner/work/stampy-ui/stampy-ui/node_modules/@tensorflow/tfjs/node_modules/@tensorflow/tfjs-core/dist/tf-core.node.js:2978:27)
    at inferShape (/home/runner/work/stampy-ui/stampy-ui/node_modules/@tensorflow/tfjs/node_modules/@tensorflow/tfjs-core/dist/tf-core.node.js:5428:9)
    at tensor1d (/home/runner/work/stampy-ui/stampy-ui/node_modules/@tensorflow/tfjs/node_modules/@tensorflow/tfjs-core/dist/tf-core.node.js:18528:25)
    at /home/runner/work/stampy-ui/stampy-ui/node_modules/@tensorflow/tfjs-node/dist/kernels/StridedSlice.js:35:51
    at /home/runner/work/stampy-ui/stampy-ui/node_modules/@tensorflow/tfjs/node_modules/@tensorflow/tfjs-core/dist/tf-core.node.js:4506:22
    at Engine.scopedRun (/home/runner/work/stampy-ui/stampy-ui/node_modules/@tensorflow/tfjs/node_modules/@tensorflow/tfjs-core/dist/tf-core.node.js:4516:23)
    at Engine.tidy (/home/runner/work/stampy-ui/stampy-ui/node_modules/@tensorflow/tfjs/node_modules/@tensorflow/tfjs-core/dist/tf-core.node.js:4505:21)
    at tidy (/home/runner/work/stampy-ui/stampy-ui/node_modules/@tensorflow/tfjs/node_modules/@tensorflow/tfjs-core/dist/tf-core.node.js:8053:19)
    at Object.kernelFunc (/home/runner/work/stampy-ui/stampy-ui/node_modules/@tensorflow/tfjs-node/dist/kernels/StridedSlice.js:34:32)
    at kernelFunc (/home/runner/work/stampy-ui/stampy-ui/node_modules/@tensorflow/tfjs/node_modules/@tensorflow/tfjs-core/dist/tf-core.node.js:4651:32)
Error: Process completed with exit code 1.

In my procrastination, tfjs-node apparently fixed this already. Looks like we're currently using version 4.6.0 in the stampy-nlp repo just fine.


KBM Type Key/Type shortcut doesn't work if Launcher is still loading

It looks like when the Xaml Island part of launcher is loading, it's window handle is in focus. Because of that the Type key logic doesn't work as expected as it's supposed to work only when the KBM window is in focus. The moment the launcher loading is done everything works fine again.

@jyuwono @saahmedm @ryanbodrug-microsoft thoughts on this? I think it's a bug that a user would only see if the start PowerToys.exe and go straight to KBM and start doing stuff before Launcher finishes loading, or if they enable launcher and start using KBM while launcher is loading.
It's seems like a bug that would happen rarely for users so I don't know if we should prioritize this for Build, unless there is a quick and easy solution to make the launcher xaml island window not be in focus while loading.

Not able to repro this anymore, so closing the issue


return executedquerystring when error

currently, when there's an error (for example an error in the sql query), we do not get back the executed-query-string into grafana's query inspector:

in QueryDB we do construct a dataframe with the required metadata: https://github.com/grafana/sqlds/blob/3c409475badfb0bad9e76f547fa01a676d351e0a/query.go#L64
but in handleQuery, when QueryDB ends with an error, we do not return the "returned" dataframe in the general case

this PR returns the result, but i'm not sure if this is the best approach. could there be some leak of info with this that we want to avoid? WDYT?

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

I think it's just confusion with the sdk that you can return a frame with an error, and an error.  And when should a frame be returned with an error vs an error.  And if a frame is returned with an error, do we also return an error.


RUMM-1498 update nightly tests for background RUM events

What does this PR do?
Following #652, update the nightly test to handle cases when background RUM events are tracked/not tracked.

Codecov Report

Merging #654 (6e50861) into master (8a8cee1) will increase coverage by 0.13%.
The diff coverage is n/a.


Non option to connect to the mainnet of Temple Wallet

Hi there,
Great project. I've found some issues though.
I've been testing the connection to several wallets via Beacon and I've found the following issues:

Temple is the most intuitive one but, now somehow now I can not connect to the Wallet Mainnet. (Please see the image attached). I can't select the mainnet instead of the Edo2testnet.

Galleon: seems to work fine but when I try to connect it to Minter it throws an error "Beacon network mismatch"

Kukai. I haven't found a way to get the secret key. So I can't add it to the config files.

Thank's in advance,
Aleix

Hey Aleix,
Minter should automatically select the correct network based on the Minter configuration. Is your minter installation set to run against mainnet or edo2net?

Hi mycodecrafting,
My minter installation was set to run against the mainnet. Although this issue has been solved with the last release.
Thanks,
Aleix


Add --prefix arg to enable prefix in Ludown

Proposed Changes
Add --prefix flag to enable prefix in messages.
Testing
Several tests were added to test the addition of the prefix

Pull Request Test Coverage Report for Build 1273

20 of 20 (100.0%)  changed or added relevant lines in 13 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.09%) to 68.884%

Totals

Change from base Build 1268:
0.09%

Covered Lines:
3887

Relevant Lines:
5410

💛  - Coveralls


WARP: Update README with macOS library path instructions

The README said to use DEP_BINARYNINJACORE_PATH but this was not referenced.
I updated the build.rs to use this variable, and as a treat I added the default install locations from the User Guide
I think this works on Windows too, but I do not have a Windows install to test. Thank you for the hard work on WARP I am excited to use it! 💜💜🥷

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Hi, thank you for the PR.
I don't know why the README says to set DEP_BINARYNINJACORE_PATH, that env var is only used on a separate branch.
Anyways what you want is to set BINARYNINJADIR instead.
Also this might interest you: https://github.com/Vector35/warp/pull/22/ (sigem docker image)

Correct the README: https://github.com/Vector35/binaryninja-api/commit/487fa4b240ae2c2d5a712dd6e00071f8de57d0fc
Add auto linking: https://github.com/Vector35/binaryninja-api/commit/5239ad6e8345e691fde31fd9f513fcf200e942aa
Let me know if those two suffice.
FWIW we store a lastrun artifact on your system which can be used instead of guessing for the install.
Thank you for the PR!

Correct the README: 487fa4b Add auto linking: 5239ad6
Let me know if those two suffice.
FWIW we store a lastrun artifact on your system which can be used instead of guessing for the install.
Thank you for the PR!

Lastrun is documented here for those curious:
https://docs.binary.ninja/guide/index.html#user-folder (it lives in your user folder)

Correct the README: https://github.com/Vector35/binaryninja-api/commit/487fa4b240ae2c2d5a712dd6e00071f8de57d0fc
Add auto linking: https://github.com/Vector35/binaryninja-api/commit/5239ad6e8345e691fde31fd9f513fcf200e942aa
Let me know if those two suffice.
FWIW we store a lastrun artifact on your system which can be used instead of guessing for the install.
Thank you for the PR!

This looks much better, thank you!

Closing as author is satisfied with the changes mentioned above, re-open if that isn't the case. Thanks again!


Background URLSessions support

Is your feature request related to a problem? Please describe.
If the app gets suspended or terminated, the URLSession cannot continue.
Describe the solution you'd like
Either the OpenAI class itself or some other class that would take the responsibility of making the calls would have to be a subclass of NSObject and implement the URLSessionDelegate protocol, then there would have to be a way to register these delegates, associate them with individual calls, and have a way for the AppDelegate to bring them to life and finish the original task even after the app was suspended.
https://developer.apple.com/documentation/foundation/urlsessiondelegate/1617185-urlsessiondidfinishevents
We can model this on the AWSS3TransferUtility:
https://github.com/aws-amplify/aws-sdk-ios/blob/main/AWSS3/AWSS3TransferUtility.m
Describe alternatives you've considered
Not sure if we have an alternative. The main issue is that the background sessions do not support closures so we have to use URLSession delegates.
Additional context
This is about robustness of the system - low connectivity and people leaving the app. Imagine you have to transcribe 1 GB audio file. Even if you split it to smaller parts, you still need to upload them somehow if the user goes to the background. I don't think real users will have the patience to look at a progress bar for 10 minutes or however long it will take.

hi @kutakmir , did you go ahead with this?

after some experimentation, i find 3 key takeaways (and a bonus 4th):

URLSession must be created with a delegate, and configured to run in background.

URLSession(
    configuration: .background(withIdentifier: UUID().uuidString),
    delegate: self,
    delegateQueue: nil)

Requires keeping track of completionHandler closures, and running them when system invokes urlSession methods. Recommend dictionaries keyed by UUID, which can be recovered from URLSession.configuration.identifier. We need to do this due to the runtime error: "Completion handler blocks are not supported in background sessions. Use a delegate instead."
URLSessionDelegate.urlSession implementations (and related classes) must be invoked on main thread, e.g. using DispatchQueue.main.sync {} when in the background.
Extra Credit: urlSessionDidFinishEvents method doesn't actually come into play, although it does get called when the app is in the background.

All this is demonstrated in this fairly minimal example. Make a new iOS project, delete all .swift files, and add just this. It runs for me in iOS 15 simulator.
Perhaps this would help someone more familiar with MacPaw/OpenAI networking accomplish this request, or maybe I'll dig into that now that I have this minimum working example, as that suggested aws codebase is pretty fracking heavy for what we've got here.
import SwiftUI

#if os(macOS)
class MyAppDelegate: NSObject, NSApplicationDelegate {

    func applicationDidFinishLaunching(_ notification: Notification) {
    }

    func applicationWillTerminate(_ notification: Notification) {
        // Insert code here to tear down your application
    }

    func applicationShouldTerminateAfterLastWindowClosed(_ sender: NSApplication) -> Bool {
        return true
    }
}
#endif

@available(iOS 14.0, *)
@main
struct URLSessionDelegateExampleApp: App {
    @Environment(\.scenePhase) private var scenePhase

    var body: some Scene {
        WindowGroup {
            ContentView()
        }
        .onChange(of: scenePhase) { _, newPhase in
            switch newPhase {
            case .active:
                // App is now active
                DataModel.instance.isInForeground = true
            case .inactive:
                // App is now inactive (e.g., transitioning to the background)
                DataModel.instance.isInForeground = false
            case .background:
                // App is now in the background
                DataModel.instance.isInForeground = false
            @unknown default:
                break
            }
        }
    }
}

class DataModel: ObservableObject {

    static var instance = DataModel()
    private init() {}
    @Published var text = "Hello, \(#function)!"
    @Published var isInForeground = true
}

@available(iOS 13.0, *)
struct ContentView: View {
    @ObservedObject var dm = DataModel.instance

    var body: some View {
        VStack {
            Button(action: {
                UIPasteboard.general.string = dm.text
            }) {
                Text(dm.text)
                    .onAppear {
                        MyURLSessionDelegate
                            .instance
                            .urlSession
                            .downloadTask(
                                with: URLRequest(url: URL(string: "https://duck.com")!),
                                completionHandler: { [dm] url, urlResponse, error in
                                    if let url = url ?? urlResponse?.url {
                                        if let data = try? Data(contentsOf: url) {
                                            dm.text = String(data: data, encoding: .utf8) ?? data.base64EncodedString()
                                        } else {
                                            dm.text = url.absoluteString
                                        }
                                    } else if let error {
                                        dm.text = error.localizedDescription
                                    } else {
                                        dm.text = "downloaded https://duck.com"
                                    }
                                })
                            .resume()
                    }
            }
        }
    }
}

@available(iOS 13.0, *)
class MyURLSessionDelegate: NSObject {
    static let instance = MyURLSessionDelegate()
    private override init() {
        super.init()
    }

    var completionHandlers = [String: @Sendable (URL?, URLResponse?, Error?) -> Void]()

    var urlSession: URLSession { get {
        URLSession(
            configuration: .background(withIdentifier: UUID().uuidString),
            delegate: self,
            delegateQueue: nil)
    }}

}

@available(iOS 13.0, *)
extension MyURLSessionDelegate: URLSessionDelegate {
    // Tells the delegate that all messages enqueued for a session have been delivered.
    /*
     In iOS, when a background transfer completes or requires credentials, if your app is no longer running, your app is automatically relaunched in the background, and the app’s UIApplicationDelegate is sent an application(_:handleEventsForBackgroundURLSession:completionHandler:) message. This call contains the identifier of the session that caused your app to be launched. You should then store that completion handler before creating a background configuration object with the same identifier, and creating a session with that configuration. The newly created session is automatically reassociated with ongoing background activity.
     When your app later receives a urlSessionDidFinishEvents(forBackgroundURLSession:) message, this indicates that all messages previously enqueued for this session have been delivered, and that it is now safe to invoke the previously stored completion handler or to begin any internal updates that may result in invoking the completion handler.
     */
    public func urlSessionDidFinishEvents(forBackgroundURLSession: URLSession) {
#if DEBUG
        print(#function)
#endif
        // Handle the completion of events for background URL session here
        // Notify the app's completion handler, if any
        DispatchQueue.main.sync {
        }

    }

    public func urlSession(_ urlSession: URLSession, didBecomeInvalidWithError: Error?) {
        if let completionHandler = self.completionHandlers.removeValue(forKey: urlSession.configuration.identifier!) {
            if DataModel.instance.isInForeground {
                DispatchQueue.main.async {
                    completionHandler(nil, nil, didBecomeInvalidWithError)
                }
            } else {
                DispatchQueue.main.sync {
                    completionHandler(nil, nil, didBecomeInvalidWithError)
                }
            }
        }
    }
}

@available(iOS 13.0, *)
extension MyURLSessionDelegate: URLSessionDownloadDelegate {

    public func urlSession(_ urlSession: URLSession, downloadTask: URLSessionDownloadTask, didFinishDownloadingTo location: URL) {
        if let completionHandler = self.completionHandlers.removeValue(forKey: urlSession.configuration.identifier!) {
            if DataModel.instance.isInForeground {
                DispatchQueue.main.async {
                    completionHandler(location, downloadTask.response, downloadTask.error)
                }
            } else {
                DispatchQueue.main.sync {
                    completionHandler(location, downloadTask.response, downloadTask.error)
                }
            }
        }
    }
}

extension URLSession {

    public func downloadTask(with request: URLRequest, completionHandler: @escaping @Sendable (URL?, URLResponse?, Error?) -> Void) -> URLSessionDownloadTask {
        MyURLSessionDelegate.instance.completionHandlers.updateValue(completionHandler, forKey: self.configuration.identifier!)
        return downloadTask(with: request)
    }

    public func downloadTask(with url: URL, completionHandler: @escaping @Sendable (URL?, URLResponse?, Error?) -> Void) -> URLSessionDownloadTask {
        MyURLSessionDelegate.instance.completionHandlers.updateValue(completionHandler, forKey: self.configuration.identifier!)
        return downloadTask(with: url)
    }

    public func downloadTask(withResumeData resumeData: Data, completionHandler: @escaping @Sendable (URL?, URLResponse?, Error?) -> Void) -> URLSessionDownloadTask {
        MyURLSessionDelegate.instance.completionHandlers.updateValue(completionHandler, forKey: self.configuration.identifier!)
        return downloadTask(withResumeData: resumeData)
    }

}


A question about qk_scale

Hello, I have a question about this:
https://github.com/microsoft/Swin-Transformer/blob/2622619f70760b60a42b996f5fcbe7c9d2e7ca57/models/swin_transformer.py#L123
what's the function of scale ? Or I can find the reason in paper?
Thanks.

You can find the reason in paper Attention is All you Need. In Section 3.2.1, the author says "...We suspect that for large values of dk, the dot products grow large in magnitude, pushing the softmax function into regions where it has extremely small gradient. To counteract this effect, we scale the dot products by 1/sqrt(dk)."

You can find the reason in paper Attention is All you Need. In Section 3.2.1, the author says "...We suspect that for large values of dk, the dot products grow large in magnitude, pushing the softmax function into regions where it has extremely small gradient. To counteract this effect, we scale the dot products by 1/sqrt(dk)."
Hello @zeliu98 ,
I have got it. Thanks for your reply!
The issue can be closed.


Duplicate tags when having multiple @Api annotations with the same value

When having more than one class that shares the same @Api annotation, the tags will be duplicated in the list of tags. Here's a sample generated with v 2.0.1-SNAPSHOT today:
tags: [
   {
      name: "/activities"
   },
   {
      name: "/activities"
   }
]

Something's not right for sure. It shouldn't even look like a path. Have you by any chance created a group strategy plugin?

Sorry, @dilipkrish, it's my custom GrouppingStrategy. Feel free to close this

Closing it then 👍🏻


multi: Add setaltsig endpoint.

docs/api: Add setaltsig details.

middleware: Allow alternate vsp auth address.

If an alternate address exists for this ticket, check it first and fall
back to the commitment address if validation fails. Alternate address
failure is allowed so that other endpoints do not need to add a field to
all requests specifying whether this is the alternate address's
signature or the commitment address's signature.

webapi: Add setaltsig endpoint.

Allow setting or clearing the alternate public key that signs json
requests from the client. Save a copy of the request and signature in
the db.

This also adds another route to put a ticket in the database. Add a fee
status to indicate no fee address has been assigned yet.

db/ticket: Add AltSigAddress.

To allow signing with addresses other than the commitment address, add
an alternate signature address. In order to continue to prove that the
address was chosen by the user, add an alternate signature history. Only
allow three records per ticket to be saved to cap needed db space.

closes #279

As discussed in matrix, Will change the history to one record.

This code is running at dcrstakedinner.com

Will go ahead and squash the current squashables tomorrow.

I've been chatting with @JoeGruffins about this on Matrix. I'm not very happy with how much code this PR is currently changing/adding. This is a minor change to functionality, which is to support a small set of users, and I think we can achieve the desired functionality with a much smaller changeset, without altering too much of the existing code.
A new database bucket to store altSig, request and response (keyed with ticket hash) can be introduced. In the new /altsig endpoint, we just create an entry in that bucket. Then we update the auth middleware to use any existing values in that bucket, or fallback to using the commitment address.
I believe that would avoid all of the changes which are being made to /getfeeaddress and the existing database code. Further, isolating the new optional step in this way means that code which is executed later doesnt need to look like if optional step completed (do this) else (do that) - it can just stay as it currently is.

Let me know if this looks more like what you were thinking @jholdstock  It does not touch getfeeaddress any longer, or mess with ticket initialization logic.
I do think that maybe we should remove addresses from the database for tickets that end up not being handled, like what happens with.. tickets I think.
This comes with a database upgrade now, for a new bucket. If you test please back up your original database first.

Deleting the altsig for unmined tickets. Also completely removed a notion of max altsigs and updating. Only allow users to set once.
This may not be optimal, but in order to use the new endpoint with decrediton, I needed to set some values in the nginx file. This is mine ver-batim.

nginx.conf
user nginx;
worker_processes auto;
error_log /var/log/nginx/stakepool_error.log;
pid /run/nginx.pid;

events {
    worker_connections 1024;
}

http {
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for" "$realip_remote_addr"';

    access_log  /var/log/nginx/stakepool_access.log  main;

    sendfile            on;
    tcp_nopush          on;
    tcp_nodelay         on;
    keepalive_timeout   65;
    types_hash_max_size 2048;

    include             /etc/nginx/mime.types;
    default_type        application/octet-stream;

    # Ensure ngx_http_realip_module is available
    set_real_ip_from <IP_ADDRESS>/32;
    # If you are behind a NAT router, specify LAN
    #set_real_ip_from 10.24.0/16;
    real_ip_header X-Forwarded-For;
    real_ip_recursive on;

    server {
        listen          80 default_server;
        server_name     _;

        return 308      https://$host$request_uri;
    }

    limit_req_zone $binary_remote_addr zone=stakepool:10m rate=8r/s;

    server {
        listen       443 http2 ssl default_server;
        server_name dcrstakedinner.com; # managed by Certbot

        gzip on;
        gzip_proxied any;
        gzip_comp_level 3;
        gzip_min_length 512;
                gzip_types text/css text/* text/javascript application/x-javascript application/json
                application/xml application/atom+xml application/xaml+xml application/javascript
                application/x-font-ttf application/font-woff font/collection font/opentype
                font/otf font/ttf image/bmp image/svg+xml image/x-icon application/octet-stream;

        #ssl                     on; # only needed for older nginx where not in listen
        ssl_certificate /etc/letsencrypt/live/dcrstakedinner.com/fullchain.pem; # managed by Certbot
        ssl_certificate_key /etc/letsencrypt/live/dcrstakedinner.com/privkey.pem; # managed by Certbot

        ssl_session_cache               shared:SSL:20m;
        ssl_protocols                   TLSv1.1 TLSv1.2; # add TLSv1.3 if supported
        ssl_ciphers                     EECDH+CHACHA20:EECDH+CHACHA20-draft:EECDH+AES128:RSA+AES128:EECDH+AES256:RSA+AES256:EECDH+3DES:RSA+3DES:!MD5;
        ssl_prefer_server_ciphers       on;
        #ssl_dhparam                     /etc/nginx/dhparam.pem; # openssl dhparam -out /etc/nginx/dhparam.pem 4096
        add_header                      Strict-Transport-Security max-age=15552001;

        # Everything else is proxied to dcrstakepool.
        location / {
            # apply rate limiting
            limit_req zone=stakepool burst=16;

            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $realip_remote_addr;
            proxy_pass http://<IP_ADDRESS>:8000;
            proxy_http_version 1.0;
            if ($request_method = 'OPTIONS') {
                add_header 'Access-Control-Allow-Origin' "https://dcrstakedinner.com" always;
                add_header 'Access-Control-Allow-Credentials' 'true' always;
                add_header 'Access-Control-Allow-Headers' "Origin, X-Requested-With, Content-Type, Accept, vsp-client-signature" always;
                add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
                add_header 'Access-Control-Max-Age' 1728000;
                add_header 'Content-Type' 'text/plain; charset=utf-8';
                add_header 'Content-Length' 0;
                return 204;
            }
        }
    }
}

decrediton can be on this pr: https://github.com/decred/decrediton/pull/3533
dcrwallet for the client is currently the wrong rpcversion to work with decrediton master, so a temporary wallet commit is here: https://github.com/JoeGruffins/dcrwallet/tree/useaccoutforvoting.flat
The vspd server can be run with default dcrd/dcrwallet.

I think I got everything commented upon. Tested buying a few more tickets and looks ok so far.
how do you like this updating style? It's my first time to try, and sorta my idea I guess. Is it confusing? It feels more organized for me, keeping changes together, but it may be just me. Perhaps quoting the commit hash in replies?

The update style seems good to me, its easy to follow.
Which part of the nginx config is required exactly? Theres a lot there. I'm guessing something to do with ssl?

Which part of the nginx config is required exactly? Theres a lot there. I'm guessing something to do with ssl?

Specifically the part under if ($request_method = 'OPTIONS') { I belive. It isn't optomized. There may be something extra I should be doing in decrediton. Will look into in more unless it's something you know about...

Thanks for the great review! Will squash outstanding squash commits before addressing.

hmm, that was a botched rebase

Log levels were all over the place, sorry. Wasn't paying enough attention to those.

Will squash squashables.


[red-knot] add support for more type narrowing forms

[x] x is not None
[ ] #13715
[ ] x
[ ] bool(E) (where E is any supported narrowing expression)
[ ] isinstance(x, T) (where T is a class or tuple of classes)
[ ] not isinstance(x, T) (where T is a class or tuple of classes)
[ ] issubclass(x, T) (where T is a class or tuple of classes)
[ ] not issubclass(x, T) (where T is a class or tuple of classes)
[ ] not x
[ ] x == None
[ ] x != None
[ ] type(x) is C (where C is a class)
[ ] type(x) is not C (where C is a class)
[ ] x == T (where T is any literal type)
[ ] x != T (where T is any literal type)
[ ] x is T (where T is any literal type, incl a class or function)
[ ] x is not T (where T is any literal type, incl a class or function)
[ ] E and F (where E and F are both supported narrowing expressions)
[ ] E or F (where E and F are both supported narrowing expressions; reduces to (T & C(E)) | (T & C(F)), where T is the prior type of a variable and C(E) / C(F) are the constraints on it obtained from E and F)
[ ] apply negated narrowing from previously excluded cases to elif and else blocks

This is not an exhaustive list of what we'll eventually want to support, it's just a starter list of some useful ones that aren't currently blocked by other type system features that aren't implemented yet.
(Supporting narrowing on x.attr or x[sub] expression forms is out of scope for this issue.)
Implementation for these lives in crates/red_knot_python_semantic/src/types/narrow.rs.

https://github.com/microsoft/pyright/blob/main/docs/type-concepts-advanced.md#type-guards is pyright's documentation on supported narrowing forms.

Thanks for listing them down. There are also pattern constraints which are applied and yet to be implemented: https://github.com/astral-sh/ruff/blob/5b4afd30caebd6e2c5c27bbf5debc1663cbb28a2/crates/red_knot_python_semantic/src/types/narrow.rs#L96-L125

Is one of the tasks a good first issue? I really liked the idea behind red-knot and am looking forward to contribute

Adding support for x != None should be very good for a first issue, since it should have exactly the same effect as x is not None, which is already supported.

Some predicates narrow based on characteristics of an object that could be transient for mutable objects.
For example, if x: means that x must be truthy. This means that if x is typed as None, or Literal[False] or Literal[0], we can narrow its type to Never, since those types represent known immutable objects that are never truthy and never can be. If x is typed as a union including any of those types, we can safely eliminate them from the union. And it will be important that we can do this. But we cannot generally assume that this narrowing means x will remain truthy. If x is a list, it could be truthy one moment, and the next moment (after a .clear() call), it no longer is.
Similarly, it is not correct to represent the narrowing effect of if x == 1: as an intersection with Literal[1], because any type in Python can write an __eq__ method that compares equal to the integer 1. So x == 1 does not imply that x must actually be of type Literal[1]. Even if x were previously typed as int, it could be an instance of a subclass of int that compares equal to 1; that's not Literal[1]. But there are some types that we know represent objects that can never become equal to 1: for instance None, or any other literal type that isn't Literal[1]. It will be important that we can narrow away these types, but otherwise we can't try to preserve any knowledge that x == 1, because x could be mutable, and its equality to 1 could change any time after the narrowing, without us being aware of it.
To sum this up with examples, this is the behavior I think we will want:
x: int | None
if x:
    # this could also be `int & ~Literal[0]`?
    reveal_type(x)  # revealed: int

x: Literal[0, 1, 2]
if x:
    reveal_type(x)  # revealed: Literal[1, 2]

x: int
if x == 1:
    # not safe to narrow to `Literal[1]`, could be a subclass of `int` with custom `__eq__`
    reveal_type(x)  # revealed: int

x: Literal[1, 2, 3]
if x == 1:
    reveal_type(x)  # revealed: Literal[1]

One way to approach this, which is tempting in its generality (and previously tempted me), is to define new Type variants such as Truthy and EqualTo(T), and then return these types as the constraints on x from x or x == 1. Then we can handle these types in intersection simplification, where e.g. Truthy would be disjoint with None and Literal[False] and some other types, and EqualTo[Literal[1]] would be disjoint from all other literals.
The downsides of this approach are two-fold. One is that we then have to handle these types in all type operations, increasing the size of the matrix for binary operations especially, even though it's not clear that these types have expressive utility outside of narrowing.
The bigger problem is that these types represent possibly-transient characteristics, as discussed above. A particular list[int] may be truthy one moment, and then after a .clear() call, it is no longer truthy. This means that really the only safe use of these types is to immediately eliminate union elements that we know must always be disjoint with them; otherwise we should simply discard them. For example, after x == 1 where x is int, not only can we not narrow x to Literal[1], we shouldn't even keep its type as int & EqualTo(Literal[1]), because it may be that some subsequent method call on x makes it no longer EqualTo(Literal[1]) at all, in the same way l.clear() would invalidate an intersection like list[int] & Truthy.
I think this is sufficient reason to discard having types like EqualTo or Truthy.
I think the key observation here is that, since there are a limited set of types where we know all inhabitants are immutable with respect to equality and truthiness (all literal and singleton types), the narrowing we can apply in these cases is only a negative intersection to eliminate some of these literal types, never an intersection with a positive type. If the narrowing is represented as an intersection with some constraint type (as we currently represent it), the constraint type must be a negation type (an intersection with only negative elements in it.)
This is easy for if x:: it can be safely represented as an intersection with ~Literal[0] & ~Literal[False] & ~Literal[""] & ~None. But this approach doesn't work for narrowing if not x: or if x == 1:, because the set of all known-never-to-be-falsy types is too large to represent directly (it includes all literal types that are not 0/False/""/None), and similarly the set of all known-not-to-equal-1 types is very large (all literal types that are not Literal[1]).
Another approach would be to inspect the current type of x in if x == 1: while inferring narrowing constraints, and just build a negative intersection containing only the relevant types from the too-large-to-represent set of negative constraints. That is, if x is Literal[1, 2, 3] and we have if x == Literal[1]:, in theory we'd want to intersect x with the too-large-to-represent type "all literal types that are not Literal[1]", but in practice we could intersect it simply with ~Literal[2] & ~Literal[3] and get the same result. To some extent this means we are re-implementing some intersection simplification logic in the inference of narrowing constraints, and muddying the boundaries between inferring a constraint and applying it. But this may be a totally fine pragmatic approach.
A variant of this could be to have constraints no longer be represented simply as a Type to intersect with, but as a new enum that might be just a Type to intersect with, or might be something like EliminateNotEqualTo[Literal[1]] or EliminateNonTruthy, and then we'd apply those constraints later, in type inference, when we already have the type of x in hand. The actual application would look similar to what we'd do in the previous paragraph, we just wouldn't do it eagerly when inferring the constraint itself, so as to keep "inference of constraints from a test expression" and "application of constraints to a type" more cleanly separated.

"Not isinstance" and "elif else" can be checked off ✅

I think we also need to handle TypeIs[T] and TypeGuard[T] for narrowing. (blocked by the implementation of generics for now)

Yeah, I was thinking of TypeIs and TypeGuard as a separate feature, but it doesn't hurt to list it here.
I don't think it is blocked on generics. These are just special forms, which use bracket syntax, but they don't require user-defined generic classes or functions.


docker: pin python 3.7 instead of 3.8, try to fix image do not start

With 3.7, it starts on my VM lvupavicsdev.ouranos.ca (available for testing).
Docker image do not start under python 3.8 with this error:
Traceback (most recent call last):
  File "/opt/conda/envs/wps/bin/raven-wps", line 33, in <module>
    sys.exit(load_entry_point('raven', 'console_scripts', 'raven-wps')())
  File "/opt/conda/envs/wps/bin/raven-wps", line 25, in importlib_load_entry_point
    return next(matches).load()
  File "/opt/conda/envs/wps/lib/python3.8/importlib/metadata.py", line 77, in load
    module = import_module(match.group('module'))
  File "/opt/conda/envs/wps/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 783, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/opt/wps/raven/cli.py", line 16, in <module>
    from . import wsgi
  File "/opt/wps/raven/wsgi.py", line 5, in <module>
    from .processes import processes
  File "/opt/wps/raven/processes/__init__.py", line 1, in <module>
    from .wps_raven import RavenProcess  # isort: skip
  File "/opt/wps/raven/processes/wps_raven.py", line 7, in <module>
    from ravenpy.models import Raven
  File "/opt/conda/envs/wps/lib/python3.8/site-packages/ravenpy/models/__init__.py", line 3, in <module>
    from .base import Ostrich, Raven, get_average_annual_runoff
  File "/opt/conda/envs/wps/lib/python3.8/site-packages/ravenpy/models/base.py", line 33, in <module>
    from .importers import NcDataImporter
  File "/opt/conda/envs/wps/lib/python3.8/site-packages/ravenpy/models/importers.py", line 9, in <module>
    import geopandas
  File "/opt/conda/envs/wps/lib/python3.8/site-packages/geopandas/__init__.py", line 1, in <module>
    from geopandas._config import options  # noqa
  File "/opt/conda/envs/wps/lib/python3.8/site-packages/geopandas/_config.py", line 126, in <module>
    default_value=_default_use_pygeos(),
  File "/opt/conda/envs/wps/lib/python3.8/site-packages/geopandas/_config.py", line 112, in _default_use_pygeos
    import geopandas._compat as compat
  File "/opt/conda/envs/wps/lib/python3.8/site-packages/geopandas/_compat.py", line 202, in <module>
    import rtree  # noqa
  File "/opt/conda/envs/wps/lib/python3.8/site-packages/rtree/__init__.py", line 9, in <module>
    from .index import Rtree, Index  # noqa
  File "/opt/conda/envs/wps/lib/python3.8/site-packages/rtree/index.py", line 6, in <module>
    from . import core
  File "/opt/conda/envs/wps/lib/python3.8/site-packages/rtree/core.py", line 77, in <module>
    rt.Error_GetLastErrorNum.restype = ctypes.c_int
  File "/opt/conda/envs/wps/lib/python3.8/ctypes/__init__.py", line 394, in __getattr__
    func = self.__getitem__(name)
  File "/opt/conda/envs/wps/lib/python3.8/ctypes/__init__.py", line 399, in __getitem__
    func = self._FuncPtr((name_or_ordinal, self))
AttributeError: /opt/conda/envs/wps/bin/python3.8: undefined symbol: Error_GetLastErrorNum

One solution proposed on the net is to install libspatialindex with conda or apt.
Toblerity/rtree#64 (comment)

Let's keep this option when we can not stay on 3.7 anymore.  I'd rather not install not needed packages and grow the docker image size that is already quite big.

Ok, but I'm pretty sure it's installed in 3.7, you just didn't have to manually install it, it came bundled with another library or something like that.

That's very strange behaviour. We should have installable images for Python3.7, 3.8, and 3.9.

That's very strange behaviour. We should have installable images for Python3.7, 3.8, and 3.9.

Probably what David hinted is right.  The additional package is probably also there on 3.7 but on 3.8 it has to be installed manually so probably a broken dependency tree on 3.8.
I suspect this dependency problem will be fixed with time and so our next attempt to move to 3.8, hopefully we won't have to perform the manual work-around.
Given the Raven docker image is gigantic (4 GB), there are a lot of indirect dependencies.  The newer python, the less chance it works.  If only 1 single dependency declare it can not go to a newer version of python, the entire stack will stay behind.
That's exactly why our Jupyter env is also at 3.7 without any python pinning.  It's so gigantic (6G docker image) that 3.7 is the greatest common denominator for all the packages installed.

@Zeitsperre For PyWPS servers, we only need to support one Python version, since the core use case is being used in prod on our own infrastructure. I'm happy to stick with 3.7 until it's time to upgrade to 3.8 and then we make the move across all birds.
Those that work at the moment with 3.8 should stick to 3.8, but I don't think we need to worry about supporting multiple python versions for WPS servers.

Okay, understoud. Thanks for the explanations. If we don't need to support multiplatform/multiple version of Python for WPS, that make our lives a bit easier.


Docker run and capture output

Hi,
Is there a way to use docker.run and capture the output to a variable.
This is what I have so far:
var options = { Binds: ['/scans:/scans:rw'], NetworkMode:'mynet'}

docker.run('scanafi-reporting', ["python", "process.py", "test", "test", "1.5.6", "23 June 2018"], process.stdout,
    options,
    function (err, data, container) {
        console.log(data.StatusCode);
        console.log(container)

    });

I've had success with capturing the stream this way:
    var Writable = require('stream').Writable;
    // snip
    var myStream = new Writable();

    myStream._write = function write(doc, encoding, callback) {
      var StringDecoder = require('string_decoder').StringDecoder;
      var decoder = new StringDecoder('utf8');
      var result = decoder.write(doc);
      resolve(result);  // Note I'm wrapping this in a promise
    };

    // snip
    docker.run('myContainer', ['node', 'app.js'], myStream, {}...

However, I'm finding a limitation on the length of what can be outputted.  It seems to cap at 8192 bytes for some reason, so if anyone has a pointer there, I'd appreciate it!

The best approach is to use library such as memory-streams to deal with output.

https://github.com/apocas/dockerode/issues/401


Adds extract_hsi_tif in clip_plot function

Integrates the extract_hsi_to_tif function with clip_plot. So now hsi/hdf5/h5 data can be processed with the package.
Resolves the ISSUE #42

@MarconiS, merge this PR so that these can be reflected In the PR #46


Add scrollToTopByDefault ability

I've found this to be pretty useful... it allows the ability to prevent the current scroll position on the screen from being maintained as you navigate forward through an SPA.

Seems useful to me too. Thanks!

Cool, no problem! Is there any chance of getting a new release for it?

Yes – I just released version 1.1.0. Thanks again. :+1:

woot, thanks :star2:


Add method for validating global filters names

fixing https://bugzilla.redhat.com/show_bug.cgi?id=1381622
Add method for validating global filters' names
when saving global filter in expression editor
with the same name.
Originally, validating uniqueness of names of filters in miq_search.rb
in main manageiq repo was designed to compare names.
But name of a filter is a concatenation of its type (search_type) and its description.
There are different types of filters under Global Filters in the tree:
"global" and also "default". And this is the core of the problem.
It was compared something like "default_Analysis" to "global_Analysis".
Two different names so it didn't throw error and allowed you to save filter
with the same "name". We call it "name", but it is a :description of a filter
what we have to compare to fix the bug.
This PR just adds the method to MiqSearch.
The BZ will be completely fixed by merging this
and another PR in manageiq-ui-classic repo:
https://github.com/ManageIQ/manageiq-ui-classic/pull/327

@miq-bot add_label bug, ui

@miq-bot remove_label wip

@miq-bot add_label wip

@miq-bot remove_label wip

@miq-bot add_label wip

@miq-bot remove_label wip

@isimluk rubocop fixed, all checks have passed, and I think this is ready to merge, @gtanzillo

Thanks!


feat(orchestrator): add filter by DateArgument

Add the capability to filter by fields that are modelled as DataArgument in DataIndex.
Supported operators are EQ, IN, IS_NULL, GT, GTE, LT, LTE, BETWEEN.

Some of the operators (e.g. equal) can be used for other types too. Need to open 2 or 3 follow up PR to complete the work.


Config validation

This PR refactors type checking, and introduces config validation using Zod.
What's in this PR

Zod added as dependency
tsconfig strict compiler option enabled, a prerequisite for using Zod
Config file refactored to rely on Zod for type checking. Additional value constraints, previously not possible when using only TS, have been implemented.
Config validation before usage

great idea, thanks @iperzic !


Humidity of Heating Group not correct

Please describe your problem
I am using a HM-CC-RT-DN and a HM-TC-IT-WM-W-EU as part of a heating group in the CCU.
I have added the group with the channel
Climatcontrol_RT_Transiver
as:
HomematicThermostatAccessory
In HAP.
I can read current and to-be temperate and also manipulate the to-be temperature as well as changing the heating mode.
But I don’t get the Humidity values. They are stuck on 20%!
Humidity should be transferred from HM-TC-IT-WM-W-EU to the group and then be read by HAP and shown in home app.
I can reproduce it with another Heating group with same HM devices. It doesn’t matter if I separate humidity as a separate tile or not. If I add the HM-TC-IT-WM-W-EU additionally to HAP then this device shows the correct humidity in home app.

It seems to have changed yesterday


Add osm extract url

Checklist

[x] Appropriate branch selected (all PRs must first be merged to dev before they can be merged to master)
[x] Any modified or new methods or classes have helpful JavaDoc and code is thoroughly commented
[x] The description lists all applicable issues this PR seeks to resolve
[x] The description lists any configuration setting(s) that differ from the default settings
[x] All tests and CI builds passing

Description
Allow for downloading OSM file from a public URL instead of using the vex extract server. This
supports non-North American deployments and allows for the use of a stable OSM network for US
deployments (rather than using a new extract each time).
re ibi-group/datatools-ui#124

:tada: This PR is included in version 3.7.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


[Documentation] Need to include the expected content of sslprofiles.xml in uploading certificates through UI

Description:
In [1], we have not mentioned about that we expect the entry with * for Servers need to be included in sslprofiles.xml file. If customer adds some other entries for that file, this feature will not work.
[1] https://docs.wso2.com/display/AM210/Dynamic+SSL+Certificate+Installation
Suggested Labels:
Bug, docs
Suggested Assignees:
N/A
Affected Product Version: APIM wum updated
OS, DB, other environment details and versions:    Ubuntu
Steps to reproduce:
N/A
Related Issues:
N/A

@ruthryi
Please add a note to the documentation with the following information.
The sslprofiles.xml file is configured with * so that it can be used with any backend server. You DO NOT need to modify this.


Response.close() nullability issue.

Not a big deal, but since OkHttp now returns much more nullable / not nullable things I found that I missed some cases handling in my code.
And this lead to seeing that Response.close() directly calls body.close() with body that can be null and not tested.
The function should maybe either be marked as possible NPE throw or test for null body for completeness.

We could throw an IllegalStateException instead. Applications shouldn't ever attempt to close the response returned from a cacheResponse() or networkResponse().

Not sure to understand
From code https://github.com/square/okhttp/blob/c70e35802e984e8b9bf3fbf1479deb37c0bdfa43/okhttp/src/main/java/okhttp3/Response.java#L274
Response.close() correctly only deals with body (ResponseBody) and does not touch the others.
Finally maybe I'm wrong but in the end just adding a body != null check sound like the proper handling as there's no real error to manage in fact, a Closeable should not NPE on close.

It's an error to attempt to close a response that doesn't have a body.

It's an error to attempt to close a response that doesn't have a body.

@swankjesse how strongly do you hold this opinion? If this were not an error condition, then users could use Response in a try-with-resources statement in scenarios where the body is not considered. Take this example which checks for a successful status code and ignores the body
Response response = null;
try {
  response = client.newCall(request).execute();
  if (!response.isSuccessful()) {
    throw new IllegalStateException("HTTP server returned error code " + response.code());
  }
} finally {
  if (response != null && response.body() != null) {
    response.close();
  }
}

If this code didn't need to first check response.body != null to avoid an error condition, then we could get rid of boilerplate with a try-with-resources statement.
try(Response response = client.newCall(request).execute()) {
  if (!response.isSuccessful()) {
    throw new IllegalStateException("HTTP server returned error code " + response.code());
  }
}

Your example above is safe. You have to try hard to get a response without a body – it’s the response used by Response.networkResponse() or Response.cacheResponse().

Cool, that helps. I was running into trouble in a unit test where I was building a Response myself and I omitted the body e.g.
    final Response response =
        new Response.Builder()
            .request(success.request())
            .protocol(Protocol.HTTP_1_1)
            .message("OK")
            .code(204)
            .build();

but I can fix this by adding an empty body to the builder chain and remain confident that for my use case responses should always have a non-null body.
.body(EMPTY_RESPONSE)

thanks!


added cleanup instructions, fixed formatting for consistency

What type of PR is this?
/kind cleanup
Any specific area of the project related to this PR?
/area documentation
What this PR does / why we need it:
I added cleanup instructions for the linux lab and made some edits to make it consistent with the Kubernetes lab.
Which issue(s) this PR fixes:

Fixes #
Special notes for your reviewer:

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: mikegcoleman
Once this PR has been reviewed and has the lgtm label, please assign vjjmiras for approval. For more information see the Kubernetes Code Review Process.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

content/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: Issif, mikegcoleman
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

content/OWNERS [Issif]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

LGTM label has been added.  Git tree hash: 7aa5582ccb36c52ccc28bf2b966c1b4fd0fe2c94


Fixed output redirection in various scripts

Just a small fix about redirection in some of the scripts

thanks for this!