[feature request] can read_feather in R use skip?

I have a feature request for R's read_feather to be able to use the "skip" parameter found in read.csv. Normally when I write data to disk I also write comments for myself for how the data was generated and place them into the csv file. When reading this data back into R later on I use read.csv(..., skip = X). Will I be able to do the same with feather?
The resulting csv I would write would look like
#Note to self, this file came from ...
<<normal csv header>>
<<normal csv row>>

No, because feather doesn't support comments in that way.


[Bug]: Cannot find module with id = {xxx} It might have been removed on the Rust side.

System Info
System:
OS: macOS 14.7
CPU: (10) arm64 Apple M1 Pro
Memory: 2.76 GB / 16.00 GB
Shell: 5.9 - /bin/zsh
Binaries:
Node: 18.18.2 - ~/.nvm/versions/node/v18.18.2/bin/node
npm: 9.8.1 - ~/.nvm/versions/node/v18.18.2/bin/npm
pnpm: 9.12.3 - ~/.nvm/versions/node/v18.18.2/bin/pnpm
Browsers:
Chrome: 130.0.6723.70
Safari: 18.0.1
npmPackages:
@rspack/cli: ^1.0.14 => 1.0.14
@rspack/core: ^1.0.14 => 1.0.14
@rspack/dev-server: ^1.0.9 => 1.0.9
Details
{       test: /\.scss$/,       use: [         rspack.CssExtractRspackPlugin.loader,         {           loader: 'css-loader',           options: { modules: false }         },         {           loader: 'builtin:lightningcss-loader',           options: {             targets: ['last 2 versions', 'not ie < 11', '> 1%']           }         },         {           loader: 'sass-loader',           options: {             api: 'modern-compiler',             implementation: require.resolve('sass-embedded')        }       } }

for Plugins:
new rspack.CssExtractRspackPlugin({ ignoreOrder: true })
I dont konw why, it make me confuse
Reproduce link
No response
Reproduce Steps
There was no problem with the first compilation, but when I modified the code, the console directly reported an error

I used style-loader to replace  rspack.CssExtractRspackPlugin.loader, it works well

It's possible that you are calling some methods on a Module on the JavaScript side. The panic occurs when you call methods on a Module that has been removed from the Compilation in Rust.

its my errors, thanks


Discussion: Improve memory use for queries

This is a tracker issue for memory utilisation in Thanos to serve Queries and the steps we want to take to make things better.
Core of the issue:

Both Querier and Store Gateway allocate unbounded amounts of memory during query execution.
This can and does lead to OOM in resource-limited container environments (k8s etc).

What we want is:

A list of ideas to consider and try;
Status of each idea based on discussions, tests etc;

Repro
Base build
thanos, version 0.7.0 (branch: master, revision: ed045447fef8c4b5ff62697559463549ff522f31)
  build user<EMAIL_ADDRESS>  build date:       20191010-13:52:26
  go version:       go1.13.1

Data

synthetically generated (uses thanosbench blockgen)
data size:

metrics: 200
targets: 100
sample interval: 15s
total time interval: 10 hours

Query

count({__name__=~".+"}) by (__name__)
time span: all generated data
curl:

curl 'http://localhost:11902/api/v1/query_range?query=count(%7B__name__%3D~%22.%2B%22%7D)%20by%20(__name__)&dedup=true&partial_response=true&start=1568634900&end=1569844500&step=4838&max_source_resolution=0s&_=1570622122482'

Memory profile

MacBook Pro 16GB RAM
Store Gateway: 20GB peak
Querier: 10GB peak before time-out
See chart

Chart

Green is Storage Gateway; Red is Querier
Note that the query timed out so does not show Querier in its entirety.

List of ideas
Per-query limits and flow optimisations looks to be the way to go.
[1.] Simple global memory limiter in Querier. Rejected. See https://github.com/thanos-io/thanos/pull/1631 .
[1.1] Simple global memory limiter in SG? I've just realised that the OOM problem actually may be in SG that causes most pain, not in Querier.  Worth considering?
[2.] Limit concurrent queries, max samples, and series(?). As per Cortex: https://github.com/thanos-io/thanos/pull/1631#issuecomment-541616487
[3.] Frontend query caching. See https://github.com/thanos-io/thanos/issues/1006

users start to use frontend query caching. This changes a characteristic of traffic against Querier a lot, as it splits queries by day - maybe we can take that as an advantage?

[4.] Result streaming.
why does it not stream out the results? is it something to do with deduplication perhaps? aggregation?

@bwplotka: Mainly because of SeriesSet API PromQL requiring full series per series iteration.

but in querier.go, we see comment which we way want to investigate further?

// TODO(@fabxc ): this could potentially pushed further down into the store API
// to make true streaming possible.
sortDedupLabels(resp.seriesSet, q.replicaLabels)

[5.] Prevent knowingly bad queries?
[6.] Forking.
Maybe crazy idea, but what if we fork process for each query with simple mem limits?
This could be a way to have per-query memory limits etc.
[7.] (@bwplotka) Count allocations (roughly) per user / query.
[8.] Chop [tmin, tmax] in Querier. Can we chop the requested [tmin, tmax] into smaller (1h?) chunks [tmin, T1], [T1, T3], ... [Tn, tmax], request those from Store Gateway one by one, assemble results in Querier like we do now, and then send over to client?  Is this similar to what Frontend query caching would do?  Not worth it, see https://github.com/thanos-io/thanos/pull/1667
[9.] (@bwplotka) can we loosen StoreAPI a bit to allow series being aligned differently? This means mean the returned values could be unordered thus opening up way to stream chunks.
[10.] Intern labels (Labels []Label). These probably consume tons of memory (strings). And there are probably lots of duplicate identical lists. Why keep them all in memory as distinct objects? Why don't we do this at the point close to proto unmarshal:
func internLables(labels []Label) []Label {
  h := hash(labels)
  if interned, found := lookup(h); if found {
    return interned
  }

  addToMap(h, labels)
  return labels
}

possibly look at other things we can intern?
[11.] Related work in Prometheus codebase to unify labels package and which hopefully will allow/lead to fewer allocs (alloc-free casts etc). See https://github.com/prometheus/prometheus/issues/6029
[12.] Optimise for special case of:  Querier -> 1x Store Gateway -> 1x S3 bucket.  This way we essentially can do straight streaming without having to merge the results, right? [What about dedup?]
[12.1] Taking [12] further and look at this case: Querier -> 1x Store Gateway -> [1x S3 bucket + 1x Prometheus]. Can we assert that for a lot of queries the data will be either in S3 or Prometheus thus allowing optimisation [12]?
[12.2] Can we take this optmisation further and do smarter query partitioning and merging when required?

[999.] Other. Inventive use of OS/k8s/other products which may make queries better.

What about just.. chunk limit? It's similar to 2. but we operate on chunks rather then samples on storeAPI level really.

why does it not stream out the results? is it something to do with deduplication perhaps? aggregation?

Mainly because of SeriesSet API PromQL requiring full series per series iteration. This is the same for StoreAPIs but you can imagine that with fanout, we need all results to merge (essentially MergeSort) them into series per series sorted by labels. Not much we can do here. On top of it deduplication is applied but it works on per series basis so that one is streaming technically.
Also 8. Count allocations per user (:

Count allocations (roughly) per user (:

Are we talking about memory accounting per query here?

Did a super quick trial of [10] with label interning in Querier. Not sure it made much difference with my test data set but but it was a super dirty quick job. Will look into this in more detail and in SG. That may be more promising avenue + see if there are easy wins using [11].

Just to ensure it is transparent. The data we used for your tests @ppanyukov :

Data

synthetically generated (uses thanosbench blockgen)
data size:
- metrics: 200
- targets: 100
- sample interval: 15s
- total time interval: 10 hours

.. had many chunks with *single sample each (due to some misconfiguration in the initial version of thanosbench), so while they can repro OOM they are not really representative. (:
Still, it is extremely useful to see those ideas. I aggregated an umbrella issue for those here: https://github.com/thanos-io/thanos/issues/1705 I copied most of your ideas there! Let's propose ideas one by one in the next issues (:
Closing this in favor of https://github.com/thanos-io/thanos/issues/1705


feat(core): ViewContainerRef.createComponent respects mocks #4742

closes #4742
closes #4693

v14.6.0 has been released and contains a fix for the issue. Feel free to reopen the issue or to submit a new one if you meet any problems.


shows deadname in another language.

I don't know if it's supposed to happen but when I put the deadname in another language it didn't work.

Hi @omeralef, thanks for reporting this!
Could you provide an example of a deadname and name to be replaced? Doesn't have to be your deadname specifically. Is it possible that the deadname has a special character in it that isn't in the Latin alphabet? That would be my first guess as to why the replacement isn't working, and if so, I believe that will be fixed automatically through the next version of the extension.
/remind me to close this issue if no response in 1 week


Multiple system workloads doesn't start on Fedora Nodes

Hi,
I'm trying to create a cluster using mixed Fedora (29) and Ubuntu (18.04) nodes, but am unable to bring up all System workloads on them.
Ubuntu nodes works well through.
Here is a test cluster:

After waiting for a long time, my System workloads are still creating, with no progression (all the spining workloads are on the Fedora Node).

On the Fedora node, docker ps gives the following result
CONTAINER ID        IMAGE                                COMMAND                  CREATED             STATUS              PORTS               NAMES
bd7536d21c40        rancher/hyperkube:v1.13.5-rancher1   "/opt/rke-tools/en..."   19 minutes ago      Up 19 minutes                           kubelet
c7cf08c1bfaa        rancher/hyperkube:v1.13.5-rancher1   "/opt/rke-tools/en..."   19 minutes ago      Up 19 minutes                           kube-proxy
554dee5858f4        rancher/rancher-agent:v2.2.4         "run.sh --server h..."   19 minutes ago      Up 19 minutes                           wizardly_murdock
dea460872711        rancher/rancher-agent:v2.2.4         "run.sh --server h..."   22 minutes ago      Up 22 minutes                           determined_clarke
2950c4307bb5        rancher/rancher-agent:v2.2.4         "run.sh --server h..."   About an hour ago   Up About an hour                        clever_jepsen
b516ca3f98d8        rancher/rancher-agent:v2.2.4         "run.sh --server h..."   About an hour ago   Up About an hour                        trusting_golick
86aec3bb4d13        rancher/rancher-agent:v2.2.4         "run.sh --server h..."   About an hour ago   Up About an hour                        keen_heisenberg
42ed737bc7d5        rancher/rke-tools:v0.1.28            "nginx-proxy CP_HO..."   About an hour ago   Up About an hour                        nginx-proxy
16e37ab98370        rancher/rancher-agent:v2.2.4         "run.sh --server h..."   About an hour ago   Up About an hour                        upbeat_boyd

On the Ubuntu node, docker ps gives me:

CONTAINER ID        IMAGE                                COMMAND                  CREATED             STATUS              PORTS               NAMES
1abb008c9875        f0fad859c909                         "/opt/bin/flanneld -…"   24 minutes ago      Up 24 minutes                           k8s_kube-flannel_canal-zrn2v_kube-system_6ae02d80-98bf-11e9-94ed-fa163e1c8b60_0
442b980a83da        a89b45f36d5e                         "start_runit"            24 minutes ago      Up 24 minutes                           k8s_calico-node_canal-zrn2v_kube-system_6ae02d80-98bf-11e9-94ed-fa163e1c8b60_0
776963e320a0        2817cb463960                         "/bin/sh -c 'kube-ap…"   24 minutes ago      Up 24 minutes                           k8s_kube-api-auth_kube-api-auth-bg78b_cattle-system_6b7ec3a6-98bf-11e9-94ed-fa163e1c8b60_0
a0eb7167989b        2b37f252629b                         "/entrypoint.sh /ngi…"      24 minutes ago      Up 24 minutes                           kubelet
a3003df2bf23        rancher/hyperkube:v1.13.5-rancher1   "/opt/rke-tools/entr…"   24 minutes ago      Up 24 minutes                           kube-proxy
8362a2eecd05        rancher/rke-tools:v0.1.28            "nginx-proxy CP_HOST…"   29 minutes ago      Up 28 minutes                           nginx-proxy

Useful
Info

Versions
Rancher v2.2.4 UI: v2.2.70

Route
containers.index

Well... I created a new Fedora instance, it works on this one... Still not working on the other instance. I'll try to investigate further, and will reopen this issue if needed.


Limit the subquery results by memory usage

Description
Overview
Currently, in the ClientQuerySegmentWalker, when the data sources get inlined, they can be limited by the number of rows to prevent a query (subquery) from hogging up the broker's memory. This however doesn't have a proper correspondence with the memory used, since a row can have multiple columns with varying amounts of data in them. Therefore it would be better if a memory limit is also available, which prevents the subquery's results from exploding beyond a certain memory limit.
This PR aims to use the Frame which was introduced along with the MSQ to store the inline results. Since the Frames are backed by memory, we can fetch the memory used by the frame, and correspondingly the data source to estimate the size that is consumed by the inline data source. This is a close estimate of the size taken by the subquery results.
Configuration

User can set the maxSubqueryBytes key in the query context with the value that is the upper bound on the number of bytes the subquery's results can take.
Often, the result types of the query's results are unknown to the broker. Materializing to frames require that the types be present, which can cause results to not be materialized. Therefore there is an additional undocumented parameter useNestedForUnknownTypeInSubquery parameter that by default is set to false but can be set to true, which controls how to handle the column's whose types are unknown (i.e. the columnType is empty/null). If the parameter is set, then the null types are serded as nested JSON data, which should handle most of the common cases.

Behaviour
As proposed in the review comment if

if maxSubqueryBytes is not set

We use maxSubqueryRows to limit the results of the subquery by the number of rows. Also, we execute the older code path, which doesn't materialize the results to frames

if maxSubqueryBytes is set

if we can materialize the results of all the subqueries to frames

We limit the results of the subquery by the number of bytes

if we can't materialize the results of all the subqueries to frames

We default to the old code path and limit the results of the subquery by the number of rows (if set).

Proposed changes

ClientQuerySegmentWalker#toInlineDataSource now has distinct code paths which convert the results respectively to frames (new changes) or the iterable of rows (old changes)
The query tool chests have been updated to materialize the results as frames if possible.

Supporting changes

FrameBasedIndexedTable - An indexed table that works on the FrameBasedInlineDataSource. It indexes the key columns and provides a way to extract the columnReader for the columns of the data source
FrameBasedInlineDataSource - Inline data source which is based upon an underlying list of frames. Frames can be written using individual row signatures. The datasource itself has its own signature which is "cumulative" signature of the underlying frames.
IterableRowsCursorHelper - Creates a cursor from an iterable representing the rows of a datasource.
ConcatCursor - Cursor representing concatenation of multiple underlying cursors.
ScanResultValue has been updated to provide the row signature whenever present in the underlying datasources.
FrameBasedInlineDataSourceSerializer - Serializes a FrameBasedInlineDataSource as if it was the traditional InlineDataSource. This is done to keep the communication protocol between the broker and the data servers (historicals) unchanged.

Testing

The set of changes has been tested on the existing CalciteQueryTests stack.
The changes have been tested on a local deployment

Impact on existing deployments

This change shouldn't affect the existing deployments. The communication protocol between the broker and the historical is unchanged (the frame-based datasources, inline as existing traditional datasources).
The change to ScanResultValue shouldn't affect the upgrade process since historicals are updated before the brokers in the proper upgrade method, therefore brokers can always assume that the ScanResultValue should contain the updated information (of signature). Broker's currently don't break even if the fetched ScanResultvalue has rowSignature as null, therefore existing deployments shouldn't be affected
The new code paths are only deployed when the memory limit is mentioned by the user in the queries, therefore existing queries should continue to work as is, without any performance impacts.

Follow up

Create an automatic user-friendly configuration based on the guidelines.

Release note
Users can now add a guardrail to prevent subquery’s results from exceeding the set number of bytes by setting druid.server.http.maxSubqueryRows in Broker's config or maxSubqueryRows in the query context. This feature is experimental for now and would default back to row-based limiting in case it fails to get the accurate size of the results consumed by the query.

This PR has:

[x] been self-reviewed.

[x] using the concurrency checklist (Remove this item if the PR doesn't have any relation to concurrency.)

[x] added documentation for new or modified features or behaviors.
[x] a release note entry in the PR description.
[x] added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
[ ] added or updated version, license, or notice information in licenses.yaml
[x] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
[x] added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
[ ] added integration tests.
[x] been tested in a test Druid cluster.

Thanks for the reviews!
Post the latest batch of changes, here are the major things that I have updated:

Create a FrameBasedIndexedTable to create an indexed table based on Frame datasource. This also changed the Frame type to COLUMNAR since to index the table we need to get COLUMNs corresponding to the indexed keys.
Use a memory allocator factory to allow flexibility in converting the inline results to the frames
Added configurations, need to provide the fallback code paths as well.
Refactored the code and stylistic changes

Since this PR is liable to break due to merge conflicts, going ahead and merging this.
@LakshSingla Please address the logging feedback from @abhishekagarwal87 as part of a separate PR.

Thanks for the contribution @LakshSingla !!


Vertical Reduction

Is it possible to add 3x and 4x options to Vertical Reduction?
73 de K4RCG
Bob

It'll require a new settings file (can't use the old one), but it seems to work now!
Version 1.1.2
https://swharden.com/software/FSKview/#latest-version
Let me know what you think!

Hi, Scott and group,
I found a "bad" bug.
In FT8 not the received calls but the called ones are displayed.
The schema is the same as in SSB or CW
Call de Caller, so the second callsign must be displayed
For the selected spots, for example
CT1DHR calls for ZL1BRL. But I only hear CT1DHR, which then has to be labelled.
73 Joe
Von: Scott W Harden<EMAIL_ADDRESS>Gesendet: Donnerstag, 6. August 2020 04:33
An: swharden/FSKview
Cc: Subscribed
Betreff: Re: [swharden/FSKview] Vertical Reduction (#39)
It'll require a new settings file (can't use the old one), but it seems to work now!
Version 1.1.2
https://swharden.com/software/FSKview/#latest-version
Let me know what you think!
https://user-images.githubusercontent.com/4165489/89484030-87eab580-d76b-11ea-8234-97fffe5ff1a3.png image
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub https://github.com/swharden/FSKview/issues/39#issuecomment-669647124 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AENOD2BANLMPULRGUUTHRWTR7II6BANCNFSM4PSQJWXQ .  https://github.com/notifications/beacon/AENOD2BY46REKQPVEAOBL73R7II6BA5CNFSM4PSQJWX2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOE7VACFA.gif
--
Diese E-Mail wurde von AVG auf Viren geprüft.
http://www.avg.com

Guten Morgen Joe,
Good “catch”! Scott is working on this app and we’ll see what it
continues to do.   I really like being able to identify digi mode traces by
callsigns. I think this is a very cool feature in addition to QRSS.
A thought Scott: IF QRSS Tx stations added some sort of digital ID bits
to their signals we could have call their signs superimposed on the QRSS
traces as well. This would be great to identify QRSS signals without
stacking. I wonder if the Reed Solomon ID protocol from other digi modes
could be a clue??  Or maybe just what WSJT USES?? Ha, I’m making this all
too complex!!!  Easy for me to suggest work!!! LOL
73
Bob
On Thu, Aug 6, 2020 at 04:14 df2jp<EMAIL_ADDRESS>wrote:

Hi, Scott and group,
I found a "bad" bug.
In FT8 not the received calls but the called ones are displayed.
The schema is the same as in SSB or CW
Call de Caller, so the second callsign must be displayed
For the selected spots, for example
CT1DHR calls for ZL1BRL. But I only hear CT1DHR, which then has to be
labelled.
73 Joe
Von: Scott W Harden<EMAIL_ADDRESS>Gesendet: Donnerstag, 6. August 2020 04:33
An: swharden/FSKview
Cc: Subscribed
Betreff: Re: [swharden/FSKview] Vertical Reduction (#39)
It'll require a new settings file (can't use the old one), but it seems to
work now!
Version 1.1.2
https://swharden.com/software/FSKview/#latest-version
Let me know what you think!
<
https://user-images.githubusercontent.com/4165489/89484030-87eab580-d76b-11ea-8234-97fffe5ff1a3.png>
image
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub <
https://github.com/swharden/FSKview/issues/39#issuecomment-669647124> ,
or unsubscribe <
https://github.com/notifications/unsubscribe-auth/AENOD2BANLMPULRGUUTHRWTR7II6BANCNFSM4PSQJWXQ>
. <
https://github.com/notifications/beacon/AENOD2BY46REKQPVEAOBL73R7II6BA5CNFSM4PSQJWX2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOE7VACFA.gif

--
Diese E-Mail wurde von AVG auf Viren geprüft.
http://www.avg.com
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/swharden/FSKview/issues/39#issuecomment-669783023,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AQEHCPIY4FDTPZG2V5TM2CLR7JQ6VANCNFSM4PSQJWXQ
.

In FT8 ... CT1DHR calls for ZL1BRL. But I only hear CT1DHR

Thanks for reporting this Joe! Consider this FT8 log line:
191010_124800     7.074 Rx FT8    -14  0.1 2066 KG5JJ W4POT EL99

Currently the first word after the audio offset (2066) is what is displayed in FSKview. It sounds like you'd like all words after the offset to be displayed instead. For this line, that would be KG5JJ W4POT EL99 not just KG5JJ. I will make this change this evening.
Bob I like your idea (having QRSS TX'ers transmit a digital ID), but it seems unlikely to happen as it would mean reprogramming all the QRSS transmitters out there to transmit some new mode. For QRSS I think we're pretty entrenched transmitting only FSK CW, with the occasional animal on the spectrogram here and there =]

I'll leave this issue closed and track progress toward Joe's suggestion in issue #40

Concur Scott!  The random balloons and animals will suffice!  :-)
Bob
On Thu, Aug 6, 2020 at 1:44 PM Scott W Harden<EMAIL_ADDRESS>wrote:

I'll leave this issue closed and track progress toward Joe's suggestion in
issue #40 https://github.com/swharden/FSKview/issues/40
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/swharden/FSKview/issues/39#issuecomment-669935376,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AQEHCPOVN5Y4XJRLR7PS2QDR7KXU5ANCNFSM4PSQJWXQ
.

FYI, Joe, the full message should now be displayed in version 1.1.3
https://swharden.com/software/FSKview/#latest-version
Feel free to post here or in #40 if you have any trouble!
Scott


Mixing beta(n,1) with point mass outside of [0,1] is broken

https://preview.squiggle-language.com/playground#code=eNpNjM0KwjAQhF9lyamFlDYHLz14EZ%2FAY%2BshJVtdSLKSbPAP391iFZzbDPN9T5XPfD2UEGy6q15SQf2Z9o6E029xONviZccOVa8C3aQkrMYISyYUW220qTW0LXCiE0XrwVGWRFMR4qiXE0xcokMHFGHotDmudOQUrK9Mp8FgY7rVYn3gLHBhigLB5gxcJJND4PmfHswXW3rdbFcZPVC93v4cRko%3D

This also can happen when \alpha and \beta are strictly higher than 1:
https://preview.squiggle-language.com/playground#code=eNo1yMEKgzAMANBfCTkpZNB6FDyJX7DjtkNLMhRMC12LjuG%2FW4S94%2FvhZ47bvai69MU%2BpyJ01cRLjuk%2FLG9X1jxGFuyRAwwQYlK3NtYYsu0zsK%2FnJbtrujr1dNnr6t5wIGBP8LBy6wyBfbV4nFSqJ0k%3D


java.sql.SQLException: Method not supported when 'nulls' in resulset

Hi, we has RImpala version 0.1.6 and we receive this error:
Error in .jcall("RJavaTools", "Ljava/lang/Object;", "invokeMethod", cl,  :
  java.sql.SQLException: Method not supported

When try to make, for example, this query with nulls in some rows:
age_group <- rimpala.query(Q="select nom, type from aa_lc_pp.basic")

Simple query working in Hue, but if we not put 'where type is not null' the query show this error.
Any ideas about where we should look to solve this problem?

I cant create other branch/pull request to talk about fixed, i create a fork and modify the code for fix nulls problems in Java, build.sh script for fix routes and delete unnecesary files, you can see it in:
https://github.com/Worvast/RImpala
Regards


Project Working Fine

Your Project is Good But You Need Fix Touch In Another Device

it is an imgui scaling issue when screen game resolution not the same with device resolution, for now there is no fix for it


#131 fixed

This pr fixes issue created in #124 as mentioned in #131

Maybe I made changes following the PR #121. That's why this issue was not appearing for me.
This way of selecting the selected value of the select field is clearly more accurate. I tried to work around with the previously used method but seems to have created an issue.

Nice.


fix(examples): supply chain image boot goes silent for 10 minutes

Describe the bug
The container appears as if it hung because it takes a long time to download the AIO images and it's not saying anything in the logs for that period of time.
To Reproduce
Launch the example container as instructed by the readme and watch how it appears to hang for about 10 minutes while it is actually hard at work setting everything up.
Expected behavior
Logs should show a little more liveness so people don't abandon the example assuming that it's broken.
Logs/Stack traces
N/A
Screenshots
N/A
Cloud provider or hardware configuration:
Dev machine
Operating system name, version, build:
N/A
Hyperledger Cactus release version or commit (git rev-parse --short HEAD):
main @ 0.4.1
Hyperledger Cactus Plugins/Connectors Used
N/A
Additional context
N/A
cc: @takeutak @sfuji822 @hartm @jonathan-m-hamilton @AzaharaC @jordigiam @kikoncuo @jagpreetsinghsasan

I'm not sure this counts as a bug, but it would be a nice QoL fix.  Thanks Peter!

Another thing:  more documentation around what conditions, exactly, the demo needs to work would be great.  This probably isn't the place to say this, but in the long run, a demo that business people could easily run on standard windows corporate laptops would be great.


stdin: is not a tty

I encountered this error during vagrant up "stdin: is not a tty"
I didn't cause any detectable problem but just for the record i'm adding two of the solutions i found :
https://github.com/Varying-Vagrant-Vagrants/VVV/issues/517
https://coderwall.com/p/qtbi5a/prevent-stdin-is-not-a-tty-error-in-vagrant

Since this only shows up on Windows / when Ansible isn't installed, I hadn't seen it in a very long time. It is slightly jarring to see a red message before all the green starts showing up, but since there are a lot of other red messages (warnings, nothing's breaking during the provisioning), I'm not worried about it, and would rather not add any additional shell provisioner just to fix a warning.
Thanks for referencing the upstream changes—the main issue highlights all the details (and why it's not a big deal) here: https://github.com/mitchellh/vagrant/issues/1673


BadRequestExeption returned as 500 error by Nest

Bug Report
500 error from Nest v9.0.0 when using TypedParam.
A similar problem for another library was reported here:
nestjs/nest#5241
Suggestion was:

Shared libraries should list @nestjs/core and @nestjs/common as peerDependencies. Otherwise, httpException instanceof HttpException in which httpException is thrown from shared library will give false value.

Which makes sense, as I noticed that Nestia has these nest libraries listed as dependencies, not as peerDependencies. Do you think the adjustment could be made?

You mean that @nestia/core has to assign same properties from dependencies to peerDependencies, right?
If possible, can you make a demo repo that reproducing the bug?
Also, about the peerDependencies definition, I'll wait your PR.

Upgrade, then be fixed
npm i -D nestia@latest
npx nestia setup


Zend\Db: Allow adding native predicates to sql queries, respecting nextPredicateCombineOrder

Fix for #6800

Merged to develop for release with 2.4.


fix: long version of go dependency unable to inserted into software table (CVE-2020-36569)

Fixes Issue
Fix: long version of go dependency unable to inserted into software table (CVE-2020-36569)
Description of Change
Alter's column size to varchar 75
Please add a description of the proposed change
Alters to a minimum possible value of version columns to 75 characters. Could be more or less as well.
Have test cases been added to cover the new functionality?
no

commit message should start with a lowercase fix otherwise the pipeline breaks @srini00088

@gregoranders thanks, have amended the commit message

Any tentative timelines on when this fix can get released? It's critical.

AFAICS this ain't complete yet. You need to modify dependencycheck.properties as per https://github.com/jeremylong/DependencyCheck/blob/main/core/src/main/resources/dependencycheck.properties#L21-L24.

Wouldn't it be safer to trim any data read from CVE to not risk database length violations? Otherwise any faulty data in CVE entries will make the plugin fail again?

I will update this PR and release an update today.

see https://github.com/jeremylong/DependencyCheck/pull/5229

Could you please upgrade gradle plugin to 7.4.4 as well? Latest is still 7.4.3 https://plugins.gradle.org/plugin/org.owasp.dependencycheck

Could you please upgrade gradle plugin to 7.4.4 as well? Latest is still 7.4.3 https://plugins.gradle.org/plugin/org.owasp.dependencycheck

Looks like it's showing 7.4.4 now.

@jeremylong could you update the jenkins plugin too please?

@jeremylong same here, please update this to jenkins please

Ah, I missed that.  Maybe the plugin didn't need updating.  Excellent.


Format import statement

Description

Improve import statement order

Functional Acceptance Criteria

Application still work as before

Checklist (Not required for Documentation update)

[x] I have performed a self-review of my code.
[ ] I have added tests to cover my changes.
[ ] I have updated the documentation, if applicable.
[x] My code follows the Coding Guidelines.

Type of Change

[ ] Bug fix
[ ] New feature
[ ] Documentation update
[ ] Other (Enchancement)

Wow 3 approve


Fix Flashdata time comparison for PHP 8

In PHP 8, string to number comparison works differently (see https://www.php.net/manual/en/migration80.incompatible.php)
This means that Flashdata now no longer clears after each request, and instead persists indefinitely.  The author of the Session library helpfully left a comment stating that the string 'old' < time() which under PHP 8 is no longer the case.

It looks same as #6007.

It looks same as #6007.

Sorry, didn't notice that.  However that PR seems to address multiple things, and the one for the Flash data check isn't quite correct

Sorry, didn't notice that.  However that PR seems to address multiple things, and the one for the Flash data check isn't quite correct

So, how to fix it ?

Yes, why you fix it?

This is an issue in newer version of php . You can fix it by replacing line 420 in system/libraries/Session/Session.php:
elseif ($value < $current_time)
with
elseif ($value === 'old' || $value < $current_time)


GateFilter bug

Not every Py-ART Radar object features fields with masks. They may just be standard ndarrays. I'm not sure what controls this, but in my experience I've had to design workarounds for when this occurs. Thus, if you request the mask attribute from such a field (such as with a GateFilter), you will get an AttributeError, like so:
Traceback (most recent call last):
File "//anaconda/lib/python2.7/site-packages/artview/plugins/gatefilter.py", line 378, in apply_filters
self.Vradar.value.fields[field]['data'].mask)
AttributeError: 'numpy.ndarray' object has no attribute 'mask'
ARTview needs to be able to account for this, either with better exception handling or by constructing the mask from scratch. Below is an example of how I've worked around this issue in PyTDA. With this function, I am attempting to pull standard ndarrays from Py-ART fields. However, in addition to just pulling the data out of a masked array, I need to deal with occasions when there is no mask, or if there is and there is no _FillValue dictionary item:

def get_sweep_data(radar, field_name, sweep_number):
    # Check if _FillValue exists
    try:
        fill_value = radar.fields[field_name]['_FillValue']
    except KeyError:
        fill_value = BAD_DATA_VAL
    # Check if masked array
    try:
        return radar.fields[field_name]['data'][
            radar.sweep_start_ray_index['data'][sweep_number]:
            radar.sweep_end_ray_index['data'][sweep_number]+1][:].filled(
                fill_value=fill_value)
    except AttributeError:
        return radar.fields[field_name]['data'][
            radar.sweep_start_ray_index['data'][sweep_number]:
            radar.sweep_end_ray_index['data'][sweep_number]+1][:]

The recommended method for accessing the mask of MaskedArrayed (including those where the mask is nomask) which also works with ordinary ndarrays is to use the numpy.ma.getmaskarray function which will return a boolean array of the correct size.
In the same manner numpy.ma.filled can be used to obtain a ndarray from a ndarray or MaskedArray where any masked elements are filled with a given value.
In general using the function in the numpy.ma namespace is a good idea.

I'll submit a PR to fix this bug this evening. Been meaning to get into the ARTView code base and this is a good excuse.

thank you for reporting this bug @tjlang.
@jjhelmus is there a significant overhead of making all pyart arrays masked?

@tjlang thanks for the report. I thought that I tested for this in the original code, but I guess not...
Thanks to @jjhelmus for tackling this.

From what I recall most of the Py-ART's read functions return a MaskedArray for the field data, but there may be a few that return ndarrays.  The one case that comes to mind is when reading Cf/Radial NetCDF files where the underlying variable does not have a missing_value on _FillValue attribute.  That data will likely be returned as an ndarray.
There is no compelling reason not to make all field data returned as masked arrays but I'm not sure if can be guaranteed.  Users can put whatever they please in the 'data' key of a field dictionary but hopefully most of the time they will put either a MaskedArray or a ndarray.   Py-ART tries to cover both of these two cases in the processing and analysis routines which at least in my opinion has not been especially burdensome.


onDefinition handler is not called when onHover handler is

I'm writing a language server. I've got so far setting up symbols, context (range) determination and hovering working. The definition provider is basically also working but there is a strange problem breaking things.
First the relevant code:

onHover: https://github.com/LordOfDragons/vscode-langext-dragonscript/blob/7018805eefff777117219d9c73879b0970d13cb2/server/src/server.ts#L331
onDefinition: https://github.com/LordOfDragons/vscode-langext-dragonscript/blob/7018805eefff777117219d9c73879b0970d13cb2/server/src/server.ts#L378C12-L378C24

And here the screenshot images referenced below:

Image (1) is the test code the language server is applied to.
Image (2) is onHover called for the toInt() function.
Image (3) is onHover called for the ++ operator.
Image (4) is onDefinition called for the toInt() function.
Image (5) is onDefinition not called for the ++ operator.
The bug is thus image (5). For toInt() function both onHover and onDefinition are called but for the ++ operator only onHover is called and onDefinition is never called. The range is correct since onHover is called for ++. Why does the VSCode Node based language server not call onDefinition for some ranges onHover is called for and not others? Is a different logic used for calling onHover and onDefinition?

@jrieken although LSP I think this is the default behavior in VS Code. I assume we only call on Definition only on word and not on ++ operator. Is this correct?

Wes

Hm... that's kinda annoying since overloading operators is a common practise in various languages.

@jrieken  Any plans on changing this in the future?

@LordOfDragons all I can recommend is to open an issue against VS Code since there is little I can do in the library to fix this.

I will close the issue since there is nothing I can do without having this fixed in VS Code.


Fix ODS-2206, ODS-2226 for ODH 2.6

The Pipelines-related UI has changed in the Dashboard.

The PR is incomplete, at the time of posting it fails on
1) Element with locator 'xpath=//tr[td[@data-label="Name"]//*[text()="dsp-ldap-user2"]]/td[@class="pf-c-table__action"]/div/button[@aria-label="Actions"]' not found.

I'm missing some wait, possibly?

@diegolovison I had a discussion about the "placeholder when no pipelines defined test". What the dashboard does (not showing any test) is desired behavior. I am updating test for that.

I thought I'm done fixing the last thing, but actually there is a bit more to the test that I've realized. Fixing more.

I thought I fixed all there is to this, but actually there is more.


Update README.md

Update reproduction.live image to be up to date with how the site looks now. Also, this one is more visually appealing than the last.

@googlebot I signed it!


Create runners for golang

Right now, we have support for running tests written in javascript or typescript frameworks in jest, mocha or jasmine using test-at-scale-js-runners. We can create test-runners for golang as well targeting native golang testing framework and integrate it in test-at-scale docker image

Hey , I'm wiling to work on this issue :D


[Feature Request] 悬浮显示信息

Checklist

[X] I searched exising issues and no one else requests similar feature.
[X] I think that 25%+ users are positive towards this feature.

Describe the feature
todo
Additional context
No response

主要是在输入框文本过长或者显示不了全部内容时出现悬浮显示框

ToolTip 暂不考虑


Helm chart server SSL certificate error https://releases.rancher.com/server-charts/stable

Attempting to download helm charts for rancher from https://releases.rancher.com/server-charts/stable or https://releases.rancher.com/server-charts/latest results in an SSL certificate error:
$ curl -v https://releases.rancher.com/server-charts/stable
...
* Server certificate:
*  subject: CN=*.storage.googleapis.com
*  start date: Oct 18 09:44:18 2021 GMT
*  expire date: Jan 10 09:44:17 2022 GMT
*  subjectAltName does not match releases.rancher.com
* SSL: no alternative certificate subject name matches target host name 'releases.rancher.com'
* Closing connection 0

Also:
$ helm repo add rancher-latest https://releases.rancher.com/server-charts/latest
Error: looks like "https://releases.rancher.com/server-charts/latest" is not a valid chart repository or cannot be reached: Get "https://releases.rancher.com/server-charts/latest/index.yaml": x509: certificate is valid for *.storage.googleapis.com, *.googleapis.com, commondatastorage.googleapis.com, *.commondatastorage.googleapis.com, storage.googleapis.com, storage.mtls.googleapis.com, *.appspot.com.storage.googleapis.com, *.content-storage.googleapis.com, *.content-storage-p2.googleapis.com, *.content-storage-upload.googleapis.com, *.content-storage-download.googleapis.com, *.storage-upload.googleapis.com, *.storage-download.googleapis.com, not releases.rancher.com

Same here

Same here as well

same here

Same for me.

Same here

FYI, the following works as a temporary fix:
helm repo add rancher-latest http://releases.rancher.com/server-charts/latest

this just prevented us from migrating a prod environment

Same error:
helm fetch rancher-latest/rancher
Error: Get "https://releases.rancher.com/server-charts/latest/rancher-2.6.2.tgz": x509: certificate is valid for *.storage.googleapis.com, *.googleapis.com, commondatastorage.googleapis.com, *.commondatastorage.googleapis.com, storage.googleapis.com, storage.mtls.googleapis.com, *.appspot.com.storage.googleapis.com, *.content-storage.googleapis.com, *.content-storage-p2.googleapis.com, *.content-storage-upload.googleapis.com, *.content-storage-download.googleapis.com, *.storage-upload.googleapis.com, *.storage-download.googleapis.com, not releases.rancher.com
helm repo update
Hang tight while we grab the latest from your chart repositories...
...Unable to get an update from the "rancher-latest" chart repository (https://releases.rancher.com/server-charts/latest):
Get "https://releases.rancher.com/server-charts/latest/index.yaml": x509: certificate is valid for *.storage.googleapis.com, *.googleapis.com, commondatastorage.googleapis.com, *.commondatastorage.googleapis.com, storage.googleapis.com, storage.mtls.googleapis.com, *.appspot.com.storage.googleapis.com, *.content-storage.googleapis.com, *.content-storage-p2.googleapis.com, *.content-storage-upload.googleapis.com, *.content-storage-download.googleapis.com, *.storage-upload.googleapis.com, *.storage-download.googleapis.com, not releases.rancher.com

Got the same error on the install-docker scripts.
wget https://releases.rancher.com/install-docker/19.03.9.sh
--2021-11-09 16:18:02--  https://releases.rancher.com/install-docker/19.03.9.sh
SSL_INIT
Loaded CA certificate '/etc/ssl/certs/ca-certificates.crt'
Resolving releases.rancher.com (releases.rancher.com)... <IP_ADDRESS>, 2a00:1450:400a:808::2010
Connecting to releases.rancher.com (releases.rancher.com)|<IP_ADDRESS>|:443... connected.
The certificate's owner does not match hostname ‘releases.rancher.com’

Same issue here:
+ helm_v3 install --namespace cattle-system rancher https://releases.rancher.com/server-charts/latest/rancher-2.6.2.tgz --values /config/values-01_HelmChart.yaml
Error: failed to download "https://releases.rancher.com/server-charts/latest/rancher-2.6.2.tgz" (hint: running `helm repo update` may help)

I think this is a global problem:
> openssl s_client -connect releases.rancher.com:443 </dev/null 2>/dev/null | openssl x509 -noout -text | grep DNS:

DNS:*.storage.googleapis.com, DNS:*.googleapis.com, DNS:commondatastorage.googleapis.com, DNS:*.commondatastorage.googleapis.com, DNS:storage.googleapis.com, DNS:storage.mtls.googleapis.com, DNS:*.appspot.com.storage.googleapis.com, DNS:*.content-storage.googleapis.com, DNS:*.content-storage-p2.googleapis.com, DNS:*.content-storage-upload.googleapis.com, DNS:*.content-storage-download.googleapis.com, DNS:*.storage-upload.googleapis.com, DNS:*.storage-download.googleapis.com

Looks like san is fixed
❯ openssl s_client -connect releases.rancher.com:443 </dev/null | openssl x509 -inform pem -text | grep DNS
depth=2 C = US, O = Amazon, CN = Amazon Root CA 1
verify return:1
depth=1 C = US, O = Amazon, OU = Server CA 1B, CN = Amazon
verify return:1
depth=0 CN = releases.rancher.com
verify return:1
DONE
                DNS:releases.rancher.com, DNS:*.releases.rancher.com

And helm repo add works correctly
❯ helm repo add rancher-stable https://releases.rancher.com/server-charts/stable
"rancher-stable" has been added to your repositories

DNS for rancher.com was moved and CloudFlare was no longer providing the SSL cert/termination in front of the the google bucket.  CloudFront is doing that now.


Change the rule of the Release Sims CI in rcx/ and release/ branch

Summary

Change the rule of the Release Sims CI in rc*/* and release/* branch
Problem Definition

It takes very long time in any PR of release/1 and rc*/*` branch.
We need to reduce the CI checking time.
Proposal

I recommend to change the rule that Release Sims just run only if release tag is added in PR

For Admin Use

[ ] Not duplicate issue
[ ] Appropriate labels applied
[ ] Appropriate contributors tagged
[ ] Contributor assigned/self-assigned

I think it would be good if the release tag is added regardless of a specific branch.


how to infer in python

Hi all!
Before I start I would like to mention that I am very new to Tensorflow, so please pardon me if my question is very basic.
I am trying to infer BERT model after exporting it using the following piece of code:
Export the model
def serving_input_fn():
with tf.variable_scope("sorting_hat"):
feature_spec = {
"input_ids": tf.FixedLenFeature([MAX_SEQ_LENGTH], tf.int64),
"input_mask": tf.FixedLenFeature([MAX_SEQ_LENGTH], tf.int64),
"segment_ids": tf.FixedLenFeature([MAX_SEQ_LENGTH], tf.int64),
"label_ids": tf.FixedLenFeature([], tf.int64),
}
serialized_tf_example = tf.placeholder(dtype=tf.string,
shape=[None],
name='input_example_tensor')
receiver_tensors = {'examples': serialized_tf_example}
features = tf.parse_example(serialized_tf_example, feature_spec)
return tf.estimator.export.ServingInputReceiver(features, receiver_tensors)
EXPORT_DIR = 'gs://{}/export/{}'.format(BUCKET, TASK_VERSION)
estimator._export_to_tpu = False  # this is important
path = estimator.export_savedmodel(EXPORT_DIR, serving_input_fn)
print(path)
I want to know how can the saved model.pb file be used for inference in python code(not using the saved_model_cli )

Does using TensorFlow Serving work for you?

Not exactly, as i have to integrate in an application. So using a TF session and inferring on the graph would make more sense according to my application

In that case, try looking at the run_classifier.py script when do_predict==True.

Hey, in that code estimator is already loaded and is being used directly. For my case, I need to know how to use a graph.pb file and load it as the estimator to perform estimator.predict run_classifier.py

It's been awhile since I last used tensorflow with a Session. I'm afraid I can't help now.

Does using TensorFlow Serving work for you?

how to do for tensorflow-serving in online prediction ? thx

@smartcatdog what do you mean online prediction?

@smartcatdog what do you mean online prediction?

@hsm207 online prediction is the meaning of predicting a request from web in production environment. For example, a text user input is "how do you do" from web, and then return results.

@smartcatdog
In that case, I would suggest you set up tensorflow serving and use it's rest API to send user input to it and to get the corresponding prediction. The rest api for tensorflow serving is documented here.

@smartcatdog
In that case, I would suggest you set up tensorflow serving and use it's rest API to send user input to it and to get the corresponding prediction. The rest api for tensorflow serving is documented here.

@hsm207 yeah, thx ~however, i used session.run() and feed_dict when i got input by tensorflow-serving, it will take me about 300ms, it's too long to meet the online prediction just,,,,

is doing the prediction on a more powerful machine an option?


Sessions do not delete when DELETE /api/v2/sessions

After send DELETE /api/v2/sessions I can use old JWToken for creating new order and other actions.

This one is known issue we need to store JWT until expiration time


Async support

Allow modifyResponse's callback return a promise

@zaaack I'll be realize it in a few days, and if you're in a hurry, create a PR for it.

@zaaack You can wrap a layer of modifyResponse outside the promise

@zaaack I've supported the promise in v0.1.4


Yarn Install command is failing on a new box install

Description of Issue
When tring to install on a new box yarn install fails.
Errors or Screenshots of Issue
`C:\Users\Administrator\Desktop\Servers\Spaghetti-Bot>yarn
yarn install v1.22.5
info No lockfile found.
[1/4] Resolving packages...
warning squad-server > gamedig > cheerio ><EMAIL_ADDRESS>Use cheerio-select instead
warning squad-server > sqlite3 > node-gyp ><EMAIL_ADDRESS>request has been deprecated, see https://github.com/request/request/issues/3142
warning squad-server > tedious > adal-node ><EMAIL_ADDRESS>request has been deprecated, see https://github.com/request/request/issues/3142
warning squad-server > sqlite3 > node-gyp > request ><EMAIL_ADDRESS>this library is no longer supported
warning<EMAIL_ADDRESS>standard 16.0.0 and eslint-config-standard 16.0.0 no longer require the eslint-plugin-standard package. You can remove it from your dependencies with 'npm rm eslint-plugin-standard'. More info here: https://github.com/standard/standard/issues/1316
warning workspace-aggregator-1848a0b2-ca7e-4876-b247-afbacdc3b33b ><EMAIL_ADDRESS>standard 16.0.0 and eslint-config-standard 16.0.0 no longer require the eslint-plugin-standard package. You can remove it from your dependencies with 'npm rm eslint-plugin-standard'. More info here: https://github.com/standard/standard/issues/1316
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
[1/2] ⠁ sqlite3
error C:\Users\Administrator\Desktop\Servers\Spaghetti-Bot\node_modules\sqlite3: Command failed.
Exit code: 1
Command: node-pre-gyp install --fallback-to-build
Arguments:
Directory: C:\Users\Administrator\Desktop\Servers\Spaghetti-Bot\node_modules\sqlite3
Output:
node-pre-gyp info it worked if it ends with ok
node-pre-gyp info using<EMAIL_ADDRESS>node-pre-gyp info using<EMAIL_ADDRESS>| win32 | x64
node-pre-gyp WARN Using request for node-pre-gyp https download
node-pre-gyp info check checked for "C:\Users\Administrator\Desktop\Servers\Spaghetti-Bot\node_modules\sqlite3\lib\binding\napi-v6-win32-x64\node_sqlite3.node" (not found)
node-pre-gyp http GET https://mapbox-node-binary.s3.amazonaws.com/sqlite3/v5.0.1/napi-v6-win32-x64.tar.gz
node-pre-gyp http 403 https://mapbox-node-binary.s3.amazonaws.com/sqlite3/v5.0.1/napi-v6-win32-x64.tar.gz
node-pre-gyp WARN Tried to download(403): https://mapbox-node-binary.s3.amazonaws.com/sqlite3/v5.0.1/napi-v6-win32-x64.tar.gz
node-pre-gyp WARN Pre-built binaries not found for<EMAIL_ADDRESS>and<EMAIL_ADDRESS>(node-v88 ABI, unknown) (falling back to source compile with node-gyp)
node-pre-gyp http 403 status code downloading tarball https://mapbox-node-binary.s3.amazonaws.com/sqlite3/v5.0.1/napi-v6-win32-x64.tar.gz
gyp info it worked if it ends with ok
gyp info using<EMAIL_ADDRESS>gyp info using<EMAIL_ADDRESS>| win32 | x64
gyp info ok
gyp info it worked if it ends with ok
gyp info using<EMAIL_ADDRESS>gyp info using<EMAIL_ADDRESS>| win32 | x64
gyp ERR! configure error
gyp ERR! stack Error: Command failed: C:\Python39\python.EXE -c import sys; print "%s.%s.%s" % sys.version_info[:3];
gyp ERR! stack   File "", line 1
gyp ERR! stack     import sys; print "%s.%s.%s" % sys.version_info[:3];
gyp ERR! stack                       ^
gyp ERR! stack SyntaxError: invalid syntax
gyp ERR! stack
gyp ERR! stack     at ChildProcess.exithandler (node:child_process:333:12)
gyp ERR! stack     at ChildProcess.emit (node:events:376:20)
gyp ERR! stack     at maybeClose (node:internal/child_process:1063:16)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (node:internal/child_process:295:5)
gyp ERR! System Windows_NT 10.0.14393
gyp ERR! command "C:\Program Files\nodejs\node.exe" "C:\Users\Administrator\Desktop\Servers\Spaghetti-Bot\node_modules\node-gyp\bin\node-gyp.js" "configure" "--fallback-to-build" "--module=C:\Users\Administrator\Desktop\Servers\Spaghetti-Bot\node_modules\sqlite3\lib\binding\napi-v6-win32-x64\node_sqlite3.node" "--module_name=node_sqlite3" "--module_path=C:\Users\Administrator\Desktop\Servers\Spaghetti-Bot\node_modules\sqlite3\lib\binding\napi-v6-win32-x64" "--napi_version=7" "--node_abi_napi=napi" "--napi_build_version=6" "--node_napi_label=napi-v6"
gyp ERR! cwd C:\Users\Administrator\Desktop\Servers\Spaghetti-Bot\node_modules\sqlite3
gyp ERR! node -v v15.5.1
gyp ERR! node-gyp -v v3.8.0
gyp ERR! not ok
node-pre-gyp ERR! build error
node-pre-gyp ERR! stack Error: Failed to execute 'C:\Program Files\nodejs\node.exe C:\Users\Administrator\Desktop\Servers\Spaghetti-Bot\node_modules\node-gyp\bin\node-gyp.js configure --fallback-to-build --module=C:\Users\Administrator\Desktop\Servers\Spaghetti-Bot\node_modules\sqlite3\lib\binding\napi-v6-win32-x64\node_sqlite3.node --module_name=node_sqlite3 --module_path=C:\Users\Administrator\Desktop\Servers\Spaghetti-Bot\node_modules\sqlite3\lib\binding\napi-v6-win32-x64 --napi_version=7 --node_abi_napi=napi --napi_build_version=6 --node_napi_label=napi-v6' (1)
node-pre-gyp ERR! stack     at ChildProcess. (C:\Users\Administrator\Desktop\Servers\Spaghetti-Bot\node_modules\node-pre-gyp\lib\util\compile.js:83:29)
node-pre-gyp ERR! stack     at ChildProcess.emit (node:events:376:20)
node-pre-gyp ERR! stack     at maybeClose (node:internal/child_process:1063:16)
node-pre-gyp ERR! stack     at Process.ChildProcess._handle.onexit (node:internal/child_process:295:5)
node-pre-gyp ERR! System Windows_NT 10.0.14393
node-pre-gyp ERR! command "C:\Program Files\nodejs\node.exe" "C:\Users\Administrator\Desktop\Servers\Spaghetti-Bot\node_modules\node-pre-gyp\bin\node-pre-gyp" "install" "--fallback-to-build"
node-pre-gyp ERR! cwd C:\Users\Administrator\Desktop\Servers\Spaghetti-Bot\node_modules\sqlite3
node-pre-gyp ERR! node -v v15.5.1
node-pre-gyp ERR! node-pre-gyp -v v0.11.0
node-pre-gyp ERR! not ok
Failed to execute 'C:\Program Files\nodejs\node.exe C:\Users\Administrator\Desktop\Servers\Spaghetti-Bot\node_modules\node-gyp\bin\node-gyp.js configure --fallback-to-build --module=C:\Users\Administrator\Desktop\Servers\Spaghetti-Bot\node_modules\sqlite3\lib\binding\napi-v6-win32-x64\node_sqlite3.node --module_name=node_sqlite3 --module_path=C:\Users\Administrator\Desktop\Servers\Spaghetti-Bot\node_modules\sqlite3\lib\binding\napi-v6-win32-x64 --napi_version=7 --node_abi_napi=napi --napi_build_version=6 --node_napi_label=napi-v6' (1)
`
Squad Information

SquadJS Version: beta branch current
Squad Version: current

System Information

Operating System: 10 server 2016
NodeJS Version: current
Yarn Version: currnet

I'd try some of the suggestions listed in here: https://github.com/mapbox/node-sqlite3/issues/612

I'd try some of the suggestions listed in here: https://github.com/mapbox/node-sqlite3/issues/612

i couldnt get it to work. yarn doesnt work all the way. No lock file but it does spin up. I think its an issue with the server its currently on.

i couldnt get it to work. yarn doesnt work all the way. No lock file but it does spin up. I think its an issue with the server its currently on.

rn I have a working version of the node and yarn.lock if I have to ill use that.

rn I have a working version of the node and yarn.lock if I have to ill use that.


[5.6] Use binary format for UUIDs in MySQL grammar

It's faster and takes up less space: http://mysqlserverteam.com/storing-uuid-values-in-mysql-tables/

I'll link it with internal issue: https://github.com/laravel/internals/issues/350

Same changes should be done for SQLite, it will be better to make as separate PR?

@a-komarev: Separate PR IMO.

How do you address this concern?

@Dylan-DPC I see here 2 soluitions.

Make changes without backward compatibility and write about it in changelog. Additionally make method stringUuid() which will work as old method uuid().
Name method binaryUuid() to not introduce backward compatibility.

@Dylan-DPC write in the upgrade guide that people who don't want new column type should change their UUID definitions to $table->char(36)?

@taylorotwell My thoughts as well.

It's faster and takes up less space: http://mysqlserverteam.com/storing-uuid-values-in-mysql-tables/

It is smaller, but it is not faster. The blog itself shows how the speed increase is due to the shuffling of the guid bytes to make the sequential.
Persisting uuids as binary is okay, but changing the existing $table->uuid() and ignoring backward compatibility? Seriously? Just call it uuidAsBinary() and avoid all the conflicts. There's nothing that requires breaking the existing code. Is there any arguments at all for breaking existing code?
Also, see the discussions in https://github.com/laravel/internals/issues/645

@sisve maybe because this method on other engines not using strings?

Those other engines has an actual real proper uuid data type that accepts strings, and is displayed as a string. Mysql does not have an uuid data type. And this change doesn't take us closer to a proper data type, it just changes how they are stored, and makes it much harder to work with them out-of-the-box.
This change provides nothing to the developers to use this binary format. Not only does everyone have to change their existing migrations if they want to keep the old behavior, they would also have to change all their code to use binary primary keys if they wanted to use these new things.
How does binary values work in the model binding?
Will this require a new cast in Eloquent to parse these?

If we will implement new method uuidAsBinary how it will be handled by PostgreSQL? As uuid working right now or it should produce binary column too?

Yes, all engines would implement it as a binary column because that is what you've asked the migration system to.
The important part is that the models should work similar; and doing something like User::find($id) should be possible no matter what data type $id is. Doing this change, as current implemented, means that you would send in a string to a binary column and not getting a match. You would have to convert $id to binary, or use database functions, and thus break how the code should look on engines that supports uuids.
This PR is incomplete no matter what we call the method. We need to cast the readable uuids to binary somewhere so that User::find($id) works. And no, I will not accept that the PR title is "change this line in one file", this is about seeing the full picture and provide the users what is needed to actually use these new binary uuids in some portable way.

But there is no point to store UUID in engines which has special type optimized for it. As far as I know only MySQL & SQLite don't have such data types.
Maybe it will be better to name it more abstract: $table->optimizedUuid()? Then we wouldn't bind to binary format and MySQL + SQLite will use binary type and this wouldn't affect on engines which has native UUID type implementation (they will call $this->uuid() under the hood?

@a-komarev You need to read https://github.com/laravel/internals/issues/645. There's nothing optimized about this specific change, except storage size. When people talk about "optimized uuids" they talk about sequential ids and how they are stored internally in mysql. So the only thing related to the migration system would be a "store this uuid as binary", which binaryUuid() or uuidAsString() tells.
Regarding your point about specialized types; keep in mind that the current solution means that we can do things like ...
SELECT * FROM users WHERE id = 'cfb8dc8f-cd76-4dd1-88a1-28edbd2ff9a8';

$user = Users::find('cfb8dc8f-cd76-4dd1-88a1-28edbd2ff9a8');

This syntax works with strings and it works with uuid data types. Every database engine that has support for a uuid data type supports automatic parsing/casting/whatnot from strings to uuids. So it makes sense for $this->uuid('id') to either produce a uuid column, or something that works as close to it as possible. Just as $this->json('col') falls back to a string field, because it is the closest match for the way you store and fetch the values.
Imagine changing this fields to a gzipped binary for SQL Server just because SQL Server doesn't have support for json fields natively...

@sisve but what about joining and comparisons which are not spending resources on encode UUID string representation to binary?
select *
  from `table1`
  join `table2`
    on `table1`.`uuid` = `table2`.`uuid`

I've worked on a PHP project where all entities had UUID strings as primary keys. All this stuff were stored in MySQL. It was working great until records count reached millions. Then system start dying from this queries.
Isn't it an optimization?

@a-komarev It's a space optimization. I bet you were storing your uuids with the utf8mb4 collation too, instead of ascii. Using utf8mb4 would take up 4x more space than ascii. I am simplifying here since it depends on many things. But there's a relative higher space saving going from utf8mb4 to ascii (-75% change), than going from ascii to binary (-50% change).
They way you've presented your problem makes it hard to tell what actually went wrong. Debugging databases requires much more than just "started dying" and "worked better".

Did you only change to binary? Then it was probably a caching issue where you started hitting the database more. What MERGE_THRESHOLD are you using?
Did you also change to sequential/"optimized" uuids? That would have changed the locality of the data which would be relevant if these queries focused mostly on recent data, like showing activity feeds.

It's hard to say what made it better in your case without details.
What we have so far are links to blogs that talks about storage improvements, both data size and index sizes. A look at https://www.percona.com/blog/2014/12/19/store-uuid-optimized-way/ shows that they are all about changing around the binary representation of the uuids so they become sequential, and the graph shows the performance difference of these sequential uuids compared to the "normal" binary ones. It's hard to use that link as a argument in this PR since this PR doesn't in anyway attempt to introduce sequential uuids.
If we focus on http://mysqlserverteam.com/storing-uuid-values-in-mysql-tables/ we find another extremely important part;

So, in a sequence of UUIDs continuously generated by a single machine, all UUIDs have different first bytes. Inserting this sequence into an indexed column (in binary or text  form) will thus modify a different index page each time, preventing in-memory caching. So it makes sense to re-arrange the UUID, making the rapidly-changing parts go last, before we store into id_bin. Again, note that this idea applies only to UUIDs of version 1.

They are all about sequential ids too. So it is a grave mispresentation when this PR uses that blog post as something that would back up these changes. This PR does not implement the stuff that the blog post focuses on, at all.
Basically, all this PR does is breaking a migration type that has worked for two years. In fact, it was even known when it was implemented in https://github.com/laravel/framework/pull/10674 that using a char(36) wasn't the most space efficient solution, but it was merged anyway. And it's important that the migration system is stable since it has lots of code and database structures that lives far longer than any Laravel release. It's way easier to add new features to it, than change existing, if we want to keep backward compatibility.
We can absolutely try and implement the entire sequential uuid solution discussed in https://github.com/laravel/internals/issues/645. But it would make sense to first close this PR and then move the discussion of implementation details over to the internals issue, and then get back with a complete PR.

If we're only interested in look-up speed we can already generate time-ordered UUIDs using Ramsey's UUID library which is already used elsewhere in the framework. Those will provide much more efficient look-ups. I would even be fine with making it easier to use such IDs in Eloquent with a simple opt-in configuration item or something. I think I will hold off on this change though since it's mainly focused on storage size and would require other changes to make it user friendly IMO.


Issue

Hi,
I'm trying to run the code on a ESP32S3, I hade issues with the BLE scanner finding advertiser before and solved by upgrading the library dependecies to ^1.4.0:
<EMAIL_ADDRESS>Now I get the advertisers address reported, but after the address of the nuki lock I see "No service data found":
I NimBLEScan: Updated advertiser: 54:d2:72:3f:4a:f5
I NimBLEAdvertisedDevice: No service data found
Basically not finding the advertisment for the pairing service.
Is there something special to be done for ESP32S3 (BLE 5.0) to correctly operate?

Problem is fixed by upgrading dependencies in platformio to ^1.0.0


remove Instrument and InstrumentValue comments

#1443

this will be unnecessary when the following is merged:

#1468


Add conj, real, transpose, adjoint on numbers

Fix #73
Needs tests

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 70.43%. Comparing base (8b85f1a) to head (1877394).

Additional details and impacted files
@@           Coverage Diff           @@
##             main      #75   +/-   ##
=======================================
  Coverage   70.43%   70.43%
=======================================
  Files          14       14
  Lines         592      592
=======================================
  Hits          417      417
  Misses        175      175

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

Is this now tested with #76?

Feel free to merge a minor PR like this one on your own when it's ready.

The operators themselves will be tested but not the dot product which was our aim. I'll add something to that effect


Fresh install fails

I just cloned a fresh install and I get the following when I do make install
osboxes@osboxes:~/permacoop$ sudo make install
cp server/ormconfig.json.dist server/ormconfig.json
cp server/.env.dist server/.env
cp client/config.js.dist client/config.js
docker run -it --rm -v /server:/app -w /app node npm i
npm WARN saveError ENOENT: no such file or directory, open '/app/package.json'
npm WARN enoent ENOENT: no such file or directory, open '/app/package.json'
npm WARN app No description
npm WARN app No repository field.
npm WARN app No README data
npm WARN app No license field.
up to date in 0.421s
found 0 vulnerabilities
docker run -it --rm -v /client:/app -w /app node npm i
npm WARN saveError ENOENT: no such file or directory, open '/app/package.json'
npm WARN enoent ENOENT: no such file or directory, open '/app/package.json'
npm WARN app No description
npm WARN app No repository field.
npm WARN app No README data
npm WARN app No license field.
up to date in 0.391s
found 0 vulnerabilities
make start
make[1]: Entering directory '/home/osboxes/permacoop'
docker-compose -p permacoop up -d
Creating permacoop_database_1 ... done
Creating permacoop_api_1      ... done
Creating permacoop_client_1   ... done
make[1]: Leaving directory '/home/osboxes/permacoop'
make api-build-dist
make[1]: Entering directory '/home/osboxes/permacoop'
docker-compose -p permacoop exec api npm run build
ERROR: No container found for api_1
Makefile:36: recipe for target 'api-build-dist' failed
make[1]: *** [api-build-dist] Error 1
make[1]: Leaving directory '/home/osboxes/permacoop'
Makefile:10: recipe for target 'install' failed
make: *** [install] Error 2
docker ps shows the database ok but the client and the api containers exit immediately. - Cheers, Dave

Hi,
I'm going to take a look at this, thank you for the feedback.

Hi @gridworkz,
Can you try again?
make stop
make rm
docker volume prune (ro remove database volume)
make install

I have added a proxy, so you can access to the api with the url http://localhost/api and the client  with http://localhost.
Let me know if you are still having problems.

Hi Mathieu:
I got the client/server started: Couple things to note -

change host to localhost in ./server/ormconfig.json (connect to Pg)
change host to localhost in ./client/config.js (connect to Redis)

So after a fresh install I still don't see<EMAIL_ADDRESS>in the database. I
can see the queries being executed and migrated OK during the server
startup. If I exec into the db docker - 'select * from user' just returns
docker and 'select * from user_administravie' is empty. I tried to login
with<EMAIL_ADDRESS>/ john and still no luck. I'm still a little uncertain
where the api token comes in during login.
Cheers,
Dave
On Sun, Oct 4, 2020 at 4:26 AM Mathieu MARCHOIS<EMAIL_ADDRESS>wrote:

Hi @gridworkz https://github.com/gridworkz,
Can you try again?
make stop
make rm
docker volume prune (to remove database volume)
make install
I have added a proxy, so you can access to the api with the url
http://localhost/api and the client with http://localhost.
Let me know if you are still having problems.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/fairnesscoop/permacoop/issues/131#issuecomment-703221345,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AEVXNSGSBVMPW6WKEA7F6YLSJAWRVANCNFSM4R6SC4VA
.


Irritating "Image dimension" log message

I am processing a 4000x6000 image using ocrd-eynollah-segment and get - among other messages - this message:
14:32:10.541 INFO eynollah - Image dimensions: 448x672

Should this read "Patch dimensions" and maybe get a log level of DEBUG?

👀 @vahidrezanezhad

maybe get a log level of DEBUG?

@vahidrezanezhad knows best about the contents but I converted all the print logging to log.INFO, so it's still visible in a default logging setup. These could certainly be more expressive and/or just be DEBUG stmts.

@vahidrezanezhad Can you please clarify/confirm what is the exact meaning of the log message here, i.e. whether  Image dimensions: 448x672 does in fact refer to the size of the patches?

@cneud Yes. They are actually the patch size (input size) of each model.

Thanks. So let's change the log message accordingly for clarity then, ok?

sure

It should still be DEBUG not INFO though.

I've created PR #71 to downgrade the log message to loglevel DEBUG.


Documentation: How to update the installed PHP-Dynamic-Qr-code?

How to update the installed PHP-Dynamic-Qr-code? I did not used docker while installing, but a section explaining the update process would be great in documentation. Will just a "git pull" gonna work?

You can see the standard installation here: https://giandonatoinverso.it/documentation/#installation

Hi, @giandonatoinverso. There is a section named "INSTALLATION" as you referred. But I mean, after I installed with standard installation and there is a new version, it is not clear how to update to the new PHP-Dynamic-Qr-code version. Maybe in the installation document, an "UPDATING" section could be added. Or a fifth step could be added to the "INSTALLATION" section like this:
INSTALLATION
Follow these steps to install the script:
1. Clone the project from GitHub
2. Edit the yaml file for docker ...
4. Finish
5. Updating the installation
   To update the installation just do a "git pull" ...

Maybe as a user, I couldn't understand how to update. Anyway, PHP-Dynamic-Qr-code is amazing. Many thanks for your hard work.

At the moment git pull is sufficient as a future version will be backwards compatible with the QR codes already created


compose up build fail for permission denied

Hi
when trying to follow the readme and run docker-compose up --build
i get permission denied on the postgress_data folder
tried changing the owner to be my user - but then subdirectories have the same error.
with which user one should run docker-compose ?
thanks
Shlomi

I have the same problem. Did somebody find a solution? I added my user to a docker group.

I have the same problem. Did somebody find a solution? I added my user to a docker group.

Found the answer. If you want to run it a second time you have to clean progres as desccribed in the makefile: rm -rf postgres_data


[PDI-12717] - Race condition in database connection creation

make connections' map a CHM instance
eagerly initialize singleton
improve concurrency correctness by marking counters with 'volatile'
add an integration test

@mattyb149, @brosander, review it please

@mattyb149, @brosander, review it please

@mattyb149, @brosander, @deinspanjer, review it please

@mattyb149, @brosander, @deinspanjer, review it please

@akhayrutdinov please fix checkstyle violations and ensure unit tests are passing

@brosander, I've done both. Tests passed and checkstyle had no violations against classes I had changed


Switch Window command doest not respect Snap windows

VSCode Version: 1.31 Stable and 1.32 Insiders
OS Version: Windows 10

Steps to Reproduce:

Open a VSCode instance with any folder/workspace
Snap this window to right (Win + Right)
Open another VSCode instance with any other folder/workspace
Snap this window to left (Win + Left)
Now you have two VSCode instances side by side
In any of those windows, execute the Switch Window command and choose the other window
The other window looses its Snapped state, being restored to a normal state. It should not restore the window, but simply focus the other window, just like Alt + Space does

Does this issue occur when all extensions are disabled?: Yes

Different issue from the bot link, but possibly the same or a similar root cause.

@sbatten the switch window command unsnaps the window, might be something to do with the title bar?


Capture the Spark

Secure the Enemy's Spark and bring it back to your spawn.
Holders personal knockback is decreased, Team Launches are incentivized
(Personally, could do without, but this could be interesting)

What about naming it Capture the Jug


Cronjob

Issue
Set up Celery Beat or at least a cronjob for now to capture data.
Notes
Keep an eye on cost.

cronjob running locally ✅


Clarify intent of providing a configuration as a dependency

The behavior of declaring a Configuration as a dependency using
the dependency handler DSL was confusing. Using a direct notation,
this is equivalent to calling extendsFrom: it doesn't effectively
add a new dependency to the graph.
However, since we added a Provider taking dependency notation,
it was possible to supply a Configuration in the provider, in
which case it would be resolved and the files would be added
as dependencies.
This behavior is not intended and certainly confusing. This commit
deprecates the DSL which allows declaring configurations as dependencies,
and makes the build fail if a Configuration is used in a Provider.
Future Gradle releases may include different DSL methods to support:

adding a list of dependencies lazily (using a ListProvider)
adding a file collection lazily

Closes #13977

@bot-gradle test this

OK, I've already triggered ReadyForMerge build for you.

@bot-gradle test this

OK, I've already triggered ReadyForMerge build for you.

@bot-gradle test this

OK, I've already triggered ReadyForMerge build for you.

@bot-gradle test RFM

OK, I've already triggered ReadyForMerge build for you.

@bot-gradle test this

OK, I've already triggered ReadyForMerge build for you.

@bot-gradle test this

OK, I've already cancelled the old build and triggered a new ReadyForMerge build for you.

Posted this on StackOverflow today regarding using configuration as a dependency so I could control ordering. Pointers if there is some better way to do that using extendsFrom or otherwise would be appreciated


The LargeScreen and SmallScreen widgets are missed

The LargeScreen and SmallScreen widgets mentioned in the article are missed.

They are inside the Body, my fault...


Don't wrap StubPropertySource

Hi, I've been analyzing an issue in my project (https://github.com/gavlyukovskiy/spring-boot-data-source-decorator/issues/55) that was reproducing only when jasypt starter is added. The issue was that I have ConfigurationProperties that references class, which was not available at runtime. I've spent some time to find the reason why and here what I found:

As a performance optimization when Spring Boot binds ConfigurationProperties before attempting to bind a field it checks if property source contains the property itself or any descendants of the property. If there is no properties of specific prefix Spring Boot shortcuts to resolving whole field as null without trying to recursively resolve object tree.
This check goes into ConfigurationPropertySource#containsDescendantOf that depends on particular implementation of PropertySource - that either wraps a Map or is a EnumerablePropertySource
While iterating over property sources Spring Boot ignores StubPropertySource - the property source that does not have any properties and is a placeholder for properties that might be resolved in the future.
When Jasypt wraps all property sources it also wraps StubPropertySource into EncryptablePropertySourceWrapper so that this property source is neither ignored nor enumerable, which leads to Spring Boot to always resolve recursively all ConfigurationProperties classes.

While the problem was 100% on my side for having class that is not available at runtime in properties, I think it makes sense to not wrap StubPropertySource and let Spring Boot optimize this use case. It should be safe as it doesn't have any properties to be encrypted and in case it's replaced with other property source during context refresh - that property source will be wrapped by your bean post processor.
I have tried locally to apply the change in EncryptablePropertySourceConverter:43 (or use jasypt.encryptor.skip-property-sources=org.springframework.core.env.PropertySource$StubPropertySource) and I saw small improvement in startup time - from 7.3 - 7.4s to 7.1 - 7.2s. Obviously is not a reliable benchmark :)

Ok, thanks for the heads up. I'll make the change

@ulisesbocchio I have opened PR with a fix - #265, not sure if there more places with such check, but this one popped up in my debugging session.

3.0.4 released


Spinner animation for remote data loading is not smooth

Describe the bug
When remote data is used, the animation for the spinner keeps restarting when writing the search term. I believe it is because the spinner is reinserted with each keystroke.
To Reproduce
Create an example on JSFiddle, CodePen or similar service and outline the steps for reproducing the bug.

Go to https://tom-select.js.org/examples/remote/
Open the select and start writing
See error as the animation keeps restarting.

Expected behavior
Once the spinner is inserted, the animation should smoothly run until the spinner is removed.
Additional context
Add any other context about the problem here.

OS: Windows
Browser Chrome, FireFox, probably all the browsers

<!--
Thanks for taking the time to submit your bug report,
and don't forget to star our repo!

Still an issue 🤷♂

Still an issue 🙁

Still an issue 🎉


Remove huge deleted files (demo.gif) from Git's history

The size of the repository is huge: 140 MB. Most of it is due to a few demo.gif files that have been deleted but still are in Git's history:
Commands to find them:
git verify-pack -v .git/objects/pack/pack-*.idx | sort -k 3 -n | tail -10

git rev-list --all --objects | grep <object-sha1>

Commands to remove them safely:
git filter-repo --invert-paths --path images/demo.gif --path demo.gif --path client/public/demo.gif --path public/demo.gif

(From https://github.com/newren/git-filter-repo/)
Down to 7.6 MB!

Thanks for bringing this to my attention.
I can confirm it reduces the size locally, and it did seem to rewrite history when i tried a force push. however, when i try to clone the repo again, it's still at 140 mb, any idea why? the push did remove a commit as far as i could tell

Ran this twice, on second time, with local file size removed again and history rewrite, the push said all up to date.

is it because the other branches have it in their git history maybe? hmm

No worries, I'll try again once the other branches are merged

After trying this again with no other branches in the repo, I believe this is something on github's end needing to garbage collect on the repo, which is done every few weeks. I'll check back in a month.. When I clone down the repo, I can view the commit with git rev-list --all --objects | grep demo.gif but it is not existent on github

@vibl I just wanted to let you know I finally got this fixed. I talked to github support to reset the cache, and after they did so, I realized the problem was the image still existing in the tags. The repo will clone down at around 8 MB now!

Great :-) Thank you for your work!


fix(ingest/json-schema): convert non-string enums to strings

Prior to this change, the schema enums were processed without being converted to their string representations. This caused an ingestion failure for schemas with non-string enum elements.
To address this, the code has been refactored to ensure that all elements in the schema's enum list are now converted
to their string representations using the repr() function.
This change adds quotation marks around string elements, which clearly delineates strings from other special values like integers or None types. (similar to the json-schema-for-humans library: 1, 2)
Checklist

[ ] The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
[ ] Links to related issues (if applicable)
[ ] Tests for the changes have been added/updated (if applicable)
[ ] Docs related to the changes have been added/updated (if applicable). If a new feature has been added a Usage Guide has been added for the same.
[ ] For any breaking change/potential downtime/deprecation/big changes an entry has been made in Updating DataHub

Related issue: https://github.com/datahub-project/datahub/issues/8480

Thanks for the contribution!
Could you add a test case for this here?
https://github.com/datahub-project/datahub/blob/master/metadata-ingestion/tests/unit/schema/test_json_schema_util.py

Could you add a test case for this here?

Hey @shirshanka, I've added a test -- let me know if there's anything else I'm missing.


Allow collapse / hiding  of response models and move below the request

Usability Request.
In the swagger UI for a controller, each operation contains 3 sub-headers; Implementation Details, Response Class, Parameters. I think the logical order is to show the response after parameters (and the Try It button). Also, response schemas can get quite long, which makes for a lot of scrolling. Can the display of models and model schema be collapsed or hidden by default?

I think this issue belongs in the Swagger UI repo: https://github.com/swagger-api/swagger-ui
(Swashbuckle creates the Swagger definition that is used by Swagger UI. It does not alter the looks of the UI.)

Agreed - this issue belongs in the swagger-ui repo.


fix(CLI): contract id parsing

What
Allow parsing Contract Addresses
Why
Part of the move from Bytes<32> to Address anywhere a contract id is used.
Known limitations
[TODO or N/A]

Part of https://github.com/stellar/soroban-tools/issues/403


PlayerUnitEvents issue

Hi,
I am trying to update my map to use 3.0 version and i have encountered issue with PlayerUnitEvents.
When trying to register UnitTypeEvent without unit id, it fails
For example: PlayerUnitEvents.Register(UnitTypeEvent.IsAttacked, () => Console.WriteLine("Attacked"));
after adding try catch, it is following error: (PlayerUnitEvents.EnableDebug(); does not catch this error)
"Unable to cast event filter to required type System.Int32 (Parameter 'filterObj')"

EnableDebug just makes it add try/catch blocks when the entry point is WCSharp (i.e., an event managed by WCSharp goes off, then WCSharp calls your code). If the entry point is your code (i.e. an event managed by you goes off, then you call WCSharp), then there is no try/catch block, since that's your job. So EnableDebug not handling this is expected and intended.
Anyways, I've fixed this issue in v3.0.1 https://github.com/Orden4/WCSharp/commit/8509e0052d7dd4edbdd535a2ed3fb01420312b39


pywinauto can't get items from MSFLexGrid

Hi
I've tried getting the info inside this table "MSFlexGrid" to copy paste in an excel
The win32 backend show the class and title in "print_control_identifiers) as:
MSFlexGrid - ''    (L193, T350, R1170, B703)
| ['MSFlexGrid', 'Consulta LotesMSFlexGrid']
| child_window(class_name="MSFlexGridWndClass")
I can use "click_input" on it
I followed and quick fix adding like a listview wrapper in my common_controls.py , with this class name:
friendlyclassname = "MSFlexGrid"
windowclasses = [
".MSFlexGrid.",
"MSFlexGridWndClass.*",
]
With this I'm supposed to be able to use the listview wrapper to get info from my "MSFlexGrid"
But the method : item_count, data, cols ...  all shows as output:    [] or 0  column,  0 items
I've read that it could happens because it's not an unicode app (i don't know if this refers to my python script running pywinauto or the ERP written in VC6 that I'm trying to get the data)
I'm stucked in this step!
I've changed to "uia" backend
it can detect the "MSFlexGrid"  control, just the "scrollbars" of the "Pane"
but if I use the command : ctrl3 = dlg4.child_window(class_name="MSFlexGridWndClass").click_input()
it can click inside the Grid as well
My ERP is in portuguese Brazil (PT_BR) language and I've faced other problems with this unicode characters
I've used the command   is_unicode  (similar to this) and it returns : False
I'm trying to get info inside this control!
I've searched a lot and  nothing worked!

Hi @EduardoP777 is it WinForms application? To check this, you can run Inspect.exe, switch it to "UI Automation" mode instead of "MSAA" and look for FrameworkId property.
Inspect.exe is part of Windows SDK.

Hi Vasily thanks for your help
it's
FrameworkId:	"Win32"
ClassName:	"MSFlexGridWndClass"
Inside  win32 backend using my_wrapper(it's a listview clone wrapper in common controls.py) I get item count, col count, data as 0, [ ] , empty, but it shows object handle.
Waiting for your feedback to proceed...

Well, backend="uia" seems useless here. The only working way I could find is using DLL injection approach:
https://social.msdn.microsoft.com/Forums/en-US/e0406017-7473-4267-9702-94dd96430845/msflexgrid-uiautomation-c-api?forum=windowsaccessibilityandautomation
It is technically doable using module pywinauto/injected which is not part of pywinauto currently. We plan to make it separate Python package with wheels on PyPI. But it will take unpredictable time.

I see MSFlexGrid is a IDispatch COM component. And this code works for me:
import win32com.client as com
grid = com.Dispatch("MSFlexGridLib.MSFlexGrid")
print(grid.hWnd) # prints 0

I would suggest to try grid.hWnd = your_wrapper.handle and play with other properties and methods of grid object. If it magically helps, it could be implemented as MSFlexGridWrapper for backend="win32" without any "uia" things.

No, setting grid.hWnd seems wrong way. We need to figure out how to connect to existing instance of MSFlexGrid.
Something like AtlAxGetDispatch(hContainer) from here: https://cplusplus.com/forum/windows/46242/ but I couldn't quickly find a Python analogue of AtlAxGetDispatch, maybe com.Dispatch(...) additional params would be enough.

It also depends if MSFlexGrid is implemented as in-proc or out-of-proc COM server on the application side. For the out-of-proc (out-of-process) server it should be easy to automate it from separate Python process like pywinauto script. Otherwise it may require DLL injection which is more complicated.

I've tried the way similar to Excel.Application object: http://timgolden.me.uk/python/win32_how_do_i/attach-to-a-com-instance.html
But it fails:
grid = com.GetObject(Class="MSFlexGridLib.MSFlexGrid")
Traceback (most recent call last):
  File "<interactive input>", line 1, in <module>
  File "C:\Python27\lib\site-packages\win32com\client\__init__.py", line 70, in GetObject
    return GetActiveObject(Class, clsctx)
  File "C:\Python27\lib\site-packages\win32com\client\__init__.py", line 79, in GetActiveObject
    dispatch = pythoncom.GetActiveObject(resultCLSID)
com_error: (-2147221021, 'Operation unavailable', None, None)

But maybe I don't have any instances of flex grid on my desktop. @EduardoP777 please try it on your side.

is there any available solution for the issue ?

hey @vasily-v-ryabov how I can reach to you, I have the same issue but I couldn't find a proper solution for it, and I have been searching for a solution for days without any hope.

There are reports in https://github.com/pywinauto/pywinauto/issues/1293 https://github.com/pywinauto/pywinauto/issues/1170 https://github.com/pywinauto/pywinauto/issues/1030 about "cells" and "grids".
I think it is certainly more difficult for UIAutomation to manipulate such elements than others.
Have you read those reports?

yes I have went through all the mentioned reports. However, there are no potential solution to be used.
I am open to use alternative solutions


Recovering systemd docker compose gotchas

pull image - rate limit hit - systemd docker compose

https://github.com/movementlabsxyz/movement/blob/c43e7770346a2e1284fe7aefce8cb24ae7a26553/docs/movement-node/run/ansible/follower-node/mainnet/movement-full-follower.service.j2#L19
missing doc for AWS_* envs in ansible setup
https://github.com/movementlabsxyz/movement/blob/c43e7770346a2e1284fe7aefce8cb24ae7a26553/docs/movement-node/run/manual/follower-node/README.md?plain=1#L39-L41
Fixes

Add AWS_* into .env, use EnvironmentFile=/home/{{ user }}/movement/.env
use ExecStartPre, ExecStop and StartLimitIntervalSec

# movement-full-follower.service.j2
[Unit]
...
StartLimitIntervalSec=200
...

[Service]
...
EnvironmentFile=/home/{{ user }}/movement/.env
ExecStartPre=/usr/bin/docker compose --env-file .env -f /home/{{ user }}/movement/docker/compose/movement-full-node/docker-compose.yml -f /home/{{ user }}/movement/docker/compose/movement-full-node/docker-compose.follower.yml pull
ExecStart=/usr/bin/docker compose --env-file .env -f /home/{{ user }}/movement/docker/compose/movement-full-node/docker-compose.yml -f /home/{{ user }}/movement/docker/compose/movement-full-node/docker-compose.follower.yml up --force-recreate --remove-orphans
ExecStop=/usr/bin/docker compose --env-file .env -f /home/{{ user }}/movement/docker/compose/movement-full-node/docker-compose.yml -f /home/{{ user }}/movement/docker/compose/movement-full-node/docker-compose.follower.yml stop
...

As suggested by docker CLI, the pull limit could be overcome by authenticating with docker login. Upgrading to a paid plan on Docker Hub may help further, but it's not strictly necessary, just authenticating raises the limit twofold.
It looks to me that the docker pull limit and AWS setup are different issues which need their own tracking, comment thread, etc.. Can you file the AWS issue separately? I'm closing this issue with the login advice above.

On a better thought, we need an advice on how to configure docker authentication in this example setup. However, pulling the images for a container revision is a one-off, you should not hit the pull limit repeatedly unless you either change CONTAINER_REV many times within 6 hours, or lots of other docker images get pulled at the same time from the same IP address.

Thanks @pratikbin, these are good points. I agree that we should use ExecStartPre in the short term and rethink our deployment recipe in the long term.

Shall create PR for same. let me know to which branch should i do the PR?


Feature: Automatically use lto in release

Just automatically add:
[profile.release]
lto = "fat"

No argument was provided to explain why this would be valuable.
When thinking about this, I can only come up with an argument against such feature: cargo diet is about keeping crates packages small, it isn't concerned with binary size.
Please feel free to provide arguments on the closed issue though, maybe I am not seeing something.

Well, if you can make the binary smaller, isn't the crate smaller? Or am I missing something?

Well, if you can make the binary smaller, isn't the crate smaller?

The crate is a package containing source code to build the binary or other artifacts, not every crate produced binaries. cargo diet only means to slim down the amount of source code shipped in these crates and leaves configuring artifacts entirely to the crate owners. Changing build settings really isn't anything where one size fits all.

I see, I think I must've misunderstood a few things along the line.. Mb!


Namespace / PSR-4 Autoloading

Composer!
You might need to change the version number and such, though.

Thanks! :smiley:


collection object locality: in wrong country when close to border

For example a geo-referenced object near the Iguazú Falls in NE Argentina:

This is in fact in "Argentina: Misiones" instead of "Brazil: Paraná" as recorded (the object a little further west is correct). I guess the underlying shape file is too coarse. Would it be possible to access the polygon and add more angles so that it would follow more closely the course of the Iguazú River? Or perhaps run it through some algorithm to adapt to the complex border (e.g. mapshaper)?

Can we put soft validation instead of hard validation that the point is out of geographic area. Or put some buffer zone of few km along the border line?

There are at least 3 issues being raised here, some of which have been answered elsewhere (See TaxonWorks doc) very recently.
@typophyllum If the label says two different things this is a problem with the label, I"m not sure if that's the underlying issue in this specific question.
In general though:

Users will not be allowed to edit GeographicAreas.  They will always represent existing, static datasets.
We don't supply the base maps, others do (Google, Open Street Maps).
If you have a point georeference you do not need a GeographicArea.  Everything needed to display is calcuable from the point.  For example, see the DWC export for your object in which county, state, and country are populated with text values.
If you have a point reference everythign you need to compute, for curation purposes is done.
The whole point of moving to a spatial system is make use of it.  Asserting two conflicting things is a) not necessary, and b) contrary to the point of adopting an quantitative approach.

So, if you have conflict, don't include the GeographicArea, you lose nothing, and make no conflicting assertions.  Win, win.

@mjy in this particular example if only point is stored (without geographic area) it will be resolved in Brazil by the parser, but we can visually see that the point is in Argentina. It will be an issue for many marginal points. In those cases the having geographic area is beneficial. Yes, the geographic area contradict the point, because the geographic area is not precise enough. That is why I suggested a soft validation or a buffer border zone. The point is still in Argentina if it is few km from the area border.

@proceps @mjy the actual purpose of this issue was to point out that some shape must be wrong in the database and because of that the coordinates are incorrectly auto-assigned to Brazil (during import in this case).

Related: https://github.com/SpeciesFileGroup/taxonworks/issues/1474

The NE shapes are indeed rough, inteneded for displaying quick and dirty maps.
The solution to @LocoDelAssembly issues is to update (may already have this in the method) the option to assign Country/State/County based on a preferred Gazetteer set.  However, because no gazeeter has the global coverage we'd like, this will never be a completely satisfactory solution.
Allowing two contradictory pieces of data is NOT sustainable, period.  How will we know which one is intended to be correct if the curator provides both?
The point is- if a point georeference is supplied, we will consider that truth.  If multiple ones are supplied, the first in the list is truth.  Any other solution solves one problem, and creates another.
The real solution- we need to do better with our point references- match them to more accurate (potentially new) GeographicArea sets or also explore external georeferencing APIs.

I also experience few cases when the point is not precise enough. Degrees and minutes, no seconds, and it bounce across the border, but the country of collecting is known. Within certain error radius, I can estimate the locality, but if the point is deleted, I bounce to the country or state shape file. And if point is provided, it is resolved in a wrong country. Soft validation and map preview are the best solution in my point of view. Of cause the confidence levels could be used not to verify the locality every time.

Thanks for all the responses. I like Dmitry's idea of a buffer border zone and soft validation, perhaps with a special task that would find discrepancies between point and area data. In this particular case it is a pretty precise point (for sure on the forest trail shown on Hernán's screenshot that runs parallel to the border-forming river, +/-200 m north/south), imported from OSF.

I'm strongly against soft validation. Permitting sloppy data rather than addressing the root problem, i.e. finding and adding better Gazetteers, or improving our reverse lookups, kicks the can down the road rather than puts pressure on us to do it right.

Mental note for me - consider testing distance from point to nearest border and if too close then leaving record uncertain/empty.

Another note- priority may be testing against NE rather than GADM, switching would likely make significant difference.

Decision at April 6th Open Session to change (expand) the fuzziness when a point is near a border (county, state, country, river) so that Country will not be changed.