RFC: item() to return scalar for arrays with exactly 1 element.

def item(self) -> Scalar:
     """If array contains exactly one element, retun it as a scalar, else raises ValueError."""

Examples:

numpy.ndarray.item
torch.Tensor.item
pandas.Series.item
pandas.Index.item
polars.Series.item
xarray.DataArray.item

Demo:
import pytest
import xarray as xr
import pandas as pd
import polars as pl
import numpy as np

@pytest.mark.parametrize("data", [[], [1, 2, 3]])
@pytest.mark.parametrize(
    "array_type", [torch.tensor, np.array, pd.Series, pd.Index, pl.Series, xr.DataArray]
)
def test_item_valueerror(data, array_type):
    array = array_type(data)
    with pytest.raises(ValueError):
        array.item()

@pytest.mark.parametrize(
    "array_type", [torch.tensor, np.array, pd.Series, pd.Index, pl.Series, xr.DataArray]
)
def test_item(array_type):
    array = array_type([1])
    array.item()

Currently, only torch fails, because it raises RuntimeError instead of ValueError.

This was discussed in #710 , along with the more general to_list, which works also for ND arrays.

item() is a bit different from to_list, and honestly I find it confusing that a method named to_list can return something that is not a list.

.item() is more constrained than to_list indeed, and a bit cleaner. I checked other libraries - NumPy, PyTorch, JAX and CuPy implement .item(), Dask does not. (TF doesn't have it in the docs, so probably also not - but I can't check). CuPy/JAX do the transfer to CPU if the ndarray is on GPU.
This is a minor convenience method though, since float() & co work as well. They are clearer, since type-stable, and it also work for Dask. The only downside is that if you want some dtype-generic implementation to return a single element, you have to write a little utility for it to call int/float/complex/bool as appropriate. Something like:
def as_pyscalar(x):
    if xp.isdtype(x, 'real floating'):
        return float(x)
    elif xp.isdtype(x, 'complex floating'):
        return complex(x)
    elif xp.isdtype(x, 'integral'):
        return int(x)
    elif xp.isdtype(x, 'bool'):
        return bool(x)
    else:
        # raise error, or handle custom/non-standard dtypes if desired

Static typing of such a function, and of .item(), would also be a little annoying as it requires overloads.

item also works on arrays with multiple dimensions, whereas we decided to make it so float does not.
>>> np.array([1]).item()
1

We discussed this in a call today, and concluded that this fell into a bucket of functionality that is useful, but also easy to implement on top of what's already in the standard. In addition, there are problems with trying to add this: a item() method is hard, because it's missing in some libraries and missing methods cannot be worked around in array-api-compat. If we'd do this, a function would be the way to go - but since that's not present in any libraries, it'd be new - hence more work, and likely to incur resistance from array library maintainers.
Outcome:

Create the array-api-extra package where this kind of function can live, and add it there (probably as as_pyscalar or a similarly descriptive name, not as item)
Only reconsider adding it to the standard itself in the future if most/all array libraries have already added that function.

On a very fundamental level, I believe .item() makes no sense on DataFrame-like objects (pandas.DataFrame, polars.DataFrame, pyarrow.Table, etc.) because these are designed to represent heterogeneous data types.
From a mathematical PoV, item() acts on array-like data with homogeneous type, as a representation of the natural isomorphism V →K, when V is a 1-dimensional vector space over K.

Is this usage guaranteed?
If so, should it be added somewhere to the specification?  I looked for it here.
FWIW I like a bare item function (def item(x: Array, /) -> complex | bool).

Yes, __float__ and so on are guaranteed. See https://data-apis.org/array-api/latest/API_specification/generated/array_api.array.float.html#array_api.array.float. Though Ralf's helper should include a if x.ndim != 1 or x.size != 1: raise ValueError check.


%matplotlib inline interaction?

I did notice one thing: if I put %matplotlib inline to also use mpl plots (cause legacy stuff) then the px stops rendering - it makes a blank space in the notebook of about the size the figure would be.

From https://www.reddit.com/r/Python/comments/b3doya/introducing_plotly_express/ej0p664/

Hi, I made that reddit post linked to above, just clarifying that it doesn't seem to have anything to do with using `%matplotlib inline%, I've getting that behaviour in new, blank notebooks as well.
So its the as issue 17.
I'm using Anaconda 2018.12, installed plotly_express using pip install plotly_express.
I'm testing this out in a new conda env with just pip installed and then doing pip install plotly_express.
it imports fine:
import plotly_express as px
print(px.data.iris.__doc__)
iris = px.data.iris()```

prints out info and I get a iris data frame, but doing this:

`px.scatter(iris, x="sepal_width", y="sepal_length")` or

fig = px.scatter(iris, x="sepal_width", y="sepal_length")
fig
gives me a blank area the size of the plot.

It seems to be in offline mode as `fig.offline_initialized` returns True.

OK, we'll take a look! which versions of Python and JupyterLab are you using?

I've updated the README to note that the plotly-extension needs to be installed if using JupyterLab via jupyter labextension install @jupyterlab/plotly-extension ... This resolved #17 apparently.


Column and table aliasing for Data Library tables

Should display sync table names imported from the Data Library on maps/datasets dashboard and in the editor by transforming lower case, underscore separated names to caps case, space separated names, e.g. 'united_states_tornadoes_2014' -> 'United States Tornadoes 2014'. Care should be taken to attempt to not capitalize prepositions such as and/or.
This sample code might be a good start for the caps case portion of this task, MIT licensed and seems fairly straightforward, with a regex for words to ignore: https://github.com/gouch/to-title-case/blob/master/to-title-case.js

Also, I believe (but you'll need to double check) the check you want to trigger this aliasing is something like vis.get('type') === 'remote'. There's also a method on the vis model isSyncTable() that might work too.

@mbforr RE: data management process
Initial Dataset

Import new dataset (foo_rev1.shp) into common data account (make sure it is set to private so it doesn’t propagate to end-users yet)
Modify table/column names to preference via editor UI
Make dataset public so that other accounts can see it
Connect dataset from other account, map/table will show updated table/column names

The common dataset table/column names cannot be modified after a user has imported the dataset. If it happens, all users that imported the older version will get SQL errors when viewing the vis/table in the UI. The derived dataset in a user's account can still be deleted and reimported though, and will get the updated column names if reimported.
Data Updates

Import updated dataset (foo_rev2.shp) into common data account, keeping it private
Update foo_rev1 with foo_rev2 data using the SQL console (sample query below)
Delete foo_rev2

Data changed within the common data table will be properly reflected in user  accounts immediately after the update is done.
UPDATE foo_rev1
SET store_locations = b.abc123
FROM foo_rev2 b
WHERE foo_rev1.id = b.id

thanks let me check and i'll let you know asap
On Wed, Aug 17, 2016 at 12:12 PM, jbranigan<EMAIL_ADDRESS>wrote:

@mbforr https://github.com/mbforr RE: data management process
Initial Dataset

Import new dataset (foo_rev1.shp) into common data account (make
sure it is set to private so it doesn’t propagate to end-users yet)
Modify table/column names to preference via editor UI
Make dataset public so that other accounts can see it
Connect dataset from other account, map/table will show updated
table/column names

The common dataset table/column names cannot be modified after a user has
imported the dataset. If it happens, all users that imported the older
version will get SQL errors when viewing the vis/table in the UI. The
derived dataset in a user's account can still be deleted and reimported
though, and will get the updated column names if reimported.
Data Updates

Import updated dataset (foo_rev2.shp) into common data account,
keeping it private
Update foo_rev1 with foo_rev2 data using the SQL console (sample
query below)
Delete foo_rev2

Data changed within the common data table will be properly reflected in
user accounts immediately after the update is done.
UPDATE foo_rev1SET store_locations = b.abc123FROM foo_rev2 bWHERE foo_rev1.id = b.id
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/cartodb-org/cartodb/issues/116#issuecomment-240462985,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEmtLCzQp-i_2cNYvQVo9rdTVdscc3BHks5qgzLdgaJpZM4Jkc7r
.

--
Matt Forrest
Client Success Manager
CARTO
201 Moore St, Brooklyn, NY, 11206
P:<PHONE_NUMBER> x7508
C:<PHONE_NUMBER>
carto.com http://www.carto.com/


[SPARK-33919][SQL][TESTS] Unify v1 and v2 SHOW NAMESPACES tests

What changes were proposed in this pull request?

Port DS V2 tests from DataSourceV2SQLSuite to the base test suite ShowNamespacesSuiteBase to run those tests for v1 catalogs.
Port DS v1 tests from DDLSuite to ShowNamespacesSuiteBase to run the tests for v2 catalogs too.

Why are the changes needed?
To improve test coverage.
Does this PR introduce any user-facing change?
No
How was this patch tested?
By running new test suites:
$ build/sbt -Phive-2.3 -Phive-thriftserver "test:testOnly *ShowNamespacesSuite"

Test build #133405 has started for PR 30937 at commit ad20040.

Test build #133405 has finished for PR 30937 at commit ad20040.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/133405/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37996/

Test build #133409 has started for PR 30937 at commit 798510e.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38000/

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38000/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38000/

Test build #133409 has finished for PR 30937 at commit 798510e.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/133409/

Test build #133592 has started for PR 30937 at commit 9e88e0a.

Test build #133592 has started for PR 30937 at commit 9e88e0a.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38181/

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38181/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38181/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38181/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38181/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38181/

Test build #133592 has finished for PR 30937 at commit 9e88e0a.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
class ShowNamespacesParserSuite extends AnalysisTest with SharedSparkSession

Test build #133592 has finished for PR 30937 at commit 9e88e0a.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
class ShowNamespacesParserSuite extends AnalysisTest with SharedSparkSession

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/133592/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/133592/

@cloud-fan @HyukjinKwon Could you review this PR, please.

@cloud-fan @HyukjinKwon Could you review this PR, please.

Test build #133624 has started for PR 30937 at commit 8f8751d.

Test build #133624 has started for PR 30937 at commit 8f8751d.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38213/

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38213/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38213/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38213/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38213/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38213/

Test build #133624 has finished for PR 30937 at commit 8f8751d.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test build #133624 has finished for PR 30937 at commit 8f8751d.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/133624/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/133624/

thanks, merging to master!

thanks, merging to master!


iOS image size - Doc

Maybe it's a good idea to add this issue as a tip on the tutorial to help those who are running iOS device and getting this result on Tabbar:

Or just resize the PNG icons to 30x30 (works well on the android and iOS):

Android:

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 720bf7f0-db92-f738-b864-ee149a02377b
Version Independent ID: 720bf7f0-db92-f738-b864-ee149a02377b
Content: Tutorial: Create a .NET MAUI app - .NET MAUI
Content Source: docs/tutorials/notes-app/index.yml
Product: dotnet-mobile
Technology: dotnet-maui
GitHub Login: @adegeo
Microsoft Alias: adegeo

@tuliocll This sounds like a bug with .NET MAUI. It shouldn't be allowing an image to bleed through like that. Can you file a bug on https://github.com/dotnet/maui/issues
If you look at the picture, it seems to have sized the image really low, then stretched it to an area too big. It's almost related to https://github.com/dotnet/maui/issues/6043

I'm reopening this. I'm running into more problems with this for iOS/Mac. It doesn't seem like this will be fixed anytime soon in the product so the solution is to set a base size on the icon or use OnPlatform extensions.

https://github.com/dotnet/maui/issues/11099


Update _index.md

Desculpem a intromissão, estou interessado em contribuir, esta é a minha primeira aventura. Ainda preciso seguir todos os passos, para depois submeter alguma 'issue'.

Hi @DonatoHorn, thank you for your contribution, and welcome!
Are you in the slack channel to the Portuguese team? There are some points in your PR that I can help you to fix:

Fix the conflicts
Change the target branch from main to dev-pt

Hi @DonatoHorn! again -  thanks for your contribution, looks like some of the fixes made by you in this PR are already done, could you check the current content/pt-br/style-guide/_index.md (look in the dev-pt branch) file and see If it still makes sense?
My suggestion is to close this PR and If makes sense open a new one, we can do it together, just send me a message via slack :)


Flutter - add confidence score to TextBlock/Line/Element

https://developers.google.com/android/reference/com/google/mlkit/vision/text/Text.Element
Will these features be added to the ml kit - text recognition library in Flutter?
I need to get OCR confidence score in the reads.
for (final line in textBlock.lines) {
          print(' line Text: ${line.text}');
          print(' line Rect: ${line.boundingBox.toString()}');
          print(' line CornerPoints: ${line.cornerPoints.toString()}');
          print(' line confidence: ${line.confidence.toString()}');
          print(' line confidence: ${line.angle.toString()}');
}

for (final element in line.elements) {
          print(' element Text: ${element.text}');
          print(' element Rect: ${element.boundingBox.toString()}');
          print(' element CornerPoints: ${element.cornerPoints.toString()}');
          print(' element confidence: ${element.confidence.toString()}');
}

java example - https://github.com/googlesamples/mlkit/blob/master/android/vision-quickstart/app/src/main/java/com/google/mlkit/vision/demo/java/textdetector/TextGraphic.java

@EduardoDiNizo : I do not have an ETA for that, but go to our README, understand how this plugins works, learn about Flutter MethodChannels and feel free to send a PR with your contribution. Contributions are always welcome. I will review, merge your PR, and release a new version ASAP.

Let me see, if it something I can work on too.

@fbernaly , @bensonarafat . Thank you very much for your attention and agility in this task. I will be the first to test the release! :)

Can I close this issue?


Runtime error

Hi, using Delphi 10.3 with Skia 4.1.
When I execute any SKIA-based EXE I get:
Parameter AContext cannot be nil.
Some solutions suggest a replacement unit for FMX.Graphics but this does not work.
Any ideas?
Thanks,
Kim.

Hi @KBracknell, thanks to report the issue!
Have you enabled the Skia in your project?
Inside the RAD Studio > right click on your project > “Enable Skia”

Thx viniciusfbb,
Good advice... the project had been set to have Skia enabled (as it initially showed as Skia disabled. However, Enabling Skia did not initially fix the problem. It did, however, work after a reboot (I had hadn't rebooted after initial Skia install). However, now if I open the and run the the Skia4Delphi demo that comes with the install it gives heaps of compile errors (yes, Skia is Enabled). Errors are as follows:
Messages
Checking project dependencies...
Compiling Skia4Delphi.dproj (Release. Win32)
brcc32 command line for "Skia4Delphi.vrc"
c:\program files (x86)\embarcadero\studio\20.0\bin\cgrc.exe -c65001 "Skia4Delphi.vrc" -foSkia4Delphi.res dcc32 command line for "Skia4Delphi.dpr"
(dcc32 Error] Sample.Form.Filter.pas(152): E2010 Incompatible types: 'Integer' and 'string'
[dcc32 Error] Sample.Form.Filter.pas(154): E2250There is no overloaded version of 'MakeShader' that can be called with these arguments [dcc32 Error] Sample.Form.Filter.pas(197): E2250 There is no overloaded version of 'Show' that can be called with these arguments [dcc32 Error] Sample.Form.Filter.pas(278): E2250 There is no overloaded version of 'Show' that can be called with these arguments [dcc32 Error] Sample.Form.Filter.pas(327): E2250 There is no overloaded version of 'Show' that can be called with these arguments [dcc32 Error] Sample.Form.Filter.pas(371): E2010 Incompatible types: 'Integer' and 'string'
[dcc32 Error] Sample.Form.Filter.pas(373): E2250 There is no overloaded version of 'MakeShader' that can be called with these arguments [dcc32 Error] Sample.Form.Filter.pas(416): E2250 There is no overloaded version of 'Show' that can be called with these arguments [dcc32 Error] Sample.Form.Filter.pas(464): E2250 There is no overloaded version of 'Show' that can be called with these arguments [dcc32 Error] Sample.Form.Filter.pas(491): E2010 Incompatible types: 'Integer' and 'string'
[dcc32 Error] Sample.Form.Filter.pas(493): E2250 There is no overloaded version of 'MakeShader' that can be called with these arguments [dcc32 Error] Sample.Form.Filter.pas(532): E2250 There is no overloaded version of 'Show' that can be called with these arguments [dcc32 Fatal Error] Skia4Delphi.dpr(17): F2063 Could not compile used unit 'Sample.Form.Filter.pas'
Failed
Elapsed time: 00:00:02.9
Build
Output
?

It almost looks like the 4.1 version of Skia is incompatibe with Delphi 10.3?
Any ideas?
Thanks again,
Kim.

I have since unistalled Skia 4.2 and reverted to 3.4 - demo mow compiles fine. I guess there are incompatibilities with more recent Skia versions? That's a shame...
Cheers,
Kim.

There is no incompatibility. Maybe it's a conflict with some old version installed. For a secure installation, follow these steps:

Install running the setup (the manual installation is not recommended) with the IDE closed. You can choose always the same destination folder (prefer the default folder to avoid administrator permission), even if there is an old installation there.
Delete old Skia4Delphi folders that you detect on your computer.
Check in your "RAD Studio > Options > Language > Delphi > Library > Library Path >" if there is any other skia directory besides these:

Check in your "RAD Studio > Component > Install Package" if there is any other skia package besides these:

@KBracknell Recently a user had the same problem and we discovered that EtheaDev/SVGIconImageList internally adds a version of Skia4Delphi in its installation, resulting in 2 versions of the library in your RAD Studio. If you have this library, try to uninstall it, and reinstall Skia4Delphi.
I reported the issue to them few weeks ago and it's possible they already fixed it in their latest version.


[pg] Support psql as admin tool

[x] #1735

local=> \l
Status: INTERNAL_ERROR
Issues:
: Fatal: ydb/library/yql/parser/pg_wrapper/comp_factory.cpp:457: Condition violated: `f', code: 1
: Error: Query invalidated on scheme/internal error., code: 2019

local=> \dn
Status: INTERNAL_ERROR
Issues:
: Fatal: ydb/library/yql/parser/pg_wrapper/comp_factory.cpp:457: Condition violated: `f', code: 1
: Error: Query invalidated on scheme/internal error., code: 2019

local=> \df
List of functions
Schema | Name | Result data type | Argument data types | Type
--------+------+------------------+---------------------+------
(0 rows)

local=> \dv
Did not find any relations.

local=> \du
Status: GENERIC_ERROR
Issues:
:4:3: Error: SublinkExpr: unsupported link type: 6

https://github.com/ydb-platform/ydb/pull/1936

https://github.com/ydb-platform/ydb/pull/2039


reintroduce support for build.scala project?

[ ] I've read the CONTRIBUTING guidelines

steps
problem
pure build.scala project is no longer support as of sbt 1.X
expectation
project with only build.scala should be supported,because .sbt is too hard to work with if the project is complex or you try to develop/customize plugins
notes
sbt version: 1.X

I could be mistaken, but I believe that at least part of the reason that build.scala is no longer supported is because we don't want *.scala files in the project base directory. You can still write your build in pure scala and just define a trivial build.sbt file that delegates to the objects created by your scala file.
For example, you could create a file called, say, project/build.scala that looks like:
import sbt._
object Build {
  val root = project in file(".")
}

and then your build.sbt file would just be:
  val root = Build.root

Build.scala's days were numbered when we introduced auto plugins with autoImport and := macros in sbt 0.13. At that point build.sbt DSL were no longer Scala, but more like another language that can interop with Scala (like Scala can interop with Java).
We removed trait Build because it does the same thing as build.sbt, and having two ways of doing the same thing was adding both confusion to users and maintenance cost.
See https://github.com/sbt/sbt/pull/2524#issuecomment-206620041 by ralph-tice (Apr 6, 2016):

Nobody likes to see "their" feature or code path deprecated, and I personally am a big fan and proponent of Build.scala. But strictly from a maintainability standpoint, I don't think any open source project has the bandwidth to cater to a minority of users, nor should they. So I tried to find some sort of proxy for usage metrics, and the results (below) surprised me:

First, for the lulz, 1529 usages of override lazy val settings since it came up in the previous comment
https://github.com/search?q="override+lazy+val+settings"+extension%3Ascala+path%3A%2Fproject&type=Code
14,152 usages of extends Build (in *.scala)
https://github.com/search?o=desc&q="extends+build"+extension%3Ascala+path%3A%2Fproject&s=indexed&type=Code
and 39,580 usages of scalaVersion (in *.sbt)  https://github.com/search?o=desc&q="scalaVersion"+extension%3Asbt+path%3A%2F&s=indexed&type=Code

I think these searches are representative in that they count roughly similar things, and the data surprised me. I probably would not be using Build.scala today if there were some usage metrics available to help me choose what the community has de-facto voted for, and for me the schism between Build.scala and build.sbt is more confusing than it would have been to learn the build DSL in the first place.

As of today (April 2019), the gap is even wider:

13,449 usages of extends Build (in *.scala)
95,229 usages of scalaVersion (in *.sbt)

If you have a few minutes, see also this segment from Scala Days 2015 talk The road to sbt 1.0 is paved with server (21:49).


emails: Improve "contact us" line in onboarding emails.

Improve "contact us" line in onboarding emails
as per the issue:
-> make macros
-> use different lines for self hosted and cloud
-> alter the line in follow-up html
Fixes:
https://github.com/zulip/zulip/issues/24260

Screenshots and screen captures:
Self hosted alternative

confirm registration

Follow up day 1

Invitation

realm reactivation

Cloud Alternative

confirm registration

Follow up day 1

Invitation

Realm reactivation

Self-review checklist

[x] Self-reviewed the changes for clarity and maintainability
(variable names, code reuse, readability, etc.).

Communicate decisions, questions, and potential concerns.

[ ] Explains differences from previous plans (e.g., issue description).
[ ] Highlights technical choices and bugs encountered.
[ ] Calls out remaining decisions and concerns.
[ ] Automated tests verify logic where appropriate.

Individual commits are ready for review (see commit discipline).

[ ] Each commit is a coherent idea.
[x] Commit message(s) explain reasoning and motivation for changes.

Completed manual review and testing of the following:

[ ] Visual appearance of the changes.
[ ] Responsiveness and internationalization.
[ ] Strings and tooltips.
[ ] End-to-end functionality of buttons, interactions and flows.
[ ] Corner cases, error conditions, and easily imagined bugs.

I have made appropriate changes in the commit as well

The screenshots look good to me! @laurynmm could you please mark this PR for integration review when it's ready from your perspective?

I will take a look at the documentation again and go through the example commit.

@UjjwalAggarwal-1 - This looks good. I'm tagging for integration review. My one note is that the commit message could be simplified ...
Several onboarding emails sent to new users/organizations used the
same "contact us" language. Adds new macros for the "contact us"
line for Zulip Cloud and self-hosted realms to eliminate repetition.

The suitable macro for Zulip Cloud vs self-hosted organizations is
decided conditionally, by checking corporate_enabled.

Fixes #24260.

@zulipbot add "integration review"
@timabbott - My one open question is about the strings generated for translation. The followup_day1.source.html has a variant for the Zulip Cloud version of the "contact us" macro, and I wasn't sure if that would get flagged for translation twice on what's a very similar string.

Made the changes in the commit message. I was planning to keep the details as is in the issue statement, but one could always look at the changes itself

Unfortunately, I think the macros here don't work for translations -- here's what ./scripts/setup/inline_email_css.py generates in templates/zerver/emails/compiled/followup_day1.html:
{% trans %}Questions? Contact us — we'd love to help!{% endtrans %}

And similarly for templates/zerver/emails/compiled/macros.html.
Since the content inside trans is not identical to the pre-trans content, I'm pretty sure this isn't going to translate anything.
Hmm, but this seems to be a potential existing issue; I see in templates/zerver/emails/compiled/missed_message.html the following:
{% trans url="https://zulip.readthedocs.io/en/latest/production/email-gateway.html" %}
    Do not reply to this email. This Zulip server is not configured to accept incoming emails (help).
    {% endtrans %}

So either Django somehow is handling this despite the styling tags appearing, or translations are broken for many other strings in our emails.
Unfortunately, the answer appears to be the existing footer content is not translated properly; here's a development environment test:

Started https://chat.zulip.org/#narrow/stream/58-translation/topic/email.20notification.20footers/near/1512669 to discuss how to fix that.

what should I do in this case?

I think there's nothing to do on this PR until that conversation comes to a good resolution; I expect we'll merge this PR with minimal changes once we've figured that out.

i will do that, require a bit time

@UjjwalAggarwal-1 Would you be able to resolve the conflicts on this PR in the next couple of days?

Seems like the author is busy, I could take over in the upcoming days if the author is still inactive.

The rebase was handled by git rebase automatically -- just some renamed files.
This is a good checkpoint, and I'm going to merge it as written since it is a definite improvement and will make it easier to do further work from here, but it doesn't update the plain-text version of the emails (which also seem to have had some typos in them). I've opened https://github.com/zulip/zulip/issues/25198 for the remaining work to complete this project for this release and assigned it to you @Joelute. Thanks @UjjwalAggarwal-1 and @Joelute!


Tiled deferred / Forward +

Do some research and see which is better.

Z-binning was implemented in #193 .


Add attributes for ok/cancel labels in DatePicker

The attribute names are chosen to be the same as in material-ui
This fixes #805.

Do you want me to fix the lint errors related to DatePicker and DatePickerDialog?

Yeah please! May you do the same changes for TimePicker??

Sure

Okay, done!

Thanks!


logging: respect ctx

The ctx is cancelled on SIGTERM
Fix #2726 (regression in #2683)

Also is it possible to add a test that can capture the regression and demonstrate the fix? For example, test sudo nerdctl run --rm hello-world can be finished in, e.g., 5s. (not sure if this will be flaky).

Added TestRunRmTime to test this


[SPARK-19634][ML] Multivariate summarizer - dataframes API

What changes were proposed in this pull request?
This patch adds the DataFrames API to the multivariate summarizer (mean, variance, etc.). In addition to all the features of MultivariateOnlineSummarizer, it also allows the user to select a subset of the metrics.
How was this patch tested?
Testcases added.
Performance
Resolve several performance issues in #17419, further optimization pending on SQL team's work. One of the SQL layer performance issue related to these feature has been resolved in #18712, thanks @liancheng and @cloud-fan
Performance data
(test on my laptop, use 2 partitions. tries out = 20, warm up = 10)

Vector size/records number
1/10000000
10/1000000
100/1000000
1000/100000
10000/10000

Dataframe
15149
7441
2118
224
21

RDD from Dataframe
4992
4440
2328
320
33

raw RDD
53931
20683
3966
528
53

Test build #80126 has started for PR 18798 at commit 2860390.

@WeichenXu123 thanks! Can you post some performance numbers as well?

Test build #80126 has finished for PR 18798 at commit 2860390.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
  class SummarizerBuffer(
 trait TraversableIndexedSeq

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80126/
Test PASSed.

performance data attached. cc @thunterdb @jkbradley

Thank you for the performance numbers @WeichenXu123 , I have a couple of comments:

you say that SQL uses adaptive compaction. How bad is that? I assume it adds some overhead.
did you just run each experiment once? I would be interested in error bars on these numbers, as it can take up to 30 seconds for the JVM to warm up and optimize the byte code. You should report the geometric mean or the median time of running these experiments to make sure that you are skewed by outliers. Some others will probably have some good advice as well.
from the performance numbers, there are 2 different regimes: small vectors, and big vectors (for which even the DataFrame -> RDD conversion is faster than working straight with DataFrames). I would be curious to know the bottlenecks for each case.

If we trust these numbers, the overall conclusion is that the SQL interface adds a 2x-3x performance overhead over RDDs for the time being. @cloud-fan @liancheng are there still some low hanging fruits that could be merged into SQL?
This state of affair is of course far from great, but I am in favor of merging this piece and improve it iteratively with the help of the SQL team, as this code is easy to benchmark and representative of the rest of MLlib, once we start to rely more on dataframe and catalysts, and less on RDDs.
@yanboliang @viirya @kiszk what are your thoughts?

cc @hvanhovell as well.

@thunterdb

The dataframe deserialize from binary data will add overhead, (maybe there is compaction or not, it depends on the datatype, cc @liancheng ) about 1x performance in my test.
My test try 20 times, the first 5 times used as warm up, the numbers in the table is the mean of the 15 times results. You can trust the result I think.
When the vector size is small (1~10), the bottleneck is at many small java object creation I think. So the performance result is different than the larger vector.

@WeichenXu123 @thunterdb
Thanks for this great work, we are always happy to see improvement which can help us to migrate MLlib workload to Dataset based API.
Here are my two cents:
1, Form the performance test result, there are performance degradation from 3 -> 2 -> 1.
The difference between case 3 and case 2 is the deserialization cost of each instance, I suspect we can't skip this step, so it maybe more hard to optimize it. However, the difference between case 2 and case 1 is the aggregate implementation. The aggregate on Dataset will use user-defined type as aggregate buffer, so I think we should make some effort to improve TypedImperativeAggregate. I guess it maybe not so hard compared with the previous bottleneck, but this is just my intuition, let’s check with SQL guys. cc @liancheng  @cloud-fan @hvanhovell
2, @WeichenXu123 In the current experiment, I saw you only use two partitions. Is there any difference if we run against more partitions or more dataset?
3, @thunterdb I agree that we can get this in and make improvement continuously. When we get desirable result, we can start to migrate other MLlib workload on top of Dataset/catalysts. But we should add comment to let users know(if we can’t get desirable result before
2.3.0) that the performance of Dataset-based multivariate statistic is not good enough currently. Thanks.

Test build #80359 has started for PR 18798 at commit 4f32e27.

Test build #80363 has started for PR 18798 at commit 6053d0e.

Test build #80359 has finished for PR 18798 at commit 4f32e27.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80359/
Test PASSed.

Test build #80363 has finished for PR 18798 at commit 6053d0e.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80363/
Test PASSed.

Merged build finished. Test PASSed.

Test build #80403 has started for PR 18798 at commit b02db42.

Test build #80403 has finished for PR 18798 at commit b02db42.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80403/
Test FAILed.

Jenkins, test this please.

Test build #80407 has started for PR 18798 at commit b02db42.

Test build #80407 has finished for PR 18798 at commit b02db42.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80407/
Test PASSed.

Merged build finished. Test PASSed.

@yanboliang do you feel comfortable to merge this PR? I think that all the questions have been addressed.

@thunterdb I'm on travel these days, will do a final pass and merge it on next Monday/Tuesday. Thanks.

Sorry can we make the performance data clear? Currently it doesn't say what the unit of the numbers is.

@viirya Sure! comment updated.

@WeichenXu123 Thanks! Looks good.

@WeichenXu123 I left some minor comments, otherwise, LGTM. Thanks.

@yanboliang I will update ASAP, thanks!

Test build #80668 has started for PR 18798 at commit 7540c4c.

Test build #80668 has finished for PR 18798 at commit 7540c4c.

This patch fails Scala style tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80668/
Test FAILed.

Test build #80669 has started for PR 18798 at commit b081fc3.

Test build #80671 has started for PR 18798 at commit c82958f.

Test build #80669 has finished for PR 18798 at commit b081fc3.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80669/
Test PASSed.

Test build #80671 has finished for PR 18798 at commit c82958f.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
sealed abstract class SummaryBuilder

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80671/
Test PASSed.

Thank you @yanboliang.

Merged into master, thanks for all.


Creates fileCache impl per ctx, not engine

This allows the reuse of the context with the same file cache pass across multiple wazero.Runtime, and
this makes the compilation cache is pseudo-thread-safe in one process.
Signed-off-by: Takeshi Yoneda<EMAIL_ADDRESS>

Still, using the same path for multiple contexts and passing these contexts to multiple wazero.Runtime are not safe, but I think out of common usecase

now go test ./internal/gojs/... -count=100 runs without crash


please make sure rectangular select & copy is implemented

Is your feature request related to a problem? Please describe.
I often need to select rectangular areas out of shell content and copy them elsewhere.
Normal line-based select doesn't work for this.
Describe the solution you'd like
Rectangular selection like you get in macOs Terminal with Alt+drag'n'select
Describe alternatives you've considered
There aren't any really.

Closed due to complete rewrite as part of bringing the project back to life, please create a new issue if still relevant. Thank you!


[FLINK-8887][tests] Add single retry in MiniClusterClient

What is the purpose of the change
This PR presents a test workaround for race-conditions in FLIP-6 (most notably FLINK-8887). Basically, every MiniClusterClient call is retried once after 500ms in case of certain exceptions.
This is only a band-aid until a proper fix is in place so we can finally continue merging more test ports.
Brief change log

add guardWithSingleRetry convenience method
add ScheduledExecutor to MiniClusterClient
guard all calls to the MiniCluster

Verifying this change
The change can be verified by cherry-picking this branch and running the AbstractOperatorRestoreTestBase. Before this change there was always 1-2 tests failing, whereas now none should fail.
/cc @aljoscha @GJL

+1 I think this makes sense if it unblocks merging test ports to master. I'm guessing we'll leave FLINK-8887 open until we have the proper fix?

merging.


Migrating default branch from master to main

LLVM made the move to main being the default branch, we should probably do it too for ONNX-MLIR

FYI, we are planning to rename our master branch to main, around Friday 19th. Thereafter, you will have to rebase on master. Let us know if there are issues with this migration.
Done to conform with the new wordings that are recommended for computer science projects

done


Is there any way to list all the compositions ?

I wonder if I can list all the compositions so that I can select one and then render?

You’d achieve this through a preprocessing step with a JSX script; I don’t have an actual solution written for that, however.
As an alternative, you could parse a list using the report generated when you collect the project files, since it lists all the comps there (assuming you are collecting your projects)


Enhancement - Account Creation Workflow - add targets to terraform plan/apply

User Story

Currently, when a new account is specified in /environments/*.json an account creation workflow is triggered. The triggered workflow runs a Terraform plan and updates the PR based on using terraform with -target module.vpc. When the PR is updated with a Terraform plan outcome, it does not show an accurate representation of the changes being made.
Additional targets should be added that include all modules except the one causing the dependancy issue.
Value

We require this change to ensure that are changes that Terraform will make are visible.
Questions / Assumptions

Definition of done

[ ] additional targets are added to Terraform code for plan/apply when accounts are created/updated

Reference
How to write good user stories

Additional targets have now been added to the Terraform Plan that updates the PR. This will ensure changes being carried out are visable before a PR is merged.
Closing ticket


pylance warning - Argument of type "list[LGBMRegressor]" cannot be assigned to parameter "models" of type "Models" in function "init"

Description
I'm encountering a type mismatch issue when passing a list of models (regressors) to the MLForecast class. The documentation states that the models parameter can be either a single regressor or a list of regressors as in https://github.com/Nixtla/mlforecast/blob/main/mlforecast/forecast.py#L126. However, when passing a list of models or the model alone, I receive a type error indicating that Argument of type "list[LGBMRegressor]" cannot be assigned to parameter "models" of type "Models" in function "init"
Python 3.10.12
mlforecast 0.13.4
According to the documentation:
Parameters
models : regressor or list of regressors
Models that will be trained and used to compute the forecasts.
Sample code:

# Import necessary libraries
from mlforecast import MLForecast
import lightgbm as lgb
from sklearn.linear_model import LinearRegression
import pandas as pd

# Create a simple DataFrame to simulate data
train_df = pd.DataFrame({
    'ds': pd.date_range(start='2020-01-01', periods=100, freq='D'),
    'y': range(100),
})

# Define a list of models
models = [
    lgb.LGBMRegressor(random_state=0, verbosity=-1),
    LinearRegression(),
]

# Initialize MLForecast with the list of models
try:
    fcst = MLForecast(models=models, freq='D')
    print("Model initialized successfully.")
except TypeError as e:
    print(f"Encountered an error: {e}")

the above code works fine without warning, but when using
models = [lgb.LGBMRegressor(random_state=0, verbosity=-1)]

or
models = lgb.LGBMRegressor(random_state=0, verbosity=-1)

it gives respective pylance warnings as below in vscode:
Argument of type "list[LGBMRegressor]" cannot be assigned to parameter "models" of type "Models" in function "__init__"
  Type "list[LGBMRegressor]" is incompatible with type "Models"
    "list[LGBMRegressor]" is incompatible with "BaseEstimator"
    "list[LGBMRegressor]" is incompatible with "List[BaseEstimator]"
      Type parameter "_T@list" is invariant, but "LGBMRegressor" is not the same as "BaseEstimator"
      Consider switching from "list" to "Sequence" which is covariant
    "list[LGBMRegressor]" is incompatible with "Dict[str, BaseEstimator]"Pylance[reportArgumentType](https://github.com/microsoft/pyright/blob/main/docs/configuration.md#reportArgumentType)

or

Argument of type "LGBMRegressor" cannot be assigned to parameter "models" of type "Models" in function "__init__"
  Type "LGBMRegressor" is incompatible with type "Models"
    "LGBMRegressor" is incompatible with "BaseEstimator"
    "LGBMRegressor" is incompatible with "List[BaseEstimator]"
    "LGBMRegressor" is incompatible with "Dict[str, BaseEstimator]"Pylance[reportArgumentType](https://github.com/microsoft/pyright/blob/main/docs/configuration.md#reportArgumentType)

Could you please look into why that is happening despite the documentation?
Link
No response

Hey @sumamjose, thanks for using mlforecast. This seems to be related to LightGBM, it seems that the linter picks up the dummy classes defined in the compat module instead of the scikit learn ones. Here's a minimal example (without mlforecast) where we can see the issue:

Feel free to open an issue there, it seems that what XGBoost does (defining those dummy base classes as object instead) works.


Refactoring Boundary Class

Codecov Report
Attention: Patch coverage is 94.73684% with 1 lines in your changes are missing coverage. Please review.

Project coverage is 81.58%. Comparing base (567fd28) to head (e52dfe5).
Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #4536   +/-   ##
=======================================
  Coverage   81.57%   81.58%
=======================================
  Files         110      110
  Lines       53594    53594
=======================================
+ Hits        43721    43725    +4
+ Misses       9873     9869    -4


catch numpy arrays in attrs before converting to dict

Makes it easier to dump to json (after conversation on #917)

Are the names time_check and np_check specific enough for utils.py?

Something more descriptive would probably be a good idea. Maybe ensure_us_time_resolution and decode_numpy_dict_values?

Let's add a test case including a numpy scalar in an attribute? Otherwise, looks good to me.

Thanks!


Certficate Name Check Should be Case-INsensitive

Since DNS names are not case sensitive, the comparison of the certificate names to the host name (or the certificate name specified on the command line) should also be case-insensitive - www.google.com should match www.Google.com.

I've been hitting this horribly as well. Looking into it at the moment, as I now have people clamoring (rightfully) about this.

This should now be closable.


fix NPE converting from string to date/time in mimir

Also, in repl don't exit in case a bug is hit but show the exception instead

I'm a little confused. How are we getting nulls? Like at all?

That is a valid question though: what data would produce null and not a DateTimeParseException?

@alissapajer I feel like it should only happen if we're calling apply on the results of the CF1P when the row is undefined.

@rintcius my current thinking is that there might be an underlying problem causing us to get nulls which we should address. For example what @djspiewak suggested above. So before just merging this, we should figure out where the nulls are coming from.

@djspiewak @alissapajer
Let me know if I'm misunderstanding you, but these are fixes in isDefinedAt, not in apply.  See e.g.
https://github.com/rintcius/quasar/blob/6bd6a22e5f4d73093b422d8364763e715f4e04fc/mimir/src/main/scala/quasar/mimir/TimeLib.scala#L195
Does that unconfuse you or is there still confusion left?

@rintcius Oh, I see what's going on. This is the wrong way around though. In fact, this is actually going to yield incorrect results for isDefinedAt. That's not the fault of your fix; the function was just always wrong.
isDefinedAt here needs to delegating to c.isDefinedAt(row). If the result from that function is false, then we shouldn't do the datetime conversion at all. This will avoid ever seeing null values, and will also correct this function for other cases.
For example, I'm relatively certain that the following query will result in incorrect results right now:
select localtime(select foo from bar where foo != "somedatetimestring")

Imagine real values there. The "somedatetimestring" result will magically reappear in the converted dataset. This bug, as well as the bug where we see null values at all, will be fixed by correctly delegating to c.isDefinedAt(row). There may be a utility which does this for us (like Op1SomethingOrOther).

@djspiewak the filter will be run before calling localtime, so those values will already have gone to undefined meaning that I think catching the NullPointerException would be enough. But that's a good test to add to our suite in any case :)
And I agree with @djspiewak that checking c.isDefinedAt(row) is the better way to solve this; else we'd have to catch NullPointerExpection in all of our mimir functions.

@alissapajer Sending a value to undefined doesn't make it null. In this case, c.isDefinedAt(row) will be false, but c(row) will not be null, which is why the value will "reappear".

@djspiewak oh I see what you're saying. Agreed. The BitSet filter is what gives us the holes.

@rintcius following up, I think that the query that @djspiewak suggested
select localtime(select foo from bar where foo != "somedatetimestring")

won't be doing the right thing, and that's a query that could conceivably be generated by the frontend. So we should fix this for v0.1.0.

@alissapajer indeed that's the same underlying issue to what I posted on discord

BTW, we really need to fix the isDefinedAt issue on these functions before 0.10 if we expect them to be used at all. It's simply broken.

@djspiewak @alissapajer PR ready for re-review
As discussed with Daniel, I couldn't reproduce the incorrect result regarding select localtime(select foo from bar where foo != "somedatetimestring")
(v53.1.0) 💪 $ select offsetdatetime(select str from `days0.data` where str != "2014-08-17T06:00:00Z")
{ "$offsetdatetime": "2015-09-18T07:00:00.000000000Z" }
(v53.1.0) 💪 $ select * from `days0.data`
{ "num": 1, "str": "2014-08-17T06:00:00Z" }
{ "num": 2, "str": "not something dateish" }
{ "num": 3, "str": "2015-09-18T07:00:00Z" }
{ "num": 4 }

This is using https://github.com/slamdata/quasar/pull/3691 to get the results of the correct type when selecting date/times & on top of that a catch NPE instead of deferring to c.isDefinedAt(r)  (i.e. my first impl). The query shows the correct results though.
So I'm leaving that query out of this PR now (also because it needs #3691). The other comments I have processed, I think.

[ch493]


#325 fix: accept feb-29th as birthday

The bot previously used the year 1970 to return the date so feb 29th automatically rolled over to march 1st. Instead we can use year 1972 which was a leapyear to return and save the date.

Why not using the year the people give?

Because people don't give us a year and we don't really care about the age of people, so we don't need the year.


Add darwin arm64 target

This PR add support for the darwin arm64 target along with the following changes:

Update Go to v1.16.2
Update fyne cli to v2.0.1
Add darwin arm64 target
Add the darwin-image command to build the darwin docker image
Add dedicated docker image for windows
Remove darwin 386 target
Remove the dependency from the docker/golang-cross image for the base image

Fixes #33
To test the darwin image:

Clone this branch
compile fyne-cross from this branch
Download Command Line Tools for Xcode >= 12.4
Run make base to build the new fyneio/fyne-cross:base-latest image with Go 1.16.x (Note this step is needed until the new base image is released and published on dockerhub)
Run: fyne-cross darwin-image --xcode-path /path/to/Command_Line_Tools_for_Xcode_<version>.dmg
build and test your app

NOTE: the creation of the image may take several minutes and may require more than 25 GB of free disk space.

To test the windows image:

Clone this branch
Run make base to build the new fyneio/fyne-cross:base-latest image with Go 1.16
Run make windows to build the fyneio/fyne-cross:windows-latest image
compile fyne-cross from this branch
build and test your app

This looks like a really great addition, thanks.
Unfortunately I don't have access to an M1 to test, which is probably a large chunk of this change.

Would it be possible to just download the Command Line Tools instead? According to Apple, they contain all the SDK and headers needed either way. This means downloading 400 MB instead of 11 GB and make our lives a lot easier.

I think you are mis-reading the information there. the 400MB file does not contain all the SDKs, it enables their use on the commandline. Those tools require XCode (and the SDKs it includes) to be present first.

Would it be possible to just download the Command Line Tools instead? According to Apple, they contain all the SDK and headers needed either way. This means downloading 400 MB instead of 11 GB and make our lives a lot easier.

I think you are mis-reading the information there. the 400MB file does not contain all the SDKs, it enables their use on the commandline. Those tools require XCode (and the SDKs it includes) to be present first.
(This is based on my understanding, and extreme skepticism that their massive SDKs are possible to compress that far!)

Ah. I see now. It looks like you probably are correct.

Perhaps some of the people from https://github.com/fyne-io/fyne/pull/1922, like @zdima and @sirkon could try this out as they have M1 machines? If so, it would be greatly appreciated :slightly_smiling_face:

@Jacalz my first understanding was the same of @andydotxyz but looking closer at the osxcross doc it seems that starting from Xcode command line tools 12.x it is known to work. Let's give it a try.

Would it be possible to have some sort of fyne-cross build-darwin pathtoxcode command to avoid having to cd into the project root and run make commands? Just a thought to make darwin builds easier. I don't think there would be much use for doing this for the base images.

Sure, in the plan this was the next step. Before implement that just want to be sure the macOS cross toolchain and image work as expected ;-)

Thanks @Jacalz going to check the error on windows.
Btw it seems from first tests that using the Command_Line_Tools_for_Xcode works, at least on complilation side (does not have macOS >= 10.12 to test it).

Btw it seems from first tests that using the Command_Line_Tools_for_Xcode works, at least on complilation side (does not have macOS >= 10.12 to test it).

If you have any patch for getting that working, I’ll happily test it out on 10.13 🙂

This might not be the most elegant solution out there (with all talk about not downloading random binaries from the internet). However, if anyone want to help out and test Darwin arm64 binaries on M1 machines without having to compile the fyne-cross images, I have a binary available for download here: https://github.com/Jacalz/wormhole-gui/issues/7#issuecomment-792925169

@Jacalz added with e072d7305d4494b4f5b29bd2bd72bf0dcd75cbb0

It is failing to start:

On Mar 8, 2021, at 12:50 PM, Jacob<EMAIL_ADDRESS>wrote:
This might not be the most elegant solution out there (with all talk about not downloading random binaries from the internet). However, if anyone want to help out and test Darwin arm64 binaries on M1 machines without having to compile the fyne-cross images, I have a binary available for download here: Jacalz/wormhole-gui#7 (comment) https://github.com/Jacalz/wormhole-gui/issues/7#issuecomment-792925169
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/fyne-io/fyne-cross/pull/34#issuecomment-792947198, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACI2FHR4OHUPQCPDW7QDOTTCUE6BANCNFSM4YYDOZ6A.

@Jacalz windows build is fixed in 1d0ae5d. The commit also introduce a dedicated docker image for windows, to test you need to run make windows first.
@zdima could you please run from the terminal the wormhole-gui.app/Contents/MacOS/warmhole-gui binary and report the error, if any ? Thanks!

@zdima could you please run from the terminal the wormhole-gui.app/Contents/MacOS/warmhole-gui binary and report the error, if any ? Thanks!

Actually I run it from the finder and from command line. It reports damaged file in both cases.

@zdima could you please run from the terminal the wormhole-gui.app/Contents/MacOS/warmhole-gui binary and report the error, if any ? Thanks!

Actually I run it from the finder and from command line. It reports damaged file in both cases.

I checked the system log file. Seems like MAC security is not allowing this command to run:
default	13:35:54.152787-0500	kernel	ASP: Security policy would not allow process: 4186, /Users/zdima/Downloads/wormhole-gui.app/Contents/MacOS/wormhole-gui
Is this process trying to access some protected data?

Another user reported a similar error on M1. https://github.com/Jacalz/wormhole-gui/issues/7#issuecomment-792975064

@zdima could you please run from the terminal the wormhole-gui.app/Contents/MacOS/warmhole-gui binary and report the error, if any ? Thanks!

Actually I run it from the finder and from command line. It reports damaged file in both cases.

I checked the system log file. Seems like MAC security is not allowing this command to run:
default 13:35:54.152787-0500 kernel ASP: Security policy would not allow process: 4186, /Users/zdima/Downloads/wormhole-gui.app/Contents/MacOS/wormhole-gui
Is this process trying to access some protected data?

It shouldn’t, as far as I know. We should probably try with something else than my project to be sure (fyne_demo perhaps). I just added a binary as I was compiling it to test fyne-cross.

@lucor If we are stopping to use the docker/golang-cross image for Darwin and Windows, should we maybe stop using it altogether? The Linux support shouldn’t be too hard to wire up as is and FreeBSD doesn’t really use the either way. Just a thought 🙂

Another user reported a similar error on M1. Jacalz/wormhole-gui#7 (comment)

The latest BigSur OS is very strict. You can't even exec ps command from the app.
I suspect the network layer may get restriction as well as I see many times security warnings like "app xyzy is trying access network share". A simple test app can confirm if this is the case.

@lucor If we are stopping to use the docker/golang-cross image for Darwin and Windows, should we maybe stop using it altogether? The Linux support shouldn’t be too hard to wire up as is and FreeBSD doesn’t really use the either way. Just a thought slightly_smiling_face

Updated the PR descrition, we already do this :) The dependency from docker/golang-cross hase been removed from the base image.

Another user reported a similar error on M1. Jacalz/wormhole-gui#7 (comment)

The latest BigSur OS is very strict. You can't even exec ps command from the app.
I suspect the network layer may get restriction as well as I see many times security warnings like "app xyzy is trying access network share". A simple test app can confirm if this is the case.

Agreed, test with a minimal "hello world app" would be great.

I ran make windows and got this error:
/bin/sh: -c: line 1: syntax error near unexpected token `('
/bin/sh: -c: line 1: `docker build -f /home/jacob/Downloads/fyne-io-fyne-cross-darwin-arm64 (1)/docker/base/Dockerfile -t fyneio/fyne-cross:base-latest .'
make: *** [Makefile:4: base] Error 2

Here is a test compile of the hello world example of https://github.com/fyne-io/fyne/blob/develop/cmd/hello/main.go:
fyne-hello-test-darwin-arm64.zip
If any M1 users want to check if it works.

Here is a test compile of the hello world example of https://github.com/fyne-io/fyne/blob/develop/cmd/hello/main.go:
fyne-hello-test-darwin-arm64.zip
If any M1 users want to check if it works.

No luck.
However the logs are different but yet related to security.

default	14:46:08.783782-0500	lsd	Non-fatal error enumerating at <private>, continuing: Error Domain=NSCocoaErrorDomain Code=260 "The file “PlugIns” couldn’t be opened because there is no such file." UserInfo={NSURL=PlugIns/ -- file:///Users/zdima/Downloads/Test.app/Contents/, NSFilePath=/Users/zdima/Downloads/Test.app/Contents/PlugIns, NSUnderlyingError=0x14fe22b90 {Error Domain=NSPOSIXErrorDomain Code=2 "No such file or directory"}}
default	14:46:08.783852-0500	lsd	- 45683955: Checking whether application is managed at file:///Users/zdima/Downloads/Test.app//io.fyne.hello
default	14:46:08.816973-0500	lsd	SecTranslocateCreateSecureDirectoryForURL: created /private/var/folders/0l/3mf0k_35553cjfstjtm5sgj40000gn/T/AppTranslocation/24A68833-65E1-4BEC-A05F-0E2A469487CD/d/Test.app
default	14:46:08.848562-0500	kernel	AMFI: '/private/var/folders/0l/3mf0k_35553cjfstjtm5sgj40000gn/T/AppTranslocation/24A68833-65E1-4BEC-A05F-0E2A469487CD/d/Test.app/Contents/MacOS/Test' has no CMS blob?
default	14:46:08.848581-0500	kernel	AMFI: '/private/var/folders/0l/3mf0k_35553cjfstjtm5sgj40000gn/T/AppTranslocation/24A68833-65E1-4BEC-A05F-0E2A469487CD/d/Test.app/Contents/MacOS/Test': Unrecoverable CT signature issue, bailing out.
default	14:46:08.859123-0500	tccd	AUTHREQ_ATTRIBUTION: msgID=139.35, attribution={accessing={identifier=a.out, pid=4421, auid=501, euid=501, binary_path=/private/var/folders/0l/3mf0k_35553cjfstjtm5sgj40000gn/T/AppTranslocation/24A68833-65E1-4BEC-A05F-0E2A469487CD/d/Test.app/Contents/MacOS/Test}, requesting={identifier=com.apple.syspolicyd, pid=139, auid=0, euid=0, binary_path=/usr/libexec/syspolicyd}, },
default	14:46:08.862232-0500	runningboardd	Acquiring assertion targeting [app<application.io.fyne.hello.7158211.7158215(501)>:4421] from originator [daemon<com.apple.coreservices.appleevents(55)>:309] with description <RBSAssertionDescriptor| "AE/Test" ID:162-309-2950 target:4421 attributes:[
	<RBSDomainAttribute| domain:"com.apple.launchservicesd" name:"LSNotification" sourceEnvironment:"(null)">
	]>
default	14:46:08.864725-0500	loginwindow	-[PersistentAppsSupport applicationReady:] | App: Test, ready, updating active tracking timer
default	14:46:08.864787-0500	loginwindow	-[ApplicationManager checkInAppContext:eventData:] | ApplicationManager: Checked in app : Test
default	14:46:08.904558-0500	lsd	Non-fatal error enumerating at <private>, continuing: Error Domain=NSCocoaErrorDomain Code=260 "The file “PlugIns” couldn’t be opened because there is no such file." UserInfo={NSURL=PlugIns/ -- file:///var/folders/0l/3mf0k_35553cjfstjtm5sgj40000gn/T/AppTranslocation/24A68833-65E1-4BEC-A05F-0E2A469487CD/d/Test.app/Contents/, NSFilePath=/var/folders/0l/3mf0k_35553cjfstjtm5sgj40000gn/T/AppTranslocation/24A68833-65E1-4BEC-A05F-0E2A469487CD/d/Test.app/Contents/PlugIns, NSUnderlyingError=0x152e0a770 {Error Domain=NSPOSIXErrorDomain Code=2 "No such file or directory"}}
default	14:46:08.904797-0500	lsd	- 45683955: Checking whether application is managed at file:///var/folders/0l/3mf0k_35553cjfstjtm5sgj40000gn/T/AppTranslocation/24A68833-65E1-4BEC-A05F-0E2A469487CD/d/Test.app//io.fyne.hello
default	14:46:08.921931-0500	lsd	Non-fatal error enumerating at <private>, continuing: Error Domain=NSCocoaErrorDomain Code=260 "The file “PlugIns” couldn’t be opened because there is no such file." UserInfo={NSURL=PlugIns/ -- file:///var/folders/0l/3mf0k_35553cjfstjtm5sgj40000gn/T/AppTranslocation/24A68833-65E1-4BEC-A05F-0E2A469487CD/d/Test.app/Contents/, NSFilePath=/var/folders/0l/3mf0k_35553cjfstjtm5sgj40000gn/T/AppTranslocation/24A68833-65E1-4BEC-A05F-0E2A469487CD/d/Test.app/Contents/PlugIns, NSUnderlyingError=0x154046810 {Error Domain=NSPOSIXErrorDomain Code=2 "No such file or directory"}}
default	14:46:08.922158-0500	lsd	- 45683955: Checking whether application is managed at file:///var/folders/0l/3mf0k_35553cjfstjtm5sgj40000gn/T/AppTranslocation/24A68833-65E1-4BEC-A05F-0E2A469487CD/d/Test.app//io.fyne.hello
default	14:46:08.924581-0500	tccd	-[TCCDAccessIdentity staticCode]: static code for: identifier io.fyne.hello, type: 0: 0x10a812ec0 at /var/folders/0l/3mf0k_35553cjfstjtm5sgj40000gn/T/AppTranslocation/24A68833-65E1-4BEC-A05F-0E2A469487CD/d/Test.app
default	14:46:08.967648-0500	trustd	could not enable test hierarchy: no UAT pinning preferences set
default	14:46:08.988074-0500	trustd	could not enable test hierarchy: no UAT pinning preferences set
default	14:46:11.163599-0500	kernel	ASP: Security policy would not allow process: 4421, /Users/zdima/Downloads/Test.app/Contents/MacOS/Test

Looks like the binaries require some sort of signing to be openable. I think I’ve read about that somewhere before. That might need to be hooked up into fyne-cross, I recon (if it isn’t already and I just didn’t sign it).

Looks like the binaries require some sort of signing to be openable. I think I’ve read about that somewhere before. That might need to be hooked up into fyne-cross, I recon (if it isn’t already and I just didn’t sign it).

Was the codesign command executed as part of the packaging?
I saw it was executed when Fyne package command run for my app.

Looks like the binaries require some sort of signing to be openable. I think I’ve read about that somewhere before. That might need to be hooked up into fyne-cross, I recon (if it isn’t already and I just didn’t sign it).

Was the codesign command executed as part of the packaging?
I saw it was executed when Fyne package command run for my app.

It might have, I’m honestly not sure. All I did was do fyne-cross darwin -arch arm64. If it doesn’t sign it, it probably should do so.

Yep, it looks like need further investigation :(
Btw it seems that osxcross was already signing the binary, if so latest commit is not needed.
@zdima or any other M1 user, just to retrieve further info could you please  run codesign -dv --verbose=4 <binary_name>
against the binaries from the following ?

https://github.com/fyne-io/fyne-cross/files/6103732/fyne-hello-test-darwin-arm64.zip
https://github.com/fyne-io/fyne-cross/files/6111293/fyne-cross-test2-darwin-arm64.zip
the binary obtained from the compilation of https://github.com/fyne-io/fyne/blob/develop/cmd/hello/main.go on the M1

@zdima or any other M1 user, just to retrieve further info could you please run codesign -dv --verbose=4 <binary_name>
against the binaries from the following ?

https://github.com/fyne-io/fyne-cross/files/6103732/fyne-hello-test-darwin-arm64.zip
$ codesign -dv --verbose=4 ./Test.app/Contents/MacOS/Test
Executable=/Users/zdima/Downloads/Test.app/Contents/MacOS/Test
Identifier=a.out
Format=app bundle with Mach-O thin (arm64)
CodeDirectory v=20400 size=74366 flags=0x20002(adhoc,linker-signed) hashes=2321+0 location=embedded
VersionPlatform=1
VersionMin=720896
VersionSDK=721152
Hash type=sha256 size=32
CandidateCDHash sha256=1513bceff1161c17a06bd0ae30d651a04fb13d6c
CandidateCDHashFull sha256=1513bceff1161c17a06bd0ae30d651a04fb13d6c1ea3aaa51469e9546f0ff564
Hash choices=sha256
CMSDigest=1513bceff1161c17a06bd0ae30d651a04fb13d6c1ea3aaa51469e9546f0ff564
CMSDigestType=2
Executable Segment base=0
Executable Segment limit=3309568
Executable Segment flags=0x1
Page size=4096
CDHash=1513bceff1161c17a06bd0ae30d651a04fb13d6c
Signature=adhoc
Info.plist=not bound
TeamIdentifier=not set
Sealed Resources=none
Internal requirements=none

https://github.com/fyne-io/fyne-cross/files/6111293/fyne-cross-test2-darwin-arm64.zip
$ codesign -dv --verbose=4 ./Test.app/Contents/MacOS/Test
Executable=/Users/zdima/Downloads/Test.app/Contents/MacOS/Test
Identifier=Test
Format=app bundle with Mach-O thin (arm64)
CodeDirectory v=20400 size=74429 flags=0x0(none) hashes=2321+2 location=embedded
VersionPlatform=1
VersionMin=720896
VersionSDK=721152
Hash type=sha256 size=32
CandidateCDHash sha1=067e57cc2e8181fccda214f4da6859252601fc54
CandidateCDHashFull sha1=067e57cc2e8181fccda214f4da6859252601fc54
CandidateCDHash sha256=38eb332d398446daacc4b3e0c52e35e41fb7e410
CandidateCDHashFull sha256=38eb332d398446daacc4b3e0c52e35e41fb7e4106964a0d3e0a860790641529a
Hash choices=sha1,sha256
CMSDigest=1c68ce4b88d818b8db1a91328f90fe7b2b110a7d15b0db51fa0adb6d6eb9c7b6
CMSDigestType=2
Page size=4096
CDHash=38eb332d398446daacc4b3e0c52e35e41fb7e410
./Test.app/Contents/MacOS/Test: no signature
Info.plist=not bound
TeamIdentifier=not set
Sealed Resources=none
Internal requirements count=1 size=120

the binary obtained from the compilation of https://github.com/fyne-io/fyne/blob/develop/cmd/hello/main.go on the M1
$ codesign -dv --verbose=4 ./test
Executable=/System/Volumes/Data/Network/Servers/mars.local.cloud/mnt/sweep/workfolder/zdima/test/test
Identifier=a.out
Format=Mach-O thin (arm64)
CodeDirectory v=20400 size=93438 flags=0x20002(adhoc,linker-signed) hashes=2917+0 location=embedded
VersionPlatform=1
VersionMin=720896
VersionSDK=721152
Hash type=sha256 size=32
CandidateCDHash sha256=7790214727ea9fd0516017e0fa05701a8bdd51db
CandidateCDHashFull sha256=7790214727ea9fd0516017e0fa05701a8bdd51db96eac56a771163dcd43c6031
Hash choices=sha256
CMSDigest=7790214727ea9fd0516017e0fa05701a8bdd51db96eac56a771163dcd43c6031
CMSDigestType=2
Executable Segment base=0
Executable Segment limit=3112960
Executable Segment flags=0x1
Page size=4096
CDHash=7790214727ea9fd0516017e0fa05701a8bdd51db
Signature=adhoc
Info.plist=not bound
TeamIdentifier=not set
Sealed Resources=none
Internal requirements=none

Came we get Go upgraded to 1.16.1 while we’re at it? It includes some rather important security fixes https://mobile.twitter.com/golang/status/1369690242681278464.

Updated in 27c55af

Updated to Go v1.16.2 in c95deca

Root cause could be the wrong macos min version used to compile for arm64, updated in 0d36b99
Feedback using that commit for fyne-cross and the latest docker image with 1.16.2 would be great.
If possible please test using  the --no-string-debug flag too:

fyne-cross darwin --app-id com.example.test --arch arm64 --no-cache
fyne-cross darwin --app-id com.example.test --arch arm64 --no-cache --no-strip-debug

Here are two new M1 test applications for testing. One is without debug and the other is with debug symbols:
fyne-cross-darwin-arm64-test-no-debug.zip
fyne-cross-darwin-arm64-test-with-debug.zip

Things are looking much better now. I am getting the same deprecation warnings as https://github.com/fyne-io/fyne/issues/1833, which I take as a good sign that the binaries are correctly built :slightly_smiling_face:

Here are two new M1 test applications for testing. One is without debug and the other is with debug symbols:
fyne-cross-darwin-arm64-test-no-debug.zip
fyne-cross-darwin-arm64-test-with-debug.zip
A bit different this time:
default	08:10:50.334913-0500	kernel	ASP: Security policy would not allow process: 12374, /Users/zdima/Downloads/Test-2.app/Contents/MacOS/Test

Are these binaries signed with developer profile?
There could be a latest OS restriction to not run any app unless there is a developer profile behind it.
These are only options OS allowed, and I assume app will not run unless there is a signature that can be verified.

Check this article. Perhaps adding entitlement  com.apple.developer.endpoint-security.client will solve security issue.
https://developer.apple.com/forums/thread/666123

Oh, great! So it looks like is working :-)
With regard to the warning above, this is expected. Go is using an ad-hoc signature, so unless you sign again the binary with a cert with a vality identity it cannot pass through Gatekeeper (see https://github.com/golang/go/issues/42684#issuecomment-730037844 and the official docs https://developer.apple.com/documentation/macos-release-notes/macos-big-sur-11_0_1-universal-apps-release-notes#Code-Signing for details)
Details on how open a developer-signed or notarized app can be found on the apple support site: https://support.apple.com/en-us/HT202491
The plan is to found a way to sign the binary using a valid cartificate directly from linux, but I'd say that this is another issue.
On macOS you would sing using a developer certificate using either the fyne release command or directly the codesign tool.
@zdima could you please confirm if it runs after allowing the execution from an unidentified developer?

That is the thing. Latest Big Sur don't have an option to run the app from unidentified source. I am not getting "Open anyway" option and can't find if I can enable it.

Did you tried by any chance the following from the macOS user guide: https://support.apple.com/guide/mac-help/open-a-mac-app-from-an-unidentified-developer-mh40616/mac ?

To override your security settings and open the app anyway follow these steps:

In the Finder on your Mac, locate the app you want to open.
Don’t use Launchpad to do this. Launchpad doesn’t allow you to access the shortcut menu.
Control-click the app icon, then choose Open from the shortcut menu.
Click Open.
The app is saved as an exception to your security settings, and you can open it in the future by double-clicking it just as you can any registered app.

Note: You can also grant an exception for a blocked app by clicking the Open Anyway button in the General pane of Security & Privacy preferences. This button is available for about an hour after you try to open the app.
To change these preferences on your Mac, choose Apple menu  > System Preferences, click Security & Privacy, then click General.

Yes. I did. Multiple times.
I think without developer certificate this not going to work. At least on M1 with latest OS.
Do we know if it works on Mac with x86-64?
I believe the developer certificate can be obtained for free once you register as developer on apple dev site. Unless you want to publish the app in the store, no payment require.

I found this article https://www.kencochrane.com/2020/08/01/build-and-sign-golang-binaries-for-macos-with-github-actions/. It might be kind of useful. However, it still looks like you need a paid developer account to be able to get signing certificates https://developer.apple.com/support/compare-memberships.

Yes. I did. Multiple times.
I think without developer certificate this not going to work. At least on M1 with latest OS.

It could be, but it is hard to debug without a physical access to an M1 :-(
@zdima To check if the if the signature is valid could you please run: codesign -v -vvvv —ignore-resources <path/to/binary> on both binaries above ?

Do we know if it works on Mac with x86-64?

Yes, it works as reported by @Jacalz  above. The restriction applies only to latest macOS on M1

Do we know if it works on Mac with x86-64?

Yes, it works as reported by @Jacalz above. The restriction applies only to latest macOS on M1

Though, we should keep in mind that it was tested on MacOS 10.13. Someone running Big Sur on an amd64 machine should probably test as well. I think @andydotxyz has one?

@zdima To check if the if the signature is valid could you please run: codesign -v -vvvv —ignore-resources <path/to/binary> on both binaries above ?

This is output:
/Applications/Test-2.app/Contents/MacOS/Test: valid on disk (not all contents verified)
/Applications/Test-2.app/Contents/MacOS/Test: satisfies its Designated Requirement```

@zdima To check if the if the signature is valid could you please run: codesign -v -vvvv —ignore-resources <path/to/binary> on both binaries above ?

This is output:
/Applications/Test-2.app/Contents/MacOS/Test: valid on disk (not all contents verified)
/Applications/Test-2.app/Contents/MacOS/Test: satisfies its Designated Requirement

Thanks @zdima. This is weird...  as per the official doc the signature is valid for the policy:

To check the validity of a signature, run codesign -v -vvvv —ignore-resources . If the command indicates “valid on disk (not all contents verified)” and “satisfies its Designated Requirement”, the code is valid under this policy.

Let's try o recap where we are so far:

fyne application is cross compiled  for arm64 using Go 1.16.2 against the SDK 1.11
the application is signed using an ad hoc certificate by Go
the signature is valid when checked with codesign
the application is downloaded from internet
fyne apps built natively on arm64 are known to work out of the box.
cross-compiled app runs on macOS 10.13 amd64, we expect it should also run on macOS 11.x amd64 since codesigning is required only on arm64.

Issue: we are unable to run on macOS 11.x arm64 a cross-compiled app with fyne-cross on macOS amd64 due to Gatekeeper policy issues. We have not found a way yet to open an app from an unidentiefied developer despite the doc says it should be possible.
Things comes in mind to debug the issue above:

build a fyne app natively on arm64 -> upload somewhere -> download and run on arm64
build a fyne app with fyne-cross on arm64 for arm64 -> run locally
check additional steps that would allow installation of apps from unidentiefied developer (and downloaded from internet ?)

Missed something ? Any additional ideas ?

Though, we should keep in mind that it was tested on MacOS 10.13. Someone running Big Sur on an amd64 machine should probably test as well. I think @andydotxyz has one?

Current build tools work on BigSur for x86_64 yes - no signing required (though you will get warnings if the file was downloaded without signing).

Obviously on an x86_64 BigSur those built M1 binaries won't run.
However I ran fyne-cross to build for darwin and that ran on BigSur for me (x86_64).

Thanks @lucor. That works now :+1:

Here is a new M1 test applications for testing: test.zip
It tries a possible fix for the linker.


Office Hours - 19/12/2024

Date
December 19th - 2024 - 15:30 GMT
Meeting notices

FINOS Project leads are responsible for observing the FINOS guidelines for running project meetings. Project maintainers can find additional resources in the FINOS Maintainers Cheatsheet.

All participants in FINOS project meetings are subject to the LF Antitrust Policy, the FINOS Community Code of Conduct and all other FINOS policies.

FINOS meetings involve participation by industry competitors, and it is the intention of FINOS and the Linux Foundation to conduct all of its activities in accordance with applicable antitrust and competition laws. It is therefore extremely important that attendees adhere to meeting agendas, and be aware of, and not participate in, any activities that are prohibited under applicable US state, federal or foreign antitrust and competition laws. Please contact<EMAIL_ADDRESS>with any questions.

FINOS project meetings may be recorded for use solely by the FINOS team for administration purposes. In very limited instances, and with explicit approval, recordings may be made more widely available.

Agenda
Office Hours is a meeting for maintainers to get together to discuss active development and upcoming priorities.

[ ] Open floor to discuss on-going issues/demos.
[ ] @yt-ms to give brief demo of Flox
[x] @karlmoll - any updates on the FINOS automated attendee Github action?

Zoom info
Join Zoom Meeting
https://zoom.us/j/95349776375
Meeting ID: 953 4977 6375
Passcode: 373428

Ross Maden - Morgan Stanley

Matthew Auburn - Morgan Stanley

Yan Tordoff /  Morgan Stanley

Leigh Finegold / Morgan Stanley

Karl Moll / FINOS

Denis Coffaro / Morgan Stanley

Luigi Bulanti / Morgan Stanley

Aidan McPhelim / Morgan Stanley

Graham Packer / Morgan Stanley

Jim Gough @jpgough-ms


releasenotes: fix opengl.extraPackages option name

Things done

Built on platform(s)

[x] NixOS
[ ] OS X
[ ] Linux

[x] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[x] Fits CONTRIBUTING.md.

Should be merged into master and release-16.03 I guess

Pushed to master (31fa7eda70bc2388014bd606a94582863da83bf3) and release-16.03 (e2deaf77c18d23f4877dc3e8b8cd8c9df4fa7cc3). Thanks!


Domain resolution always fails on the second use

The following Crystal code fails for me:
require "socket"

p TCPSocket.new("google.com", 80)
p TCPSocket.new("google.com", 80)

It always raises on the second connection:
#<TCPSocket:fd 12>
Error connecting to 'google.com:80': Network is unreachable (Errno)
[4364583] *CallStack::unwind:Array(Pointer(Void)) +87
[4364474] *CallStack#initialize<CallStack>:Array(Pointer(Void)) +10
[4364426] *CallStack::new:CallStack +42
[4372071] *Exception +39
[4371992] *Errno#initialize<Errno, String>:CallStack +120
[4371841] *Errno::new<String>:Errno +113
[4414852] *TCPSocket +324
[4411046] *TCPSocket#initialize<TCPSocket, String, Int32, Nil, Nil>:Nil +1206
[4409816] *TCPSocket::new<String, Int32, Nil, Nil>:TCPSocket +184
[4409627] *TCPSocket::new<String, Int32>:TCPSocket +27
[4341583] ???
[4349481] main +41
[139866884900224] __libc_start_main +240
[4339577] _start +41
[0] ???

I'm having a hard time trying to understand why this happens. The exactly equal ruby code works as expected. And the error is reproduced every time I run: The first works, the second fails. The domain name might be relevant, with some others the error changes to getaddrinfo: Non-recoverable failure in name resolution (Socket::Error), but I found no pair of domains that work.
The odd thing is that I was unable to reproduce it on other machines. But still looks to be a crystal issue because every other program works and I don't think I have any fancy configuration.
I'm running Linux, Fedora 23, 64-bit.

Similar to #2562, The ruby code takes 0.137 seconds to work, and the crystal version takes 6.054 seconds to fail. Consistent with the 3 seconds for one connection reported there.

@lbguilherme it works for me without issues:
$ crystal --version
Crystal 0.16.0 [fc89c1a] (Thu May  5 16:38:39 UTC 2016)
$ crystal 6.cr
#<TCPSocket:fd 9>
#<TCPSocket:fd 10>

Most of these issues are network related issues, perhaps caused by cached DNS entries in your machine or your local network and sometimes fail to respond.
Any more details that you can provide will be appreciated.

Since this happens on a single machine, and can't be reproduced elsewhere makes it complicated to help debug.
We use libevent's DNS resolver (evented), not getaddrinfo or gethostbyname syscalls that are blocking the whole process. This explains why other languages are unaffected: Ruby and Curl both use getaddrinfo. Maybe the installed libevent has a bug that is triggered with your network or OS configuration?
There has been some discussions (in another related issue) to use alternative, but not portable, syscalls like getaddrinfo_r that are non blocking, but this will require non trivial and per target work (Linux, Darwin), and sometimes there are no such alternatives (FreeBSD).

Closing in favor of #2660


Steps to Reproduce

helm repo add nsm https://helm.nsm.dev/
helm install nsm nsm/nsm -n nsm

Context

Kubernetes Version: 1.17
Helm version: 3.2.0

Failure Logs
$ kubectl get pod

nsm-admission-webhook-6cb9df6d7d-mc2mv        1/1     Running            0          13m
nsm-vpp-forwarder-2bd2x                       0/1     Running            0          13m
nsm-vpp-forwarder-mcks8                       0/1     Running            0          13m
nsmgr-drdfp                                   2/3     CrashLoopBackOff   7          13m
nsmgr-rlkxh                                   2/3     CrashLoopBackOff   7          13m

$ kubectl describe pod nsmgr-drdfp
...
Events:
  Type     Reason     Age                  From               Message
  ----     ------     ----                 ----               -------
  Normal   Scheduled  <unknown>            default-scheduler  Successfully assigned nsm/nsmgr-drdfp to worker2
  Normal   Pulled     16m                  kubelet, worker2   Container image "docker.io/networkservicemesh/nsmdp:v0.2.0" already present on machine
  Normal   Created    16m                  kubelet, worker2   Created container nsmdp
  Normal   Started    16m                  kubelet, worker2   Started container nsmdp
  Normal   Started    16m                  kubelet, worker2   Started container nsmd-k8s
  Normal   Created    16m                  kubelet, worker2   Created container nsmd-k8s
  Normal   Pulled     16m                  kubelet, worker2   Container image "docker.io/networkservicemesh/nsmd-k8s:v0.2.0" already present on machine
  Normal   Killing    16m                  kubelet, worker2   Container nsmd failed liveness probe, will be restarted
  Normal   Started    15m (x2 over 16m)    kubelet, worker2   Started container nsmd
  Normal   Created    15m (x2 over 16m)    kubelet, worker2   Created container nsmd
  Warning  Unhealthy  15m (x5 over 16m)    kubelet, worker2   Liveness probe failed: Get http://<IP_ADDRESS>:5555/liveness: dial tcp <IP_ADDRESS>:5555: connect: connection refused
  Warning  Unhealthy  15m (x7 over 16m)    kubelet, worker2   Readiness probe failed: Get http://<IP_ADDRESS>:5555/readiness: dial tcp <IP_ADDRESS>:5555: connect: connection refused
  Normal   Pulled     11m (x6 over 16m)    kubelet, worker2   Container image "docker.io/networkservicemesh/nsmd:v0.2.0" already present on machine
  Warning  BackOff    101s (x35 over 10m)  kubelet, worker2   Back-off restarting failed container

$ kubectl describe pod nsm-vpp-forwarder-2bd2x
...
Events:
  Type     Reason     Age                   From               Message
  ----     ------     ----                  ----               -------
  Normal   Scheduled  <unknown>             default-scheduler  Successfully assigned nsm/nsm-vpp-forwarder-2bd2x to worker2
  Normal   Pulled     18m                   kubelet, worker2   Container image "docker.io/networkservicemesh/vppagent-forwarder:v0.2.0" already present on machine
  Normal   Created    18m                   kubelet, worker2   Created container vppagent-forwarder
  Normal   Started    18m                   kubelet, worker2   Started container vppagent-forwarder
  Warning  Unhealthy  3m15s (x90 over 18m)  kubelet, worker2   Readiness probe failed: HTTP probe failed with statuscode: 503

Project currently requires helm 2.0, as I learned the hard way.

@qlyoung #2075 has been merge :)  We should now support helm 3 :)

Thank you for helping.
But I don't think this error is related to Helm version. Looking at pod logs the spire-agent was unable to talk to spire-server due to network reasons, specifically the spire-agent was unable to resolve the name "spire-server", ending with
nc: bad address 'spire-server'
For what I saw, the spire-agent has this problem because it is using
hostNetwork: true;  following this https://github.com/kubernetes/kubernetes/issues/56934 I installed Calico, and now it seems to be working

Yes, thanks


templatetags caching fix

fix for #19

Looks like this broke the build. Please resubmit with a passing build. Thank you!

Fixed as of 266240250a


C# Support

Many thanks for this functionality.
Many have been waiting for this since 2016 =]
https://github.com/microsoft/vscode/issues/16221
As far as I understand, support for С# is planned?
Thank You !
Regards,
Anton.

Yes, its planned to be released

still waiting 😅

Also still waiting ;-;

Hi I'm busy with implementing tree sitter, when this is finished, C# will be added.


Add Packages to Pro API, and a few Typing Enhancements

Pro Package API

Add Pro API Package model.
Add ProApi.get_packages_v1() with overloads.
Add allowed sort & filter fields for Package request.
Update documentation for Package model.
Add integration tests for Package model.

Typing Enhancements

Add files parameter to pro_api_request() (create policies with icons!)
Add overloads for get_computer_inventory_v1().
Resolve type-checking issues when package is installed as editable.
Resolve several other miscellaneous type-checking issues.

Can you discuss the inclusion of overloads? I'm not seeing the value they're bringing to the two interfaces you've implemented them with.

Sure!  It allows the type checker to infer what will be returned by the method, based on whether or not pagination is being used.  This doesn't do much within the API itself, but it improves the ergonomics of code that consumes the SDK.

Sure! It allows the type checker to infer what will be returned by the method, based on whether or not pagination is being used. This doesn't do much within the API itself, but it improves the ergonomics of code that consumes the SDK.

Not two minutes after I sent that did it click that it made the return type deterministic.
...I am not against this. I'd want to get it added to the remaining Pro API methods though and included in the dev docs as a part of the style guide.

This is an overall very solid PR with good enhancements. 🎉

I'd want to get [overloads] added to the remaining Pro API methods though and included in the dev docs as a part of the style guide.

Overloads added.  Haven't updated the style guide yet.

Haven't updated the style guide yet.

Never mind, went ahead and did that as well.  Also added a recommendation for Optional[] usage.


Lumen support

This PR fixes several issues, that i found working with Lumen.

Lumen don't have cache() helper.
This issue.

Thanks @nope7777 for submitting this PR! I will look into it as soon as possible -- that being said it might be a few weeks before it can be integrated.

@mikebronner , great, thanks.


zipfile builder does not account for the .conversion convention in the data directory

I found a problem that I have not anticipated.
Some entries e.g. /mcfs/data/materialscommons/9a/fe/1de0f87b-9afe-418e-a88f-d624eefd6b2b
are simple, the contents of the file. However, other entries seem to have a different structure.
For example, 465f4b22-9a51-456f-93c3-afe5b8a0624c does not exist at /mcfs/data/materialscommons/9a/51/
but there is a file in /mcfs/data/materialscommons/9a/51/.conversion/ called 465f4b22-9a51-456f-93c3-afe5b8a0624c.jpg
So, I assume that my tar task was underspecified. That is, I made the assumption that given a file id, like
465f4b22-9a51-456f-93c3-afe5b8a0624c, I need only use the 4 digits of the second part to find the file.
That is, in this case, I could tar up the file in /9a/51/465f4b22-9a51-456f-93c3-afe5b8a0624c using
the file name from the datafile record and my job was done.
However, it appears, in this case, that I must first check for the existence of /9a/51/.conversion and determine
if a file with an unknown extension and a filename of 465f4b22-9a51-456f-93c3-afe5b8a0624c exists in that directory
and (if it does), check the extension against the filename (from the record) and if all is in agreement, copy the file
to the tar stream.

Not an issue


How to test Quota Monitor once deployed

What were you initially searching for in the docs?
How to test that the Quota Monitor is working once deployed

Is this related to an existing part of the documentation? Please share a link
No
Describe how we could make it clearer
It would be nice to have specific steps that lay out how to test the service, especially a method to test it that doesn't involve spending more money to saturate the quotas
If you have a proposed update, please share it here
One way might be to provide low quota limits that the service can be switched to temporarily to help during the testing phase. This is more of a feature request than documentation but that is one approach that might work.

Hi,
We have planned our next release and plan on including something similar in the next release, our plan is slightly different in that we were going to provide a method to simulate a test event so that it can be confirmed that the system, including the notification process works, this would make it so that arbitrary resources don't need to be created, and as you said, money does not need to be spent, in order to test that the solution works. We do not have an exact date for the release at the moment but will keep you up to date as best we can.


Best way to handle MCP23017-pins with LED-machine?

I would like to use an led-machine (only EVT_ON and EVT_OFF needed) with 16 pins controlled through mcp23017. I'm using Adafruit-MCP23017 library:
#include "Adafruit_MCP23017.h"
Adafruit_MCP23017 mcp1;

// in setup:
mcp1.begin( 1 );
  for ( int i=0; i<=15;i++ ){
    mcp1.pinMode( i, OUTPUT );
}

// somewhere in loop:
mcp1.digitalWrite( 6, HIGH);

I'm thinking about building an own machine derived from led-machine to handle different way of initializing and writing to pins. But before I start: is there a better way to achieve this?
gTom

First you need to be clear about what you want to achieve. If you just want to switch the individual leds on and off without a time element involved (like with blinking or fading) then it's questionable that you need a state machine at all.
It might be nice to have an Automaton-like interface to the leds, but you could achieve that even with a dummy state table. The machine would basically be sleeping all the time.
const static state_t state_table[] PROGMEM = {
   /*               ON_ENTER      ON_LOOP  ON_EXIT  ELSE */
   /*    IDLE */          -1,   ATM_SLEEP,      -1, IDLE,
};

You could then create an interface like this:
mcp1.begin( 1 );
mcp1.trigger( EVT_LED1_ON );
mcp1.trigger( EVT_LED2_ON );
mcp1.trigger( EVT_LED2_ON );
mcp1.on( 1 );
mcp1.off( 1 );

By just implementing your own trigger(), on() and off() methods.

Thanks, I will think about it. When I need some timing within this, I think I will try to setup a new machine.
gTom


single vs. double precision

in _check_input we're currently converting to float32 due to the PyTorch models. Maybe better to use double precision where possible and single precision only where necessary, e.g. for deep learning models / models using GPU's. Thanks @edaub.

closing as PyTorch requires float32


feat: horizontally align lightbox images

Very long vertical images used to be on the very left of the screen. With this change they are now centered.

👍

This branch has conflicts that must be resolved

@demedos 😂

closing since it has been added in v0.9.0


Doc

Codecov Report

Merging #34 (ac42e47) into dev (5124d65) will decrease coverage by 0.05%.
The diff coverage is 83.33%.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
@@            Coverage Diff             @@
##              dev      #34      +/-   ##
==========================================
- Coverage   62.62%   62.57%   -0.05%
==========================================
  Files          60       60
  Lines        1343     1336       -7
==========================================
- Hits          841      836       -5
+ Misses        502      500       -2

Impacted Files
Coverage Δ

src/Flandre.Core/Messaging/MessageContext.cs
87.50% <ø> (ø)

src/Flandre.Core/Utils/StringParser.cs
85.89% <ø> (ø)

...rc/Flandre.Framework/Attributes/OptionAttribute.cs
100.00% <ø> (ø)

src/Flandre.Framework/Common/CommandContext.cs
100.00% <ø> (ø)

src/Flandre.Framework/Common/MiddlewareContext.cs
100.00% <ø> (ø)

src/Flandre.Framework/Common/Plugin.cs
55.55% <ø> (ø)

...rc/Flandre.Framework/Events/CommandInvokedEvent.cs
53.33% <ø> (ø)

...andre.Framework/Extensions/FlandreAppExtensions.cs
0.00% <ø> (ø)

...ramework/Extensions/ServiceCollectionExtensions.cs
0.00% <ø> (ø)

.../Flandre.Framework/Extensions/SessionExtensions.cs
100.00% <ø> (ø)

... and 9 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Dose this repo has been abandoned?

We can see many open issue submitted at a long time ago there,and the commit activity ended at Mar 2021.

Good question, if so:
https://docs.gunicorn.org/en/latest/faq.html?highlight=Hey#how-might-i-test-a-proxy-configuration
Let's ping gunicorn.

I wonder what does this repo do and/or what are the highlights of this tool ? For instance comparison to ab or k6

@imcom well, for starters, ab benchmark tools is pretty shit when simulating a lossy network or firewall application since HTTP request can be timed out (3-way handshake blocked, lossy network, ...) and the ab benchmark tools refuse to give any statistics if that happens.

it's 2023 now and still no response @phriscage you were the last committer. Can you tell us if this project is dead?

This is too good of a tool to let die!

Ping ?
This tool is great, damn simple and efficient. Take me 3 min to build from scratch and 10sec to understand how to use will not even a Golang people !
Maybe an official fork or good replacement suggestion ?

Check out https://github.com/hatoo/oha


Make stunner-auth replicas configurable

As far as I know, it should be safe to run more than one replica of stunner-auth, but unfortunately we can't currently configure that with the helm chart.
Currently, it's hard coded to 1:
https://github.com/l7mp/stunner-helm/blob/9b1c55349d60f654ea36fb042bac6212f189a693/helm/stunner-gateway-operator/templates/stunner-auth-service.yaml#L15
Happy to make a PR if there is nothing I'm missing w.r.t. running more than one replica.

Thx, this seems useful. Pls submit a PR, happy to merge.


[android] Extend validity of the certificate

For uploading a apk to Google Play a minimun of 25 years (9125 days) is needed for the validity of the key.
More info:
https://developer.android.com/studio/publish/app-signing
Pending to improve the makefiles: the actual makefiles for android in raylib generate a new key in a new keystore each time it is compiled. This is not valid for Google Play, they request to use always the same keysotre.

Thanks @danimartin82! Actually, the key for GooglePlay needs to be configured with the credentials of the user. The generated key is just for testing.