Improvement in figure_6_3_TD_methods_performance.py

Hello!
I think I found a subtle mistake in figure_6_3_TD_methods_performance.py
In line 19, get_expectation has a subtle mistake.
That part is about expected SARSA.
I assume action value of any action should be multiplied by epsilon / (nA)
The original code is
def get_expectation(Q: np.ndarray, state: int, epsilon: float):
    # Non greedy actions
    result = (Q[state] * epsilon).sum()
    # Greedy action
    result += Q[state].max() * (1 - epsilon)
    return result

And I think we can change like this
def get_expectation(Q: np.ndarray, state: int, epsilon: float):
    nA = Q.shape[-1]
    # Any action
    result = Q[state].sum() * (epsilon / nA)
    # Greedy action
    result += Q[state].max() * (1 - epsilon)
    return result

p.s. Thank you so much for sharing your code!! Your code helped me a lot in my studies!

Hi, I appreciate you pointing this issue out, and I believe you are absolutely correct. Would you mind creating a PR so that I can merge your change? Thanks!


Fullscreen & Second Monitor

OS: Mac OS X 10.11.6
--
Issue Descriptions: The app does not Resume. See User Interface Preservation on The Core App Design.
When quitting the application on Mac, it does not appear to use the Resume functionality (or custom window sizing, window state of fullscreen, display window was on restoration).
I have a second monitor and definitely use the app on the second screen in Fullscreen ⌘ + ⌃ + F all the time. The app is never out of Fullscreen and on my main iMac display.
While it's a pain to always have to move the app over and put it in Fullscreen it's not the biggest deal if it can't automatically do this. I do wonder if this is an issue with the site-to-native-app framework being used. Meaning it's not directly GPMDP's fault.
With that, is it possible to add Mac's native Resume functionality for managing the player window upon quit and launch?
--
Steps to Reproduce:
(This requires a second display and a macOS device)

Launch GPMDP
Move the main player window to the second display
Enter Mac's native fullscreen mode (Using Mission Control, maximize button, ⌘ + ⌃ + F, Menu Bar > View > Toggle Fullscreen)
Quit the application
Observe that the app "Resumes" on the main display and not in Fullscreen

Size, location and screen are all persisted by GPMDP. The only thing we don't persist is full screen state as I didn't feel it was appropriate (other apps didn't appear to do it).
The follow up issue you are experiencing is that when in fullscreen the X, Y coords and size are "invalid" for normal usage so are reset on relaunch of the app.
I can look into supporting persisted full screen state but I still think the UX will be janky at best.

So I just had a go at persisting the full screen state and I stand by my initial guess that it would be janky UX.  I personally don't like it doing it, and I don't think others will either.
Although this experience is sub-optimal there isn't much I can do about it, newer versions of macOS appear to report fullscreen coords a little more logically though so if you can update your macOS you should see a small improvement in window position restoration. 👍

That is not correct. Which is why I linked to the documentation. The Restore functionality is for both window coordinates and window state, including Fullscreen.
I don't know what janky UX means. I do know that every single app I use on my computer is in Fullscreen. Apps that don't do this I do submit to the developer to support Resume correctly.
If this is an issue for you, then please add an option to turn it off for yourself, but leave Resume on by default as that is the expected behavior on any Mac version.
I see no reason to not implement the correct behavior of a Mac application even if it's "janky." Even the dying Chrome apps opened as windows as their own application support Resume for display, position, and Fullscreen.

As it currently is, an update recently broke the UX. So I see no reason why not to have Fullscreen support when the app is already broken.
The window chrome at the top is not displayed anymore and it shows a black bar at the bottom.
When the UI refreshes on the homepage, it's blaring white and not the dark theme.
When the app is in dark mode—albums with light backgrounds don't show the appropriate contrasting overlay to make text readable.
So, janky UX is already a thing as of the current state. I can ignore those above, but the Fullscreen thing is starting to bug me as I am tired of having to move it over when it's just a music app; no reason to have a complicated window system. I know it's a free app, but I might as well not use it since the Fullscreen thing is breaking it entirely to show up on the wrong display anyways.

That is not correct. Which is why I linked to the documentation. The Restore functionality is for both window coordinates and window state, including Fullscreen. As seen in apps that quit in Fullscreen, will Resume in Fullscreen.

That certainly isn't the case in several of the apps I tested, it appears to be developer prerogative as to whether or not the fullscreen state is restored. Also the documentation is for macOS apps only and specifically targets macOS native API's.  This change would affect all platforms.

The window chrome at the top is not displayed anymore and it shows a black bar at the bottom.

Fixed in master

When the UI refreshes on the homepage, it's blaring white and not the dark theme.

Fixed in master

When the app is in dark mode—albums with light backgrounds don't show the appropriate contrasting overlay to make text readable.

Fixed in master

I might as well not use it since the Fullscreen thing is breaking it entirely to show up on the wrong display anyways.

That is entirely your choice to make, if you truly feel that the fullscreen state of this app is "breaking it entirely" then you do what you must 😄

I see no reason to not implement the correct behavior of a Mac application

Nowhere in the documentation you linked does it say that macOS apps have to use the resume functionality, if they did it would be enforced at an OS level, not be an optional API that people can use.  That API isn't even accessible in our codebase anyway and we use a custom window state preserver: https://github.com/MarshallOfSound/Google-Play-Music-Desktop-Player-UNOFFICIAL-/blob/master/src/main/features/core/persistAppState.js#L12

All that being said I can sense this becoming a thorn in my side so I'm just going to do it anyway...  If any issues get raised about this being unexpected (as I believe it is) it will be removed 👍

Thanks for doing this


add instructions / code for one machine multi gpus

possibilities:

setup correctly a local spark session with N gpus
create a multiprocessing strategy

should be much easier to setup than multinode

tried this:
            spark = (
                SparkSession.builder.config("spark.driver.memory", "16G")
                .config("spark.task.resource.gpu.amount", "1")
                .config("spark.executor.resource.gpu.amount", "2")
                .config("spark.worker.resource.gpu.amount", "2")
                .config("spark.driver.resource.gpu.amount", "2")
                .config("spark.driver.resourcesFile", "/home/ubuntu/gpufile")
                .config("spark.executor.resourcesFile", "/home/ubuntu/gpufile")
                .config("spark.worker.resourcesFile", "/home/ubuntu/gpufile")
                .config("spark.executor.resource.gpu.discoveryScript", "/home/ubuntu/clip-retrieval/getGpusResources.sh")
                .config("spark.worker.resource.gpu.discoveryScript", "/home/ubuntu/clip-retrieval/getGpusResources.sh")
                .config("spark.driver.resource.gpu.discoveryScript", "/home/ubuntu/clip-retrieval/getGpusResources.sh")
                .master("local[" + str(2) + "]")
                .appName("spark-stats")
                .getOrCreate()
            )

but not working
there's probably a way with spark
probably possible to just do a multiprocessing strategy as well

Hi @rom1504, is there any solution for a one-node multiple GPUs scenario for distributed inference?

Yes advise to follow the same instructions as for multi machines
On Sun, Mar 24, 2024, 9:17 PM Mohammad Mahdi Derakhshani <
@.***> wrote:

Hi @rom1504 https://github.com/rom1504, is there any solution for a
one-node multiple GPUs scenario for distributed inference?
—
Reply to this email directly, view it on GitHub
https://github.com/rom1504/clip-retrieval/issues/115#issuecomment-2016931192,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAR437SJQPUH7RR643YQ4JTYZ4YGHAVCNFSM5O32I7GKU5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TEMBRGY4TGMJRHEZA
.
You are receiving this because you were mentioned.Message ID:
@.***>

Thanks for the reply, @rom1504. Where can I get access to the /home/ubuntu/gpufile file? Any link would be appreciated.

https://github.com/rom1504/clip-retrieval/blob/main/docs/distributed_clip_inference.md#download-spark-on-workers
has the instruction to create it
On Sun, Mar 24, 2024, 9:45 PM Mohammad Mahdi Derakhshani <
@.***> wrote:

Thanks for the reply, @rom1504 https://github.com/rom1504. Where can I
get access to the /home/ubuntu/gpufile file? Any link would be
appreciated.
—
Reply to this email directly, view it on GitHub
https://github.com/rom1504/clip-retrieval/issues/115#issuecomment-2016937771,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAR437WROOFALD53GYGAENTYZ43NLAVCNFSM5O32I7GKU5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TEMBRGY4TGNZXG4YQ
.
You are receiving this because you were mentioned.Message ID:
@.***>


Include result in params to extension method in ShoppingCartControlle…

…r::add()

Merge failure is pre-existing. I'll resolve that.


Fleshing out fmu implementation

This still has a number of pieces to be completed, but it's not too far, and I want to get it into PR to avoid losing anything.

Need to determine the correct inputs for the modelDescription xml file
Need to wire up the required functions to actually step the model through time
Determine if the model can step backward
Figure out building the zipped .fmu file structure in CMake
Modify the fmpy driver to actually call the built FMU via CTest

💯


Unable to create a private connection

What happened?
In GCP, I am trying to create a private connection to a vpc I have to then deploy a CloudSQL instance (which requires a network with private access enabled).
I have the Network, a GlobalAddress, and try to create the Connection, but the script fails reporting that the address cannot be found. If I make a query using gcloud, the address is present.
Steps to reproduce
This is part of the script I am running.

vpc = gcp.compute.Network(
    "default",
    name="default",
    project=project,
    auto_create_subnetworks=False,
    # to allow vpn broadcasting routes
    routing_mode="GLOBAL")

ipv4_address = gcp.compute.GlobalAddress(
  "ipv4-address",
  description="IP address range to be used for private connection",
  network=vpc.id,
  project=project,
  address_type="INTERNAL",
  purpose="PRIVATE_SERVICE_CONNECT",  # Correct ?
)

private_vpc_peering = gcp.servicenetworking.Connection(
  "private-vpc-peering",
  network=vpc.id,
  service="servicenetworking.googleapis.com",
  reserved_peering_ranges=[ipv4_address.name],
)

Expected Behavior
A Connection object should be created.
Actual Behavior
The script fails with the following error:
* Error waiting for Create Service Networking Connection: Error code 9, message: Allocated IP range 'ipv4-address-f7f5e3e' not found in network.
    Help Token: xxx

But if I execute:
NAME                                    ADDRESS/RANGE  TYPE      PURPOSE                  NETWORK  REGION       SUBNET  STATUS
ipv4-address-f7f5e3e                    <IP_ADDRESS>    INTERNAL  PRIVATE_SERVICE_CONNECT  default                       RESERVED
...

Versions used
CLI
Version      3.31.0
Go Version   go1.17.9
Go Compiler  gc
Plugins
NAME           VERSION
gcp            6.25.0
google-native  0.19.1
kubernetes     3.19.2
python         unknown
Host
OS       ubuntu
Version  20.04
Arch     x86_64
This project is written in python (/nix/store/5dpzdxcq5vc4dh1kyg9igcailvi1qmlk-python3-3.9.12-env/bin/python3 v3.9.12)
Current Stack: loadtest
TYPE                                     URN
pulumi:pulumi:Stack                      urn:pulumi:loadtest::::pulumi:pulumi:Stack::-loadtest
pulumi:providers:gcp                     urn:pulumi:loadtest::::pulumi:providers:gcp::default
pulumi:providers:gcp                     urn:pulumi:loadtest::::pulumi:providers:gcp::default_6_25_0
gcp:compute/network:Network              urn:pulumi:loadtest::::gcp:compute/network:Network::default
gcp:compute/subnetwork:Subnetwork        urn:pulumi:loadtest::::gcp:compute/subnetwork:Subnetwork::default
gcp:compute/router:Router                urn:pulumi:loadtest::::gcp:compute/router:Router::router
gcp:container/cluster:Cluster            urn:pulumi:loadtest::::gcp:container/cluster:Cluster::-cluster-loadtest
gcp:compute/routerNat:RouterNat          urn:pulumi:loadtest::::gcp:compute/routerNat:RouterNat::nat
gcp:compute/instance:Instance            urn:pulumi:loadtest::::gcp:compute/instance:Instance::bastion
gcp:compute/firewall:Firewall            urn:pulumi:loadtest::::gcp:compute/firewall:Firewall::allowiap
gcp:compute/globalAddress:GlobalAddress  urn:pulumi:loadtest::***::gcp:compute/globalAddress:GlobalAddress::ipv4-address
Found no pending operations associated with loadtest
Backend
Name           DESKTOP-UOHVP03
URL            gs://xxxx
User           bruno
Organizations
NAME                  VERSION
pip                   22.1.1
pulumi-gcp            6.25.0
pulumi-google-native  0.19.1
pulumi-kubernetes     3.19.2
setuptools            62.3.2
wheel                 0.37.1
Pulumi locates its logs in /tmp by default
warning: A new version of Pulumi is available. To upgrade from version '3.31.0' to '3.33.1', visit https://pulumi.com/docs/reference/install/ for manual instructions and release notes.
Additional context
No response
Contributing
Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

👍

:+1:

Does this fail even if you try running the above again (after the initial failure)? I suspect some sort of an eventual consistency issue. The same seems to be alluded to here on the terraform provider: https://github.com/hashicorp/terraform-provider-google/issues/11100#issuecomment-1040863151

@brunoripa have you found a solution to this?

Hi there,
I'm currently facing the same isssue. I have setup a vpc network, a GlobalAddress and it can't be found in the network.

This should work:
gcp_config = pulumi.Config("gcp")
project = gcp_config.require("project")
vpc = gcp.compute.Network(
    "default-812",
    name="default-812",
    project=project,
    auto_create_subnetworks=False,
    # to allow vpn broadcasting routes
    routing_mode="GLOBAL",
)

us_west1_subnetwork = gcp.compute.Subnetwork(
    "us-west1-subnetwork",
    name="default-812-us-west1",
    project=project,
    region="us-west1",
    ip_cidr_range="<IP_ADDRESS>/16",  # Example CIDR range for the subnetwork
    network=vpc.id,
    private_ip_google_access=True,  # Allow Google API access
)

ipv4_address = gcp.compute.GlobalAddress(
    "ipv4-address",
    description="IP address range to be used for private connection",
    network=vpc.id,
    project=project,
    address_type="INTERNAL",
    prefix_length=16,
    purpose="VPC_PEERING",  # Correct ?
)

private_vpc_peering = gcp.servicenetworking.Connection(
    "private-vpc-peering",
    network=vpc.id,
    service="servicenetworking.googleapis.com",
    reserved_peering_ranges=[ipv4_address.name],
)```


[Web Interface Issue]

Thanks for this great project
I have followed the readme and

flashed my ESP32
Setup the Wifi on it via the Window that Popped Up
Now I am trying to navigate to the esp32's web interface (Found the ip address in my router's client page)

But thats it. There is just nothing coming up. What am I doing wrong?

Nothing, as long as you have the right IP address it should work. You could try adding http:// in front of the IP address, so the browser doesn't try a https connection.

Tried Firefox Developer, Chrome and Safari. With http, just ip. Nothing. Just a spinner of death

It's really hard to tell why it doesn't work, especially if you were able to configure the Wifi in the first place. Can you ping the device? If possible, please connect to the serial line and post the logs (115200 Baud rate)

Solved. As my ESP-32 was 2.4G I connected it to the Guest Network which has rstictions on the LAN!

ok that makes sense :)


Add support for tiled indices via padding/alignment aware embedding kernel (tiled indices only)

Ticket
#14915
Problem description
uint32 untilize not accurate, so padding aware kernel was needed as opposed to standard conversion
What's changed
A lot of changes to look at, first is the split_work_to_cores_aligned function.
I made this function to help split work to cores such that the units per core is aligned with some value. In the new embedding kernel, I align with the face_height of a tile. After developing this kernel further, I believe it's possible to use the standard split_work function for more parallelism, but I figured I'd keep it for now as it could prove useful for perf in the future where we want to make sure our split work is aligned. (You can use the new aligned function as an immediate drop in replacement for the same return values.)
As for the new program factory and kernel, the principle of weights writing is identical to the RM kernel, and this means we want to read the indices such that we are accessing the tensor in an RM fashion, you could compare it simply to a parallelized print. A lot of the work is simply offset algebra, and I tried to make sure to use const expr for any tile or face heights, do let me know if there's anywhere I don't!
With this change, sweeps for embedding go from 46% -> 92% due to the support of tiled indices.
Checklist

[ ] Post commit CI passes
[ ] Model regression CI testing passes

Looks like this code was forked/copied before my recent pr https://github.com/tenstorrent/tt-metal/pull/16237 was merged.
Would be good to copy over the cleanup/refactoring, especially for the kernel side.


remove old binaries from rpm_repos folder and rework for current project environment

rpm_repos is legacy stuff, and needs to be cleaned up.  minimally, we should remove old rpms that reference a non-existant repo.  we can stand up a new repo, or find a suitable existing repo to make goldstone-server accessible without using localinstall.  The ES and logstash repos should be bumped, and we shouldn't need the epel repo since there is an epel-release package available to centos7.

This is resolved in the 0.6.0 release.


Improve homepage links accessibility

Description
Improve accessibility in homepage cards (CreatorCard & ZPageCard), This allows users to "open in new tab" the links and also enables to interact with the images trough the native right click menu
This pull request includes the following changes:

Replaced CardMedia with an imgHTML tag to enhance accessibility.
Ensured the CardActionArea functions as an <a> tag for better navigation.
Quit the navigateTo action from the HoverCard

Samples
Before

After

Issue
Closes #63

Nice, looks good! Thanks! https://test.free2z.com/


chore(turing): remove conversion of k8sConfig to json

Motivation
The GitHub PR addresses two separate issues, with two separate changes:

The Merlin chart dependency in Turing is (very) outdated (0.10.21)-> Bump up Merlin version (0.11.4)
Fixing the cannot overwrite table with non table for merlin.imageBuilder.k8sConfig (map[]) warning when running helm template . or helm lint . in helm-charts/charts/turing -> Convert the imageBuilder.k8sConfig struct into a YAML map just like for Merlin

The first issue's self evident so nothing much to talk there. As for the second, it turns out after some debugging that the recursive coalesceTablesFullKey function (from Helm) called onto the YAML configs in the parent Turing values file actually causes imageBuilder.k8sConfig (an empty string) to be matched with imageBuilder.k8sConfig (a map) in the Merlin chart, so when it attempts to merge those two values, that warning occurs. However, that's not an issue because in the end merlin.imageBuilder.k8sConfig (a map) of the Turing chart values still gets matched with imageBuilder.k8sConfig (a map) in the Merlin chart, and the coalesceTablesFullKey function works properly as expected and merges both maps.
The bug's ultimately something on Helm's end and it affects reading a main chart's values that also happens to be exactly in the same schema as the subchart but it seems like they might've addressed this in one of their recent commits though it hasn't been released yet: https://github.com/helm/helm/commit/0a5148faffb7110bab58a466a52be0686a69947c (didn't really read into this but it mentioned fixing the priority of merging subchart values). So with that, we can temporarily silence the warning, refactor the schemas of both charts to make them different or, simply make both blocks have the same schema.
Anyway, with the refactoring of the Merlin configs from environment variables to a configs file, the k8sConfig struct has already been refactored from a JSON object to a YAML map. Turing however, still uses k8sConfig in JSON for consistency though that is no longer needed since Merlin no longer stores any of such structs in JSON. It thus makes complete sense to just modify the Turing chart to expect a k8sConfig struct in YAML just like in Merlin to address the second issue and to avoid that bug altogether in the process.
This PR thus implements the fix mentioned above.
No changes are expected downstream to the Turing app since it already expects a YAML map for the k8sConfig.
Modification

charts/turing/values.yaml - Default value for imageBuilder.k8sConfig is now changed to an empty map and not an empty string
charts/turing/templates/_helpers.tpl - Since a JSON-string is no longer expected, there's no need for the fromJson function
scripts/generate-cluster-creds.sh - Simplification of the cluster credentials generation script since we no longer did conditional handling for both Merlin and Turing

Checklist

[x] Chart version bumped
[x] README.md updated

Merging this now; will be bumping up the CaraML chart version in a separate MR. Thanks for the quick review!


New GENSC Null Value Controlled Vocab

We should implement this in Qiita. IMHO, this also underscores the need for an easier way to update studies in EBI (after they have already been submitted) to deal with the "temporarily obscured" and "to be reported" values here, in addition to other use cases. This is verbatim from the email:

We would like to bring to your attention that from May 2015 the ENA will support a new null value vocabulary in ENA sample metadata checklists.
Based on recent discussions with the expert/research communities and feedback from submitters the ENA will from May 2015 support reporting of null values with a controlled vocabulary that we believe suits well needs of the user community we serve as well as development and service needs of the database.
The null value vocabulary aims to describe the following cases (concepts):
not applicable (metadata information is inappropriate to report, can indicate that the standard itself fails to model or represent the information appropriately)
not available: not collected (metadata information of an expected format was not given because it has not been collected)
not available: to be reported (metadata information of an expected format was not given, a value will be given at the later stage)
not available: restricted access (metadata information exists but can not be released openly)
obscured (metadata information has been provided deliberately with low precision)
temporarily obscured (metadata information has been provided deliberately with low precision, a higher precision value will be provided at a later stage)
The last two, which consider level of precision, will be used, for example, for geo-references (latitude, longitude and country/sea).

Marking low priority for now, since the changes are still being discussed and are not due out until May anyways.

This has been in place for some time now, closing.


Full folder logic does not work with package nesting depth > 1 in custom /../ namespace

I just restructure our package structure, it looks now like follows

/CQSE/ABAP_DEV is the root package used for sync with abapGit. Everything worked fine with full folder logic before adding the three marked packages. But with the additional level of nesting in the package structure, the contents of the three new packages do not get pushed to the git repository.
I did a few tests with some dummy packages and it seems that the problems are related to the /../ custom namespace. It seems to work fine with packages in Z namespace
Here the result of such a package structure in Z namespace: https://github.com/pfaller/abapgit-test/tree/nested-packages  - everything looks as expected
And here the same for /CQSE/ namespace: https://github.com/pfaller/abapgit-test/tree/nested-pacakges-2 after staging / pushing the following objects:

The marked programs are missing in the git repository.
That's how it looks like in abapGit after staging / pushing the changes:

The programs in question still appear as if these would be new.
I also tried to create the package structure manually in the git but didn't help. Also I did a little bit debugging, but couldn't locate the bug.
Here the debug info of abapGit:
abapGit version: v1.33.4

XML version: v1.0.0

GUI version: 7400.1

LCL_TIME:<PHONE_NUMBER> +0200

SY time: 20170411<PHONE_NUMBER>

Supported objects: ACID, AUTH, CLAS, DDLS, DOCT, DOCV, DOMA, DTEL, ENHO, ENHS, ENQU, ENSC, FUGR, IARP, IASP, IATU, INTF, MSAG, NROB, PARA, PINF, PROG, SFBF, SFBS, SFPF, SFPI, SFSW, SHI3, SHLP, SICF, SMIM, SPLO, SSFO, SSST, SUSC, SUSO, TABL, TOBJ, TRAN, TTYP, TYPE, VCLS, VIEW, W3HT, W3MI, WDYA, WDYN, WEBI, XSLT

Browser: Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.2; WOW64; Trident/7.0)
Frontend time: Tue Apr 11 2017 09:25:41 GMT+0200 (Mitteleuropäische Sommerzeit)

thanks for the bug report
how does the staging page look? try pressing stage in the above example

looks quite normal to me:

Also behaves like staging would work normal, the objects can be added and commited but in the end these are sill missing in the Git.

I think I have reproduced the issue, will try to fix
  METHOD namespaces.

    FIELD-SYMBOLS: <ls_tree> LIKE LINE OF mt_trees.

    append( iv_path = '/src/#foo#a/#foo#a1/'
            iv_name = 'a1.txt' ).

    append( iv_path = '/src/#foo#a/#foo#a2/'
            iv_name = 'a2.txt' ).

    mt_trees = lcl_git_porcelain=>build_trees( mt_expanded ).

    cl_abap_unit_assert=>assert_equals(
      act = lines( mt_trees )
      exp = 5 ).

    LOOP AT mt_trees ASSIGNING <ls_tree>.
      cl_abap_unit_assert=>assert_not_initial( <ls_tree>-data ).
    ENDLOOP.

  ENDMETHOD.

@pfaller try this, https://github.com/larshp/abapGit/pull/712

Sorry for the late reply .. #712 did work! Many thanks for your great support and for providing this awesome tool!


Enhancement: Tweak border styles and positioning

This PR applies a couple of visual tweaks similar to those found in statamic/cms:8807 and statamic/cms:8822. Summary of changes:

New active class in the Vue component, for conditional styling (eg, bottom radius only when inactive)
Styles updated to account for borders and radii in various scenarios
Typo cleanup on the component and its directory
Swapped in grp-scoped Tailwind styles to the addon's stylesheet
Note: (this was a subjective change—feel free to revert, if you prefer to manually style the additions made in this PR)

B&A: Inactive

B&A: Active

This is awesome!


CI config specifies Scala version 2.13.10 but project uses 2.13.11

CI fails with
[error] Switch failed: no subprojects list "2.13.11" (or compatible version) in crossScalaVersions setting.
[error] If you want to force it regardless, call ++ 2.13.11!

crossScalaVersions settings with correct Scala version added


Scrollbar "current character" show

Is there an existing issue for this?

[X] I have searched the existing issues
[X] I have read the guide to filing a bug

Use case
When #52207 is completed, Scrollbar would always know the right size of the scrolling content (except on infinite scroll, but that's another thing entirely, I believe).
Could then this be an entrance for an option to highlight the first character for the current first item on screen? Like when you scroll in Google Contacts the first letter appears at the side of the Scrollbar to help the user in a big list to find something easier.
Proposal
I propose an API where Scrollbar gives you the current index of the list so that we could return a character (or maybe a widget?) to show in a popup that follows the Scrollbar for helping users navigate big lists.

Thanks for the report @FMorschel
Are you looking for a toast message or something similar that'll show current index or page out of total ? If so, have you already checked https://pub.dev/packages/scroll_to_index and possibly https://stackoverflow.com/a/63511502 to see if it helps in your case ?

Hi! Actually no, I was thinking more like this on the right that follows the trackbar:
https://github.com/flutter/flutter/assets/52160996/8bd80441-07c8-48c8-ae78-4c7f4c9b5469

Thanks for the update. Treating this as a potential proposal to add the requested feature for scrollbar api.

Revisiting this issue and I have feeling that there might be community package already which could be offering the feature, for example: check https://pub.dev/packages/azlistview and see if it helps to achieve your use case.

I'll try it out this weekend and relay my thoughts to you!

This one may also provide the requested feature: https://pub.dev/packages/draggable_scrollbar
We're probably unlikely to implement this in the framework. The scrollbar does not know about the content of the scrolling widget, so the user would probably have to provide label information through a new API.

Thanks, Piinks! The draggable_scrollbar is mostly what I was looking for!
I see what you mean. Thanks for helping me anyway! We could then close this if you agree.

We could then close this if you agree.

Thanks for the update @FMorschel. I will close this then.


Unable to connect to the Hub Database in DevTools.

Describe the bug
A clear and concise description of what the bug is.

I can't connect to the Hub Database in DevTools. Always in loading state.

Or only display the view title, clicking the refresh button will result in an error.

After multiple attempts, there is a small probability that the data information can be displayed correctly, but there is currently no stable successful outcome.

Steps to reproduce
Steps to reproduce the behavior:

I created the Nuxt Hub project following the official way. https://hub.nuxt.com/docs/getting-started/installation#quickstart

"dependencies": {
  "@nuxt/eslint": "^0.3.12",
  "@nuxthub/core": "^0.5.17",
  "nuxt": "^3.11.2"
},
"devDependencies": {
  "@nuxt/eslint-config": "^0.3.12",
  "eslint": "^9.2.0",
  "vue-tsc": "^2.0.18",
  "wrangler": "^3.56.0"
}

Two new message data entries have been added, but the corresponding message table data still cannot be displayed in the Hub Database of DevTools.

Expected behavior
A clear and concise description of what you expected to happen.
The two new data entries I added should be displayed in the database table "message", but they are not.

I think I have probably found the reason for the problem: network issues...
I'm sorry, I originally thought it was an issue with NuxtHub, but it turns out it's a problem with my own network. I am trying to resolve it better.


Reduce number of tone labels for Yongning Na from 7 to 5

There are three level tones: High, Mid, Low, and two combinations: Low-to-High and Mid-to-High. In the first version of the training set there were some stray instances of two other combinations, which were used phonetically, not phonemically: {"˩˧", "˧˩"}. So the model included those.
Cleanup of texts was verified today. I searched for {"˩˧", "˧˩"} in all the texts (sentence-level <FORM>) and found only 1 occurrence of ˧˩, which I corrected. Occurrences of ˩˧ had been removed already. So the texts are 'clean' now (I think), as of commit b7e1824a10b91736011942c289fa7b9e79deadd8 (CNRS/HimalCo repository).
The change reducing the number of tone labels for Yongning Na from 7 to 5 is to be done on lines 56 to 58 of persephone/datasets/na.py
UNI_TONES = {"˩", "˥", "˧"}
BI_TONES = {"˧˥", "˩˥", "˩˧", "˧˩"}
TONES = UNI_TONES.union(BI_TONES)
The set of BI_TONES can be simplified by removing {"˩˧", "˧˩"}
Thus:
UNI_TONES = {"˩", "˥", "˧"}
BI_TONES = {"˧˥", "˩˥"}
The earlier training sets had those tones, so I don't know how this is to be handled technically: could the earlier version of the na.py script be stored together with the earlier version of the data set, for colleagues interested to replicate the earlier experiments?

Hi Alexis, thanks for raising this. I had noted the extreme infrequency of [˩˧] and [˧˩]. It's good to now understand that it was a phonetic distinction vs phonemic.

The earlier training sets had those tones, so I don't know how this is to be handled technically: could the earlier version of the na.py script be stored together with the earlier version of the data set, for colleagues interested to replicate the earlier experiments?

Yes, this is no problem. The old approach will be kept on its own branch (lrec_final, for replicating LREC results) for posterity, and all new changes to Na preprocessing can be made on the development branch. (Though performance-wise this change should have effectively no impact since the tones we'll remove were so infrequent the model learnt to just never predict them)


Basis' GetEuler function returns wrong angles under certain conditions

Godot version
4.0-beta1_mono_win64
System information
Windows 10
Issue description
While lerping a basis and getting its euler angle I got results that visually were wrong and also seemed wrong in numbers.
Basis : [X: (0, 0,040403813, 0,9991835), Y: (1, 1,1920929E-07, 0), Z: (-1,1920929E-07, 0,9991835, -0,040403843)]
Euler: (-1,5303826,    -3,1415896,    1,5707934)
Next frame:
Basis:  [X: (0, 0, 1), Y: (1, 0, 0), Z: (0, 1, 0)]
Euler: (-1,5707964,    1,5707964,     0)
The basis are very close, but by getting the euler of the second one I expect a value more like (-1.57, 0, 1.57...).
Out of the 24 rotations I'm working with only 4 to 8 seem to be affected by this issue.
Steps to reproduce
Code:
Basis basis1 = new Basis( new Vector3(0, 0.040403813f, 0.9991835f),  new Vector3(1, 0, 0), new Vector3(0, 0.9991835f, -0.040403843f));
Basis basis2 = new Basis(  new Vector3(0, 0, 1),  new Vector3(1, 0, 0),  new Vector3(0, 1, 0) );
GD.Print("basis1:");
GD.Print( basis1 );
GD.Print("basis1.GetEuler():");
GD.Print(basis1.GetEuler());
GD.Print("basis2:");
GD.Print( basis2 );
GD.Print("basis2.GetEuler():");
GD.Print(basis2.GetEuler());

Prints:
basis1:
[X: (0, 0,040403813, 0,9991835), Y: (1, 0, 0), Z: (0, 0,9991835, -0,040403843)]
basis1.GetEuler():
(-1,5303826, 3,1415927, 1,5707964)
basis2:
[X: (0, 0, 1), Y: (1, 0, 0), Z: (0, 1, 0)]
basis2.GetEuler():
(-1,5707964, 1,5707964, 0)
Minimal reproduction project
No response

Just for fun can you try cubic interpolate with? We did more work there.

While lerping a basis and getting its euler angle I got results that visually were wrong and also seemed wrong in numbers.
Basis : [X: (0, 0,040403813, 0,9991835), Y: (1, 1,1920929E-07, 0), Z: (-1,1920929E-07, 0,9991835, -0,040403843)] Euler: (-1,5303826, -3,1415896, 1,5707934)
Next frame:
Basis: [X: (0, 0, 1), Y: (1, 0, 0), Z: (0, 1, 0)] Euler: (-1,5707964, 1,5707964, 0)
The basis are very close, but by getting the euler of the second one I expect a value more like (-1.57, 0, 1.57...).

It should rather result in something like (-1.57, -1.57, 0).

Close to the first basis ((-1,5303826, -3,1415896, 1,5707934) ~= (-PI / 2, -PI, PI / 2)):

The incorrect result ((-1,5707964, 1,5707964, 0) ~= (-PI / 2, PI / 2, 0)):

Your suggested result ( (-1.57, 0, 1.57) ~= (-PI / 2, 0, PI / 2)):

Would-be-proper result ((-1.57, -1.57, 0) ~= (-PI / 2, -PI / 2, 0)):

In GDScript get_euler seems to work fine. This:
@tool
extends EditorScript

func _run():
	var basis1 := Basis(Vector3(0, 0.040403813, 0.9991835), Vector3(1, 0, 0),Vector3(0, 0.9991835, -0.040403843))
	var basis2 := Basis(Vector3(0, 0, 1), Vector3(1, 0, 0), Vector3(0, 1, 0) )
	print("basis1:")
	print( basis1 )
	print("basis1.get_euler():")
	print(basis1.get_euler())
	print("basis2:")
	print( basis2 )
	print("basis2.get_euler():")
	print(basis2.get_euler())

Outputs (by default get_euler() returns YXZ Euler angles so these are the same as above):
basis1:
[X: (0, 0.040404, 0.999183), Y: (1, 0, 0), Z: (0, 0.999183, -0.040404)]
basis1.get_euler():
(-1.530383, 3.141593, 1.570796)
basis2:
[X: (0, 0, 1), Y: (1, 0, 0), Z: (0, 1, 0)]
basis2.get_euler():
(-1.570796, -1.570796, 0)

Seems different Euler angles conventions were not ported to dotnet/mono, its GetEuler takes no parameter and is claimed to work in YXZ convention (but as proven above it's not even working properly):
https://github.com/godotengine/godot/blob/62792eeb9fac4f44272f59f95ec8eb3abc0432a7/modules/mono/glue/GodotSharp/GodotSharp/Core/Basis.cs#L246-L286
Porting Basis changes from #54084 to dotnet/mono should probably fix these issues.
Not sure if there's anything wrong with some interpolation, haven't checked none of these.

Woops my bad I sometimes wrongly think pi is a whole turn when it's just half.
The interpolation part is indeed irrelevant, it is just extra context to show where I noticed the weird behavior. Thanks for confirming the problem!

Woops my bad I sometimes wrongly think pi is a whole turn when it's just half.

I find it much easier to think and write expressions in terms of TAU (which is supported in Godot in both GDScript and C#)

Oh that's perfect! Thank you :D

Was lerping a basis and getting its euler angle incorrect caused by Godot Engine? Can I close?

Was lerping a basis and getting its euler angle incorrect caused by Godot Engine?

According to @Maveyyl's comment lerping/interpolating is irrelevant, it was just when they spotted the issue. And yes, getting incorrect euler angles from a basis is caused by the Godot Engine, check my previous comment for more details.

Can I close?

Yes, after it will be fixed (so not yet). :slightly_smiling_face:
What needs to be done? I'll quote myself:

Porting Basis changes from #54084 to dotnet/mono should probably fix these issues.
Basis.GetEuler needs to be fixed in 3.x as well.

To be clear: it seems to be (C#/dotnet/mono)-only issue.


[CLOSED] Include proper error handling and warnings

 Issue by IQAndreas
Friday Sep 13, 2013 at 01:56 GMT
Originally opened as https://github.com/IQAndreas/github-issues-import/issues/6

Allow for better error handling for the following cases:

There is a problem with the config options
The GitHub API returns an error
The target issue already exists in the repository (should at least warn)

I also want to prompt the user "You are about to import all 42 issues into hello-world. Are you sure you want to continue?"

 Comment by IQAndreas
Friday Sep 13, 2013 at 02:01 GMT

@dliuzzi has already done this for warnings thrown by the GitHub API: https://github.com/dliuzzi/tools
All that needs to be done is filter through the commits and make the changes (I'm pretty sure this code is too far-gone to do a straight up merge).

 Comment by IQAndreas
Friday Sep 27, 2013 at 05:00 GMT

Fixed in the following commits:

Add proper error handling for HTTP requests
Add error if the config file cannot be found
Add error if missing required config options


pd: mock pd-server to test pd mod

Now we start a real PD server to test pd mod, but it is not convenient, and most of times, we may ignore this test in local machine.
Maybe we can mock a pd server to test HTTP handshake + network RPC communication.

Yet another mockup https://github.com/overvenus/rust-pd

embed this in tikv test directly.

FYI, #1591 includes a rust PD mockup.


Parsing playlists with Podcasts

Hello,
First of all, thanks for sharing your work. It's great!
Trying to use the python code to ingest the Spotify API I found an issue due to having podcasts in some of my playlists. I've written a workaround that may be useful (yet not elegant).
In line 67, replace:
if len(json_items["items"]) > 0
with:
if json_items is not None:
    if len(json_items["items"]) > 0 and not any(i['track'].get('episode',False) for i in json_items["items"]):

and indent the following code.
Hope it helps!

Uh

Uh

Great contribution!😂


Lovelace Resources not migrated

Updating from 2.4.7 to beta 15 doesn't copy the lovelace resources in the folder /config/www/icloud3. After migration there is no folder created nor any cards copied. What remains is /config/www/custom_cards

According to installation note #2 this should happen automatically:
iCloud3 installs the Event Log card program into the config/www/icloud3 directory and updates it when a new version is available.
 iCloud3 checks the Lovelace Resource when it starts. The directory containing the icloud3-event-log-card.js file is added or updated so you should not have to do anything.
Currently this is what is shown:

Attached the migration log
icloud3-migration.log

I did a manual move to /config/www/icloud3 and changed according to this text:
Note: Using another custom card directory - If you move the Event Log card to another directory, the Lovelace Resources should automatically be changed. If something happens and it is not changed, change the directory (icloud3) in the URL statement on the above screen to the new directory name. Then select the new directory name in the Event Log Directory field on the iCloud3 Configurator > Menu Page 2 > Other Parameters screen.

But it resulted in the same screen. The log is actually not working at all. It remains empty and all sensors are not showing anything.

Having copied the content from the zipfile directory /event_log_card/ into /config/www/icloud3 made the card work.
Had to delete and re-add all devices in order for them work. So migration actually doesn't (at least in my case) migrate the devices properly.

When it migrates, it uses the directory from v2.4.7 (custom _cards) so the resources do hot have to be upgraded. I’ll make a note to clarify this in the docs.
Try refreshing your browser.

I did see that all devices were inactive, but the devices were all there (sensor.jeroens_iphone, etc.) they didn't show the right value, battery state was 0% for example. Even after activating them and filling in the right things with family sharing and tracking, they still didn't show the right value. Not untill I deleted the devices and added them again, did they show the value.

In the lovelace card, the battery level of the iphone is very strange. It jumps constantly 100%. So from the actual % to the 100%. It gives this log:

What is causing this? As you can see in the log, it worked yesterday, but today it says 100%, while it clearly isn't.

I found a problem where an old iosapp battery level would overwrite a newer famshr battery level. The iOS app battery level is checked every 15-secs so it would cause a bouncing like you are seeing. I have added an update Time so old info will not replace newer info. This will be on the next release in a few days.

Hi, I noticed my locations and battery levels were not updating and when I went to check the log with the lovelace card it looks just like JeroenTuinstra's screenshot above, showing a partial header but no contents.
I recently switched to the iCloud3.0.0 beta 15 from the HA iCloud integration and it seemed to be working perfectly for a while (weeks?).  I'm not sure what changed on my end before it stopped updating locations/etc. and when it stopped showing most of the lovelace card contents, perhaps after HA system restart or two.
Am I bringing this up in the right issue#? Should I make a new one?
Log notes:

It looks like I have icloud3-event-log-card.js  in the /config/www/custom_cards/ and  /config/www/icloud3/ folders. I also found it in /config/custom_components/icloud3/ folder. I did not migrate from an earlier version but it's possible I tried iCloud3 a long time ago on my system, if that explains why I have the file in both folders. I installed v3.0.0 b15 from HACS.

The last line in /config/icloud3-debug-1.log has a timestamp of 2023-04-15 00:00:00 which was midnight yesterday at this point. I did not find any error messages in icloud3-debug-1.log that stand out, mostly just occasional Waze connection errors.

/config/icloud3-debug.log shows today's log starting at 2023-04-16 12:38:16 but again, it shows nothing remarkable (just creating sensor entities, setting up zones, etc.). There were no errors in that file.

icloud3-event-log_2023.03.23-21.57.51.log seems normal although the date in the filename was about three weeks ago. Perhaps that was the date I installed iCloud3 v3.0.0 b15? There are no icloud3-event-log~~~.log files in the /config/ folder that are more recent.

I also have a /config/icloud3-migration.log file that seems normal with no errors.

Do I wait patiently for the next beta version to roll out or would it help if I provided more information or full logs?

I submitted iCloud3 v3 to be added to HACS several weeks ago but, as far as I can tell, it has not been added yet. Doing a search in HACS only lists iCloud3 v2. How and where did you find v3 b15?

I added the v3 beta custom repository to HACS and installed through that.
It was recommended to me on a GitHub message chain for one of the issues on the official iCloud integration that mentioned how they should remove it since it wasn't being maintained anymore. Someone suggested the v3 beta of iCloud3 because the GUI would make it easier to manage my family's phones than with v2 (my kids constantly change their phone names so I was manually reconfiguring iCloud3 v2 a lot when I initially tried it a year or so ago). So far the beta has worked great other than this issue's minor hiccup which is very minor for a beta! Thank you for making it!

b16 released


IPv6 unittest failure

System: Vibe 0.7.20 -version=VibeWin32Driver, DMD 2.065.0, Windows XP 32-bit

In vibe.core.net there is a unittest that runs vibe.core.win32.resolveHost() with an IPv6 address (line 215):
test("102:304:506:708:90a:b0c:d0e:f10");

which fails with "Invalid IP address string", and is the only test that fails on my system.
Investigation reveals that resolveHost() first calls WSAStringToAddress() with AF_INET, which is the IPv4 flag, and fails because the address is IPv6. Then it tries again with AF_INET6, which also fails, presumably due to this behaviour specified on the WSAStringToAddress() reference page:

Support for IPv6 addresses using the WSAStringToAddress function was added on Windows XP with Service Pack 1 (SP1) and later. IPv6 must also be installed on the local computer for the WSAStringToAddress function to support IPv6 addresses.

Then, since resolveHost() was called with use_dns = false, it hits enforce(use_dns, …) and fails. Without this line it tries GetAddrInfoEx() which fails as well with:

Failed to look up host 102:304:506:708:90a:b0c:d0e:f10: The requested name is valid and was found in the database, but it does not have the correct associated data being resolved for.

I'm not really sure what the best solution would be. If IPv6 support is supposed to be a dependency then perhaps a more explicit test for IPv6 support should be run when the event-driver starts. Otherwise something like version (IPv6) {…} might be sufficient. Or, to at least get the unittest working, perhaps a workaround function that emulates the proper behaviour of WSAStringToAddress() in the absence of IPv6 support, since all it does is parse the address string and fill a sockaddr structure.

Won't fix because XP is EOL now.


added HttpClient java11 instrumentation for sync requests

closes #1278

[x] sync requests
[x] async requests
[x] changelog
[x] update docs
[x] test on application and add screenshots(java 11, attach to java7 jvm)

:grey_exclamation: Build Aborted

The PR is not allowed to run in the CI yet

Expand to view the summary

Build stats

Build Cause: [Pull request #1307 opened]

Reason: The PR is not allowed to run in the CI yet

Start Time: 2020-07-22T17:57:24.691+0000

Duration: 4 min 52 sec

Commit: c321d68d0eafb1a4017f9e73eefb5f8de9ebfe86

Steps errors
Expand to view the steps failures

Name: Error signal

Description: githubPrCheckApproved: The PR is not allowed to run in the CI yet. (Only users with write permission

Duration: 0 min 0 sec

Start Time: 2020-07-22T18:01:13.216+0000

log

Log output
Expand to view the last 100 lines of log output

[2020-07-22T18:00:48.827Z] Cleaning workspace
[2020-07-22T18:00:48.846Z] Using shallow fetch with depth 3
[2020-07-22T18:00:48.846Z] Pruning obsolete local branches
[2020-07-22T18:00:48.791Z]  > git config remote.origin.url git@github.com:elastic/apm-agent-java.git # timeout=10
[2020-07-22T18:00:48.797Z]  > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
[2020-07-22T18:00:48.808Z]  > git config remote.origin.url git@github.com:elastic/apm-agent-java.git # timeout=10
[2020-07-22T18:00:48.825Z]  > git rev-parse --verify HEAD # timeout=10
[2020-07-22T18:00:48.831Z] No valid HEAD. Skipping the resetting
[2020-07-22T18:00:48.831Z]  > git clean -fdx # timeout=10
[2020-07-22T18:00:48.846Z] Fetching upstream changes from git@github.com:elastic/apm-agent-java.git
[2020-07-22T18:00:48.847Z] using GIT_SSH to set credentials GitHub user @elasticmachine SSH key
[2020-07-22T18:00:48.851Z]  > git fetch --no-tags --progress --prune -- git@github.com:elastic/apm-agent-java.git +refs/pull/1307/head:refs/remotes/origin/PR-1307 +refs/heads/master:refs/remotes/origin/master # timeout=15
[2020-07-22T18:00:49.518Z] Merging remotes/origin/master commit 4de87a80b3daf6c187bfc972ad72dee904ba37fb into PR head commit c321d68d0eafb1a4017f9e73eefb5f8de9ebfe86
[2020-07-22T18:00:50.145Z] Merge succeeded, producing ea357f76e315580c130732d96eaf1b291e37f18f
[2020-07-22T18:00:50.145Z] Checking out Revision ea357f76e315580c130732d96eaf1b291e37f18f (PR-1307)
[2020-07-22T18:00:49.520Z]  > git config core.sparsecheckout # timeout=10
[2020-07-22T18:00:49.528Z]  > git checkout -f c321d68d0eafb1a4017f9e73eefb5f8de9ebfe86 # timeout=15
[2020-07-22T18:00:49.812Z]  > git remote # timeout=10
[2020-07-22T18:00:49.816Z]  > git config --get remote.origin.url # timeout=10
[2020-07-22T18:00:49.824Z] using GIT_SSH to set credentials GitHub user @elasticmachine SSH key
[2020-07-22T18:00:49.827Z]  > git merge 4de87a80b3daf6c187bfc972ad72dee904ba37fb # timeout=10
[2020-07-22T18:00:50.124Z]  > git rev-parse HEAD^{commit} # timeout=10
[2020-07-22T18:00:50.143Z]  > git config core.sparsecheckout # timeout=10
[2020-07-22T18:00:50.146Z]  > git checkout -f ea357f76e315580c130732d96eaf1b291e37f18f # timeout=15
[2020-07-22T18:00:53.719Z] Commit message: "Merge commit '4de87a80b3daf6c187bfc972ad72dee904ba37fb' into HEAD"
[2020-07-22T18:00:53.719Z] First time build. Skipping changelog.
[2020-07-22T18:00:53.719Z] Cleaning workspace
[2020-07-22T18:00:53.791Z]  > git --version # timeout=10
[2020-07-22T18:00:53.798Z] fatal: bad object 6c62d3f53b77d98a3e133635021db7fb14d4c2b2
[2020-07-22T18:00:53.716Z]  > git rev-parse --verify HEAD # timeout=10
[2020-07-22T18:00:53.721Z] Resetting working tree
[2020-07-22T18:00:53.721Z]  > git reset --hard # timeout=10
[2020-07-22T18:00:53.741Z]  > git clean -fdx # timeout=10
[2020-07-22T18:00:54.596Z] Masking supported pattern matches of $JOB_GCS_BUCKET or $NOTIFY_TO
[2020-07-22T18:00:54.668Z] Timeout set to expire in 1 hr 0 min
[2020-07-22T18:00:54.696Z] The timestamps step is unnecessary when timestamps are enabled for all Pipeline builds.
[2020-07-22T18:00:54.993Z] [INFO] Number of builds to be searched 10
[2020-07-22T18:00:55.240Z] [INFO] 'shallow' is forced to be disabled when running on PullRequests
[2020-07-22T18:00:55.265Z] Running in /var/lib/jenkins/workspace/_java_apm-agent-java-mbp_PR-1307/src/github.com/elastic/apm-agent-java
[2020-07-22T18:00:55.304Z] [INFO] gitCheckout: Checkout SCM PR-1307 with some customisation.
[2020-07-22T18:00:55.339Z] [INFO] Override default checkout
[2020-07-22T18:00:55.405Z] Sleeping for 10 sec
[2020-07-22T18:01:05.543Z] using credential f6c7695a-671e-4f4f-a331-acdce44ff9ba
[2020-07-22T18:01:05.620Z] Cloning the remote Git repository
[2020-07-22T18:01:05.629Z] Cloning repository git@github.com:elastic/apm-agent-java.git
[2020-07-22T18:01:05.652Z]  > git init /var/lib/jenkins/workspace/_java_apm-agent-java-mbp_PR-1307/src/github.com/elastic/apm-agent-java # timeout=10
[2020-07-22T18:01:05.668Z] Using reference repository: /var/lib/jenkins/.git-references/apm-agent-java.git
[2020-07-22T18:01:05.669Z] Fetching upstream changes from git@github.com:elastic/apm-agent-java.git
[2020-07-22T18:01:05.669Z]  > git --version # timeout=10
[2020-07-22T18:01:05.675Z] using GIT_SSH to set credentials GitHub user @elasticmachine SSH key
[2020-07-22T18:01:05.679Z]  > git fetch --tags --progress -- git@github.com:elastic/apm-agent-java.git +refs/heads/*:refs/remotes/origin/* # timeout=10
[2020-07-22T18:01:07.439Z] Checking out Revision c321d68d0eafb1a4017f9e73eefb5f8de9ebfe86 (origin/PR-1307)
[2020-07-22T18:01:06.651Z]  > git config remote.origin.url git@github.com:elastic/apm-agent-java.git # timeout=10
[2020-07-22T18:01:06.655Z]  > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
[2020-07-22T18:01:06.666Z]  > git config remote.origin.url git@github.com:elastic/apm-agent-java.git # timeout=10
[2020-07-22T18:01:06.678Z] Fetching upstream changes from git@github.com:elastic/apm-agent-java.git
[2020-07-22T18:01:06.679Z] using GIT_SSH to set credentials GitHub user @elasticmachine SSH key
[2020-07-22T18:01:06.682Z]  > git fetch --tags --progress -- git@github.com:elastic/apm-agent-java.git +refs/pull/1307/head:refs/remotes/origin/PR-1307 +refs/heads/master:refs/remotes/origin/master # timeout=10
[2020-07-22T18:01:07.426Z]  > git rev-parse origin/PR-1307^{commit} # timeout=10
[2020-07-22T18:01:07.436Z]  > git config core.sparsecheckout # timeout=10
[2020-07-22T18:01:07.444Z]  > git checkout -f c321d68d0eafb1a4017f9e73eefb5f8de9ebfe86 # timeout=10
[2020-07-22T18:01:07.621Z] Commit message: "added HttpClient java11 instrumentation for sync requests"
[2020-07-22T18:01:07.621Z] First time build. Skipping changelog.
[2020-07-22T18:01:07.630Z]  > git --version # timeout=10
[2020-07-22T18:01:07.639Z] fatal: bad object 6c62d3f53b77d98a3e133635021db7fb14d4c2b2
[2020-07-22T18:01:08.522Z] Masking supported pattern matches of $GIT_USERNAME or $GIT_PASSWORD
[2020-07-22T18:01:09.113Z] + git fetch<EMAIL_ADDRESS>+refs/pull/*/head:refs/remotes/origin/pr/*
[2020-07-22T18:01:10.105Z] Archiving artifacts
[2020-07-22T18:01:10.799Z] + git rev-parse HEAD
[2020-07-22T18:01:11.163Z] + git rev-parse HEAD
[2020-07-22T18:01:11.474Z] + git rev-parse origin/pr/1307
[2020-07-22T18:01:11.532Z] [INFO] githubEnv: Found Git Build Cause: pr
[2020-07-22T18:01:11.831Z] Masking supported pattern matches of $GITHUB_TOKEN
[2020-07-22T18:01:12.789Z] [WARN] githubApiCall: The REST API call https://api.github.com/repos/elastic/apm-agent-java/pulls/1307/reviews return 0 elements
[2020-07-22T18:01:12.864Z] [INFO] githubPrCheckApproved: Title: wip: added HttpClient java11 instrumentation for sync requests - User: kananindzya - Author Association: CONTRIBUTOR
[2020-07-22T18:01:13.234Z] ERROR: githubPrCheckApproved: The PR is not allowed to run in the CI yet
[2020-07-22T18:01:13.234Z] ERROR: githubPrCheckApproved: The PR is not allowed to run in the CI yet. (Only users with write permissions can do so.)
[2020-07-22T18:01:13.321Z] [INFO] Let's stop build #1. The PR is not allowed to run in the CI yet
[2020-07-22T18:01:13.346Z] Sleeping for 5 sec
[2020-07-22T18:01:14.335Z] Stage "Build" skipped due to earlier failure(s)
[2020-07-22T18:01:14.520Z] Stage "Tests" skipped due to earlier failure(s)
[2020-07-22T18:01:14.705Z] Stage "Unit Tests" skipped due to earlier failure(s)
[2020-07-22T18:01:14.717Z] Stage "Smoke Tests 01" skipped due to earlier failure(s)
[2020-07-22T18:01:14.728Z] Stage "Smoke Tests 02" skipped due to earlier failure(s)
[2020-07-22T18:01:14.738Z] Stage "Benchmarks" skipped due to earlier failure(s)
[2020-07-22T18:01:14.750Z] Stage "Javadoc" skipped due to earlier failure(s)
[2020-07-22T18:01:14.906Z] Failed in branch Unit Tests
[2020-07-22T18:01:14.915Z] Failed in branch Smoke Tests 01
[2020-07-22T18:01:14.923Z] Failed in branch Smoke Tests 02
[2020-07-22T18:01:14.931Z] Failed in branch Benchmarks
[2020-07-22T18:01:14.941Z] Failed in branch Javadoc
[2020-07-22T18:01:15.182Z] Stage "Integration Tests" skipped due to earlier failure(s)
[2020-07-22T18:01:15.273Z] Stage "AfterRelease" skipped due to earlier failure(s)
[2020-07-22T18:01:15.313Z] Stage "AfterRelease" skipped due to earlier failure(s)
[2020-07-22T18:01:15.613Z] Running on Jenkins in /var/lib/jenkins/workspace/_java_apm-agent-java-mbp_PR-1307
[2020-07-22T18:01:15.764Z] [INFO] getVaultSecret: Getting secrets
[2020-07-22T18:01:15.862Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2020-07-22T18:01:16.731Z] + chmod 755 generate-build-data.sh
[2020-07-22T18:01:16.731Z] + ./generate-build-data.sh https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-agent-java/apm-agent-java-mbp/PR-1307/ https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-agent-java/apm-agent-java-mbp/PR-1307/runs/1 ABORTED 231771
[2020-07-22T18:01:16.982Z] INFO: curl https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-agent-java/apm-agent-java-mbp/PR-1307/runs/1/steps/?limit=10000 -o steps-info.json

Hi @felixbarny ,
I need advice to instrument asyncMethod
HttpClient.sendAsync(request, HttpResponse.BodyHandlers.ofString()).get();

returns CompletableFuture. I am confuse what to do with it )

You can register a callback by calling future.whenComplete. In that callback, you can end the span. Note that with indy plugins, you don't have to duplicate code in the advices anymore. You can just share common code between HttpClientAsyncInstrumentation and HttpClientInstrumentation in a helper class. You don't need a helper class manager for that and you can use framework-specific types everywhere in an indy plugin, not just the advice methods.
Also, you can return HttpRequest instead of Object and get rid of the typing = Assigner.Typing.DYNAMIC.

You can register a callback by calling future.whenComplete. In that callback, you can end the span. Note that with indy plugins, you don't have to duplicate code in the advices anymore. You can just share common code between HttpClientAsyncInstrumentation and HttpClientInstrumentation in a helper class. You don't need a helper class manager for that and you can use framework-specific types everywhere in an indy plugin, not just the advice methods.
Also, you can return HttpRequest instead of Object and get rid of the typing = Assigner.Typing.DYNAMIC.

ok, thanks. I will try to implement this today

Rather than cloning the whole HttpRequest to add the trace headers, you could just instrument HttpRequest#headers() and return a copy of the HttpHeaders that include the trace headers.
Another simplification you may want to implement is to remove GlobalThreadLocal<Span> and call tracer.getActiveExitSpan() on the exit advice instead.

Rather than cloning the whole HttpRequest to add the trace headers, you could just instrument HttpRequest#headers() and return a copy of the HttpHeaders that include the trace headers.
Another simplification you may want to implement is to remove GlobalThreadLocal<Span> and call tracer.getActiveExitSpan() on the exit advice instead.

added HttpRequest#headers instrumentation, removed GlobalThreadLocal

screenshots

Hi @kananindzya ,
I've pushed some changes in a PR on your branch (https://github.com/kananindzya/apm-agent-java/pull/7), as it was easier than explaining them.
That's mostly cosmetic changes, this new plugin is really great :rocket: .

Hi @kananindzya ,
I've pushed some changes in a PR on your branch (kananindzya#7), as it was easier than explaining them.
That's mostly cosmetic changes, this new plugin is really great rocket .

Merged. Thanks you :)

Javadoc fails:

[2020-08-26T08:12:25.703Z] [ERROR] Failed to execute goal org.apache.maven.plugins:maven-javadoc-plugin:3.1.1:javadoc (default-cli) on project apm-agent-plugin-sdk: An error has occurred in Javadoc report generation:
[2020-08-26T08:12:25.703Z] [ERROR] Exit code: 1 - javadoc: error - The code being documented uses packages in the unnamed module, but the packages defined in https://docs.oracle.com/en/java/javase/11/docs/api/ are in named modules.

Maybe related to https://github.com/elastic/apm-agent-java/pull/1237?

That might be a side effect of compiling test code to Java 11 (from 10 initially), I'll investigate that.

@SylvainJuge @kananindzya,  in which agent version this will be available ?

It has been merged 3 weeks ago and is included in the latest available agent 1.18.0


[Issue #25] Added ScanBasedSourceOperator

@chenlica @JavierJia ... Implemented ScanBasedSourceOperator and refactored a few things. I changed the design a little by making the interfaces IScanBasedSourceOperator and IIndexBasedSourceOperator as classes. Please review the code and please let me know if anything needs to be changed.

looks very good to me. :+1:

Let's discuss this pull request when we meet on Friday.  We will make the structure right, with the right class names and test cases.

@chenlica @JavierJia ..1.  Added ScanBasedSourceOperatorTest class to test scan operator …
2. Updated textdb pom to use lucene.version property
3. Cleaning up the data in Lucene after the test case is run.
Please review the changes.

I will let Jianfeng review first if possible.  I will also do a review on the weekend.

LGTM :+1:  I left some minor comments. The only concern is that we should extract the required interface from the SampleDataStore and put it into the textdb-storage project. However, it could be finished in a separate PR.

I added a few more comments.  Please take care of them.

@chenlica @JavierJia .. Please review the changes. Will make the changes related to data store and mutiple fields support in the next PRs.

LGTM

I made a minor comment.  Please fix it and do the merge.
It will be good to do the other PRs quickly since we want other teams to use this operator to come up with their first scan-based implementation this week.
Thank you!


[modify] cms/elasticsearch/indexer : opendata_dataset and opendata_app convertor

…p convertor

Coverage decreased (-1.4%) to 83.277% when pulling 0703d7305775e584f59b994118d7133c138620c2 on fix/cms-elasticsearch into c874691301c81d735d8e4979310775897e2772f3 on master.

Coverage decreased (-1.4%) to 83.277% when pulling 0703d7305775e584f59b994118d7133c138620c2 on fix/cms-elasticsearch into c874691301c81d735d8e4979310775897e2772f3 on master.

Coverage decreased (-1.4%) to 83.277% when pulling 0703d7305775e584f59b994118d7133c138620c2 on fix/cms-elasticsearch into c874691301c81d735d8e4979310775897e2772f3 on master.

Coverage decreased (-1.4%) to 83.277% when pulling 0703d7305775e584f59b994118d7133c138620c2 on fix/cms-elasticsearch into c874691301c81d735d8e4979310775897e2772f3 on master.

Coverage decreased (-1.4%) to 83.277% when pulling 0703d7305775e584f59b994118d7133c138620c2 on fix/cms-elasticsearch into c874691301c81d735d8e4979310775897e2772f3 on master.


Add Shadow to Footer

Like header I would like to add the shadow to footer.

@abhijeet007rocks8 Please assign me this issue.


Extremely slow coverage with v8, and nestjs while using testing modules

Describe the bug
When running coverage on a nestjs project which uses @nestjs/testing, the report is very slow to report.
On my machine (Macbook pro M2), it takes around 1m for one test with one mock.
It looks like #5252, but the PR still doesn't fix the issue on my machine (if I linked the project correctly on my machine).
Maybe there is something I don't understand with nestjs that makes coverages extremely slow?
Reproduction

Clone and install https://github.com/Seboran/nest-js-slow-coverage
run pnpm vitest run --coverage
wait for coverage to take close to a minute even tho there is only one test that does one thing
check https://github.com/Seboran/nest-js-slow-coverage/blob/main/src/tools/test/test.service.spec.ts and see that the test doesn't do a lot

System Info
System:
    OS: macOS 14.2.1
    CPU: (10) arm64 Apple M2 Pro
    Memory: 304.98 MB / 16.00 GB
    Shell: 5.9 - /bin/zsh
  Binaries:
    Node: 20.5.0 - ~/.nvm/versions/node/v20.5.0/bin/node
    Yarn: 1.22.19 - /opt/homebrew/bin/yarn
    npm: 10.1.0 - ~/.nvm/versions/node/v20.5.0/bin/npm
    pnpm: 8.5.1 - /opt/homebrew/bin/pnpm
    bun: 1.0.18 - ~/.bun/bin/bun
  Browsers:
    Safari: 17.2.1
  npmPackages:
    @vitest/coverage-v8: ^1.3.1 => 1.3.1
    @vitest/ui: ^1.3.1 => 1.3.1
    vitest: ^1.3.1 => 1.3.1

Used Package Manager
pnpm
Validations

[X] Follow our Code of Conduct
[X] Read the Contributing Guidelines.
[X] Read the docs.
[X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
[X] Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
[X] The provided reproduction is a minimal reproducible example of the bug.

It looks like https://github.com/vitest-dev/vitest/issues/5252, but the PR still doesn't fix the issue on my machine (if I linked the project correctly on my machine).

This is indeed duplicate of #5252. By patching the fix from #5259 your reproduction works fast.

package.json

  "pnpm": {
    "patchedDependencies": {
      "<EMAIL_ADDRESS>"patches/<EMAIL_ADDRESS>    }

patches/<EMAIL_ADDRESS>
diff --git a/dist/provider.js b/dist/provider.js
index 408c1c8d7c5053bd9d2d053717ea6d86140b34e9..0a15cfc5c641f3ecf4f1320d88a71524c8a2bac7 100644
--- a/dist/provider.js
+++ b/dist/provider.js
@@ -170,7 +170,7 @@ function cleanUrl(url) {

 const WRAPPER_LENGTH = 185;
 const VITE_EXPORTS_LINE_PATTERN = /Object\.defineProperty\(__vite_ssr_exports__.*\n/g;
-const DECORATOR_METADATA_PATTERN = /_ts_metadata\("design:paramtypes"(\s|.)+?]\),/g;
+const DECORATOR_METADATA_PATTERN = /_ts_metadata\("design:paramtypes", \[[^\]]*?\]\),*/g;
 const DEFAULT_PROJECT = Symbol.for("default-project");
 const debug = createDebug("vitest:coverage");
 let uniqueId = 0;

$ time pnpm run test:cov --run

><EMAIL_ADDRESS>test:cov /workspaces/nest-js-slow-coverage
> vitest --coverage "--run"

The CJS build of Vite's Node API is deprecated. See https://vitejs.dev/guide/troubleshooting.html#vite-cjs-node-api-deprecated for more details.

 RUN  v1.3.1 /workspaces/nest-js-slow-coverage
      Coverage enabled with v8

 ✓ src/tools/__test__/test.service.spec.ts (1)
   ✓ ToolsService (1)
     ✓ createTool (1)
       ✓ should create a new tool

 Test Files  1 passed (1)
      Tests  1 passed (1)
   Start at  07:37:35
   Duration  898ms (transform 114ms, setup 0ms, collect 513ms, tests 12ms, environment 0ms, prepare 132ms)

 % Coverage report from v8
-------------------|---------|----------|---------|---------|-------------------
File               | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
-------------------|---------|----------|---------|---------|-------------------
All files          |   46.83 |       25 |   16.66 |   46.83 |
 src               |       0 |        0 |       0 |       0 |
  app.module.ts    |       0 |        0 |       0 |       0 | 1-15
  main.ts          |       0 |        0 |       0 |       0 | 1-15
 src/schemas       |     100 |      100 |     100 |     100 |
  tool.schema.ts   |     100 |      100 |     100 |     100 |
 src/tools         |      80 |      100 |   33.33 |      80 |
  tools.service.ts |      80 |      100 |   33.33 |      80 | 14-15,18-19
 src/tools/mock    |       0 |        0 |       0 |       0 |
  MockToolModel.ts |       0 |        0 |       0 |       0 | 1-8
-------------------|---------|----------|---------|---------|-------------------

real    0m2.411s
user    0m2.099s
sys     0m1.313s

Oops sorry for the duplicate and thank you for the answer 😄


running bootstrap queries back to back fails

Running a bootstrap query two times back to back fails.  For example:
metrics_gdf = eval.metrics.query(
    include_metrics=include_metrics,
    group_by=["primary_location_id"],
    order_by=["primary_location_id"]
).to_geopandas()
metrics_gdf

The results are printed. But run again:
metrics_gdf = eval.metrics.query(
    include_metrics=include_metrics,
    group_by=["primary_location_id"],
    order_by=["primary_location_id"]
).to_geopandas()
metrics_gdf

and you get an error about the wrong number of arguments.
PythonException:
  An exception was thrown from the Python worker. Please see the stack trace below.
Traceback (most recent call last):
  File "/home/matt/repos/teehr/.venv/lib/python3.10/site-packages/pyspark/python/lib/pyspark.zip/pyspark/worker.py", line 1247, in main
    process()
  File "/home/matt/repos/teehr/.venv/lib/python3.10/site-packages/pyspark/python/lib/pyspark.zip/pyspark/worker.py", line 1239, in process
    serializer.dump_stream(out_iter, outfile)
  File "/home/matt/repos/teehr/.venv/lib/python3.10/site-packages/pyspark/python/lib/pyspark.zip/pyspark/sql/pandas/serializers.py", line 470, in dump_stream
    return ArrowStreamSerializer.dump_stream(self, init_stream_yield_batches(), stream)
  File "/home/matt/repos/teehr/.venv/lib/python3.10/site-packages/pyspark/python/lib/pyspark.zip/pyspark/sql/pandas/serializers.py", line 100, in dump_stream
    for batch in iterator:
  File "/home/matt/repos/teehr/.venv/lib/python3.10/site-packages/pyspark/python/lib/pyspark.zip/pyspark/sql/pandas/serializers.py", line 463, in init_stream_yield_batches
    for series in iterator:
  File "/home/matt/repos/teehr/.venv/lib/python3.10/site-packages/pyspark/python/lib/pyspark.zip/pyspark/worker.py", line 1070, in mapper
    result = tuple(f(*[a[o] for o in arg_offsets]) for (arg_offsets, f) in udfs)
  File "/home/matt/repos/teehr/.venv/lib/python3.10/site-packages/pyspark/python/lib/pyspark.zip/pyspark/worker.py", line 1070, in <genexpr>
    result = tuple(f(*[a[o] for o in arg_offsets]) for (arg_offsets, f) in udfs)
  File "/home/matt/repos/teehr/.venv/lib/python3.10/site-packages/pyspark/python/lib/pyspark.zip/pyspark/worker.py", line 454, in <lambda>
    return lambda *a: (wrapped(*a), arrow_return_type)
  File "/home/matt/repos/teehr/.venv/lib/python3.10/site-packages/pyspark/python/lib/pyspark.zip/pyspark/worker.py", line 451, in wrapped
    result = f(*series)
  File "/home/matt/repos/teehr/.venv/lib/python3.10/site-packages/pyspark/python/lib/pyspark.zip/pyspark/util.py", line 83, in wrapper
    return f(*args, **kwargs)
TypeError: create_gumboot_func.<locals>.bootstrap_func() takes 3 positional arguments but 4 were given

Completed/fixed in PR #246


Provide default main.js for km use case

We should have a default modules/main.js so users do not need to supply one when using default Kepler Mapper.
@yaodong, it looks like the user's main.js is loaded here:
https://github.com/mappercore/mappercore/blob/133eef004390accadf8aba44ca434b9acad13fd6/static/modules/main.js#L69-L73
Any suggestions on the best way to route this to a default module if modules/main.js does not exist?

I'm working on a way to eliminate the requirements for main.js.
It should be able to run with a single Python file.


Use fixed-point notation in toString

This fixes an issue where toString $ toSeconds $ nanoseconds 1 would produce "1e-9s" instead of "0.000000001s".
> Decimal.toString $ Decimal.fromNumber 0.000000001
"1e-9"

> Decimal.toFixed 9 $ Decimal.fromNumber 0.000000001
"0.000000001"

I think we need to revert this.
Unsurprisingly, this is now producing "1.000000000s" instead of "1s", etc.


SvelteKit + Electron = 💔

Describe the bug
There's an issue when working from within ElectronJS, as there are non-web routes involved: they're file-based.
For starters, there's one issue inside generated index.html file, there's a line like:
__sveltekit_wgqb1w = {
  base: new URL(".", location).pathname.slice(1, -1)
};

On Windows, it returns url something like that:
'/C:/Users/Jerry/projects/electron-svelte/out/app.asar/.vite'

So the fact that it starts from /C:/ is already an issue since it then return an error:
app.qFRTHcTk.js:7 Error: Not found: /C:/Users/Jerry/projects/electron-svelte/out/app.asar/.vite/index.html

I tried to intervene into Svelte building pipeline, so when building I make corresponding changes to index.html:
-new URL(".", location).pathname.slice(0, -1)
+new URL(".", location).pathname.slice(1, -1)

Now the url is formed properly, but now I'm stuck into some infinite loop by some reason:
Throttling navigation to prevent the browser from hanging. See https://crbug.com/1038223. Command line switch --disable-ipc-flooding-protection can be used to bypass the protection

It tries to load some initial code it seems, but I'm not sure where and why it breaks – since no error occurs. Only a warning:

There's a lot of blinking involved, so clearly: the thing is looped. It's broken. And it doesn't render any html as you can see.
But if I'm not running from statics, instead from web version like this:
mainWindow.loadURL('http://localhost:5173')

Then it works perfectly fine:

So, the issue only seems related to when running from local static files like this:
mainWindow.loadFile(path.resolve(app.getAppPath(), '.vite/index.html'))

Reproduction
Largely I use this minimal electron-forge repo that explains how to use electron with esm:
https://github.com/jdms754/electron-forge-vite-esm-test
It works perfectly for dev and production: i.e. both from mainWindow.loadUrl and mainWindow.loadFile.
So I cloned it, added Svelte, and it works... But only in dev environment (mainWindow.loadUrl).
When it gets to building an executable with statics, I need to load statics like this:
mainWindow.loadFile(path.resolve(app.getAppPath(), '.vite/index.html'))

Then here, the problem occurs. All as described above: first with wrong base path, then with infinite loop.
Logs
No response
System Info
Windows 10 (22H2)
Node: 20.11.1
Electron: 29.1.4
@sveltejs/adapter-auto: ^3.0.0 => 3.1.1
@sveltejs/adapter-static: 3.0.1 => 3.0.1
@sveltejs/kit: ^2.0.0 => 2.5.3
@sveltejs/vite-plugin-svelte: ^3.0.0 => 3.0.2
svelte: 5.0.0-next.80 => 5.0.0-next.80
vite: ^5.0.3 => 5.1.5

Severity
blocking all usage of SvelteKit
Additional Information
Questions to approach to a resolution:
• At which point the infinite loop happens?
• Can we make changes to resolve that?
And please don't suggest to NOT use SvelteKit: I might see this suggestion coming. Because in Electron if you don't have url section so you basically don't need routing right? Just use basic Svelte? But if I want to support both web-version and electron-version of my app, I still would like to use single codebase with SvelteKit – so in web version I can have this cool routing system that SvelteKit provides, and electron-version: it just works. Plus, even without ability to change url in electron-version I still prefer the routing system of SvelteKit rather than pure Svelte – it helps to organize code so much better!

My guess, something is wrong here, starting at if (hydrate) ...:

It seems it constantly does goto(location.href... and that's the reason of infinite loop.
I'm not too much into the codebase, I don't know all details, but on a glance I can't even imagine how it even works in other use-cases, other than file:// protocol. It almost seems like it should go into infinite loop in every single use-case because it always passes exactly two arguments to kit.start(app, element), without hydrate argument, where at this point it's looped:

There's obviously some magic happening that I don't see, because it's not infinitely looped in other cases than file:// protocol.

I'm sorry for pinging but it seems quite important one, @Rich-Harris can you give some feedback? Like your thoughts of how/where it might go wrong that it goes into infinite loop? Or even more broadly: if you ever plan/want SvelteKit to support when used in conjunction with Electron? Or how the hydration works here. Any feedback appreciated, really. Thanks.

As far as I know SvelteKit apps have never worked when you run them from the disk, you always need a web server of some sort. Have you checked any of the available templates such as https://github.com/FractalHQ/sveltekit-electron ?

For bundling, they use electron-serve, which is basically a file server.
Still, I'd prefer to run statics without any http wrapper.
I am not sure how viable this idea of running statics as is. Maybe it's not viable at all.
I'm sorry if I'm proposing some dumb idea that would take eternity to refactor SvelteKit just to this need.
And, actually, thank you for feedback. I will consider their approach as workaround, maybe it's not that bad as I think.

I think this limitation is mostly around the fact that JS modules (which SvelteKit uses heavily) can't be loaded from file:// urls:

You need to pay attention to local testing — if you try to load the HTML file locally (i.e. with a file:// URL), you'll run into CORS errors due to JavaScript module security requirements. You need to do your testing through a server.

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Modules#other_differences_between_modules_and_standard_scripts
There is a Vite plugin called vite-plugin-singefile that is able to wrap all the CSS and JS into a single file, this can run from a single index.html file. You can use this in Svelte, but I haven't seen anyone have luck with this in SvelteKit (probably due to the complicated nature of SK bundling)

It’s no surprise file:// protocol protected by cors, it wouldn’t be safe for browsers to allow requesting file:// urls from other files: a lot of nasty stuff could be done just from opening a local html page, or worse – from a web page. Web security standards is on high stakes.
But electron is another story. It isn't just a web app, it's an executable app only then a web app, – and for executables it's already assumed they can do much more than simple html file, including file access. Thus, security standards are different. Chrome engine allows to pass --allow-file-access-from-files which negates cors issue for file access. And in Electron, it seems to be enabled by default. So, from an Electron app, I can fetch via file:// protocol no problem.
Here's my little experiment in devtools, inside of electron app:

As you can see, I was able to access index2.html from index1.html both from file:// protocol.
It works both for relative paths, and for absolute paths like:
await fetch('file:///C:/Users/Jerry/projects/electron-template/.vite/renderer/index2.html')
await fetch('index2.html') // equivalent to this

So, this shouldn't be a problem to JS modules. Might require some code changes here and there in SvelteKit, still. Because currently SvelteKit fully assumes it is run from http. But it doesn't have to. I'd love to have Electron as first-class citized in SvelteKit.

@jerrygreen try setting ssr = true in your layout config: https://github.com/mhkeller/sveltekit-electron/

@jerrygreen try setting ssr = true in your layout config: https://github.com/mhkeller/sveltekit-electron/

This doesn't work.

I had problem loading sveltekit in Electron on Mac (svelte with vite works).

The error is also related to goto


Do not block menu-items that contain submenus

Great tool. Just one improvement that would make it even better:
When clicking on a menu-item, you would expect that click to just open up the menu, showint the sub-items, and without popping you the block question.
Best regards and have a nice day!

Duplicate of https://github.com/bsgriggs/mendix-unsaved-changes-message/issues/2


how long did u trained the model when get a reasonable result?

is this loss normal on 1 GPU?
[03/11 14:38:25 c2.utils.events]: eta: 3 days, 15:40:16  iter: 22020  total_loss: 6.084  loss_cls: 4.101  loss_box_wh: 1.746  loss_center_reg: 0.247  time: 0.4291  data_time: 0.0049  lr: 0.002500  max_mem: 4794M

and here is my solver settings:
 SOLVER=dict(
        OPTIMIZER=dict(
            NAME="SGD",
            BASE_LR=0.0025,
            WEIGHT_DECAY=8e-4,
        ),
        LR_SCHEDULER=dict(
            GAMMA=0.1,
            STEPS=(81000, 108000),
            MAX_ITER=826000,
            WARMUP_ITERS=1000,
        ),
        IMS_PER_BATCH=8,
    ),

So have to training 3 days to get a converge result?

is this loss normal on 1 GPU?
[03/11 14:38:25 c2.utils.events]: eta: 3 days, 15:40:16  iter: 22020  total_loss: 6.084  loss_cls: 4.101  loss_box_wh: 1.746  loss_center_reg: 0.247  time: 0.4291  data_time: 0.0049  lr: 0.002500  max_mem: 4794M

and here is my solver settings:
 SOLVER=dict(
        OPTIMIZER=dict(
            NAME="SGD",
            BASE_LR=0.0025,
            WEIGHT_DECAY=8e-4,
        ),
        LR_SCHEDULER=dict(
            GAMMA=0.1,
            STEPS=(81000, 108000),
            MAX_ITER=826000,
            WARMUP_ITERS=1000,
        ),
        IMS_PER_BATCH=8,
    ),

So have to training 3 days to get a converge result?

This loss value seems abnormal and I never tried trained CenterNet on 1 GPU.
2w iter with batchsize 8 equals 1.2k iter on 8GPU with batchsize 128. 1200 iter with loss 6  might be correct.
by the way, since you use 1 GPU but not 8 and use a small batchsize, your MAX_ITER and STEPS should multiply 16.

Yes, 16 OOM on my 12G card. so I am using 8. But does BASE_LR effect the loss curve? So does WARMUP_ITERS?
since origin centernet using epoch to change learining strategy, I experiements this version converge much more slow than original centernet, it can achieve a reasonable result over one night on my single GPU, but this one one night and a half day got results like this 89999 iters:

which locate seems normal, but class totally wrong.
I tested your pretrained model, it's correct. Just Don't how many time it must cost to get a reasonable result.

Yes, 16 OOM on my 12G card. so I am using 8. But does BASE_LR will effect the loss curve too? So does WARMUP_ITERS?
since origin centernet using epoch to change learning rate strategy, I experiemented this version converge much more slow than original centernet, it can achieve a reasonable result over one night on my single GPU, but this one one night and a half day got results like this 89999 iters:

which locate seems normal, but class totally wrong.
I tested your pretrained model, it's correct. Just Don't know how many time it must cost to get a reasonable result.

on eight 2080Ti machine, resnet18 requires about about 20h or less, resnet50 1d, resnet101 nearly 1.5d
anyway, I will try 4-gpu  and 1-gpu version and if it succeed, I will release it.

I am currently have some seems right lr params (resnet50) which can narrow down total loss to 2.0. Results seems reasonable so far:

it cost 2 day already and need more iterations I think

I am currently have some seems right lr params (resnet50) which can narrow down total loss to 2.0. Results seems reasonable so far:

it cost 2 day already and need more iterations I think

Could you please share your parameters on resnet50-backbone resnet? I am struggling with the 6.5 loss


feat: docs search

Docs with Algolia. A bit of a custom implementation so we can customise the search button and the modal. Overrides are as global CSS, and due to the lack of control over the markup, it's not really the cleanest approach. But visually looks good!
UI/design approved by Mikey.

Note: The favourite/remove functionality isnt working. I reached out to algolia but no response yet. I believe it will work only on the production url.

I noticed that the way we're handling section names within each page is causing search results like this:

@markdalgleish the quality of the results is something we need to configure within the crawler. the crawler only crawls the production site. this PR has made a few small changes to offer extra selectors for the crawler, so it should get better


locator failure

When all services fail, a list of failures should be available.  Forex #67 should return these in the error details.

{
  "message": "All services failed for fxrates",
  "details": {
    "fixer.io": "just a test"
  }
}


Added File functionality

I generate a File functionality just to obtain

random file extension
random mime type
random fileName with a few options (file path, name, extension and directory separator)

👍

Merged in 2bf828f


Skip saml_monitor script run if SAML is not used as patron auth protocol

Description
Added check if SAML is being used as patron auth protocol. If it is not being used the saml_monitor script skips the fetching, parsing, validating and saving of the new SAML federated metadata.
Motivation and Context
saml_monitor script is being run every day and it is updating SAML federated metadata in the database even for sites that are not using SAML as a patron authentication mechanism. Idea is to only update SAML federated metadata if SAML is used as a authentication mechanism to save some resources.
Notion ticket
How Has This Been Tested?
Manually run saml_monitor script with and without SAML patron auth service.
Checklist

[ ] I have updated the documentation accordingly.
[x] All new and existing tests passed.

Switching to draft since I realised ticket wanted to skip updating metadata if the federation is not used, not just if SAML is not used.

Blocked, see ticket comment section.


On host page, labels don't show up until refresh

Rancher Version: master 10/19
Steps to Reproduce:

go to host page
add a host
edit the host and add a label

Results: The label does not show up on host until you refresh the browser
Expected:  Label should show up after saving the edit

https://github.com/rancher/rancher/issues/6225


TypeError: can't pickle int objects

ubuntu@ubuntu:~/binwalk-master$ binwalk /home/AthSDK/image/DAP1520A1_FW106B04.bin
Traceback (most recent call last):
  File "/usr/local/bin/binwalk", line 57, in <module>
    main()
  File "/usr/local/bin/binwalk", line 36, in main
    elif not modules.execute():
  File "/usr/local/lib/python2.6/dist-packages/binwalk/core/module.py", line 785, in execute
    obj = self.run(module)
  File "/usr/local/lib/python2.6/dist-packages/binwalk/core/module.py", line 804, in run
    obj = self.load(module, kwargs)
  File "/usr/local/lib/python2.6/dist-packages/binwalk/core/module.py", line 835, in load
    argv.update(self.dependencies(module, argv['enabled']))
  File "/usr/local/lib/python2.6/dist-packages/binwalk/core/module.py", line 862, in dependencies
    depobj = self.run(dependency.module, dependency=True, kwargs=dependency.kwargs)
  File "/usr/local/lib/python2.6/dist-packages/binwalk/core/module.py", line 804, in run
    obj = self.load(module, kwargs)
  File "/usr/local/lib/python2.6/dist-packages/binwalk/core/module.py", line 836, in load
    return module(self, **argv)
  File "/usr/local/lib/python2.6/dist-packages/binwalk/core/module.py", line 254, in __init__
    process_kwargs(self, kwargs)
  File "/usr/local/lib/python2.6/dist-packages/binwalk/core/module.py", line 1009, in process_kwargs
    kwargs = m.kwargs(obj, kwargs)
  File "/usr/local/lib/python2.6/dist-packages/binwalk/core/module.py", line 972, in kwargs
    arg_value = copy(module_argument.default)
  File "/usr/lib/python2.6/copy.py", line 87, in copy
    rv = reductor(2)
  File "/usr/lib/python2.6/copy_reg.py", line 70, in _reduce_ex
    raise TypeError, "can't pickle %s objects" % base.__name__
TypeError: can't pickle int objects

I grabbed a copy of what I assume is the same DAP1520A1_FW106B04.bin firmware from here: http://files.dlink.com.au/Products/DAP-1520/REV_A/Firmware/v1.06b04_06012015/
Using the latest binwalk with that firmware, extraction works as expected:
dg@ubuntu:~/workspace/binwalk_issue_326$ binwalk DAP1520A1_FW106B04.bin

DECIMAL       HEXADECIMAL     DESCRIPTION
--------------------------------------------------------------------------------
0             0x0             uImage header, header size: 64 bytes, header CRC: 0xBA3B64BA, created: 2015-01-22 03:48:48, image size: 909200 bytes, Data Address: 0x80000000, Entry Point: 0x8000C310, data CRC: 0x310BA125, OS: Linux, CPU: MIPS, image type: OS Kernel Image, compression type: lzma, image name: "Linux Kernel Image"
64            0x40            LZMA compressed data, properties: 0x5D, dictionary size: 33554432 bytes, uncompressed size: 2798288 bytes
983040        0xF0000         Squashfs filesystem, little endian, version 4.0, compression:xz, size: 3460300 bytes, 649 inodes, blocksize: 65536 bytes, created: 2015-01-22 03:48:54

dg@ubuntu:~/workspace/binwalk_issue_326$ binwalk -e DAP1520A1_FW106B04.bin

DECIMAL       HEXADECIMAL     DESCRIPTION
--------------------------------------------------------------------------------
0             0x0             uImage header, header size: 64 bytes, header CRC: 0xBA3B64BA, created: 2015-01-22 03:48:48, image size: 909200 bytes, Data Address: 0x80000000, Entry Point: 0x8000C310, data CRC: 0x310BA125, OS: Linux, CPU: MIPS, image type: OS Kernel Image, compression type: lzma, image name: "Linux Kernel Image"
64            0x40            LZMA compressed data, properties: 0x5D, dictionary size: 33554432 bytes, uncompressed size: 2798288 bytes
983040        0xF0000         Squashfs filesystem, little endian, version 4.0, compression:xz, size: 3460300 bytes, 649 inodes, blocksize: 65536 bytes, created: 2015-01-22 03:48:54

Likely binwalk was not properly installed, or this bug has been squashed by a fix since this ticket was opened


Feature Request: Support mount options in volume mount

Is your feature request related to a problem? Please describe.
I am trying to setup a PostgreSQL container app with persistent data mounted from Azure Files. However, PG-SQL only allows that the data folder is owned by a non-root user. Moreover, the doc of PGSQL specifically recommends hard in the mounting option. So, it would be very helpful if user can manually specify the mounting option of volume mounts.
Describe the solution you'd like.
Add a field to the volumeMounts structure in yaml config. The snippet should look like
      volumeMounts:
      - volumeName: postgresql-volume
        mountPath: /var/lib/postgresql/data
        mountOptions: [ "uid=999", "gid=999", "hard" ]

The specified mounting options can be passed to k8s config transparently, or manually appended to the existing mounting options, depending on the implementation.

Another mountOptions that is required is nobrl.
Microsoft states that you should set nobrl when mounting Azure Files volumes. Without this setting, SQLite fails with database is locked.
The attached Bicep file tries to deploy Mealie, which use SQLite in the API container. If you deploy it, you will see that the API fails to start because SQLite throws "database is locked" - as is expected when nobrl is not set.
meailedeploy.zip

Seeing as this feature request might really help us out with resolving issue #534 , are there any concrete plans on including this request in the Container Apps roadmap @anthonychu ?

This feature is being tracked at #765.
It's not available in the portal yet, but you can use ARM or Bicep to add a mountOptions property to the volume.
We'll announce this feature in public preview once portal supports it.
@flytzen I've tested out your Mealie example and it looks like it's working now (shows the login screen, at least). Had to make a couple of other adjustments to the Bicep:

Add mountOptions
Change API version to 2023-04-01-preview
Correct API_URL in frontend
Remove http probe from API (the base path returns 404 and the probe fails, removing the probe results in app using a default TCP probe that checks for connectivity but not status)

param location string = resourceGroup().location

param deploymentName string = 'flytzen-mealie'

// TODO: https://stackoverflow.com/questions/53226642/sqlite3-database-is-locked-in-azure
// https://github.com/MicrosoftDocs/azure-docs/issues/82464
// https://learn.microsoft.com/en-us/troubleshoot/azure/azure-kubernetes/mountoptions-settings-azure-files

resource storageAccount 'Microsoft.Storage/storageAccounts@2022-05-01' = {
  name: '${replace(deploymentName, '-','')}sa'
  location: location
  sku: {
    name: 'Standard_GRS'
  }
  kind: 'StorageV2'

  resource fileShareService 'fileServices' = {
    name: 'default'
    resource fileShare 'shares' = {
      name: '${deploymentName}-fs'
    }
  }
}

resource logAnalytics 'Microsoft.OperationalInsights/workspaces@2021-06-01' = {
  name: '${deploymentName}-la'
  location: location
  properties: {
    sku: {
      name: 'PerGB2018'
    }
  }
}

resource acaenvironment 'Microsoft.App/managedEnvironments@2022-06-01-preview' = {
  name: '${deploymentName}-cae'
  location: location
  sku:{
    name: 'Consumption'
  }
  properties: {
    appLogsConfiguration: {
      destination: 'log-analytics'
      logAnalyticsConfiguration: {
        customerId: logAnalytics.properties.customerId
        sharedKey: logAnalytics.listKeys().primarySharedKey
      }

    }
  }
  resource sharedFiles 'storages@2022-06-01-preview' = {
    name: '${deploymentName}mount'
    properties: {
      azureFile: {
        accountName: storageAccount.name
        accountKey: storageAccount.listKeys().keys[0].value
        accessMode: 'ReadWrite'
        shareName: storageAccount::fileShareService::fileShare.name
      }
    }
  }
}

resource apicontainer 'Microsoft.App/containerApps@2023-04-01-preview' = {
  location: location
  name: '${deploymentName}-api-aca'
  properties: {
    managedEnvironmentId: acaenvironment.id
    configuration: {
      ingress: {
        external: false
        targetPort: 9000
        allowInsecure: true
        traffic: [
          {
            latestRevision: true
            weight: 100
          }
        ]
      }
    }
    template: {
      // revisionSuffix: 'firstrevision'
      containers: [
        {
           name: 'mealie-api'
           image: 'hkotel/mealie:api-v1.0.0beta-5'
           volumeMounts: [
            {
              mountPath:'/app/data'
              volumeName: 'api-volume'
            }
          ]
          resources:{
            memory: '1Gi'
            cpu: json('0.5')
          }
          env: [
            {
              name: 'ALLOW_SIGNUP'
              value: 'true'
            }
            {
              name: 'PUID'
              value: '1000'
            }
            {
              name:'PGID'
              value:'1000'
            }
            {
              name:'TZ'
              value: 'Europe/London'
            }
            {
              name:'MAX_WORKERS'
              value:'1'
            }
            {
              name:'WEB_CONCURRENCY'
              value:'1'
            }
            {
              name:'BASE_URL'
              value: 'https://${deploymentName}-aca.${acaenvironment.properties.defaultDomain}'
            }
          ]
        }
      ]
      volumes:[
        {
          name:'api-volume'
          storageType:'AzureFile'
          storageName:acaenvironment::sharedFiles.name
          mountOptions: 'uid=1000,gid=1000,nobrl,mfsymlinks,cache=none'
        }
      ]
      scale: {
        minReplicas: 0
        maxReplicas: 1
        rules: [
          {
            name: 'http-requests'
            http: {
              metadata: {
                concurrentRequests: '10'
              }
            }
          }
        ]
      }
    }

  }
}

resource frontendcontainer 'Microsoft.App/containerApps@2022-06-01-preview' = {
  location: location
  name: '${deploymentName}-frontend-aca'
  properties: {
    managedEnvironmentId: acaenvironment.id
    configuration: {
      ingress: {
        external: true
        targetPort: 3001
        allowInsecure: false
        traffic: [
          {
            latestRevision: true
            weight: 100
          }
        ]
      }
    }
    template: {
      // revisionSuffix: 'firstrevision'
      containers: [
        {
          name: 'mealie-frontend'
          image: 'hkotel/mealie:frontend-v1.0.0beta-5'
          env: [
            {
              name: 'API_URL'
              value: 'https://${deploymentName}-api-aca.internal.${acaenvironment.properties.defaultDomain}'
            }
          ]
          volumeMounts: [
            {
              mountPath:'/app/data'
              volumeName: 'frontend-volume'
            }
          ]
        }
      ]
      volumes:[
        {
          name:'frontend-volume'
          storageType:'AzureFile'
          storageName:acaenvironment::sharedFiles.name
        }
      ]
      scale: {
        minReplicas: 0
        maxReplicas: 1
        rules: [
          {
            name: 'http-requests'
            http: {
              metadata: {
                concurrentRequests: '10'
              }
            }
          }
        ]
      }
    }
  }
}

This feature is being tracked at #765.
It's not available in the portal yet, but you can use ARM or Bicep to add a mountOptions property to the volume.
We'll announce this feature in public preview once portal supports it.
@flytzen I've tested out your Mealie example and it looks like it's working now (shows the login screen, at least). Had to make a couple of other adjustments to the Bicep:

Awesome, that's going far above and beyond. Thank you!

@anthonychu can we still play around with the documented „mountOptions“ field via ARM/Bicep? may I ask if this will be in private preview or public preview soon?

When editing a container app's volumes, there is a mount options field. So I guess it's available now ?

Can this be done in Azure Container Instances?