Bug: Distributed agents in JAX may be initialized with the same random key.

Hi,
I am currently implementing some distributed JAX agents in JAX following the implementation of TF D4PG agents. I am adapting the builder for this. However, I noticed that using the interface results in distributed actors being initialized with the same random key.
Here is a snippet for reproducing the problem.
from launchpad.nodes.python.local_multi_processing import PythonProcess
from complr.research.skills.lp_run import main
import haiku as hk
import launchpad as lp
import jax.numpy as jnp
import jax

class Actor:
    def __init__(self, random_key) -> None:
        self._random_key = random_key

    def run(self):
        print(self._random_key)

class DistributedAgent:
    def __init__(self, seed):
        self._key = jax.random.PRNGKey(seed)

    def actor(self):
        key, self._key = jax.random.split(self._key)
        return Actor(key)

    def build(self):
        program = lp.Program(name="program")

        with program.group("actor"):
            for _ in range(4):
                _ = program.add_node(lp.CourierNode(self.actor))
        return program

def main():
    agent = Agent(0)
    program = agent.build()
    local_resources = {
        "actor": PythonProcess(env={"CUDA_VISIBLE_DEVICES": ""}),
    }
    lp.launch(
        program,
        lp.LaunchType.LOCAL_MULTI_PROCESSING,
        terminal="current_terminal",
        local_resources=local_resources,
    )

if __name__ == "__main__":
    main()

From the terminal, you will see the following result (filtered irrelevant log messages).

[actor/0] I0709 00:07:24.056859<PHONE_NUMBER>36832 courier_utils.py:51] Binding: run
[actor/0] [4146024105  967050713]
[actor/2] I0709 00:07:24.108036<PHONE_NUMBER>21984 courier_utils.py:51] Binding: run
[actor/1] I0709 00:07:24.108819<PHONE_NUMBER>52288 courier_utils.py:51] Binding: run
[actor/2] [4146024105  967050713]
[actor/1] [4146024105  967050713]
[actor/3] I0709 00:07:24.169437<PHONE_NUMBER>70400 courier_utils.py:51] Binding: run
[actor/3] [4146024105  967050713]

From the output you can see that all actors are initialized with the same random key, despite us splitting the key in the actor method. I think the issue is because LaunchPad will (cloud)pickle the constructor and recreate that in the subprocesses. However, the pickling is lazy and happens before the method is called and consequently the pickle captures the seed in the builder before the splitting begins.
It would perhaps be useful to revisit the commit f788e7574e1e6102e81069cf5fe1ef299a95a08d which introduced these changes.

Hi Yicheng,
Your agent could be updated as follows to pass different keys to different actor nodes:
from launchpad.nodes.python.local_multi_processing import PythonProcess
from complr.research.skills.lp_run import main
import haiku as hk
import launchpad as lp
import jax.numpy as jnp
import jax

class Actor:
    def __init__(self, random_key) -> None:
        self._random_key = random_key

    def run(self):
        print(self._random_key)

class DistributedAgent:
    def __init__(self, seed):
        self._seed = seed

    def actor(self, key):
        return Actor(key)

    def build(self):
        program = lp.Program(name="program")

        key = jax.random.PRNGKey(seed)

        with program.group("actor"):
            for _ in range(4):
                actor_key, key = jax.random.split(key)
                _ = program.add_node(lp.CourierNode(self.actor, actor_key))
        return program

def main():
    agent = Agent(0)
    program = agent.build()
    local_resources = {
        "actor": PythonProcess(env={"CUDA_VISIBLE_DEVICES": ""}),
    }
    lp.launch(
        program,
        lp.LaunchType.LOCAL_MULTI_PROCESSING,
        terminal="current_terminal",
        local_resources=local_resources,
    )

if __name__ == "__main__":
    main()

Please let me know if it doesn't work for you.
Thanks,
Danila

Hi Danila,
Yeah that's exactly what did in the end. I realize that for me to write the distributed agents, they may need to be structured slightly differently compared to in tf.


You have 'and' twice in first paragraph

[Enter feedback here]

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: b7ebefef-f8c5-e429-269e-1ef7a96264fd
Version Independent ID: a8009495-9bb9-87f4-85af-0ac0074749e7
Content: What is PolyBase? - SQL Server
Content Source: docs/relational-databases/polybase/polybase-guide.md
Product: sql
Technology: polybase
GitHub Login: @MikeRayMSFT
Microsoft Alias: mikeray

@matthewcrow -- Matthew, thank you for bringing this to our attention.  I am looking into the article.

Thank you for your feedback.  I have updated the documentation and it should be live by tomorrow.


ssh key is not passed in container. expert mode

Steps to reproduce:
$ 3sdk --expert
3sdk> container start name=test
3sdk> container shell name=test

root@test:/sandbox# kosmos
JSX> j.clients.sshkey.default

Not a bug this is not needed for anything


Retrieve additional IC/TS statistics with getattribute & get_image_info()

Additional IC-wide (and TS-wide) queries for getattribute():
total_files
all_filenames

And now a variety of per-file statistics may be retrieved via
get_image_info():
broken
stat:tilesread
stat:bytesread
stat:redundant_tiles
stat:redundant_bytesread
stat:timesopened
stat:iotime
stat:mipused
stat:is_duplicate

Any opinions on this, @sorenr ?

Soren says good.


Add instructions on how to update the Hail TLS certs on Google Cloud

@violetbrina Thanks! Do you want to try following the instructions? We'll need to update the certs before the end of the month anyway, so this would be a good opportunity to check that everything is documented correctly.


python311Packages.zope-configuration: 4.4.1 -> 5.0; rename frome zope_configuration

Description of changes
Changelog: https://github.com/zopefoundation/zope.configuration/blob/5.0/CHANGES.rst
part of #245383

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[x] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 266888 at 98b74e8a run on x86_64-linux 1

8 packages failed to build:

apache-airflow
fusesoc
python310Packages.connexion
python310Packages.robotframework-selenium2library
python310Packages.robotframework-seleniumlibrary
python311Packages.connexion
python311Packages.robotframework-selenium2library
python311Packages.robotframework-seleniumlibrary

116 packages built successfully:

awslimitchecker
backblaze-b2
buildbot
buildbot-full
buildbot-ui
bump2version
certbot (python311Packages.certbot)
certbot-full
check-esxi-hardware
deja-dup
devpi-client
devpi-server
duplicity
duply
giara
home-assistant-component-tests.google
home-assistant-component-tests.local_calendar
home-assistant-component-tests.local_todo
home-assistant-component-tests.openerz
home-assistant-component-tests.rainbird
home-assistant-component-tests.reddit
mailman
mailman-hyperkitty
pdb2pqr (python311Packages.pdb2pqr)
python310Packages.aiohttp-apispec
python310Packages.approvaltests
python310Packages.b2sdk
python310Packages.certbot
python310Packages.certbot-dns-cloudflare
python310Packages.certbot-dns-google
python310Packages.certbot-dns-inwx
python310Packages.certbot-dns-ovh
python310Packages.certbot-dns-rfc2136
python310Packages.certbot-dns-route53
python310Packages.cornice
python310Packages.cypherpunkpay
python310Packages.deform
python310Packages.django-simple-captcha
python310Packages.gcal-sync
python310Packages.ical
python310Packages.logfury
python310Packages.okonomiyaki
python310Packages.openerz-api
python310Packages.pdb2pqr
python310Packages.praw
python310Packages.prawcore
python310Packages.pykwalify
python310Packages.pypugjs
python310Packages.pyradiomics
python310Packages.pyrainbird
python310Packages.pyramid
python310Packages.pyramid-chameleon
python310Packages.pyramid-exclog
python310Packages.pyramid-jinja2
python310Packages.pyramid-mako
python310Packages.pyramid-multiauth
python310Packages.pytest-golden
python310Packages.pywbem
python310Packages.scrapy
python310Packages.scrapy-deltafetch
python310Packages.scrapy-fake-useragent
python310Packages.scrapy-splash
python310Packages.simplesat
python310Packages.slack-bolt
python310Packages.sopel
python310Packages.testfixtures
python310Packages.webargs
python310Packages.west
python310Packages.zope-component
python310Packages.zope-configuration
python311Packages.aiohttp-apispec
python311Packages.approvaltests
python311Packages.b2sdk
python311Packages.certbot-dns-cloudflare
python311Packages.certbot-dns-google
python311Packages.certbot-dns-inwx
python311Packages.certbot-dns-ovh
python311Packages.certbot-dns-rfc2136
python311Packages.certbot-dns-route53
python311Packages.cornice
python311Packages.cypherpunkpay
python311Packages.deform
python311Packages.django-simple-captcha
python311Packages.gcal-sync
python311Packages.ical
python311Packages.logfury
python311Packages.okonomiyaki
python311Packages.openerz-api
python311Packages.praw
python311Packages.prawcore
python311Packages.pykwalify
python311Packages.pypugjs
python311Packages.pyradiomics
python311Packages.pyrainbird
python311Packages.pyramid
python311Packages.pyramid-chameleon
python311Packages.pyramid-exclog
python311Packages.pyramid-jinja2
python311Packages.pyramid-mako
python311Packages.pyramid-multiauth
python311Packages.pytest-golden
python311Packages.pywbem
python311Packages.scrapy
python311Packages.scrapy-deltafetch
python311Packages.scrapy-fake-useragent
python311Packages.scrapy-splash
python311Packages.simplesat
python311Packages.slack-bolt
python311Packages.sopel
python311Packages.testfixtures
python311Packages.webargs
python311Packages.west
python311Packages.zope-component
python311Packages.zope-configuration
seahub
visidata

@OfBorg build mailman


Ability to Change Profile Picture

Is your feature request related to a problem? Please describe.
I don't want my profile picture to be the same one I use for GitHub, but there doesn't seem to be an option to actually upload a picture
Describe the solution you'd like
I would like to be able to upload a picture as my profile.
Describe alternatives you've considered
...
Additional context
There is no button to add a profile picture, we should change that.

same as #1398

Closing as duplicate of #1398


Q on code sample

Dear,
The code sample on Reserved Instance Usage Summary
it says : https://management.azure.com/{scope}/Microsoft.Consumption/reservationSummaries?grain=daily&$filter=properties/usageDate ge 2017-10-01 AND properties/usageDate le 2017-11-20&api-version=2019-10-01
Shouldn't it be
https://management.azure.com/{scope}/**providers/**Microsoft.Consumption/reservationSummaries?grain=daily&$filter=properties/usageDate ge 2017-10-01 AND properties/usageDate le 2017-11-20&api-version=2019-10-01
Regards,
Thierry

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 16ccaf43-f0cb-7524-fa29-0122fc6c8389
Version Independent ID: 11a33b15-5827-8e11-10d1-67a6bfbed758
Content: Migrate from Enterprise Reporting to Azure Resource Manager APIs
Content Source: articles/cost-management-billing/costs/migrate-from-enterprise-reporting-to-azure-resource-manager-apis.md
Service: cost-management-billing
Sub-service: common
GitHub Login: @bandersmsft
Microsoft Alias: banders

@ThierryVanacker-TomTom
Our SME will review this issue and get back soon.

Hi @ThierryVanacker-TomTom Thanks for raising this issue. The sample you mentioned doesn't have providers in it because it's expected that you replace {scope} with providers/Microsoft.Billing/billingAccounts/{billingAccountId}.
I hope this helps.
Relatedly, for https://github.com/MicrosoftDocs/azure-docs/issues/85440, I made a minor update to change our example from:
https://management.azure.com/{scope}/Microsoft.Consumption/reservationSummaries?grain=daily&$filter=properties/usageDate ge 2017-10-01 AND properties/usageDate le 2017-11-20&api-version=2019-10-01
to:
https://management.azure.com/{scope}/Microsoft.Consumption/reservationSummaries?grain=monthly&$filter=properties/usageDate ge 2017-10-01 AND properties/usageDate le 2017-11-20&api-version=2019-10-01
#please-close


Json is wrong for this template

json is not correct, the extension for this template must be .bicep

@johnmarco

Can you review this PR?
IMPORTANT: When this content is ready to merge, you must add #sign-off in a comment or the approval may get overlooked.

#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team

#sign-off


openWire would allow one extra send

Looking at the test, I think it correctly reproduces the problem we were seeing. Thanks for the quick fix!


Better message for wrong ISS

Discussed here: https://stackoverflow.com/questions/46097124/keycloak-grant-validation-failed-reason-invalid-token-wrong-iss

Currently trying to track the ISS error. I could really use this improved logging.


Set up logging for the k8s pods.

Sounds like setting up Grafana?

Sounds good! It looks like the log for pods are automatically available (not sure tho)

That is what I assume too. In thay case, I think it'd make sense for us to show the users which places to look for to get the pod logs.


how to change size of emojiconSize programmatically ?

X  KW  BNQHL NPD B-) VBBNHNGHH
On Jul 7, 2017 11:05 AM, "Milad Ranjbar"<EMAIL_ADDRESS>wrote:

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/rockerhieu/emojicon/issues/165, or mute the thread
https://github.com/notifications/unsubscribe-auth/ARRKypKBXzePmmSPmZ9k2b7rn9tPhelJks5sLdHCgaJpZM4OQi70
.

@miladr0 hope it will help:
emojiconTextView.setEmojiconSize(32); //32 pixels.
Use something like (int)(32*getResources().getDisplayMetrics().density) to set size 32dp from code

@kot331107 tnx for your help


units: added network-online.target dependency for user-cloudinit-proc…

…-cmdline.service
/cc @robszumski @marineam @crawford
When dhcp server works slowly, coreos-cloudinit just fails because network is not yet ready. This commit should solve the issue.

coreos-cloudinit will retry connections for a while. Is this not long enough?

		MaxBackoff: time.Second * 5,
		MaxRetries: 15,
		Timeout:    time.Duration(2) * time.Second,

Does it mean cloudinit will wait maximum 5 sec?

The actual code is this:
	hc := &HttpClient{
		InitialBackoff: 50 * time.Millisecond,
		MaxBackoff:     time.Second * 5,
		MaxRetries:     15,
		SkipTLS:        false,
		client: &http.Client{
			Timeout: 10 * time.Second,
		},
	}

Each connection will time out after 10 seconds. After a connection fails, coreos-cloudinit will wait before reattempting. It starts at 50 milliseconds, doubling until it's clamped at 5 seconds. It does this 15 times for a grand total of 46.35s (assuming my arithmetic is correct).

Yep, it seems 45 seconds is not enough...
Fetching data from http://<IP_ADDRESS>/yaml.yaml. Attempt #1
Unable to fetch data: Get http://<IP_ADDRESS>/yaml.yaml: dial tcp <IP_ADDRESS>:80: network is unreachable
Sleeping for 100ms...
Fetching data from http://<IP_ADDRESS>/yaml.yaml. Attempt #2
Unable to fetch data: Get http://<IP_ADDRESS>/yaml.yaml: dial tcp <IP_ADDRESS>:80: network is unreachable
Sleeping for 200ms...
Fetching data from http://<IP_ADDRESS>/yaml.yaml. Attempt #3
Unable to fetch data: Get http://<IP_ADDRESS>/yaml.yaml: dial tcp <IP_ADDRESS>:80: network is unreachable
Sleeping for 400ms...
Fetching data from http://<IP_ADDRESS>/yaml.yaml. Attempt #4
Unable to fetch data: Get http://<IP_ADDRESS>/yaml.yaml: dial tcp <IP_ADDRESS>:80: network is unreachable
Sleeping for 800ms...
Fetching data from http://<IP_ADDRESS>/yaml.yaml. Attempt #5
Unable to fetch data: Get http://<IP_ADDRESS>/yaml.yaml: dial tcp <IP_ADDRESS>:80: network is unreachable
Sleeping for 1.6s...
Fetching data from http://<IP_ADDRESS>/yaml.yaml. Attempt #6
Unable to fetch data: Get http://<IP_ADDRESS>/yaml.yaml: dial tcp <IP_ADDRESS>:80: network is unreachable
Sleeping for 3.2s...
Fetching data from http://<IP_ADDRESS>/yaml.yaml. Attempt #7
Unable to fetch data: Get http://<IP_ADDRESS>/yaml.yaml: dial tcp <IP_ADDRESS>:80: network is unreachable
Sleeping for 5s...
Fetching data from http://<IP_ADDRESS>/yaml.yaml. Attempt #8
Unable to fetch data: Get http://<IP_ADDRESS>/yaml.yaml: dial tcp <IP_ADDRESS>:80: network is unreachable
Sleeping for 5s...
Fetching data from http://<IP_ADDRESS>/yaml.yaml. Attempt #9
Unable to fetch data: Get http://<IP_ADDRESS>/yaml.yaml: dial tcp <IP_ADDRESS>:80: network is unreachable
Sleeping for 5s...
Fetching data from http://<IP_ADDRESS>/yaml.yaml. Attempt #10
Unable to fetch data: Get http://<IP_ADDRESS>/yaml.yaml: dial tcp <IP_ADDRESS>:80: network is unreachable
Sleeping for 5s...
Fetching data from http://<IP_ADDRESS>/yaml.yaml. Attempt #11
Unable to fetch data: Get http://<IP_ADDRESS>/yaml.yaml: dial tcp <IP_ADDRESS>:80: network is unreachable
Sleeping for 5s...
Fetching data from http://<IP_ADDRESS>/yaml.yaml. Attempt #12
Unable to fetch data: Get http://<IP_ADDRESS>/yaml.yaml: dial tcp <IP_ADDRESS>:80: network is unreachable
Sleeping for 5s...
Fetching data from http://<IP_ADDRESS>/yaml.yaml. Attempt #13
Unable to fetch data: Get http://<IP_ADDRESS>/yaml.yaml: dial tcp <IP_ADDRESS>:80: network is unreachable
Sleeping for 5s...
Fetching data from http://<IP_ADDRESS>/yaml.yaml. Attempt #14
Unable to fetch data: Get http://<IP_ADDRESS>/yaml.yaml: dial tcp <IP_ADDRESS>:80: network is unreachable
Sleeping for 5s...
Fetching data from http://<IP_ADDRESS>/yaml.yaml. Attempt #15
Unable to fetch data: Get http://<IP_ADDRESS>/yaml.yaml: dial tcp <IP_ADDRESS>:80: network is unreachable
Sleeping for 5s...
localhost coreos-cloudinit[441]: Failed fetching user-data from datasource: Unable to fetch data.  I guess this is OK. @marineam do you foresee any issues with this change?

My only thought is that since ConditionKernelCommandLine is used to control whether this unit runs network-online.target is going to get pulled into the boot process regardless of whether that option was specified. This won't be an issue for most system but it will likely delay booting when networking is unavailable. The network online job will eventually time out so boot won't stall forever but may be problematic nonetheless. Two options: enable the unit with a generator so systemd doesn't even look at it if the option isn't there, or increase the timeout and retry count in cloudinit so it'll spin for about 3 minutes or so and not need this extra dependency.

Hi guys,
I'm the one reporting that issue on irc so I add the history here.
I'm booting my servers with ipxe and for some reason (I'm still investigating) the interface takes a lot of time to get an IP.
As the unit user-cloudinit-proc-cmdline.service is started before I have network, it times out and I can't configure the server but I think this fix will be enough to fix my problem.
And yes I'm aware that there is a problem on my network. For info, if a network guy reads this, each server is attached to two switches cisco n3k configured with a VPC and LACP between the two interfaces. There is a trunk of two vlans, one is native. I have the option : spanning-tree port type edge trunk (before I add this it was even slower). Without the trunk, everything works fine.
Here is my log :
Sep 09 15:20:25 localhost systemd[1]: Starting Load cloud-config from url defined in /proc/cmdline...
Sep 09 15:20:25 localhost kernel: IPv6: ADDRCONF(NETDEV_UP): eno4: link is not ready
Sep 09 15:20:25 localhost coreos-cloudinit[1102]: Checking availability of "proc-cmdline"
Sep 09 15:20:25 localhost coreos-cloudinit[1102]: Fetching user-data from datasource of type "proc-cmdline"
Sep 09 15:20:25 localhost coreos-cloudinit[1102]: 2015/09/09 15:20:25 Fetching data from http://mir.XXXX/internal/cor
Sep 09 15:20:25 localhost coreos-cloudinit[1102]: 2015/09/09 15:20:25 Unable to fetch data: Get http://mir.XXXX/inter
Sep 09 15:20:25 localhost coreos-cloudinit[1102]: 2015/09/09 15:20:25 Sleeping for 100ms...
Sep 09 15:20:25 localhost coreos-cloudinit[1102]: 2015/09/09 15:20:25 Fetching data from http://mir.XXXX/internal/cor
Sep 09 15:20:25 localhost coreos-cloudinit[1102]: 2015/09/09 15:20:25 Unable to fetch data: Get http://mir.XXXX/inter
Sep 09 15:20:25 localhost coreos-cloudinit[1102]: 2015/09/09 15:20:25 Sleeping for 200ms...
Sep 09 15:20:25 localhost kernel: IPv6: ADDRCONF(NETDEV_UP): eno3: link is not ready
Sep 09 15:20:25 localhost systemd[1]: Reached target Load system-provided cloud configs.
Sep 09 15:20:25 localhost systemd[1]: Starting Load system-provided cloud configs.
Sep 09 15:20:25 localhost systemd[1]: Started Run Addon /usr/share/oem.
Sep 09 15:20:25 localhost systemd[1]: Started Network Service.
Sep 09 15:20:25 localhost coreos-cloudinit[1102]: 2015/09/09 15:20:25 Fetching data from http://mir.XXXX/internal/cor
Sep 09 15:20:25 localhost coreos-cloudinit[1102]: 2015/09/09 15:20:25 Unable to fetch data: Get http://mir.XXXX/inter
Sep 09 15:20:25 localhost coreos-cloudinit[1102]: 2015/09/09 15:20:25 Sleeping for 400ms...
Sep 09 15:20:25 localhost systemd[1]: Started Garbage Collection for rkt.
Sep 09 15:20:25 localhost kernel: IPv6: ADDRCONF(NETDEV_UP): eno2: link is not ready
Sep 09 15:20:25 localhost systemd[1]: Started Extend Filesystems.
Sep 09 15:20:25 localhost sshd_keygen[1064]: ssh-keygen: generating new host keys: RSA1 RSA DSA ED25519
Sep 09 15:20:26 localhost systemd[1]: Started Install an ssh key from /proc/cmdline.
Sep 09 15:20:26 localhost kernel: IPv6: ADDRCONF(NETDEV_UP): eno1: link is not ready
Sep 09 15:20:26 localhost systemd-networkd[1093]: lo: Configured
Sep 09 15:20:26 localhost systemd[1]: Started Generate sshd host keys.
Sep 09 15:20:26 localhost systemd-logind[1098]: Watching system buttons on /dev/input/event0 (Power Button)
Sep 09 15:20:26 localhost systemd-logind[1098]: New seat seat0.
Sep 09 15:20:26 localhost systemd[1]: Starting Generate /run/issue...
Sep 09 15:20:26 localhost coreos-cloudinit[1102]: 2015/09/09 15:20:26 Fetching data from http://mir.XXXX/internal/cor
Sep 09 15:20:26 localhost coreos-cloudinit[1102]: 2015/09/09 15:20:26 Unable to fetch data: Get http://mir.XXXX/inter
Sep 09 15:20:26 localhost coreos-cloudinit[1102]: 2015/09/09 15:20:26 Sleeping for 800ms...
Sep 09 15:20:26 localhost systemd[1]: Reached target Network.
Sep 09 15:20:26 localhost systemd[1]: Starting Network.
Sep 09 15:20:26 localhost systemd[1]: Starting Network Name Resolution...
Sep 09 15:20:26 localhost systemd-resolved[1143]: Using system hostname 'localhost'.
Sep 09 15:20:26 localhost systemd[1]: Started Network Name Resolution.
Sep 09 15:20:26 localhost systemd[1]: Started Generate /run/issue.
Sep 09 15:20:26 localhost systemd[1]: Started Login Service.
Sep 09 15:20:26 localhost systemd[1]: Started Generate /run/coreos/motd.
Sep 09 15:20:26 localhost coreos-cloudinit[1102]: 2015/09/09 15:20:26 Fetching data from http://mir.XXXX/internal/cor
Sep 09 15:20:26 localhost coreos-cloudinit[1102]: 2015/09/09 15:20:26 Unable to fetch data: Get http://mir.XXXX/inter
Sep 09 15:20:26 localhost coreos-cloudinit[1102]: 2015/09/09 15:20:26 Sleeping for 1.6s...
Sep 09 15:20:28 localhost coreos-cloudinit[1102]: 2015/09/09 15:20:28 Fetching data from http://mir.XXXX/internal/cor
Sep 09 15:20:28 localhost coreos-cloudinit[1102]: 2015/09/09 15:20:28 Unable to fetch data: Get http://mir.XXXX/inter
Sep 09 15:20:28 localhost coreos-cloudinit[1102]: 2015/09/09 15:20:28 Sleeping for 3.2s...
Sep 09 15:20:29 localhost kernel: tg3 0000:01:00.1 eno2: Link is up at 1000 Mbps, full duplex
Sep 09 15:20:29 localhost kernel: tg3 0000:01:00.1 eno2: Flow control is on for TX and on for RX
Sep 09 15:20:29 localhost kernel: tg3 0000:01:00.1 eno2: EEE is disabled
Sep 09 15:20:30 localhost kernel: tg3 0000:01:00.0 eno1: Link is up at 1000 Mbps, full duplex
Sep 09 15:20:30 localhost kernel: tg3 0000:01:00.0 eno1: Flow control is on for TX and on for RX
Sep 09 15:20:30 localhost kernel: tg3 0000:01:00.0 eno1: EEE is disabled
Sep 09 15:20:30 localhost kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eno2: link becomes ready
Sep 09 15:20:30 localhost systemd-networkd[1093]: eno2: Gained carrier
Sep 09 15:20:30 localhost kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eno1: link becomes ready
Sep 09 15:20:30 localhost systemd-networkd[1093]: eno1: Gained carrier
Sep 09 15:20:31 localhost systemd-timesyncd[1031]: Network configuration changed, trying to establish connection.
Sep 09 15:20:31 localhost coreos-cloudinit[1102]: 2015/09/09 15:20:31 Fetching data from http://mir.XXXX/internal/cor
Sep 09 15:20:31 localhost coreos-cloudinit[1102]: 2015/09/09 15:20:31 Unable to fetch data: Get http://mir.XXXX/inter
Sep 09 15:20:31 localhost coreos-cloudinit[1102]: 2015/09/09 15:20:31 Sleeping for 5s...
Sep 09 15:20:32 localhost systemd-timesyncd[1031]: Network configuration changed, trying to establish connection.
Sep 09 15:20:36 localhost coreos-cloudinit[1102]: 2015/09/09 15:20:36 Fetching data from http://mir.XXXX/internal/cor
Sep 09 15:20:36 localhost coreos-cloudinit[1102]: 2015/09/09 15:20:36 Unable to fetch data: Get http://mir.XXXX/inter
Sep 09 15:20:36 localhost coreos-cloudinit[1102]: 2015/09/09 15:20:36 Sleeping for 5s...
Sep 09 15:20:41 localhost coreos-cloudinit[1102]: 2015/09/09 15:20:41 Fetching data from http://mir.XXXX/internal/cor
Sep 09 15:20:41 localhost coreos-cloudinit[1102]: 2015/09/09 15:20:41 Unable to fetch data: Get http://mir.XXXX/inter
Sep 09 15:20:41 localhost coreos-cloudinit[1102]: 2015/09/09 15:20:41 Sleeping for 5s...
Sep 09 15:20:46 localhost coreos-cloudinit[1102]: 2015/09/09 15:20:46 Fetching data from http://mir.XXXX/internal/cor
Sep 09 15:20:46 localhost coreos-cloudinit[1102]: 2015/09/09 15:20:46 Unable to fetch data: Get http://mir.XXXX/inter
Sep 09 15:20:46 localhost coreos-cloudinit[1102]: 2015/09/09 15:20:46 Sleeping for 5s...
Sep 09 15:20:51 localhost coreos-cloudinit[1102]: 2015/09/09 15:20:51 Fetching data from http://mir.XXXX/internal/cor
Sep 09 15:20:51 localhost coreos-cloudinit[1102]: 2015/09/09 15:20:51 Unable to fetch data: Get http://mir.XXXX/inter
Sep 09 15:20:51 localhost coreos-cloudinit[1102]: 2015/09/09 15:20:51 Sleeping for 5s...
Sep 09 15:20:56 localhost coreos-cloudinit[1102]: 2015/09/09 15:20:56 Fetching data from http://mir.XXXX/internal/cor
Sep 09 15:20:56 localhost coreos-cloudinit[1102]: 2015/09/09 15:20:56 Unable to fetch data: Get http://mir.XXXX/inter
Sep 09 15:20:56 localhost coreos-cloudinit[1102]: 2015/09/09 15:20:56 Sleeping for 5s...
Sep 09 15:21:01 localhost coreos-cloudinit[1102]: 2015/09/09 15:21:01 Fetching data from http://mir.XXXX/internal/cor
Sep 09 15:21:01 localhost coreos-cloudinit[1102]: 2015/09/09 15:21:01 Unable to fetch data: Get http://mir.XXXX/inter
Sep 09 15:21:01 localhost coreos-cloudinit[1102]: 2015/09/09 15:21:01 Sleeping for 5s...
Sep 09 15:21:06 localhost coreos-cloudinit[1102]: 2015/09/09 15:21:06 Fetching data from http://mir.XXXX/internal/cor
Sep 09 15:21:06 localhost coreos-cloudinit[1102]: 2015/09/09 15:21:06 Unable to fetch data: Get http://mir.XXXX/inter
Sep 09 15:21:06 localhost coreos-cloudinit[1102]: 2015/09/09 15:21:06 Sleeping for 5s...
Sep 09 15:21:11 localhost coreos-cloudinit[1102]: 2015/09/09 15:21:11 Fetching data from http://mir.XXXX/internal/cor
Sep 09 15:21:11 localhost coreos-cloudinit[1102]: 2015/09/09 15:21:11 Unable to fetch data: Get http://mir.XXXX/inter
Sep 09 15:21:11 localhost coreos-cloudinit[1102]: 2015/09/09 15:21:11 Sleeping for 5s...
Sep 09 15:21:16 localhost coreos-cloudinit[1102]: Failed fetching user-data from datasource: Unable to fetch data. Maximum retries rea
Sep 09 15:21:16 localhost coreos-cloudinit[1102]: Continuing...
Sep 09 15:21:16 localhost coreos-cloudinit[1102]: Fetching meta-data from datasource of type "proc-cmdline"
Sep 09 15:21:16 localhost coreos-cloudinit[1102]: Merging cloud-config from meta-data and user-data
Sep 09 15:21:16 localhost coreos-cloudinit[1102]: 2015/09/09 15:21:16 Ensuring runtime unit file "etcd.service" is unmasked
Sep 09 15:21:16 localhost coreos-cloudinit[1102]: 2015/09/09 15:21:16 Ensuring runtime unit file "etcd2.service" is unmasked
Sep 09 15:21:16 localhost coreos-cloudinit[1102]: 2015/09/09 15:21:16 Ensuring runtime unit file "fleet.service" is unmasked
Sep 09 15:21:16 localhost coreos-cloudinit[1102]: 2015/09/09 15:21:16 Ensuring runtime unit file "locksmithd.service" is unmasked
Sep 09 15:21:16 localhost systemd[1]: user-cloudinit-proc-cmdline.service: Main process exited, code=exited, status=1/FAILURE
Sep 09 15:21:16 localhost systemd[1]: Failed to start Load cloud-config from url defined in /proc/cmdline.
Sep 09 15:21:16 localhost systemd[1]: Dependency failed for Load user-provided cloud configs.
Sep 09 15:21:16 localhost systemd[1]: user-config.target: Job user-config.target/start failed with result 'dependency'.
Sep 09 15:21:16 localhost systemd[1]: user-cloudinit-proc-cmdline.service: Unit entered failed state.
Sep 09 15:21:16 localhost systemd[1]: user-cloudinit-proc-cmdline.service: Failed with result 'exit-code'.
Sep 09 15:21:16 localhost systemd[1]: Reached target Multi-User System.
Sep 09 15:21:16 localhost systemd[1]: Startup finished in 13.597s (kernel) + 32.578s (initrd) + 1min 492ms (userspace) = 1min 46.668s.
Sep 09 15:21:16 localhost systemd[1]: Starting Multi-User System.
Sep 09 15:21:32 localhost systemd[1]: Starting Generate /run/coreos/motd...
Sep 09 15:21:32 localhost systemd[1]: Started Generate /run/coreos/motd.
Sep 09 15:22:33 localhost systemd[1]: Starting Generate /run/coreos/motd...
Sep 09 15:22:33 localhost systemd[1]: Started Generate /run/coreos/motd.
Sep 09 15:22:39 localhost systemd-networkd[1093]: eno1: DHCPv4 address <IP_ADDRESS>/21 via <IP_ADDRESS>
Sep 09 15:22:39 localhost systemd-networkd[1093]: eno1: Configured

OK, I've fixed my network a bit. Now it takes "only" 34s once the interface is up to get an IP so I can get the config.
The Cisco tip of the day : when using trunking and bonding put all your interfaces in spanning-tree port type edge trunk and not only the Port-channel interfaces.


bug: some chains don't check for upgrades.

https://twitter.com/rhinostake/status/1698394405760377072
For some reason, umee did not find a rest_server to use to check for upgrades despite the logs showing
Found 16 rest endpoints and 16 rpc endpoints for umee
Completed fetch data for network umee

{
  "type": "mainnet",
  "network": "umee",
  "rest_server": "",
  "rpc_server": "https://rpc-umee.cosmos-spaces.cloud",
  "latest_block_height": 8179463,
  "upgrade_found": false,
  "upgrade_name": "",
  "source": "",
  "upgrade_block_height": null,
  "estimated_upgrade_time": null,
  "version": ""
}

I think the problem is bigger than umee.  We need to have checks to ensure each chain has at least one functional RPC/REST server from the CR.

This one is a problem here:
https://github.com/DefiantLabs/cosmos-upgrades/blob/a686a929fb196ad1546278d9f2134e36ad83d49f/app.py#L280-L282
A bunch of networks are returning empty health networks. This means either:

The chain.json file for the network has no rest endpoints
None of the REST servers presented in the chain.json (if any are provided) are responding to /health endpoint checks which we use to determine if the REST endpoint is healthy

I put in a guard for healthy rest endpoints checks and this happens a lot:
2023-09-04 14:17:42 No healthy REST endpoints found for network sei. Skipping...
2023-09-04 14:17:42 No healthy REST endpoints found for network acrechain. Skipping...
2023-09-04 14:17:43 No healthy REST endpoints found for network mises. Skipping...
2023-09-04 14:17:43 No healthy REST endpoints found for network arkh. Skipping...
2023-09-04 14:17:43 No healthy REST endpoints found for network likecoin. Skipping...
2023-09-04 14:17:44 No healthy REST endpoints found for network bostrom. Skipping...
2023-09-04 14:17:44 No healthy REST endpoints found for network 8ball. Skipping...

Not sure what the solution would be here, we are completely dependent on these rest endpoints.

Resolved by hitting an alternate endpoint to get a 200


Update fs2-kafka to 0.20.0

Updates com.ovoenergy:fs2-kafka from 0.19.9 to 0.20.0.
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention @scala-steward in the comments below.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "com.ovoenergy", artifactId = "fs2-kafka" } ]

labels: semver-minor

This is not a trivial upgrade.   Created https://github.com/laserdisc-io/stream-it/issues/69


Multiple Parameter Validation Functions

TODO:

[ ] Unit test
[ ] Come up with a good way to forward these to the UI / make them show up in the UI reasonably

cc: @malnick @jgehrcke who have both wanted this recently.

Replaced by #628. That includes the fix, a integration test by way of the ssh validation tests exercising this, and includes code to "squash" multi-argument things to error on all parameters for the validation failure to solve the UI problem for now.


If all options of optgroup is disabled nothings gets displayed

If all options of optgroup is disabled nothings gets displayed in that particular group if anyone is enabled then it shows all the options.

https://jsfiddle.net/thamaraiselvam/swMBL/5/
see here Europe is not shown because all or disabled . i want to show those . how can i ?

This is a limitation of Select2 3.5.x and below, it doesn't display the optgroup if nothing within it can be selected. This has been fixed in Select2 4.0.0.
https://jsfiddle.net/swMBL/6/


Do/oauth

Could you please integrate the new config from master before? Thanks!

Great! Thanks for your contribution! I hope we can now finish the Oauth task in the HowCanI project.


move browserEvent function to util

This looked like a good change, aside from the tab reformatting.  Did you close for some reason?

Yeah, I didn’t realize eslint changed everything so I closed it. I’ll try later without eslint enabled.


Accesing the wrapper formly-field from a fieldGroup.

Is there a way to access the formly-field wrapper  of fieldGroups? className works with the ngForm that contains the fieldGroups' fields, but I need to go one more step up in the hierarchy. Why? Because I need to add a class to that element, Bootstrap's row class. That way I'll be able to lay out perfectly my forms fields, and I won't need to use CSS 'hacks' in order to fix it.
Perhaps I should make a custom type? Any tips would be of paramount help.
By the way, really nice library. Thanks a lot for it

Hi @nomack84, warm welcome from Cuba! I recently had the same requirement, but un fortunately fieldGroups doesn't support telplate wrappers.
But I achieved the same result making use of the data property inside a custom template called nested, take a look: http://jsbin.com/zaqeke
Also @kentcdodds came with a slightly different approach, by using two different fields objects, look: http://jsbin.com/setupi/edit?html,css,js,output
@kentcdodds shall I submit it as an example for the website?

Seems good to me. Thanks @benoror. I hope this helps @nomack84. I'm going to go ahead and close this issue because issues are reserved for feature requests and bugs. If you need help, follow the instructions here: http://help.angular-formly.com

@benoror @kentcdodds Thanks for your help. I've found that what I needed is to add display block the the ng-form. Now is working fine.

@benoror Ben, where is this panel.html file?
	formlyConfigProvider.setWrapper({
      name: 'panel',
      templateUrl: 'panel.html'
    });

@netowp in the JSBin it's included inside an script tag, since JSBin doesn't support multiple files afaik:
<script type="text/ng-template" id="panel.html">
...
</script>

In your project you'll likely have it in a separate file, or if you'd like yo might use ES6 multi-line strings, like this:
    formlyConfigProvider.setWrapper({
      name: 'panel',
      template: `
          ...
      `
    });

Thanks, @benoror! I was so confused about that, since JSBin doesn't support multiple files. :)


3D Position of an object in real world

I need the position of an object (x,y,z coordinates in real world of chairs, refer attached image) from ZED camera in real world.
My ZED camera is stationary, placed in the corner of the room.

I already tried:

getValue() from point cloud which returns the position (x,y,z) of a pixel but its not with respect to the real world and y-axis(height) of an object varies as object moves. (need the real height from camera).
(My guess is that i have to add calibration module using camera's intrinsic/extrinsic properties as well as multiple objects of known dimensions.) is this functionality already available in SDK or do I have to write one?
getPosition() which gives me the position of the camera with reference to WORLD_FRAME, but as my ZED is stationary how it will help?

My guess is that WORLD_FRAME only refers to the tracking module of the ZED, which means the coordinates are transformed back according to the pose estimation. For you this is irrelevant since the camera is stationary.
So, let me ask you: Where is the origin of your world coordinate system in the conference room? You need to define it and then you can try to estimate the transformation of your camera system with that origin. For instance, the camera might be up 2.2 m and is pitched down by 30° and rotated around Y by 30°. With that, you can reverse-transform a point of the ZED into your actual World Space.
Hope this helps. Notice that this is not a ZED issue per se, but a transformation matrix chain problem. I wouldnt even call it a problem.

Hi,
Is a point in XYZ camera system can be transform in world space cartesian coordinate system by just do some rotation and translation transformation or I need to apply a intrinsec camera parameter to convert XYZ camera point into cartesian coordinate system ?

My guess is that WORLD_FRAME only refers to the tracking module of the ZED, which means the coordinates are transformed back according to the pose estimation. For you this is irrelevant since the camera is stationary.
So, let me ask you: Where is the origin of your world coordinate system in the conference room? You need to define it and then you can try to estimate the transformation of your camera system with that origin. For instance, the camera might be up 2.2 m and is pitched down by 30° and rotated around Y by 30°. With that, you can reverse-transform a point of the ZED into your actual World Space.
Hope this helps. Notice that this is not a ZED issue per se, but a transformation matrix chain problem. I wouldnt even call it a problem.

How can this coordinate reverse transform be done? Is there a function in pyzed. sl that allows the reverse transform using the IMU rotation values?

Hi,
You need to use RuntimeParameters::measure3D_reference_frame in world to output metrics in real world (otherwise they are in camera frame).
See here : https://www.stereolabs.com/docs/api/structsl_1_1RuntimeParameters.html#a95f5726cd4db955a263d913d310f38e4

Hi there, how about estimate the height from the red points to ground, I know the height, roll, pitch, yaw  of ZED 2i camera


Added video recording of advanced task

Link  to Video:
https://user-images.githubusercontent.com/86596711/226517504-af88b2f2-97c7-41cc-9254-9510e8125988.mp4
Link to PR : https://github.com/ruqaiyasattar/digitalstethoecopeAI/pull/17

@ruqaiyasattar have a look please.


Set-DbaAgReplica command doesnt seems to include changes for property SessionTimeout

name: Feature request/idea :bomb:
about: Suggest a new feature or enhancement

Summary of new feature

Clear/concise description of what the problem is that the new feature can solve.

Proposed technical details (if applicable)

e.g. Blog post reference that shows example code or functionality

Latest version of dbatools as of writing

Please provide the latest released version of dbatools module.

We need a bit more detail...

Get-DbaAgReplica command returns SessionTimeout metric(default value 10)
this metric is changeable, but Set-DbaAgReplica command doesnt seems to have this metric, so when I run
Set-DbaAgReplica -SqlInstance 'sql1' -Replica 'sql2' -AvailabilityGroup agSQLDB -SessionTimeout 20
I am getting "Set-DbaAgReplica : A parameter cannot be found that matches parameter name 'SessionTimeout'.
At line:1 char:123

... ash-db37,61415' -AvailabilityGroup agBigDataImport -SessionTimeout 20"
MSFT page: https://docs.microsoft.com/en-us/sql/database-engine/availability-groups/windows/change-the-session-timeout-period-for-an-availability-replica-sql-server?view=sql-server-2017
powershell command to change it:
Set-SqlAvailabilityReplica -SessionTimeout 15 `
-Path SQLSERVER:\Sql\PrimaryServer\InstanceName\AvailabilityGroups\MyAg\AvailabilityReplicas\MyReplica

For whoever wants to tackle this one, we should enforce an error or warning to be sent if they set the timeout value less than 10 seconds, and include short link to the following: https://docs.microsoft.com/en-us/sql/database-engine/availability-groups/windows/change-the-session-timeout-period-for-an-availability-replica-sql-server?view=sql-server-2017#Recommendations

We recommend that you keep the time-out period at 10 seconds or greater. Setting the value to less than 10 seconds creates the possibility of a heavily loaded system missing PINGs and declaring a false failure.


[* NumberRange] Bug in handling ranges like "1995 - 01"

Resolution is currently the interval "[1,1995)".
Wouldn't it make more sense to assume both ends are as either open or closed?
Also, would it make more sense if the interval can only be specified in the right order? I.e. 1-1995 is an interval, but 1995-1 is not?

I think in math, 1 - 1995 means [1,1995)


QEMU: enhance documentation and check for /dev/kvm

Not having KVM enabled in a GCE machine is a real issue that a user
ran into. Error handling in this case is poor, GoVM doesn't print any
error and all that the user gets is an error about timing out waiting
for ssh.
The updated README.md explains the real requirements better and warns
about this pitfall. The script itself checks very early for /dev/kvm
and bails out with an error message when it is missing.

Marcela Morales Q<EMAIL_ADDRESS>writes:

marcemq commented on this pull request.

@@ -783,11 +783,14 @@ available.
This is known to work on a Linux development host system. The user
must be allowed to use Docker.

-KVM must be enabled and the user must be allowed to use it. Usually this
-is done by adding the user to the kvm group. The
-"Install QEMU-KVM"
-section in the Clear Linux documentation contains further information
-about enabling KVM.
+KVM must be enabled. Usually this is the case when /dev/kvm exists.
+The current user does not need the privileges to use KVM and QEMU
Nitpick question, do we need to decide to use does not or
doesn't and stick to it trough the md documents? or does not
matter that much?

I learned that "doesn't" and "don't" are colloquial and not to be used
in articles or scientific papers. But it seems that documentation
nowadays leans more towards colloquial than scientific.
I think it doesn't matter that much in the end. In this case, spelling
it out has the additional advantage that it emphasizes the "not", as in
"does not need the privileges".

Don't merge, the /dev/kvm check actually fails in the CI machines. I lost access to those, so I don't know yet why that is.

I'm still not sure why the /dev/kvm check failed. I just looked at a running CI worker and the device was there. Let's try again after rebasing.
I also added a commit which enables nested virtualization. It's a small additional change that touches the same documentation section; I didn't want to create another conflicting PR for that.

@avalluri please also check the second commit.

@avalluri: ping? We currently have clean test results.

somehow ReadME section (Automated testing) links got broken with this change. Can you please fix that.

It was broken on "devel" by an an earlier PR (https://github.com/intel/pmem-csi/pull/473) and has already been fixed there by https://github.com/intel/pmem-csi/pull/481. This change here is fine, it was just based on that broken revision of "devel".
I rebased, so now "view file" shows a clean version.


Update userstore username regex to restrict *,%,?

Purpose

$subject

Related Issues

https://github.com/wso2/product-is/issues/18699
https://github.com/wso2/product-is/issues/18700

PR builder completed
Link: https://github.com/wso2/product-is/actions/runs/7486635828
Status: success

PR builder started
Link: https://github.com/wso2/product-is/actions/runs/7498370118

PR builder completed
Link: https://github.com/wso2/product-is/actions/runs/7498370118
Status: success


Crash when Playing.

I already updated to the last pattern, and when I'm playing out of nowhere my game closes by itself, maybe some other offset should be updated, I'll try a debug to get more information.

animstate: 0x3900 -> 0x3914

animstate: 0x3900 -> 0x3914

oHH, Thanks S2


[SPARK-15933][SQL][STREAMING] Refactored DF reader-writer to use readStream and writeStream for streaming DFs

What changes were proposed in this pull request?
Currently, the DataFrameReader/Writer has method that are needed for streaming and non-streaming DFs. This is quite awkward because each method in them through runtime exception for one case or the other. So rather having half the methods throw runtime exceptions, its just better to have a different reader/writer API for streams.

[ ] Python API!!

How was this patch tested?
Existing unit tests + two sets of unit tests for DataFrameReader/Writer and DataStreamReader/Writer.

Test build #60452 has started for PR 13653 at commit a59498b.

Test build #60452 has finished for PR 13653 at commit a59498b.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60452/
Test FAILed.

Test build #60460 has started for PR 13653 at commit e118631.

Test build #60460 has finished for PR 13653 at commit e118631.

This patch fails Python style tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
class DataStreamReader(object):
class DataStreamWriter(object):

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60460/
Test FAILed.

Test build #60461 has started for PR 13653 at commit bbfff70.

Test build #60461 has finished for PR 13653 at commit bbfff70.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60461/
Test PASSed.

Overall looks pretty good.  Feel free to merge after addressing comments / passing tests to avoid more conflicts.

@marmbrus Overall, I have changed the following.

Renamed writeStream.save() to writeStream.start()
Refactored writeStream.foreach() to not start the query. instead writeStream.foreach().start() starts the query
Removed writeStream.parquet(path) for now. Its good to have a single method start actually start the background query. experience says that its much easier read code and debug.

Test build #60516 has started for PR 13653 at commit 536c25e.

Test build #60516 has finished for PR 13653 at commit 536c25e.

This patch fails Python style tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60516/
Test FAILed.

Test build #60519 has started for PR 13653 at commit 29ca23b.

Test build #60519 has finished for PR 13653 at commit 29ca23b.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60519/
Test PASSed.

Merged build finished. Test PASSed.

I am merging this for now. Please keep reviewing this and I will update them in #13673


Add DataFrame transform function

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
Add a function akin to DataFrame.transform from pyspark. This gives an easy to use way to chain DataFrame transformations.
Describe the solution you'd like
It is common to write a python function that takes as it's input a DataFrame plus 0 or more arguments and return a DataFrame. It is convenient to be able to write functions this way and to chain them. For example
def add_something_cool(df: DataFrame) -> DataFrame:
    return df.with_column("the_answer", lit(42))

def add_another(df: DataFrame, col_name: str) -> DataFrame:
    return df.with_column(col_name, lit("another"))

df_original.transform(add_something_cool).transform(add_another, "second_col").show()

Describe alternatives you've considered
To do the above operation I would probably do it like
df = add_something_cool(df_original)
df = add_another(df, "second_col")
df.show()

Additional context
Documentation via Databricks for pyspark

It might be as easy as
    def transform(self, func: Callable[..., DataFrame], *args: Any) -> DataFrame:
        return func(self, args)

Seems to be working, but requires more testing


minifier removes space between & and ::after

There's a bug reported against SWC because it works differently from this plugin, but AFAICT the bug is in the plugin, not SWC.
Report: https://github.com/vercel/next.js/issues/30802#issuecomment-990960985
My response: https://github.com/vercel/next.js/issues/30802#issuecomment-992766906
Basically when the minifier encounters .globalClass & ::after it kills the space between the ampersand and the colons.

I have the same problem with & :not(:last-child). Also tested with some other selectors.
Context: In my case (https://github.com/vrk-kpa/suomifi-ui-components/blob/v7.0.0-beta.1/src/core/Form/FilterInput/FilterInput.baseStyles.tsx#L39) I'm trying to add separator between items. But after minification the selector matches to the parent element instead of the items.

Fixed by #376 and released in v2.0.7


Developer relation program presentation to the company

Have a recorded presentation before next followup? Anyone free can attend, otherwise they'll have the recorded presentation

So this issue is actually a milestone :) I'm splitting it into a few new issues before closing this one:
The @meilisearch/devrel-team will do two presentations to the MeiliSearch team members:

Presentation 1: What is DevRel and how we do it at MeiliSearch
Presentation 2: A look at our Q4 initiatives

For Presentation 1, we could:

draw the framework of the Developer Journey
sort all DevRel initiatives according to the steps in this journey (and make sure no initiative was forgotten)
create some content about what is DevRel (and what is a Developer Advocate doing)
give some info about how the scope of everyone in the DevRel team
give some perspectives on our recruiting plan
create some content about how every MeiliSearch team member is facing developers so is doing Devreloper Relations
give a very few best practices (and we could create a longer list)

For Presentation 2, we could:

show our project organization
tell what initiatives are planned for Q4 (and side activities)
elaborate on current activities

We may to update this list along the way. When needed, we wan create/update issues in the project milestone.

Issues were created in the project milestone. Closing this one.


RM New Metadb Derived Table po_lines_cost

[ ] Pull request has a complete description
[ ] Query runs without errors
[ ] Query output is correct
[ ] Query logic is clear and well documented
[ ] Query is readable and properly indented
[ ] Table and column names are in all-lowercase
[ ] Quotation marks are used only where necessary
[ ] Query begins with user documentation in comment lines
[ ] File name is listed in runlist.txt after dependencies
[ ] All columns have indexes
[ ] Table is vacuumed and analyzed

I'd agree @stdombek since it might only an echo of PR title


stages,testutil: add test/tweak for podman mount errror handling

I promised to write a small test for the podman mount error handling in https://github.com/osbuild/osbuild/pull/1571. This PR contains a new helper testutil.mock_command to create fake commands for things in PATH and a test that ensures that the error messages are handled correct from podman. This lead to a small tweak in deploy-container so that the code only podman umounts if the podman mount was successful.

I'll approve but I think @bcl accidentally added in a merge commit here.

Thanks for cleaning up @bcl.  I wish we could disable update with merge -_-


Scroll Sista doesn't work when JavaScript Scrolling is OFF.

@aqeelb, did you solve this?
I think I found the solution, although I didn't test it on any physical devices yet:
Just change this line var scrollTop = e.detail.scrollTop  to this var scrollTop = e.detail ? e.detail.scrollTop : e.target.scrollTop, on the scroll event(:279)

@Narayon  :+1:

Hi,
I tried this solution and it's working, but the scrolling smoothness is very bad. Chrome gives me a warning in the console : Deferred long-running timer task(s) to improve scrolling smoothness. See crbug.com/574343.
Any idea why this this solution gives me a very slow application ? I'm working on an Android Tablet, Galaxy Tab S, 6.0.1).

I'm guessing that's because the plugin is animating the top or bottom of the content, to compensate the translate of the header.
Animate anything rather then transform and opacity will be a performance problem.
I end up adding css classes to the elements, making a one time animation, rather then partial animations on scroll. It's not perfect, but it reduces the jank and suits my needs.
You can try using the FLIP technique, by Paul Lewis.
Share your solution if you give it a try.

Hi,
I tried @Narayon 's solution but it only works while I'm touching device. I found that unlike web browsers, native devices can only detect scroll event while I'm touching it. So, when I scroll up to the header, the header shows up a few seconds later since it detects the scroll position when it stops. I am testing iOS devices.   If you have any idea ! please help me out !

Hi,
I tried @Narayon 's solution but it only works while I'm touching device. I found that unlike web browsers, native devices can only detect scroll event while I'm touching it. So, when I scroll up to the header, the header shows up a few seconds later since it detects the scroll position when it stops. I am testing iOS devices. If you have any idea ! please help me out !


What is the connection between GFLOPs value and GPU RAM value for Mask RCNN?

Hi, when I run python tools/get_flops.py configs/mask_rcnn/mask_rcnn_r50_fpn_1x_coco.py --shape 800 command, I get above result.
==============================
Input shape: (3, 800, 800)
Flops: 181.09 GFLOPs
Params: 44.17 M
==============================

When I change shape size, Params value does not change but Floops value increases. But  when I set shape value is 2000, I get Out of Memory(GPU RAM 8 GB).
In my mind, I could not establish the connection between the Flops value and GPU RAM. So what is the Ram equivalent of Flops value according to Mask_rcnn_r50_fpn_1x_coco config? Do I need to do RAM / Flops directly? Is there anyone who can explain this to me? Thank you(Sorry for my bad English)

@zerok01 flops are related to image resolution,  model forward will be called when you are run tools/get_flops.py

@zerok01 flops are related to image resolution,  model forward will be called when you are run tools/get_flops.py

@zerok01 flops are related to image resolution, model forward will be called when you are run tools/get_flops.py

Thank you for your answer. But what I wanted to ask is KiloByte or MegaByte equivalent of the Flops value. So for example, can we say 1 GLOPS equals 1 GigaByte? Is there a formula for this? Actually, this is what I want to learn.

@zerok01 flops are related to image resolution, model forward will be called when you are run tools/get_flops.py

Thank you for your answer. But what I wanted to ask is KiloByte or MegaByte equivalent of the Flops value. So for example, can we say 1 GLOPS equals 1 GigaByte? Is there a formula for this? Actually, this is what I want to learn.

@zerok01  FLOPs is Floating-point Operations, MFLOPs is Mega FLOPs, GFLOPs is Giga FLOPs.
The formula you can reference  Pruning Convolutional Neural Networks for Resource Efficient Inference
The  code you can reference flops_to_string flops_to_string function

@zerok01  FLOPs is Floating-point Operations, MFLOPs is Mega FLOPs, GFLOPs is Giga FLOPs.
The formula you can reference  Pruning Convolutional Neural Networks for Resource Efficient Inference
The  code you can reference flops_to_string flops_to_string function

The connection between FLOPs and GPU memory cost is not very clear actually. It depends on the implementation of the model and PyTorch libraries. Usually, under similar conditions (model input etc.), models with small FLOPs should have small GPU memory cost.

The connection between FLOPs and GPU memory cost is not very clear actually. It depends on the implementation of the model and PyTorch libraries. Usually, under similar conditions (model input etc.), models with small FLOPs should have small GPU memory cost.