🐞 dagger functions can be up to 12.5x slower on certain hosts

What is the issue?
This is a follow-up to:

https://github.com/dagger/dagger/issues/8395

Given the following host:
noefetch
             /////////////                gerhard@h22
         /////////////////////            -----------
      ///////*767////////////////         OS: Pop!_OS 22.04 LTS x86_64
    //////7676767676*//////////////       Kernel: 6.9.3-76060903-generic
   /////76767//7676767//////////////      Uptime: 31 days, 7 hours, 31 mins
  /////767676///*76767///////////////     Packages: 2346 (dpkg), 13 (brew), 26 (flatpak)
 ///////767676///76767.///7676*///////    Shell: bash 5.1.16
/////////767676//76767///767676////////   Resolution: 1024x768
//////////76767676767////76767/////////   Terminal: /dev/pts/2
///////////76767676//////7676//////////   CPU: AMD Ryzen 7 5800X (16) @ 3.800GHz
////////////,7676,///////767///////////   GPU: NVIDIA 3f:00.0 NVIDIA Corporation Device 2704
/////////////*7676///////76////////////   Memory: 15210MiB / 128719MiB
///////////////7676////////////////////
 ///////////////7676///767////////////
  //////////////////////'////////////
   //////.7676767676767676767,//////
    /////767676767676767676767/////
      ///////////////////////////
         /////////////////////
             /////////////

docker info
Client: Docker Engine - Community
 Version:    27.1.1
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.16.1
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.29.1
    Path:     /usr/libexec/docker/cli-plugins/docker-compose
Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 11
 Server Version: 27.1.1
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 2bf793ef6dc9a18e00cb12efb64355c2c9d5eb41
 runc version: v1.1.13-0-g58aa920
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.9.3-76060903-generic
 Operating System: Pop!_OS 22.04 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 16
 Total Memory: 125.7GiB
 Name: h22
 ID: 1c93b978-5459-4719-a1aa-92422d716091
 Docker Root Dir: /data-2024-04-10/docker
 Debug Mode: false
 Username: infradagger
 Experimental: false
 Insecure Registries:
  <IP_ADDRESS>/8
 Live Restore Enabled: false

dagger init --sdk=go --name=foo takes 17s in an empty directory https://dagger.cloud/dagger/traces/16339e06ad4f1364f6c5ae164a7cb18b
dagger functions takes 4s the first time https://dagger.cloud/dagger/traces/621b9e3901b585cd260f62a9c79c949e
dagger functions takes 1.5s the second time https://dagger.cloud/dagger/traces/83f2d9a9459f9b62d22397ed384bd031
So far, so good.
Same host, running the same command inside a subdir of a fresh clone of the dagger/dagger repository:
cd $(mktemp -d)
git clone https://github.com/dagger/dagger
cd dagger
mkdir foo
cd foo

dagger init --sdk=go --name=foo takes 25s in this dagger/dagger/foo directory https://dagger.cloud/dagger/traces/65ff21e193c7b7681d862aaf6253b047
dagger functions takes 25s the first time https://dagger.cloud/dagger/traces/dbf4420a92e4874e38fd8fd67b73c96f
dagger functions takes 22s the second time https://dagger.cloud/dagger/traces/7f2f5efad11460ee07d0728a2cbf460c
This is completely unexpected. All that time is spent in moduleSource(refString:"./foo", relHostPath:".")

What's even more peculiar is that this behaviour is inconsistent between different hosts.
Given the following host:
noefetch
            .-/+oossssoo+/-.               gerhard@h24
        `:+ssssssssssssssssss+:`           -----------
      -+ssssssssssssssssssyyssss+-         OS: Ubuntu 24.04 LTS x86_64
    .ossssssssssssssssssdMMMNysssso.       Host: Intel Z790 Ddr5
   /ssssssssssshdmmNNmmyNMMMMhssssss/      Kernel: 6.8.0-40-generic
  +ssssssssshmydMMMMMMMNddddyssssssss+     Uptime: 7 hours, 7 mins
 /sssssssshNMMMyhhyyyyhmNMMMNhssssssss/    Packages: 1724 (dpkg), 18 (brew), 13 (snap)
.ssssssssdMMMNhsssssssssshNMMMdssssssss.   Shell: bash 5.2.21
+sssshhhyNMMNyssssssssssssyNMMMysssssss+   Resolution: 2560x1440
ossyNMMMNyMMhsssssssssssssshmmmhssssssso   Terminal: /dev/pts/0
ossyNMMMNyMMhsssssssssssssshmmmhssssssso   CPU: 13th Gen Intel i9-13900KS (32) @ 3.200GHz
+sssshhhyNMMNyssssssssssssyNMMMysssssss+   GPU: NVIDIA GeForce RTX 4090
.ssssssssdMMMNhsssssssssshNMMMdssssssss.   Memory: 2250MiB / 192932MiB
 /sssssssshNMMMyhhyyyyhdNMMMNhssssssss/
  +sssssssssdmydMMMMMMMMddddyssssssss+
   /ssssssssssshdmNNNNmyNMMMMhssssss/
    .ossssssssssssssssssdMMMNysssso.
      -+sssssssssssssssssyyyssss+-
        `:+ssssssssssssssssss+:`
            .-/+oossssoo+/-.

docker info
Client: Docker Engine - Community
 Version:    27.1.2
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.16.2
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.29.1
    Path:     /usr/libexec/docker/cli-plugins/docker-compose
Server:
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 4
Server Version: 27.1.2
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Using metacopy: false
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: systemd
Cgroup Version: 2
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 8fc6bcff51318944179630522a095cc9dbf9f353
runc version: v1.1.13-0-g58aa920
init version: de40ad0
Security Options:
apparmor
seccomp
Profile: builtin
cgroupns
Kernel Version: 6.8.0-40-generic
Operating System: Ubuntu 24.04 LTS
OSType: linux
Architecture: x86_64
CPUs: 32
Total Memory: 188.4GiB
Name: h24
ID: 2637752e-9a21-47b5-adb0-844c1c544e90
Docker Root Dir: /var/lib/docker
Debug Mode: false
Experimental: false
Insecure Registries:
<IP_ADDRESS>/8
Live Restore Enabled: false

cd $(mktemp -d)
git clone https://github.com/dagger/dagger
cd dagger
mkdir foo
cd foo

dagger init --sdk=go --name=foo takes 15s in this dagger/dagger/foo directory https://dagger.cloud/dagger/traces/e2d70104b8fb18faddc976d9d18229bc
dagger functions takes 4s the first time https://dagger.cloud/dagger/traces/9a53db92c4f4e86ad633951b450ae64e
dagger functions takes 2s the second time https://dagger.cloud/dagger/traces/29e336eb13ae7db3c55f9a6a9a144709
How can this second host be 12x slower than the first host? There is no disk or CPU contention. Both hosts are running on the same network, with access to the same WAN. Here is a side-by-side comparison (left is the slow host, right is the fast one):
https://github.com/user-attachments/assets/0dfbf8af-afe9-43fe-aa97-d47bbf5f5dd3
Dagger version
dagger v0.12.7 (registry.dagger.io/engine:v0.12.7) linux/amd64
Steps to reproduce
cd $(mktemp -d)
git clone https://github.com/dagger/dagger
cd dagger
mkdir foo
cd foo

# the following command takes 25s to run:
dagger init --sdk=go --name=foo

# the following command takes 25s on the first run:
dagger functions

# the following command takes 22s on all subsequent runs, even if nothing changes
dagger functions

Log output
No response

@marcosnils mentioned something interesting: running dagger functions from the parent directory is fast, while running it from the module directory is slow. The following repro proves it:
cd $(mktemp -d)
git clone https://github.com/dagger/dagger
cd dagger
mkdir foo
cd foo

# the following command takes 25s to run:
dagger init --sdk=go --name=foo

# the following command takes 25s on the first run:
dagger functions

# the following command takes 22s on all subsequent runs, even if nothing changes
time dagger functions
✔ connect 0.2s
✔ initialize 20.6s

Name             Description
container-echo   Returns a container that echoes whatever string argument is provided
grep-dir         Returns lines that match a pattern in the files of the provided Directory
dagger functions  0.58s user 0.36s system 4% cpu 21.356 total

# 🤨 running it from the parent directory makes it behave as expected - finishes in 2s
cd ..
time dagger -m foo functions
✔ connect 0.2s
✔ initialize 0.6s

Name             Description
container-echo   Returns a container that echoes whatever string argument is provided
grep-dir         Returns lines that match a pattern in the files of the provided Directory
dagger -m foo functions  0.26s user 0.21s system 29% cpu 1.557 total


Remember prefered faxbot

This PR performs two actions.

The faxbot GUI in mafia now remembers the last faxbot you had open, where previously it was ignored and would default to 0.
When requesting a fax from a faxbot without specifying a faxbot, the iterator starts from your prefered faxbot (2, 3, 0, 1) instead of from 0 (0, 1, 2, 3). This reuses the property from the GUI, as it's normally the users preferred faxbot.

In short, this means the user has some measure of control over what the default faxbot they use is.

I don’t know if having had the tab open is a sufficiently proactive way of
specifying a new default. What do others think?
On Sat, 9 Mar 2024 at 02:50, libraryaddict @.***> wrote:

This PR performs two actions.

The faxbot GUI in mafia now remembers the last faxbot you had open,
where previously it was ignored and would default to 0.
When requesting a fax from a faxbot without specifying a faxbot,
the iterator starts from your prefered faxbot (2, 3, 0, 1) instead of from
0 (0, 1, 2, 3). This reuses the property from the GUI, as it's normally the
users preferred faxbot.

In short, this means the user has some measure of control over what the
default faxbot they use is.
You can view, comment on, or merge this pull request online at:
https://github.com/kolmafia/kolmafia/pull/2230
Commit Summary

68ddb4e
https://github.com/kolmafia/kolmafia/pull/2230/commits/68ddb4eb96f2a9f7fadfec590527e582fe41d250
Open to last faxbot looked at in gui
be46f79
https://github.com/kolmafia/kolmafia/pull/2230/commits/be46f792794a415dea29218e364571b1b9179476
Remember preferred faxbot
cf7bd9b
https://github.com/kolmafia/kolmafia/pull/2230/commits/cf7bd9bd0d2e55c468cda1d484518460c8a029ab
Correct method ordering in test
6d25a24
https://github.com/kolmafia/kolmafia/pull/2230/commits/6d25a242a4e0b512d0f42921ba71e68b140da0f2
Merge branch 'main' into fax_prefer

File Changes
(5 files https://github.com/kolmafia/kolmafia/pull/2230/files)

M src/net/sourceforge/kolmafia/persistence/FaxBotDatabase.java
https://github.com/kolmafia/kolmafia/pull/2230/files#diff-c836a3172c38cd29da233c7aa41021ce7b20555de9c64574abc0312fbe13bcf7
(4)
M src/net/sourceforge/kolmafia/swingui/FaxRequestFrame.java
https://github.com/kolmafia/kolmafia/pull/2230/files#diff-b2449a9d940f96eebf125119f8921ef0356f6a9489858b1c4b1698925c77e9f5
(2)
M src/net/sourceforge/kolmafia/textui/RuntimeLibrary.java
https://github.com/kolmafia/kolmafia/pull/2230/files#diff-a7b6e906c6d61b624d212171a00faf4cc250872dfe45e863e109eee249643d40
(9)
M src/net/sourceforge/kolmafia/textui/command/FaxbotCommand.java
https://github.com/kolmafia/kolmafia/pull/2230/files#diff-9141f87d1ce4dbc06c8cfeecc0714b5f3fb4f32a62e62e60810a89e8b52a9b19
(8)
A
test/net/sourceforge/kolmafia/textui/command/FaxbotCommandTest.java
https://github.com/kolmafia/kolmafia/pull/2230/files#diff-29598c15f758c62d6004aba45c3306c5c09f0aaa4dc557260568ffb7b4e70580
(60)

Patch Links:

https://github.com/kolmafia/kolmafia/pull/2230.patch
https://github.com/kolmafia/kolmafia/pull/2230.diff

—
Reply to this email directly, view it on GitHub
https://github.com/kolmafia/kolmafia/pull/2230, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AABL4LUWSBRW547GDCZ3W73YXJ2H5AVCNFSM6AAAAABENXWP5WVHI2DSMVQWIX3LMV43ASLTON2WKOZSGE3TOMBSGA3TCNA
.
You are receiving this because your review was requested.Message ID:
@.***>

I almost never use the GUI to get a fax and I don't have a preferred faxbot.  My focus is on the monster - I care that I got the requested monster and not where it came from.  But I have observed that some faxbots are more reliable than others and I would set a preference to try the most reliable one first.  But continuing to try until there is success is more important than saving a few seconds and succeeding on the first bot, rather than the second.
But in the event that I do request a fax twice in a session I would prefer that the second request start with the faxbot that last succeeded.  The use case for the second call is a clan imposed desire to have a particular monster in the fax machine.  I fax and fight and then fax to restore the preferred monster.  This allows a character who cannot chat to still fight a predictable monster.

I feel kind of bad only thinking of this now, but the preference depends upon the order of the bots in faxbots.txt.  That file can change.  I seem to recall long ago discussion about being able  to support user overrides but I don't recall the resolution.  So the preference needs to be the name of the preferred faxbot and not the index that a faxbot had  in one iteration of faxbots.txt,  A circular iteration starting at the preferred bot does make sense and should be preserved.

The current PR will now store "lastSelectedFaxbot" which is used only for purposes of the GUI
And "lastSuccessfulFaxbot" which is updated when a faxbot delivers a monster.
As such, the user can no longer define a default which can be relied on.
Quick example is a monster that their preferred faxbot does not have, but another does. If the fax is successful, their new preferred faxbot would be the second faxbot.


$svg.width() and $svg.height() return zero

Sometimes the $svg.width() and $svg.width() calls return 0, even though we explicitly set width and height of the element before creating it. I cannot figure out whether this is a bug in jquery or mobile safari (iOS 7.1). In any case, I cannot reproduce it outside of my current project. The fix is simple and probably safe though, so I thought I'd upstream it:
diff --git a/RowingInMotion.Analytics/app/lib/peity/jquery.peity.js b/RowingInMotion.Analytics/app/lib/peity/jquery.peity.js
--- a/RowingInMotion.Analytics/app/lib/peity/jquery.peity.js
+++ b/RowingInMotion.Analytics/app/lib/peity/jquery.peity.js
@@ -135,16 +135,13 @@
               values = [0, 1]
           }

-          var diameter = opts.radius * 2
+          var diameter = opts.radius * 2,
+              width = opts.width || diameter,
+              height = opts.height || diameter

-          var $svg = this.prepare(
-            opts.width || diameter,
-            opts.height || diameter
-          )
+          var $svg = this.prepare(width, height)

-          var width = $svg.width()
-            , height = $svg.height()
-            , cx = width / 2
+          var cx = width / 2
             , cy = height / 2

           var radius = Math.min(cx, cy)

Interesting, the reason I read back the dimensions after manually setting them is to cater for percentage dimensions - so you can specify { width: '100%' } and it'll return the SVG element's dimensions in pixels - and it looks like that patch would break that functionality.
I did previously discover that jQuery doesn't return the dimensions of an SVG element when they're defined as a percentage (I can't quite remember the detail as it was a while ago), could this be the cause?

Hm... my definitions use pixels. I wasn't aware that peity also supported %
dimensions. Getting the width/height of an svg element accurately seems to
be non-straightforward. Can you think of an easy way to reconcile this?
(E.g. we could check if the width and height returned was zero and then use
the user defined option if it's in pixels). This is a PITA.
On Mon, May 18, 2015 at 12:34 PM, Ben Pickles<EMAIL_ADDRESS>wrote:

Interesting, the reason I read back the dimensions after manually setting
them is to cater for percentage dimensions - so you can specify { width:
'100%' } and it'll return the SVG element's dimensions in pixels - and it
looks like that patch would break that functionality.
I did previously discover that jQuery doesn't return the dimensions of an
SVG element when they're defined as a percentage (I can't quite remember
the detail as it was a while ago), could this be the cause?
—
Reply to this email directly or view it on GitHub
https://github.com/benpickles/peity/issues/68#issuecomment-103011701.

That sounds like a good solution (zero being falsy). It'd be nice to have a working test case though.

I've got the same error while generating bar charts in an Ionic2 (Angular4) app. The thing is, in our web version of that project (Angular4), it works - with exactly the same versions of peity and jQuery. I've fixed it by using:
$svg.width() || opts.width
$svg.height() || opts.height
It works for now, but I don't exactly know, if there are any side effects under any circumstances. Could you verify and apply this fix to a next version please?

Hmmm, I wonder if it's because the element hasn't yet been attached to the DOM? (Just thinking out loud...)

I really don't know. Is it important to get the dimensions from the svg element? Could we simply get it from the options? Or is it needed for dynamic appending data?

It was originally required in an old version that used <canvas>, the more I think about it the more I think there's probably a better way to do things in S(scalable!)VG.

Hmmm, I wonder if it's because the element hasn't yet been attached to the DOM? (Just thinking out loud...)

At least in my case (#105) this must be the reason. I was trying to use peity within a DataTables table and their docs says:

This callback allows you to 'post process' each row after it have been generated for each table draw, but before it is rendered into the document. This means that the contents of the row might not have dimensions ($().width() for example) if it is not already in the document.


Favicon needed

a favicon for the site is needed.
Initially, add just the CQX and in the future make it part of the setup process.

@KeeperOfMaps There is a favicon.ico located in the csb directory. Maybe this is an issue with the Apache config?

@KeeperOfMaps There is a favicon.ico located in the csb directory. Maybe this is an issue with the Apache config?

I just checked my browser and it appears to have the favicon in the titlebar:

So, I'm not sure there's a problem here. Can you check again and if it's still happening post info about your browser, etc. Thanks!

So, I'm not sure there's a problem here. Can you check again and if it's still happening post info about your browser, etc.

Tests
I have tested on Chromium, Firefox, Firefox Developer Edition, & Firefox for ios.

Chromium does not show any requests for a favicon, and I did not find a link reference for it.
Firefox & Firefox Dev call for /favicon.ico, returning a 404 Error

This is a Firefox default, not a CSB issue (see resource:///modules/FaviconLoader.jsm:186:20 on Firefox)

Firefox ios (running on webKit) did return a favicon, I'm guessing it has an if in / else ./ by default

I was unable to test on any version of Safari, although Safari does run on webKit
Possible Reason
The issue could be the favicon is not being called, and since many developers use the Safari browser, it is not noticed (@KeeperOfMaps).
Proposed Solution
Add <link rel="icon" type="image/x-icon" href="/csb/favicon.ico"> to the necessary files.


Include fix commit message when showing violations together with source

This change doesn't have a lot of impact right now, as in many cases the commit message is similar to the error message itself. However, it is more actionable and makes it more obvious in what places we can improve.
Refs: #218

I think having this new commit message is a good first step, but long term I imagine it's more useful to the user if those messages were created by the check itself. We'd have much more context there and could generate much more actionable messages, e.g.
= help: replace with `x.bar`

Yeah I was a bit torn on this (I originally added a message to Fix, but it felt off to be adding those everywhere when the rest of the check data was associated with Check). I was hoping that we could accomplish it by adding more context to the enum itself (for example, in the getattr case, we could add the property name and value, and then use that to formulate the commit). We could also allow overrides on Fix itself.

Adding more context to the enum certainly is more in line to what we're doing right now. I'll play around with that a bit, adding some context to some checks, and see how that goes. :)


Error when importing bqfetch

Hi everyone !
I'm trying to read a big table from BigQuery using python in google colab and I found bqfetch, however, when I try to import BigQueryFetcher and BigQueryTable I get an error.
I installed it by doing:
!pip install bqfetch
!pip install -r requirements.txt

But when running the second command, I get this error:
ERROR: Could not open requirements file: [Errno 2] No such file or directory: 'requirements.txt'
Then, my code is this:
from bqfetch import BigQueryFetcher, BigQueryTable

table = BigQueryTable("PROJECT", "DATASET", "TABLE")
fetcher = BigQueryFetcher('/path/to/bq_service_account.json', table)
chunks = fetcher.chunks('id', by_chunk_size_in_GB=2)

for chunk in chunks:
    df = fetcher.fetch(chunk, nb_cores=1, verbose=True)

Am I doing something wrong ? Because this is what I get:

Some help would be appreciated! Because I cannot run anything so I can't get the table I need in order to have as a dataframe in python :(
Thank you in advance!
Marina

Hello there,
Thank you for using bqfetch!

to fix ERROR: Could not open requirements file:, you should create the requirements file locally to install the dependencies. Please create the requirements.txt file in your google colab and paste the content of the requirements file present in the repo.
the import error is due to the way you are importing the module. Until now, the only way to import a class from bqfetch was by doing from bqfetch.bqfetch import BigQueryFetcher. I created a new release where it is now possible to import more intuitively using from bqfetch import BigQueryFetcher.
To fix this error, you can thus use the legacy method with from bqfetch.bqfetch import  .. or download and install the 1.1.0 release locally to use the new import system.

Before diving your big dataset into chunks of relatively small size using one of the functions available in bqfetch, did you verify that your data are independent? Namely that it is possible for you to divide the whole dataset into multiple batches that you can train independently.
If it is independent, then you can use leverage this module.
Next, for the backend, I recommend you using the default backend, so no need to set the parallel_backend argument for the moment. If this backend leads to issues, then maybe try one of the other available backends.
Then for the fetching, I recommend to fetch by chunk size as you can easily manage your memory and avoid memory overflows in your colab environment. However, you need to specify an index column in your dataset on which the dataset can be partitioned. I give some examples of index columns in the README.
If it is hard for you to find a nice index column, then try to fetch by number of chunks by estimating the chunk size with respect to the size of your dataset and your available memory. You can also start with a small chunk size, and increase more and more until you raise an overflow error.
Do not hesitate if any other questions!

I understand that you want to split your dataset into multiple parts, however it's no use to reconstruct the whole dataframe as you will still end up with a memory overflow because your dataframe is too huge to fit on your machine. What you can do to deal with this large table is to run the training loop on the parts of data instead of the whole dataframe, like in mini-batch training.
To summary:

❌ Fetch all the data => build the df => train: this leads to memory overflow as you have too much data to process.
✅ Fetch 1/n chunk of the data => build the df => train, ..., Fetch n/n chunk of the data => build the df => train: this is is feasible, but each row in your dataset should be independent, meaning that running the training loop on all the data at once or running the training loop on many small batches should approximately produce the same results. If your rows are not independent, then you need to provide a proper index column.

Your error is about the BigQuery API, can you provide the full code you used to fetch? I think your chunk size is too big and the fetching took more than 10min and raised a timeout error.

I'll think about what you said and probably come back with questions hahah thank you !!!
This is my code:
from bqfetch.bqfetch import BigQueryFetcher
from bqfetch.bqfetch import BigQueryTable

table = BigQueryTable("gcpinv-230419-ifym1p4zudsrz4zu", "prueba_lstm", "results")
fetcher = BigQueryFetcher('/content/drive/MyDrive/TFG mio/gcpinv-230419-ifym1p4zudsrz4zu-699b17de7880.json', table)
chunks = fetcher.chunks('row_num', by_chunk_size_in_GB=2, verbose=True)

for chunk in chunks:
  df = fetcher.fetch(chunk, nb_cores=-1, verbose=True)


News entries on member pages are not linked correctly

That is, you try to click on them and they don't go anywhere--they just point to #

It looks like its all the links on the right sidebar

This is now fixed and on production.


handle MetaException with glue AccessDeniedException

Description
handleGeneralException should handle MetaException with AccessDeniedException properly
24/07/03 21:34:42 INFO FlintREPL: Original exception class name: org.apache.spark.SparkException
24/07/03 21:34:42 INFO FlintREPL: Root cause exception class name: org.apache.hadoop.hive.metastore.api.MetaException

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

LGTM. but check the build failure.

LGTM. but check the build failure.

Fixed. Failed due to flint-commons refactor, will need a manual backport


Added User Authenticator System for Issue #5

Hi according to the open issue of this project I have added the user authentication system file

@bvks2020sameer the code looks good but it be be better if this code could could be integrated with existing leakylan code and i guess we dont need email and name , just an alphanumeric id and password. The password should be generated by python and displayed in commandline in server side. the user has to manually share the password with person who wants to send / recieve files.

Hi have made the changes

Also @bvks2020sameer avoid unnecessary blank lines , it feels odd.

The above mentioned changes have been done

@bvks2020sameer thanks for contributing.

Thankyou. Looking more to collaborate on . Happy Hacking


Device management seems to have a bug.

Hello, thanks for sharing your nice code.
I find a slight bug in your forward process of HeightToNormal in h2n.py. When passing a height map on CUDA device, it will encounter a exception of operating two tensors of two different devices. From your code I think that you want to dynamically adapt the input of different devices, since you have coded dx.device.
Maybe we can fix the bug by dynamically detecting the device of input, and modifying the device of self.filter_kernel and self.blur. Like:
    # Expecting input height is a tensor in shape [B, C, H, W] and in value range [0, 1]
    def forward(self, height):
        device_of_height = height.device
        self.filter_kernel = self.filter_kernel.to(device_of_height)
        self.blur = self.blur.to(device_of_height)

        x = self.filter(height)
        dx = -1.0 * x[:, 0] if self.invert_x else x[:, 0]
        dy = -1.0 * x[:, 1] if self.invert_y else x[:, 1]
        normal = torch.stack([dx, dy, torch.ones(dx.shape, device=device_of_height) / self.strength], dim=1)
        normal = torch.nn.functional.normalize(normal, dim=1)
        return normal

459aa6cc2709116ea46c20f1356f889c39f9abe2


Please invite me to the Community Organization

Name
Ajor Moses
Discord Username (if applicable) E.g John#412
No response
Questions & suggestions?
No response

Thanks for raising an issue to join this community. 🤩 Now check your Email for the invitation.


Recieved error "TypeError: Cannot read property 'concat' of null"

Describe the bug
Got the error in console when trying to run the bot for the first time.
To Reproduce
Steps to reproduce the behavior:

Open cmd (standard perms)
Download bot using git to the desktop
Install latest TypeScript version
Install dependencies and fix audit errors
Compile the source
Create .env file with only the 4 required variables 'STEAM_ACCOUNT_NAME','STEAM_PASSWORD','STEAM_SHARED_SECRET','STEAM_IDENTITY_SECRET'
Try to run bot using node dist/app.js

Expected behavior
Bot to run successfully.
Screenshots and logs

fdert13-2020-09-14.log
Additional context
Running Windows 10 2004
Version: 3.5.2

I've made a wiki entry about this error here.
Originally this one was not added as this only appeared when people updated from V2 from V3, which was released some months ago.
You need to have the ADMINS and KEEP options, and they need to be an array with a valid steamid. Ex: ["76561198144346135"]


add bayesian light-error model

Coverage decreased (-0.02%) to 18.455% when pulling 37caeb9d0583c93e92326debd934cd811db7e977 on aappling-usgs:develop into da773fa1dded1fbc176b5456098f81ad7e819f90 on USGS-R:develop.


[Feature] Building the ability for the graph-node to index private transactions priv_getLogs in private EVM-based blockchains like Hyperledger Besu

Do you want to request a feature or report a bug?
Feature
What is the current behavior?
TheGraph is not able to index the private transactions in the Hyperledger Besu EVM-based chain by enabling the privacy features.
If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem.
What is the expected behavior?
The ability to use the subgraph with our graph-node to read and index the private transaction events in EVM-based private blockchains.
More explanations:
In searching for the reasons that block the graph-node to read the events, I asked the following question in ConsenSys Quorum-Besu channel:

Isk — 11/29/2021
I'm running Quorum Dev Quickstart (https://github.com/ConsenSys/quorum-dev-quickstart) and setup a private permissioned Besu running with Tessera nodes. I've added an event into the example contract (SimpleStorage.sol)
event Set(uint amount, address sender);
And running the /quorum-test-network/smart_contracts/scripts/private_tx.jsbetween Member1 and Member3, but the issue was that I couldn't read any events happened in the set() function.
Q: Is Tessera blocking the Event emitter features in the blockchain? or how to reach and read the emitted events happened?  Discord question link

Then, got the following answer:

taccatisid — 11/30/2021
How do you fetch those events? There is a priv_getLogs method that is the equivalent of eth_getLogs for private transactions (https://besu.hyperledger.org/en/stable/Reference/API-Methods/#priv_getlogs).
Discord answer link

Basically, Besu is using a different EVM API method for the private transactions which is priv_getLogs but on the other hand, the graph-node is depending on the eth_getLogs method on reading the events from any open and public JSON RPC equivalent network.
So if this would be a useful feature to be added in the graph-node, am suggesting starting research on implementing it. After having a fast look into the code, I think the starting point would be from these scripts:

adapter.rs
chain.rs
ethereum_adapter.rs

Hi! Thanks for the write up. Two things - firstly Graph Node is currently moving towards relying on firehose based (rather than RPC based) indexing methods, so I don't think we would add an additional getLogs method here. Secondly I don't know if there is sufficient demand for private blockchain use-cases (Graph Node is predominantly focused on data from public blockchains)

Just stumbled upon this and wanted to express an interest in this. We are also starting using Besu+Tessera+Graph Node.
Thank you all for amazing work on Graph Node so far!

@azf20 Would you be interested in receiving PR on this?
@iskdrews I'm planning on working on this next week. Just wanted to check if you already started working on this?

@mfolnovic yes I am happy to collaborate with you on implementing this, I need guidance on it. I was waiting for getting an approval on implementing this feature from TheGraph team.

@iskdrews I did a quick PoC to verify your proposal and it works. But at the moment, I'm stuck with:
https://discord.com/channels/905194001349627914/938504958909747250/977063311415853067 (Hyperledger Foundation)
https://discord.com/channels/438038660412342282/791444480628490290/977054461061124097 (graphprotocol)
Any help would be appreciated! :)

@mfolnovic thanks much! I have sent you a friend request on Discord so we can start planning together on this.
I as well posted this proposal in the ConsenSys Besu channel for getting feedback: https://discord.com/channels/697535391594446898/743193040197386451/976850555584540702


table

When you click the ascending / descending order in the table, the style of the check box in front is inconsistent

Hello @niuyacong ,
This issue is resolve. Thank you for pointing it out.


feat: adding the ability for pfe-navigation to use the full width of …

…the viewport
Fixes #714
Testing instructions
Be sure to include detailed instructions on how your update can be tested by another developer.

In the demo file for pfe-navigation, add a pfe-full-width attribute to the pfe-navigation tag.

Browser requirements
Your component should work in all of the following environments:

[ x] Latest 2 versions of Edge
[ x] Internet Explorer 11 (should be useable, not pixel perfect)
[ x] Latest 2 versions of Firefox (one on Mac OS, one of Windows OS)
[ x] Firefox 60.7.2 (or latest version for Red Hat Enterprise Linux distribution)
[ x] Latest 2 versions of Chrome (one on Mac OS, one of Windows OS)
[ x] Latest 2 versions of Safari
[ ] Galaxy S9 Firefox
[ ] iPhone X Safari
[ ] iPad Pro Safari
[ ] Pixel 3 Chrome

Your repository infrastructure updates should work for at least:

[ ] Node v8.x
[ ] NPM v7.x

@castastrophe Oh... and I added the changelog entry.


I can not install it on my ubuntu 16.04

github.com/tomnomnom/assetfinder
/root/go/src/github.com/tomnomnom/assetfinder/urlscan.go:46: u.Hostname undefined (type *url.URL has no field or method Hostname)
/root/go/src/github.com/tomnomnom/assetfinder/urlscan.go:55: u.Hostname undefined (type *url.URL has no field or method Hostname)
/root/go/src/github.com/tomnomnom/assetfinder/wayback.go:38: u.Hostname undefined (type *url.URL has no field or method Hostname)
What's wrong with my machine ? any hints ?

Facing this same error on my VPS too , I am also using ubuntu 16

Hey!
This is because you are running an old version of Go. Please update to the latest version of Go and try again!
The Hostname field was added in Go 1.8 if I recall correctly, but the latest version of Go (currently 1.14) is almost always preferable.


Redo the meta_description texts (NL and EN)

These are the summaries Google reproduces in search results. Most of the ones we have now are either placeholders (e.g. first sentence of the page) or otherwise don't fit. Best to leave them blank (Google will substitute its own summary or first sentence) until they can be re-written later.

@Doppen I know what the meta_description metadata is and how it's used by Google. But how does Google use (if it does) the page_summary metadata?

This also applies to meta_keyword


Hott 854 add fallback for missing measurement units

Jira link
https://transformuk.atlassian.net/jira/software/projects/HOTT/boards/96?selectedIssue=HOTT-854
What?
I have added a fallback for when a Measurement Unit is requested which is not listed in the YML:

[x] Added an #measurement_unit method to access a specific unit, rather than going to the units hash directly
[x] #measurement_unit will now query the DB and create 'stubbed measurement unit' in the units hash if found in the DB
[x] #measurement_unit will log to Sentry with the unit code of the missing measurement unit
[x] made #measurement_units private - any public access should go via the new singular method with appropriate fallback handling

Why?
I am doing this because:

The commodities listed in ticket HOTT-825 were triggering a key error because whilst they were valid and present in the database, they had no corresponding entries in the YML file of questions (db/measurement_units.yml)

Points for discussion? (optional)

I've left the existing #measurement_units method public but there are no consumers of this method now and it would make sense to make it private since that will force consumers down the path with fallback handling (ie MeasurementUnit#measurement_unit)

Looks good man. I wish these labels could just get updated in the database :)

And not by us


Something doesn't work, lousy ticket

no description, deal with it.

@domeq can you please take a look?

No, it's too broad - please explain.


[release-0.41] fix pdb controller segfault

manual backport of the primary fix and unit test for #5849
This backport focuses only on the segfault and targeted unit test and does not include the other unit test enhancements #5849 introduced.
Fixes event recording causing a segfault in virt-controller

/retest


Validate domain resource before running introspector

Adds semantic validation of the domain resource itself. If it fails, updates the domain status. If it succeeds, allows the introspection to run.

Approving.   Pending issues for a future pull:

All status messages should be logged (not just show up in status), and logged as 'severe' if it's critical.
I18N


Show dynamic stream metadata in stream settings panel

Fixes #424

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Update metadata-json-lint.gemspec

Ship the rake_task.rb file in the gem.

Thanks!

Thanks!

Pushed!

Pushed!


Build fails on Ubuntu 18.04

i m trying sending data to kinesis and install this sdk, but get this when running cmake:
[ 66%] Performing build step for 'project_log4cplus'
/home/aiti/Downloads/compressed/amazon-kinesis-video-streams-producer-sdk-cpp-master/open-source/liblog4cplus/build/src/project_log4cplus/missing: line 81: aclocal-1.16: command not found
WARNING: 'aclocal-1.16' is missing on your system.
You should only need it if you modified 'acinclude.m4' or
'configure.ac' or m4 files included by 'configure.ac'.
The 'aclocal' program is part of the GNU Automake package:
http://www.gnu.org/software/automake
It also requires GNU Autoconf, GNU m4 and Perl in order to run:
http://www.gnu.org/software/autoconf
http://www.gnu.org/software/m4/
http://www.perl.org/
Makefile:934: recipe for target 'aclocal.m4' failed
make[3]: *** [aclocal.m4] Error 127
CMakeFiles/project_log4cplus.dir/build.make:111: recipe for target 'build/src/project_log4cplus-stamp/project_log4cplus-build' failed
make[2]: *** [build/src/project_log4cplus-stamp/project_log4cplus-build] Error 2
CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/project_log4cplus.dir/all' failed
make[1]: *** [CMakeFiles/project_log4cplus.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 2
CMake Error at CMake/Utilities.cmake:55 (message):
CMake step for liblog4cplus failed: 2
Call Stack (most recent call first):
CMakeLists.txt:77 (build_dependency)
-- Configuring incomplete, errors occurred!
See also "/home/aiti/Downloads/compressed/amazon-kinesis-video-streams-producer-sdk-cpp-master/build/CMakeFiles/CMakeOutput.log".
is there anything i missed?

The log shows aclocal is missing. Please install the build prerequisites like the compiler and the build tools. Resolving


Allow void elements to be specified in data provider

This fixes #123
Please note, this PR drops support for checking void elements keygen and menuitem these are long ago depreciated and not used it any tests. If you need these you can add them back in via a custom data provider and set "void": true.

I suggest to make isVoidElement a member of HTMLDataManager.
HTMLCompletion already gets a HTMLDataManager in the constructor.
I'd turn folding and the parser also a class that gets the HTMLCompletion as context.
Same for the selection range. It could get a parser as context

ok @aeschli I've created classes for those 3 functions. The formatting makes it look like there are more changes than there really is, but I had to indent all the functions since they are all inside classes now.

Thanks a lot @hahn-kev, nice PR!

You're welcome, happy to contribute


docs: adding an etcd v2 landing page.

Adding a page for the etcd v2 docs that can be linked from docs.md for the etcd v3 docs.

replacing /v2/README.md works for me.
tell me, please: what does 'doc is dangling' mean?

@zbwright only in repo, not connected to other docs

replaced by https://github.com/coreos/etcd/pull/8321. closing.


⚠ Bofrost.fr has degraded performance

In b3c12ce, Bofrost.fr (https://www.bofrost.fr) experienced degraded performance:

HTTP code: 200
Response time: 1540 ms

Resolved: Bofrost.fr performance has improved in dee14a5.


Assume the current python executable is the one we should spawn.

On Debian the Python executable is named python3, so the test fails
because it tries to run the Python 2 interpreter.
Using sys.executable should be the safest option as the test server
will be launched using the same interpreter that the test suite is
running under.

[X] Tests added / passed (Tests pass with default Debian python interpreter.
[X] Passes flake8 dask
[ ] Fully documented, including docs/source/changelog.rst for all changes
and one of the docs/source/*-api.rst files for new API
Not documented as there should be no visible change to anyone but people using system Python 3 interpreters on systems following the Python installation conventions for Unix like systems.

Improves #3449

Thanks @detrout !

Aha, makes sense :)


Add note on using the express transport

Here's my attempt at adding a note on express.js transports.

Well, that was a quick edit, so thank you in the first place for providing this module!

Merged. Thanks again.


Call candidates merging not set

MXJingleCall
createLocalStream(): check the merging candidates portion of code. It seems that the merged candidates are never sent.

The merged candidates are properly sent : the event content is a reference not a copy.


Some of the texts on the dashboard are out of elements.

Screenshots

Test on
Browser

Edge - Version  92.0.902.78 (Official build) (64-bit), with default zoom(100%)

Screen size

13.3"

Display Settings

hello, I have fixed the problem in the template, you can check my pull request to see it

Can you assign me this issue @Zzzul

I recommend to implement global css like this to prevent overflow text
div {
  white-space: nowrap;
  overflow: hidden;
  text-overflow: ellipsis;
}

I recommend implementing global CSS like this to prevent overflow text
div {
  white-space: nowrap;
  overflow: hidden;
  text-overflow: ellipsis;
}

I would prefer to use a simple approach, it has fairly has good browser supports, read https://developer.mozilla.org/en-US/docs/Web/CSS/overflow-wrap.
overflow-wrap: break-word;

Except for IE9, it does not recognize overflow-wrap but works fine with word-wrap, so adding both would suffice since it's much simple than using conditional rules.
word-wrap: break-word;
overflow-wrap: break-word;


Fix clipping bug

fixes #967
Fixing bug where we were only checking if a clipping shape is present via ClipRect (forgetting ClipPath and ClipRRect).

🔥


Get device_id for assist if not set

This PR adds area awareness to Assist pipelines created by StreamAssist. If the device_id for Assist was not defined via a service call, the switch will pass the device_id of its parent service to Assist which is then used by assist to get the area. I have tested this with the standard Assist pipeline as well as with the Extended OpenAI Conversation integration.
When using Extended OpenAI Conversation, the prompt should include Current Area: {{ area_name(current_device_id) }} as described in that integration's docs.
Addresses #22 and #25

Confirmed working for me.  Thanks!

Thanks! I have made a few changes to your code. I hope I didn't break anything.

Your version works for my setup but you removed the checks to see if the assist and/or device_id keys are already set. I had that to avoid overwriting a device_id or other assist settings that a user had already defined when starting StreamAssist as a service and defining the optional settings in data: -> assist:
Won't self.options["assist"] = {"device_id": self.device_entry.id} overwrite any user-defined settings for assist?

This setting can't be set from config entry. It can be changed only from service call

This setting can't be set from config entry. It can be changed only from service call

Ah, OK. I get it. Thanks!

  I see that this has been accepted and the code shows up in main but when I installed on my dev HA install I am not getting the device_id updated version.  The install is from HACS.  Does something need to happen on the HACS side for these changes to be reflected or something on your side?
I've got a few people testing my View Assist  project and they noted that device_id in SA is not working properly.  Any help is most appreciated.

And within two seconds of posting that I see there's an option in HACS to redownload and choose version as main.    I had originally just download the changed files and started using it so assumed a new release was available.
I guess the question now is if you would consider doing a point release so users can get this fix?  My assumption is that it's not best practice to point to main generally.  My View Assist project is relying heavily on this device_id to be set for proper routing so an update to Stream Assist would definitely be appreciated by me and the others that are using Stream Assist with View Assist.  Sorry for the burden.

https://github.com/AlexxIT/StreamAssist/releases/tag/v2.1.0


Fix conditional in feed workflow

Since somehow we're running on all labels

it worked? I think

let me try new package too

Looks good


Github Action fails in CI on windows-latest

Hi I've just discovered, tried and adopted right away cargo-machete ❤  Great crate !
When setting it up in Github workflow, I ran into some error on windows-latest:
error: could not compile `cargo-machete` (bin "cargo-machete") due to 1 previous error
error: failed to compile `cargo-machete v0.6.1`, intermediate artifacts can be found at `C:\Users\RUNNER~1\AppData\Local\Temp\cargo-install5qeHSQ`.
To reuse those artifacts with a future compilation, set the environment variable `CARGO_TARGET_DIR` to that path.
Error: The process 'C:\Users\runneradmin\.cargo\bin\cargo.exe' failed with exit code 101

➡ full details available in Action.
💡 A simple alternative is to install it with taiki-e/install-action, as in following commit.

note that here the workflow failed for another reason: that I had unused dependencies indeed 😄

Hope this helps !

Thanks for opening an issue. Not sure what's going on, as the error seems to be a link error, and I haven't seen it when compiling cargo-machete for the release builds, e.g. https://github.com/bnjbvr/cargo-machete/actions/runs/7991084257/job/21821428552

That could utterly be a false-positive, especially since I have a slightly exotic setup.
What about closing it for now? If ever somebody else run into it, they can just re-open it later on.


Sidestep not recognizing VPN

I have a VPN configured and saved but Sidestep is not showing it as an option in the "Available VPN Services". I've restarted Sidestep, but that hasn't made a difference. I'm running 10.9 Mavericks...anyone else seeing this?

I'm seeing this issue too.
@chetan51 are there any plans to address this? It would be awesome if you could. If not, I might see if I can find someone to take a look at the issue.

@johnbillion, i think this project has been abandoned. There hasn't been a commit for over a year and issues are not being addressed or commented on.


[LevelUp] lvlset commandxp is giving xp even when its disabled. + 2 Queries.

Hi alex,
Hope you're doing well
I've got a few things to share today.
1st Issue
Describe the bug
What's the expected behavior of lvlset commandxp?

If it's meant to give xp when you run the bot commands (if enabled) - then I think it's having some issues.
So, when it's Disabled - looks like it's still giving xp per command run. I've tried this with a test scenario which seems to be agreeing.

Cog Version*

Cog Version: 4.2.8

To Reproduce & Screenshot/Video

Steps to reproduce the behavior:
Follow the video

https://github.com/user-attachments/assets/98019073-63b3-41ae-b5d0-b874f60c7931
Expected behavior
When lvlset commandxp is set to Disabled - It shouldn't give xp.

2nd Query

How does the cog decide how much xp should be given ‒ when the xp per message is set to the default I.e.: 3- 6? - Is it completely random or something else is going on!
Meaning ‒ say, if I send a BIG message with lots of formatting and emoji included - would it get the highest xp which is 6 in this case or would it get a randomized number!
In my test scenario it looked like its random xp per message (correct me if I'm wrong here).

If it's randomized then - Is it possible for you to add a way where the bot could decide the xp by judging the msg_content as in - If the msg_content is big/better/well formatted or "good" then it'll incline towards the highest xp or give the highest xp?

(not sure if it's a good idea though, since spammers exist & they'll find ways to get more xp but that's just a handful of users and it shouldn't be a problem to deal with them as you can always blacklist them anyways ― but the main issue might be knowing which msg is a "good" msg and which msg isn't aka "bad msg". Would it be too complex or doable ― I wanna know your point of view.)

3rd Query

Is it possible to make the levelup data available (just the ‒ user's exp & weekly-exp would suffice ) via API endpoint for other bots to hook into!! ― (like how amari API works)
It'd be a game changer for development.
For example - we have a custom giveaway bot that uses the amari bot's API to get the user's amari_level and amari_weekly_exp and then uses it as as giveaway requirement - so having that data available for this cog means HUGE deal ‒ and it Not only just for us but for other people to use the cog with their own bot.

Decided to edit and add an important 3rd Query instead of making a new issue.

Your first query should be fixed now with v4.2.13
The 2nd and 3rd query are unrelated and need to be separate issues. The 2nd one is just a question and the answer is that it is random every time, if the xp is set to 3-6 then each valid message will get a random number from/including 3-6
The 3rd query would require a full rewrite so that probably wont happen within the next year at least.

Thanks
And will do


Add ELF section tags

This change will allow for someone to read the ELF sections from the Stivale 2 headers. This would be a nice feature to have, so one could read the full attributes of a section

Commits https://github.com/stivale/stivale/commit/71a61b8a0e6f206f69d20ee8cc63ca6817d72b6f and https://github.com/limine-bootloader/limine/commit/d1946b9dbceb9483948178f73840c1824df0eb6e address the issue


chore: bump f36 dependencies

@burakukula looks like there is a build error related to dependencies, can you fix it before we review?


Firestore batchGet decoding breaks

When retrieving documents from the batchGet endpoint it returns a body as such:
"[{\n  \"found\": {\n    \"name\": \"projects/xxxxx/databases/(default)/documents/xxxxx\",\n    \"fields\": {...},\n    \"createTime\": \"2018-xxxxxxxZ\",\n    \"updateTime\": \"2018-xxxxxxxZ\"\n  },\n  \"readTime\": \"2018-xxxxxxxZ\"\n}\n]"

When reaching the decoder here it breaks because of the brackets the body is wrapped in when getting here because of this.

@aaruel we've gone through a pretty significant rewrite since this came up.  Could we trouble you to verify this is still an issue with the latest version?

@aaruel We've heard no response in over two months, so I assume the latest version fixed the issue for you.  If this issue is still causing problems for you, please re-open.


fix: Pump to 6.7.0 version for generator-hipster

Motivation:
When running JHipster, a warning message was trown becase the
generator-hipster dependency version is not the latest.
Modification:
Change to the latest version.

Thank you!


duo integration

I am new to vpn land and curious that supporting duo.com. It seems that they have integration already. What do you think?

Problems with things like this is they integrate proprietary bits and then go off abandoned and neglected since nobody uses them.  Unless a critical mass of people want to use duo.com, someone active on the project steps up to maintain it and the implementation is open and flexible enough, it doesn't make sense.


[QUESTION] Not supported on A6000?

Your question
Hi,
When I run the test demo with a node consists of 2 A6000 it reports bugs:
RuntimeError: /root/opensource/flux/src/cuda/op_registry.cu:36 Check failed: arch_num == 80 || arch_num == 89 || arch_num == 90. unsupported arch: 86
So flux can only support these three GPUs (cc=90, 80, 89), correct me if I misunderstand it.
Thanks

Yes, Flux only compiled the architectures 80, 89, and 90 for now. However, I suspect that CUTLASS v2 should directly support architecture number 86. Could you try adding the corresponding arch number and recompiling to see if it works?

Thanks,
I add 86 arguments to /flux/src/cuda/op_registery.cu line 36 like this:
void
init_arch_tag() {
  int major, minor;
  cudaDeviceGetAttribute(&major, cudaDevAttrComputeCapabilityMajor, 0);
  cudaDeviceGetAttribute(&minor, cudaDevAttrComputeCapabilityMinor, 0);
  int arch_num = major * 10 + minor;
  FLUX_CHECK(arch_num == 80 || arch_num == 89 || arch_num == 90 || arch_num == 86)
      << "unsupported arch: " << arch_num;
  arch = ArchEnum{arch_num};
}
}

I recompiled it via Build from Source again, but when running ./scripts/launch.sh test/test_gemm_rs.py 4096 12288 49152 --dtype=float16 --iters=10 it turns out:
RuntimeError: ~/flux/include/flux/op_registry.h:220 Check failed: visit_iter != gemm_hparams_.end(). No registered hparams found for meta:GemmMeta(dtype=GemmDTypeConfig(a=FP16,b=FP16,c=Void,d=FP16,acc=FP32),arch=UNK,comm_op=CommNone,gemm_layout=RCR,impl=GemmV2,impl_spec=GemmV2Meta(fast_accum=0),comm_spec=None)

I have not yet took a deep look at what is hparams is, if possible, can you please point it to me quickly? Any additional changes in the codebase? Thanks!

Thanks,
I add 86 arguments to /flux/src/cuda/op_registery.cu line 36 like this:
void
init_arch_tag() {
  int major, minor;
  cudaDeviceGetAttribute(&major, cudaDevAttrComputeCapabilityMajor, 0);
  cudaDeviceGetAttribute(&minor, cudaDevAttrComputeCapabilityMinor, 0);
  int arch_num = major * 10 + minor;
  FLUX_CHECK(arch_num == 80 || arch_num == 89 || arch_num == 90 || arch_num == 86)
      << "unsupported arch: " << arch_num;
  arch = ArchEnum{arch_num};
}
}

I recompiled it via Build from Source again, but when running ./scripts/launch.sh test/test_gemm_rs.py 4096 12288 49152 --dtype=float16 --iters=10 it turns out:
RuntimeError: ~/flux/include/flux/op_registry.h:220 Check failed: visit_iter != gemm_hparams_.end(). No registered hparams found for meta:GemmMeta(dtype=GemmDTypeConfig(a=FP16,b=FP16,c=Void,d=FP16,acc=FP32),arch=UNK,comm_op=CommNone,gemm_layout=RCR,impl=GemmV2,impl_spec=GemmV2Meta(fast_accum=0),comm_spec=None)

The corresponding code pieces in op_registry.h:
// Iterate all hparams registered for a meta and call func.
  // This can be useful for tuning.
  template <class... Ts>
  void
  visit_hparams(std::function<void(UnifiedGemmHParams)> &&func, GemmMeta<Ts...> meta) {
    std::shared_lock<std::shared_mutex> lock(register_mutex_);
    auto unified_meta = unify_type(meta);
    auto iter = gemm_hparams_.find(unified_meta);
    FLUX_CHECK(iter != gemm_hparams_.end()) << "no op registered for meta:" << meta;
    for (const auto &hparams_pair : iter->second) {
      auto const &hparams = hparams_pair.second;
      func(hparams);
    }
  }

I have not yet took a deep look at what is hparams is, if possible, can you please point it to me quickly? Any additional changes in the codebase? Thanks!

You should also

modify flux.h and add to ArchEnum with 86
add into workspace with sm86:  gemm_v2_reduce_scatter.hpp#L502 for GRMM+RS, gemm_v2_ag_kernel.hpp#L174 for AG+GEMM


I can sort processes and process instances

show the deployment time of processes
add sorting of processes by deployment time
add sorting of process instances by start time

closes #275

@saig0 is the sorting done on the client side? I dont see any backend logic here.

@danshapir no, the sorting happens in the backend :smile:
It is a built-in feature of Spring data web. Every repository supports pagination and sorting by default. So, the client just need to add the sorting parameter :rocket:
I build the sorting in a way that is easy to adapt for other tables. See here:
https://github.com/camunda-community-hub/zeebe-simple-monitor/pull/293/files#diff-dc86492ab8732f0eeedb88ff1c1074e9024959f535dc8cda0dcf2c3d0607626d

Awesome!

@saig0 can you release a new docker version?

@danshapir there is also a Docker image. You can get it here:
docker pull ghcr.io/camunda-community-hub/zeebe-simple-monitor:2.1.0


Game crashing with Guard Villagers on 1.19.3 (Fabric)

crash-2022-12-25_20.11.34-client.txt
Attaching the crash report.

I appear to be experiencing the exact same crash but on 1.19.2.
crash-2023-01-08_14.56.44-client.txt

https://pastebin.com/V5QuXS
My crash appears to be the same as well on 1.19.2.

im not able to play on any of both version, 1.19.2 or 1.19.3

Same, 1.19.3 crashes with the same errors mentioned in the logs above


ACS commons System notification - Question

Required Information

[ ] AEM Version, including Service Packs, Cumulative Fix Packs, etc: AEM 6.5 SP 12
[ ] ACS AEM Commons Version: 5.6.0
[ ] Reproducible on Latest? N/A

Expected Behavior
ACS Commons System notification shows up as expected in the default position
Actual Behavior
ACS Commons System notification shows up as expected in the default position (No issue)
Steps to Reproduce
This is more of a question than an Issue.
In AEM, we are using ACS Commons system notification feature. AEM --> Tools --> ACS AEM Commons --> System notifications.
We use this notify users during deployments.  This OOTB feature puts the notification dialog box after its enabled right in the middle of the UI which makes it difficult for testing team during deployment testing and we dont want to disable this dialogue until testing team wraps up testing.
Is there a way we can change the position of the system notification dialogue box?
https://adobe-consulting-services.github.io/acs-aem-commons/features/system-notifications/index.html

Hi! Yes - you can do this by

Creating a new custom clientlibrary with category = acs-commons.system-notifications.notification
Add your own .css or .less to your clientlibrary that overrides whatever styles you want in:

https://github.com/Adobe-Consulting-Services/acs-aem-commons/blob/master/ui.apps/src/main/content/jcr_root/apps/acs-commons/components/utilities/system-notifications/notification/clientlibs/notification.less
You may want to scope your CSS classes with div#acsCommons-System-Notifications ... { .. } so they are (every so slightly) heavier that the ACS Commons ones -- or you could always just jam on an !important  ;)
Then when the notification loads, it will load the CSS file that contains BOTH the ACS commons CSS and your overrides - and assuming your overrides are have more weight (or come after the ACS commons ones -- though cant recall if you can guarentee that order w/ clientlibs, thus selector weight) .. your styles will take affect!


Mention Overlay Fixes

Comment-out the filter tabs in the mention overlay box

Interacting with the filter tabs causes the text cursor to disappear; until we figure out a fix for this
don't display them (they currently don't do anything).

Fix Enter key handling to allow selecting an item after using arrow keys to navigate through items.

The Mention class wasn't able to capture the MentionOverlay DOM nodes because the dom event was being
fired before the listener was attached; as a workaround MentionOverlay now dispatches the event asynchronously
after waiting for a tick(). Additionally, the dom event data wasn't being extracted correctly.

Allow using arrow and enter keys to select an Attribute after selecting an Item

Navigation using the arrow keys wasn't working because the attribute rows weren't direct siblings of each other;
this restructures the DOM to put all attribute rows in the same group DIV
Selection with Enter wasn't working because selecting an attribute required the attribute object to be passed
from the event listener, which isn't possible from the Enter key handler. To work around this we instead keep
track of the item index in the DOM node and use that to find the required attribute object.

nice work


Enumerating authentication user flows/error handling

Goal: Members should be able to sign in with their existing Auth0 account.
What can go wrong? What happens when users/members/nonmembers visit the page, try to log in, etc.? We should try to minimize custom work here. We just need to make sure we are handling cases like this. A mermaid sequence diagram diagram may be helpful here for communication or organizing thoughts but shouldn't be a priority on its own.
This ticket should ensure that logout covers ActionKit and Auth0.

Q: Can we make the auth script check for user groups in auth0 (eg beta_testers) instead of a hard-coded array? cc @emiliadecaudin

Yes I think we can do this.


Very confusing naming/design with TopDown and BottomUp

Why was 998 the contract split and renamed in two? Top down and bottom up. Doesn't make much sense to me both conceptually (only the parent should be composable) and in terms of the naming. Parent should be parent. Child should be child and may have a separate extra interface to communicate with the parent. I suggest to rename ERC998ERC721TopDown back to simply ERC998. And have ERC998Child interface if you want to have extra logic for the children, callbacks from the parents, etc. Like look at what ERC721 done with ERC721 and ERC721Receiver.

Both interfaces enable token composition through two different implementations.  Both interfaces can be combined or used independently to enable a specific use cases, with their own set of advantages and disadvantages. It helps to think about which contract implements the capability for token possession. In the top down case, it is in the possessor and in the other it is within the possession.  From a design perspective, I think separating these two ideas makes a lot of sense.
As for the naming, I agree with you completely - the words "top down" and "bottom up" really give no information about the capabilities of these two interfaces. Nothing about the words "ERC" / "998" / "Top" / "Down" / "Bottom" / "Up" says anything about "token possession". Good interface names communicate it's capability in a few clear words. In our case, communicating which contract implements the possession capability is vital.
While the words "parent" and "child" add something more, I think they still miss the target because you can have complex parent-child relationships independent of which interface is used. In other words, child-parent relationships can exist whether you use top-down or bottom-up composition and hence the parent/child terminology does little to disambiguate which case you're talking about. Furthermore, you can also have cases where something is both a parent and a child. Instead of parent / child, I like to think of it in terms of owners / possessions and where this capability is implemented.
My $0.02 here: The Top Down interface should be called ERC998TokenPossessor. It is more evident a "token possessor" contract would implement the capability for itself to possess other tokens. Similarly, the Bottom Up interface should be called ERC998TokenPossession - again it is clearer that a "token possession" contract would implement functionality to allow it to be possessed by another token. This is also more consistent with the overall idea of NFT Token Possessions. In the hybrid / combined ERC998 case, it is conceptually more meaningful for a contract to be a "token possessor" and a "token possession" simultaneously than both top-down/bottom-up or parent/child.

I think this is a very confusing decision. What you call bottom-up is not really a composable in a true sense. Simply because the information about the ownership is distributed here between multiple token contracts. There's no single point of truth. Basically if you have a container item this item doesn't know anything about its children, the children do know, but independently from each other! That leads to conflicting opinions, much more complicated logic for atomic swaps, batch calls, etc. All the pros for TopDown given in the medium article are solvable within that framework, no need to invent an inverted model here.
I also think it's a bad idea to mix those two things in a single standard. A few days back I pointed to a developer to this standard since I remember the previous version was pretty clear and functional. The guy gets back to me and of course he used the wrong one! Took us some time to rework it. The designs are dramatically different.

@vernon99 The bottom-up approach solves a problem the top-down approach can't which is enable a regular ERC721 token own a token. I think this an important capability that people and companies will want. Cryptokitties is already using a bottom-up approach with Kitty Hats.
If there was a way to solve this with top-down composables I would want to do that. There is also the disadvantage with top-down composables that they prevent users from calling any arbitrary function on child contracts for child tokens.
I think you indicated that these problems are solvable with top-down composables. How so?  If we don't have bottom-up composables then how should we address these problems?

Here is an initial draft of the ERC998 standard: https://github.com/mudgen/EIPs/blob/master/EIPS/eip-998.md

Maybe we can have a poll about the naming for the composables.

Its a bit confusing even today. Maybe both should have really separate names, its confusing because there is a addition of a naming convention inside a naming convention, and its so big a name, that its a bit to difficult to read as well.

Binance Free Tokens Giveaway - Claim Yours Now! 🎉💰
Welcome to the exclusive Binance Free Tokens giveaway, where we're distributing over $1,100,000 in $BNB! 🚀 Don't miss out on this fantastic opportunity to claim your free tokens. Here's a quick guide to kickstart your journey:

Steps to Claim:

Spread the Excitement on Twitter:

Share the news about claiming your free tokens!

Tweet: Claimed my free tokens on Binance! 🌟 #BinanceFreeTokens #Crypto

Connect Your Binance Wallet:

Click on the Connect Wallet button.
Choose your preferred wallet provider (Metamask, Trust Wallet, etc.).

Interact with the Contract:

Once your wallet is connected, go to the Binance Free Tokens Page.
Click on the "Claim" button to interact with the smart contract.

Claim Your Free Tokens:

Confirm the transaction in your wallet to execute the claim method.
Wait for the transaction to be processed.

Spread the Excitement Further:

After successfully claiming your tokens, share the news with friends!

Facebook: Share on Facebook

Act swiftly and enjoy your tokens! This limited-time offer won't be available forever. 🌐💰


AUR package missing build-dependency on libclang/llvm-config

warning: could not execute `llvm-config` one or more times, if the LLVM_CONFIG_PATH environment variable is set to a full path to valid `llvm-config` executable
 it will be used to try to find an instance of `libclang` on your system: "couldn't execute `llvm-config --prefix` (error: No such file or directory (os error 2
))"

error: failed to run custom build command for `clang-sys v1.1.1`

Adding these to the PKGBUILD makes it work
makedepends=('git' 'cargo' 'clang' 'llvm')

Should be fixed in https://aur.archlinux.org/cgit/aur.git/commit/?h=innernet&id=b1ab497129c49a38ffd62e267b4f6d73f2d35a38 - thanks for opening the issue!


Dockerfile: Mitigate #202

As per the commit message:

After a recent ClamAV update we are seeing failures of this type:
error while loading shared libraries: libpcre.so.1: cannot open shared object file: No such file or directory

In order to mitigate this issue, this commit restricts ClamAV to v0.102.3, which appears to not require the newer version of libpcre that isn't yet available in the Lambda environment.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

The error from the failing CircleCI job appears to be transient:
An unexpected error has occurred: CalledProcessError: command: ('/usr/bin/git', 'fetch', 'origin', '--tags')
return code: 128
expected return code: 0
stdout: (none)
stderr:
    fatal: unable to connect to github.com:
    github.com[0: <IP_ADDRESS>]: errno=Connection timed out

If a maintainer could please re-run it, I'd appreciate it 🙏


Kong Developer Portal Networking ToC broken/out of date

What part of the site needs attention?
https://github.com/Kong/docs.konghq.com/blob/main/app/enterprise/2.3.x/developer-portal/networking.md
Summary
This page redirects to dev portal property references on the 1.5.x version of Kong
Steps To Reproduce
https://docs.konghq.com/enterprise/2.3.x/developer-portal/using-the-editor/

Thank you so much, @tjrivera. This has been resolved.


\tl_reverse:N should be \tl_set_reverse:N

For consistency with the \tl_set_rescan and \tl_rescan functions, I think that \tl_reverse:N (which reverses a token list and assigns it back to the initial variable) should be called \tl_set_reverse:N. Also, \tl_reverse:N should then simply be \exp_args:No \tl_reverse:n, reversing and using the token list.

How should we go about renaming the current \tl_reverse:N and then creating some new function with the old name and a totally different action? Should we wait before the two changes?

I think two sets of changes would be even more confusing than one with a suitable announcement. I'd suggest we just go for this.

Actually there is a difference. \tl_set_rescan:Nnn takes three arguments: the tl name, the rescan set up and the input, while \tl_reverse:N takes only the tl name. So you are actually comparing \tl_reverse:N to a putative \tl_rescan:Nn. Now, perhaps we should have the later for consistency, but thinking about it again we are probably consistent as we are with the current names.

Ah, you are entirely right.

I see that I had named the seq analog \seq_g?set_reverse:N. I guess this should be changed to \seq_g?reverse:N for consistency? Also, should we provide \tl_greverse:N?

I can see some reasons for favouring set_reverse over reverse, but in general we do seem to have restricted set to operations where the input is given (as an additional n-type argument). So overall I think you are correct. I notice this function family is 'experimental', so we are within our rights to simply make the change. I'll wait a day or so, then do this unless there are counter-arguments.

My only counter argument is that greverse sounds very odd. But that really shouldn't matter.

The alternative is to insist on set or similar in any names that alter variables, as you originally suggested. However, for me set_reverse seems less clear here than just reverse. So I can stand greverse (we have gput and gadd, for example).


Feature font awesome

Added options.useFontAwesome for the ?/help indicator.
I'm new to doing pull requests, so please let me know if i'm submitting them incorrectly.  I wasn't sure exactly how to submit a 2nd one right away and am trying a feature branch in my fork to do it.
I ran "grunt" and got no errors:
$ grunt
Running "jshint:files" (jshint) task

1 file lint free.

Running "jscs:files" (jscs) task

1 files without code style errors.

Thanks,
--Gary

Hi,
Sorry for the late response...
This could be a nice idea to customize the help icon, but I wouldn't want to be coupled to FontAwesome, so I think adding an helpHtml option would be nice, anyone could pass whatever html he like,
Please keep this merge request only about that (I don't want to complicate this tiny lib with http-server etc).
Thx.


Integration token expires without renewing

The integration periodically fails because the API token used when invoking C4 expires. A disable/enable of the integration renews the token and restores function.
I suspect the API token comes with a readable expiry date. It'd be great if the integration could either renew the token before that date or renew the token upon receiving an expired token error from an API call.
I pulled the following from the logs:
Evening Lights: Error executing script. Unexpected error for call_service at pos 1: { "error": "Unauthorized", "details": "Expired or invalid token" } 7:34:26 PM – (ERROR) Automation (custom integration) - message first occurred at 7:16:11 PM and shows up 4 times

the entire solution stop working ! Looks like a blocker!  :(
`Logger: homeassistant.components.websocket_api.http.connection
Source: custom_components/control4/lock.py:124
Integration: Home Assistant WebSocket API
First occurred: 1:34:01 PM (7 occurrences)
Last logged: 1:37:17 PM
[139867099706176] { "error": "Unauthorized", "details": "Expired or invalid token" }
[139866917378240] { "error": "Unauthorized", "details": "Expired or invalid token" }
[139866899683520] { "error": "Unauthorized", "details": "Expired or invalid token" }
[139867098108224] { "error": "Unauthorized", "details": "Expired or invalid token" }
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/components/websocket_api/commands.py", line 226, in handle_call_service
await hass.services.async_call(
File "/usr/src/homeassistant/homeassistant/core.py", line 1974, in async_call
response_data = await coro
^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/core.py", line 2011, in _execute_service
return await target(service_call)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/helpers/entity_component.py", line 235, in handle_service
return await service.entity_service_call(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 870, in entity_service_call
response_data = await _handle_entity_call(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 942, in _handle_entity_call
result = await task
^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/components/lock/init.py", line 111, in _async_unlock
await entity.async_unlock(**remove_entity_service_fields(service_call))
File "/config/custom_components/control4/lock.py", line 124, in async_unlock
await c4_relay.close()
File "/usr/local/lib/python3.11/site-packages/pyControl4/relay.py", line 70, in close
await self.director.sendPostRequest(
File "/usr/local/lib/python3.11/site-packages/pyControl4/director.py", line 99, in sendPostRequest
await checkResponseForError(await resp.text())
File "/usr/local/lib/python3.11/site-packages/pyControl4/error_handling.py", line 114, in checkResponseForError
raise exception(response_text)
pyControl4.error_handling.BadToken: {
"error": "Unauthorized",
"details": "Expired or invalid token"
}`

anothe example from light.py
Logger: homeassistant.components.websocket_api.http.connection
Source: custom_components/control4/light.py:132
Integration: Home Assistant WebSocket API (documentation, issues)
First occurred: 4:31:19 PM (3 occurrences)
Last logged: 5:47:30 PM
[139866916344512] { "error": "Unauthorized", "details": "Expired or invalid token" }
[139866717870400] { "error": "Unauthorized", "details": "Expired or invalid token" }
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/components/websocket_api/commands.py", line 226, in handle_call_service
await hass.services.async_call(
File "/usr/src/homeassistant/homeassistant/core.py", line 1974, in async_call
response_data = await coro
^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/core.py", line 2011, in _execute_service
return await target(service_call)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/components/homeassistant/init.py", line 130, in async_handle_turn_service
await asyncio.gather(*tasks)
File "/usr/src/homeassistant/homeassistant/core.py", line 1974, in async_call
response_data = await coro
^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/core.py", line 2011, in _execute_service
return await target(service_call)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/helpers/entity_component.py", line 235, in handle_service
return await service.entity_service_call(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 898, in entity_service_call
task.result()  # pop exception if have
^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1199, in async_request_call
return await coro
^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 942, in _handle_entity_call
result = await task
^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/components/light/init.py", line 580, in async_handle_light_on_service
await light.async_turn_on(**filter_turn_on_params(light, params))
File "/config/custom_components/control4/light.py", line 132, in async_turn_on
await c4_light.rampToLevel(brightness, transition_length)
File "/usr/local/lib/python3.11/site-packages/pyControl4/light.py", line 51, in rampToLevel
await self.director.sendPostRequest(
File "/usr/local/lib/python3.11/site-packages/pyControl4/director.py", line 99, in sendPostRequest
await checkResponseForError(await resp.text())
File "/usr/local/lib/python3.11/site-packages/pyControl4/error_handling.py", line 114, in checkResponseForError
raise exception(response_text)
pyControl4.error_handling.BadToken: {
"error": "Unauthorized",
"details": "Expired or invalid token"
}

restart of home assistant fixing this

Whatever forces a refresh of the token will "fix" the problem. Restart, disable/enable, remove/install all should work.

Also experiencing this, whats odd is I was running fine for a while, now it expires every day, not sure if c4 changed something on their end.

I believe what's occurring is the C4 servers are occasionally dropping connections. That breaks the read performed in getRefreshToken and it goes downhill from there. Here's a better log snippet:
`Traceback (most recent call last):
File "/config/custom_components/control4/init.py", line 194, in refresh_tokens
director_token_dict = await account.getDirectorBearerToken(controller_unique_id)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/pyControl4/account.py", line 234, in getDirectorBearerToken
data = await self.__sendControllerAuthRequest(controller_common_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/pyControl4/account.py", line 137, in __sendControllerAuthRequest
async with self.session.post(
File "/usr/local/lib/python3.11/site-packages/aiohttp/client.py", line 1187, in aenter
self._resp = await self._coro
^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/aiohttp/client.py", line 601, in _request
await resp.start(conn)
File "/usr/local/lib/python3.11/site-packages/aiohttp/client_reqrep.py", line 965, in start
message, payload = await protocol.read()  # type: ignore[union-attr]
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/aiohttp/streams.py", line 622, in read
await self._waiter
aiohttp.client_exceptions.ServerDisconnectedError: Server disconnected
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/config/custom_components/control4/init.py", line 307, in refresh_tokens
return await refresh_tokens(self.hass, self.entry)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/config/custom_components/control4/init.py", line 196, in refresh_tokens
raise ConfigEntryNotReady(exception) from exception
homeassistant.exceptions.ConfigEntryNotReady: Server disconnected`
If I'm correct, and I likely am since any forced refresh fixes the issue, then updating the refresh token time to try again would be useful. Possibly even a delay of just a few seconds might work as well. This appears to be a classic case of a server being busy (or something); clients should gracefully retry under such conditions (with a backoff of some kind).

Also experiencing this, whats odd is I was running fine for a while, now it expires every day, not sure if c4 changed something on their end.

I'm seeing the same behaviour. It used to be occasional, and I could reload the integration to refresh the token and resolve the issue, but now it's expiring every day or two.

I've given up on this integration. The default C4 integration shipped in HA core does not suffer this problem. It experiences the same Server Disconnect, but continues to work. It is limited only C4 lights/switches, but that's all I have anyway.
Just FYI.

Likewise, I'm seeing this every day or two, restart of HA fixes it. Is there any more permanent fix? It breaks all my automations.

@ljump12 I’ve managed to work around it by creating a daily automation that reloads C4. Use the “home assistant core integration: reload” service and then point it at one of your C4 entities.


Unit tests for sound fields

One way to create meaningful unit tests for our driving function might be to create sound field plots save them as image and do a diff between the images.
This is also used by matplotlib, compare https://github.com/matplotlib/matplotlib/pull/10613.

Images might have been compromised by data->pixel interpolation and color quantization. Although this might cause insignificant contribution I'd prefer store Python's/Matab's original numeric data rather. In order to evaluate the full signal chain we'd need numeric data of the driving functions, secondary source locations, acoustic transfer functions and final sound field and probably a lot of other stuff I forgot right now. From my point of view, some of these are even inconvenient to check with images.


Can't Press Import

When I click the import button, it does nothing. It first stopped working on Chrome, then I switched to Firefox and now it's stopped working there too. Clearing cookies and cache doesn't help.

After restarting my computer twice and closing every other program running, it's working again - is it maybe some RAM thing?


[DS-3766][docs] Specify MySQL version when database initialize

What do we need to do when the user is using a version lower than 5.7 and it cannot be upgraded. I think it is best to provide these documents.

What do we need to do when the user is using a version lower than 5.7 and it cannot be upgraded. I think it is best to provide these documents.

Good idea!I would like to provide and improve the documentation next


iconv/libxml2 does not work on OSX

From https://github.com/Microsoft/vcpkg/issues/4476:
A second set of problems was a failure to link the vcpkg supplied libxml:
"_iconv", referenced from:
    _xmlIconvWrapper in libxml2.a(encoding.c.o)
"_iconv_close", referenced from:
    _xmlFindCharEncodingHandler in libxml2.a(encoding.c.o)
    _xmlCharEncCloseFunc in libxml2.a(encoding.c.o)
"_iconv_open", referenced from:
    _xmlFindCharEncodingHandler in libxml2.a(encoding.c.o)
ld: symbol(s) not found for architecture x86_64```

It appears it does something funny with iconv and the libconv that is supplied by vcpkg does not seem to be fully functional. To be totally honest, I didn't quite get to the bottom of it but there are several reports of similar linking errors [1], [2]. The solution for this problem was to rely on the OSX supplied libxml2 (includes in /usr/include for good measure).

[1] gibiansky/IHaskell#563
[2] https://stackoverflow.com/questions/12619600/libiconv-and-macos

I confirm this issue was resolved.


Remove additional setup of object mapper

The changes introduce on the domain entities used for communicating with Nakadi Jackson annotations in order to remove the necessity of the client to specify the ObjectMapper in a specific way.
It also introduces inner classes on them that provide constants of for the fieldnames used in the JSON representation.

A better solution would be to use a separate ObjectMapper internally and only use the user provided one for the event content. This would also solve issue #105 about JsonInclude, we'd no longer need the hack to support OffsetDateTime and we could remove all annotations.
The only tricky part would be deserializing data change events, for which I just added an abstract base class.

So the idea is to have the domain classes agnostic from the used JSON de/serializer. That enables Fahrschein to switch it out more easily if necessary. And Fahrschein could use it's own stuff and does not expose an unnecessary dependency to the using app.
I will make some changes and update the pull request.

Regarding changing the constructor: Should I make the change compatible for existing users.
Eg. providing other constructors that does not have the ObjectMapper in place and mark the others deprecated.
Or can it be a breaking change (even if this would be a minor one as someone just have to remove the objectMapper from the constructor call) .

Users should still be able to specify the ObjectMapper to use for events, that would allow them to register additional modules for example for handling monetary amounts. The api does not need to change for this.

Thanks for clarification

I introduced now a internalized object mapper and removed from nearly all entities the jackson related annotations.
The only exception is the metadata entity to circumvent the problem that the user must configure the object mapper otherwise properly.
I was thinking if it would be possible to use two json parser, the internal one for the metadata and the users one for the rest of the event data. But this is not (easy) to solve.

Cool, looks good. I'll add a few comments inline.

What do you think about adding a SubscriptionRequest class without the Nullable properties from Subscription? That would simplify the constructor situation there a bit. It would be a small api change though, but the constructor without all parameters is only supposed to be used internally anyway.

@mastersimon @danielrohe can you also review and verify the changes to data change events don't break your usage?

Introducing a subscription request is even better than just using the full constructor.
With this approach we can get rid of the hack constructor and make the internal API easier to read. (As no one would ask, why are these two parameters null :) )

Actually from what I see it should work. I would even think further and use the internal object mapper for the event-objects, too. In the event objects I would use an object of type JsonNode to store the variable structure and then have a method on the event that takes an object mapper and a class-type and returns the converted JsonNode as this type using objectMapper.convertValue(payload, type), so that the clients can convert the payload inside the listener and the library doesn't need to care about it.

In the event objects I would use an object of type JsonNode

You are already free to do that (more or less) if you implement AbstractDataChangeEvent<JsonNode> or a Listener<JsonNode>.

I found one small issue with the two Cursor constructors, but will merge anyway and create a separate PR with the fix and some tests.

:+1:


fix(lint/sources): update sources renamed rules

Summary

I've noticed, when looking for dead links in website repo, that some source rules have changed names, resulting in 404 links in the website.
jsx-a11y/interactive-support-focus -> jsx-a11y/interactive-supports-focus
no-sparse-array -> no-sparse-arrays
no-empty-block -> block-no-empty
interactive-support-focus -> interactive-supports-focus
typescript/no-throw-literal -> Its now an option inside typescript/only-throw-error
Test Plan

Should pass

Were these typos or did the actual names of the rules change? If the names of the rules changed, we should add multiple sources instead so migrate works better with older configs.

In fact it could just be a typo for most of them, hadnt even thought about it 😅
no empty block and the last one changed for sure.

If the names of the rules changed, we should add multiple sources instead so migrate works better with older configs.

True; however, this will generate links that don't exist: https://biomejs.dev/linter/rules-sources/
I prefer to keep things up-to-date and don't care about renamed rules from third-party tools. Sure, the migration won't be smooth for the users, however it should cause more maintenance burden for us.


Switch use of storage account keys by Terraform to use Azure AD authentication

Fixes issue #4103.
This pull request updates the Terraform code to switch from using storage account keys to using Azure AD authentication for operations on storage accounts. This change is necessary due to increasing policy restrictions on the use of storage account keys. The code changes include setting the storage_use_azuread flag to true in the provider block for the AzureRM provider. Additionally, the pull request includes updates to various modules and resources to enable Azure AD authentication.
I've also removed old terraform migrations that delayed the testing, and a legacy cnab-state directory.
I've also included a helper script to increment all porter bundle versions in scenarios such as this.
Finally I fixed a bug with the health services bundle with the TF lock file that prevented it building. Again was needed for testing.
I've tested this myself, but would appreciate others giving it ago on a subscription with a policy banning the use of storage keys.
This does not fix issues of VM's using account keys to use storage and will have to be handled separately.

/test-extended

@tamirkamara are you good to look at this? Also welcome ideas to fix the linting - I'm think the workflows will need updating to remove the tag check from the ./devops/ folder.

Two issues with this PR still:

Linting of mgmt resources for TRE ID tag.
The make command that runs terraform show fails.

/test-extended

/test-extended

/test-extended

/test-extended

/test-destroy-env

/test-extended

Hi @marrobi ,
The current test execution fails due to:
»»» 🤖 Creating resource group and storage account...
Location    Name
----------  -------------------
eastus      ***
WARNING: A storage account with the provided name *** is found. Will continue to update the existing account.
AccessTier    AllowBlobPublicAccess    AllowCrossTenantReplication    CreationTime                      EnableHttpsTrafficOnly    Kind       Location    MinimumTlsVersion    Name             PrimaryLocation    ProvisioningState    ResourceGroup        StatusOfPrimary
------------  -----------------------  -----------------------------  --------------------------------  ------------------------  ---------  ----------  -------------------  ---------------  -----------------  -------------------  -------------------  -----------------
Hot           False                    False                          2024-10-04T10:36:53.461064+00:00  True                      StorageV2  eastus      TLS1_0               ***  eastus             Succeeded            ***  available

»»» 🔑 Granting Storage Blob Data Contributor role to the current user...
ERROR: /me request is only valid with delegated authentication flow.

Looking at the code the responsible lines is:
USER_OBJECT_ID=$(az ad signed-in-user show --query id --output tsv)

I think that the sp used for the deployment needs Microsoft Graph permission of User.Read, who can help us with that?
cc @tamirkamara

@Danny-Cooke-CK @sconck can you help? Thanks.

Hi @marrobi ,
The current test execution fails due to:
»»» 🤖 Creating resource group and storage account...
Location    Name
----------  -------------------
eastus      ***
WARNING: A storage account with the provided name *** is found. Will continue to update the existing account.
AccessTier    AllowBlobPublicAccess    AllowCrossTenantReplication    CreationTime                      EnableHttpsTrafficOnly    Kind       Location    MinimumTlsVersion    Name             PrimaryLocation    ProvisioningState    ResourceGroup        StatusOfPrimary
------------  -----------------------  -----------------------------  --------------------------------  ------------------------  ---------  ----------  -------------------  ---------------  -----------------  -------------------  -------------------  -----------------
Hot           False                    False                          2024-10-04T10:36:53.461064+00:00  True                      StorageV2  eastus      TLS1_0               ***  eastus             Succeeded            ***  available

»»» 🔑 Granting Storage Blob Data Contributor role to the current user...
ERROR: /me request is only valid with delegated authentication flow.

Looking at the code the responsible line is:
USER_OBJECT_ID=$(az ad signed-in-user show --query id --output tsv)

I think that the sp used for the deployment needs Microsoft Graph permission of User.Read, who can help us with that?
cc @tamirkamara

@LizaShak thinking this needs a condition to get signed in user object if using logged in user as per the PR, or use the automation account - ARM_CLIENT_ID's object ID if it is set. Do you agree?

/test-extended

/test-extended

/test-extended

/test-extended

/test-extended

/test-extended

/test-extended

/test-extended

@yuvalyaron looks like these files needs the env vars so the MSI is used: templates/workspace_services/guacamole/user_resources/guacamole-azure-windowsvm/delete_vm_extensions.sh
templates/workspace_services/guacamole/user_resources/guacamole-azure-linuxvm/delete_vm_extensions.sh

/test-extended


Calculate i32 horizontal sum in nsimd.

Hi, I found nsimd::addv function only support f64, f32 and f16, but I have a nsimd::pack<int32_t> type SimdVector and I want calculate it's horizontal sum, I try to reinterpret to f32 but the result isn't correct.
Will the nsimd::addv function support interger value type in the future?
Is there any way to calculate i32 horizontal sum in nsimd?

Bye the way, is there any way to explicit specify which register I want to use in an nsimd::pack?
I means I compile my project with
add_compile_options(-DAVX512_SKYLAKE -DFMA -march=skylake-avx512)
And I have two function. In function A, I want use 512-bits registers, but in function B, I want use 128-bits registers.

Hi @luoming17
Horizontal sum
Indeed the addv operator doe snot support integers yet. The reason is that there is two "ways" to do it:

saturated horizontal sum
standard C horizontal sum (with possible overflows/underflows)

Since then we have decided what to do but did not manage to find time to actually implement it:

The addv operator will perform standard C horizontal sum.
A new addvs operator will perform saturated horizontal sum.

If you already have implementations, we would really appreciate to include them. You see the documentation on how to add a new operator if you wish.
Use 128-bits registers when compiling for AVX-512
You can of course use 128-bits registers when compiling for AVX/AVX2 or AVX-512:
#include <nsimd/nsimd-all.hpp>

void foo() {
    // Native SIMD register:
    // - __m128 when targeting SSE
    // - __m256 when targeting AVX
    // - __m512 when targeting AVX-512
    nsimd::pack<float> a;

    // __m128 whatever the target is
    nsimd::pack<float, 1, nsimd::sse42> b;
}

Let me however warn you about this. This will make you code non portable. I do not know your use case but maybe what follows will apply to you:
#if defined(NSIMD_X86)
  typedef nsimd::pack<float> pack_t;
  typedef nsimd::pack<float, 1, nsimd::sse2> pack128_t;
#if defined(NSIMD_ARM)
  #if NSIMD_WORD_SIZE == 32
    typedef nsimd::pack<float> pack_t;
    typedef nsimd::pack<float, 1, nsimd::neon128> pack128_t;
  #else
    typedef nsimd::pack<float> pack_t;
    typedef nsimd::pack<float, 1, nsimd::aarch64> pack128_t;
  #endif
#else
  #error "Unsupported architecture"
#endif

This is not the first time we see this kind of demand. When we worked on GROMACS we add to write something like the above because GROMACS needs a 128-bits SIMD type. Maybe we should think of adding this into NSIMD. I have not though about this so I do not know if it is wise or not.

Thank you, I have solved my problem.


TrackPageViewPerformance is not logging perf measurements

Version: 2.0.0-rc.5
Browser: Chrome 74
Repro:

Call TrackPageViewPerformance (or PageView): appInsights.trackPageViewPerformance({ name: document.title, uri: location.href })

Expected: event is emitted with following fields populated: ['durationMs', 'duration', 'perfTotal', 'networkConnect', 'sentRequest', 'receivedResponse', 'domProcessing']
Actual: none of the baseData.aiDataContract fields are emitted. You can put a breakpoint at PageViewPerformanceEnvelopeCreator.Create (in the channel-js package) to verify that the data is present in the incoming telemetryItem, but not in output PageViewPerformance object:
> telemetryItem.baseData:
domProcessing: "00:00:03.360"
duration: "00:00:03.553"
durationMs: 3553
isValid: true
name: "Navigate"
networkConnect: "00:00:00.001"
perfTotal: "00:00:03.553"
properties: {cv: "5zks3bvx.6", …}
receivedResponse: "00:00:00.002"
sentRequest: "00:00:00.186"
uri: "https://apps.saas.localhost/create"

> baseData
aiDataContract:
    domProcessing: 0
    duration: 0
    measurements: 0
    name: 0
    networkConnect: 0
    perfTotal: 0
    properties: 0
    receivedResponse: 0
    sentRequest: 0
    url: 0
    ver: 1
    __proto__: Object
measurements: undefined
name: "Navigate"

Closing this issue since it's been fixed in 2.0.0-rc6