Interaction terms for Categorical Variables in a RE Model

I'm trying to assess the effects on "returnAvg" of the interactions between my "SupplyClass" and "DupeClass" categorical variables, but I'm not sure I'm doing it right given that I always get the following error:
Traceback (most recent call last):
  File "F:\Python Projects\Tesi\Random Effects Regression.py", line 51, in <module>
    model = RandomEffects.from_formula("returnAvg ~ 1 + type + Color + Items + StoreRange + HasGlow + HasCutout + SupplyClass*DupeClass", data=data)
  File "F:\Python Projects\Tesi\venv\lib\site-packages\linearmodels\panel\model.py", line 2670, in from_formula
    mod = cls(dependent, exog, weights=weights, check_rank=check_rank)
  File "F:\Python Projects\Tesi\venv\lib\site-packages\linearmodels\panel\model.py", line 2616, in __init__
    super().__init__(dependent, exog, weights=weights, check_rank=check_rank)
  File "F:\Python Projects\Tesi\venv\lib\site-packages\linearmodels\panel\model.py", line 328, in __init__
    self._validate_data()
  File "F:\Python Projects\Tesi\venv\lib\site-packages\linearmodels\panel\model.py", line 479, in _validate_data
    rank_of_x = self._check_exog_rank()
  File "F:\Python Projects\Tesi\venv\lib\site-packages\linearmodels\panel\model.py", line 434, in _check_exog_rank
    raise ValueError(
ValueError: exog does not have full column rank. If you wish to proceed with model estimation irrespective of the numerical accuracy of coefficient estimates, you can set check_rank=False.
ValueError: exog does not have full column rank. If you wish to proceed with model estimation irrespective of the numerical accuracy of coefficient estimates, you can set check_rank=False.

Turning check_rank to True leads to Singular Matrix error.
I've tried looking into the documentation but I haven't found any answer on how to do it properly.
Here's how I tried inserting the interaction terms:
model = RandomEffects.from_formula("returnAvg ~ 1 + type + Color + Items + StoreRange + SupplyClass+ DupeClass + SupplyClass*DupeClass ", data=data)

Are these both pandas categorigcals?

Can you provide some more information on the structure of the data you are modeling?

I'm pulling the data from my .csv longform unbalanced panel database. When I say categorical, e.g. SupplyClass or DupeClass, I refer to an array of strings classifying each id from column "itemName" differently. I don't run into any problem using these as separated explanatory variables without creating dummies beforehand, as linearmodels recognizes them as categoricals and handles them automatically.

What size is the array?  What are the entity and time indices?


2.4.3 (with updated KDM) - calico-node keeps restarting in K8s cluster with canal networking

Steps to reproduce the problem:

Build Rancher v2.4.3 and change KDM to use branch dev-v2.4
Deploy a cluster with with canal networking, using any of the following K8s version :
1.16.10-rancher2-1
1.15.12-rancher2-2
1.17.6-rancher2-1

rancher/calico-node:v3.13.4  fails to start with following errors:
2020-05-28 06:13:22.858 [INFO][9] startup.go 309: Early log level set to info
2020-05-28 06:13:22.858 [INFO][9] startup.go 325: Using NODENAME environment for node name
2020-05-28 06:13:22.858 [INFO][9] startup.go 337: Determined node name: ip-172-31-12-10
2020-05-28 06:13:22.860 [INFO][9] startup.go 369: Checking datastore connection
2020-05-28 06:13:22.872 [INFO][9] startup.go 393: Datastore connection verified
2020-05-28 06:13:22.872 [INFO][9] startup.go 104: Datastore is ready
2020-05-28 06:13:22.876 [INFO][9] customresource.go 101: Error getting resource Key=GlobalFelixConfig(name=CalicoVersion) Name="calicoversion" Resource="GlobalFelixConfigs" error=the server could not find the requested resource (get GlobalFelixConfigs.crd.projectcalico.org calicoversion)
2020-05-28 06:13:22.881 [ERROR][9] startup.go 146: failed to query kubeadm's config map error=configmaps "kubeadm-config" is forbidden: User "system:serviceaccount:kube-system:canal" cannot get resource "configmaps" in API group "" in the namespace "kube-system"
2020-05-28 06:13:22.881 [WARNING][9] startup.go 1214: Terminating
Calico node failed to start

On fresh install v2.4.4-rc2  , calico node comes up successfully without any problems in following K8s clusters with "canal" networking:
1.16.10-rancher2-1
1.15.12-rancher2-2
1.17.6-rancher2-1


Support ODT as attachment

Hey there,
We use your API for 1 year now and it's quite frustrating to compress the ODT to zip before sending them.
Can you change that behaviour as well as the doc?

Possible extension values = gif, png, bmp, cgm, jpg, jpeg, tif, tiff, rtf, txt, css, shtml, html, htm, csv, zip, pdf, xml, doc, docx, ics, xls, xlsx, ppt, tar, and ez.

Hi @lolobosse , The support for ODT is now available.
Thanks!


How can I grant a group or a user all of permissions?

I have tried to grant a group the permission, ., but it doesn't work. Do I have to grant all of the permissions one by one? It is a huge work.

I think you might be thinking of the Astrix * wildcard
It's not recommended to use wildcards for any mod.

Thank you. If I want to grant a group all of the permissions for a plugin such as Nations, can I just grant the group Nations.*?

That's hwo that would work.  yes.

Thank you very much! 👍


"Please add the following callbacks" warning

🐛 Bug
Pytorch Lightning emits a false positive (?) warning when restoring from checkpoint during model tuning with auto_lr_find.
To Reproduce
import os

import torch
from torch.utils.data import DataLoader, Dataset

from pytorch_lightning import LightningModule, Trainer
from pytorch_lightning.callbacks import ModelCheckpoint

class RandomDataset(Dataset):
    def __init__(self, size, length):
        self.len = length
        self.data = torch.randn(length, size)

    def __getitem__(self, index):
        return self.data[index]

    def __len__(self):
        return self.len

class BoringModel(LightningModule):
    def __init__(self, lr=0.1):
        self.save_hyperparameters()
        super().__init__()
        self.layer = torch.nn.Linear(32, 2)

    def forward(self, x):
        return self.layer(x)

    def training_step(self, batch, batch_idx):
        loss = self(batch).sum()
        self.log("train_loss", loss)
        return {"loss": loss}

    def validation_step(self, batch, batch_idx):
        loss = self(batch).sum()
        self.log("valid_loss", loss)

    def test_step(self, batch, batch_idx):
        loss = self(batch).sum()
        self.log("test_loss", loss)

    def configure_optimizers(self):
        return torch.optim.SGD(self.layer.parameters(), lr=self.hparams.lr)

    def train_dataloader(self):
        return DataLoader(RandomDataset(32, 1024), batch_size=2, num_workers=8)

    def val_dataloader(self):
        return DataLoader(RandomDataset(32, 1024), batch_size=2, num_workers=8)

    def test_dataloader(self):
        return DataLoader(RandomDataset(32, 1024), batch_size=2, num_workers=8)

def run():
    model = BoringModel()

    trainer = Trainer(
        default_root_dir=os.getcwd(),
        max_epochs=1,
        auto_lr_find=True,
        callbacks=[ModelCheckpoint()]
    )
    trainer.tune(model, lr_find_kwargs={"early_stop_threshold": None})

if __name__ == "__main__":
    run()

Expected behavior
No warnings
Actual behavior
/usr/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py:1721: UserWarning: Be aware that when using `ckpt_path`, callbacks used to create the checkpoint need to be provided during `Trainer` instantiation. Please add the following callbacks: ["ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 1, 'train_time_interval': None, 'save_on_train_epoch_end': None}"].

Environment
* CUDA:
	- GPU:
		- NVIDIA GeForce GTX 1050
	- available:         True
	- version:           11.3
* Packages:
	- numpy:             1.22.3
	- pyTorch_debug:     False
	- pyTorch_version:   1.11.0+cu113
	- pytorch-lightning: 1.6.0
	- tqdm:              4.63.2
* System:
	- OS:                Linux
	- architecture:
		- 64bit
		- ELF
	- processor:
	- python:            3.10.4
	- version:           #1 SMP PREEMPT Mon Mar 28 09:16:36 UTC 2022

cc @akihironitta @borda @rohitgr7

I don't really know, why is this happening or how should this be fixed, to be honest. Also, I am really busy right now, so even if I was familiar with the internals of the checkpointing mechanism, I still wouldn't be able to contribute to this issue right now.

hey @RuRo !
can you tell how are you passing the checkpoint path here to resume?
    trainer = Trainer(
        default_root_dir=os.getcwd(),
        max_epochs=1,
        auto_lr_find=True,
        callbacks=[ModelCheckpoint()]
    )
    trainer.tune(model, lr_find_kwargs={"early_stop_threshold": None})

@rohitgr7 Sorry, I don't understand the question. The example code, that I provided, is a full example. I am not passing any checkpoint paths anywhere.
I think, that the checkpoint is created and restored automatically by auto_lr_find during tune.


DatabaseError: Access denied for user

Trying to run the example compose file for ghost from here and getting the following errors, no matter what I try:
docker-ghost-ghost-1  | [2023-03-09 23:34:20] ERROR Access denied for user<EMAIL_ADDRESS>(using password: YES)
docker-ghost-ghost-1  |
docker-ghost-ghost-1  | Access denied for user<EMAIL_ADDRESS>(using password: YES)
docker-ghost-ghost-1  |
docker-ghost-ghost-1  | "Unknown database error"
docker-ghost-ghost-1  |
docker-ghost-ghost-1  | Error ID:
docker-ghost-ghost-1  |     500
docker-ghost-ghost-1  |
docker-ghost-ghost-1  | Error Code:
docker-ghost-ghost-1  |     ER_ACCESS_DENIED_ERROR
docker-ghost-ghost-1  |
docker-ghost-ghost-1  | ----------------------------------------
docker-ghost-ghost-1  |
docker-ghost-ghost-1  | DatabaseError: Access denied for user<EMAIL_ADDRESS>(using password: YES)
docker-ghost-ghost-1  |     at DatabaseStateManager.getState (/var/lib/ghost/versions/4.48.9/core/server/data/db/state-manager.js:64:32)
docker-ghost-ghost-1  |     at DatabaseError.KnexMigrateError (/var/lib/ghost/versions/4.48.9/node_modules/knex-migrator/lib/errors.js:7:26)
docker-ghost-ghost-1  |     at new DatabaseError (/var/lib/ghost/versions/4.48.9/node_modules/knex-migrator/lib/errors.js:55:26)
docker-ghost-ghost-1  |     at /var/lib/ghost/versions/4.48.9/node_modules/knex-migrator/lib/database.js:57:19
docker-ghost-ghost-1  |     at processTicksAndRejections (internal/process/task_queues.js:95:5)
docker-ghost-ghost-1  |     at async DatabaseStateManager.getState (/var/lib/ghost/versions/4.48.9/core/server/data/db/state-manager.js:40:13)
docker-ghost-ghost-1  |     at async DatabaseStateManager.makeReady (/var/lib/ghost/versions/4.48.9/core/server/data/db/state-manager.js:73:25)
docker-ghost-ghost-1  |     at async initDatabase (/var/lib/ghost/versions/4.48.9/core/boot.js:69:5)
docker-ghost-ghost-1  |     at async bootGhost (/var/lib/ghost/versions/4.48.9/core/boot.js:414:9)
docker-ghost-ghost-1  |
docker-ghost-ghost-1  | Error: Access denied for user<EMAIL_ADDRESS>(using password: YES)
docker-ghost-ghost-1  |     at Packet.asError (/var/lib/ghost/versions/4.48.9/node_modules/mysql2/lib/packets/packet.js:728:17)
docker-ghost-ghost-1  |     at ClientHandshake.execute (/var/lib/ghost/versions/4.48.9/node_modules/mysql2/lib/commands/command.js:29:26)
docker-ghost-ghost-1  |     at Connection.handlePacket (/var/lib/ghost/versions/4.48.9/node_modules/mysql2/lib/connection.js:456:32)
docker-ghost-ghost-1  |     at PacketParser.onPacket (/var/lib/ghost/versions/4.48.9/node_modules/mysql2/lib/connection.js:85:12)
docker-ghost-ghost-1  |     at PacketParser.executeStart (/var/lib/ghost/versions/4.48.9/node_modules/mysql2/lib/packet_parser.js:75:16)
docker-ghost-ghost-1  |     at Socket.<anonymous> (/var/lib/ghost/versions/4.48.9/node_modules/mysql2/lib/connection.js:92:25)
docker-ghost-ghost-1  |     at Socket.emit (events.js:400:28)
docker-ghost-ghost-1  |     at addChunk (internal/streams/readable.js:293:12)
docker-ghost-ghost-1  |     at readableAddChunk (internal/streams/readable.js:267:9)
docker-ghost-ghost-1  |     at Socket.Readable.push (internal/streams/readable.js:206:10)
docker-ghost-ghost-1  |     at TCP.onStreamRead (internal/stream_base_commons.js:188:23)
docker-ghost-ghost-1  |
docker-ghost-ghost-1  | [2023-03-09 23:34:20] WARN Ghost is shutting down

The config file looks like this:
version: '3.1'

services:

  ghost:
    image: ghost:4-alpine
    restart: always
    ports:
      - 8080:2368
    environment:
      # see https://ghost.org/docs/config/#configuration-options
      database__client: mysql
      database__connection__host: db
      database__connection__database: ghost
      database__connection__user: ghost
      database__connection__password: ghost
      # this url value is just an example, and is likely wrong for your environment!
      url: http://localhost:8080
      # contrary to the default mentioned in the linked documentation, this image defaults to NODE_ENV=production (so development mode needs to be explicitly specified if desired)
      #NODE_ENV: development

  db:
    image: mysql:8.0
    restart: always
    environment:
      MYSQL_USER: ghost
      MYSQL_PASSWORD: ghost
      MYSQL_DATABASE: ghost

Did you at any point start the db service, stop it, and then later start it but with different environment values?
Short answer: docker-compose rm -fv (assuming you have no important data already in mysql or ghost)
Long answer: https://github.com/docker-library/postgres/issues/203#issuecomment-255200501 (about postgres, but mysql behaves the same for this case)

Yes, I did try to change the credentials because the default example with unmodified variables has a different error related to not being able to find the db at all as far as I remember. I tried doing docker-compose rm -fv and even did a full docker system prune -a which re created the db and prompted me to add MYSQL_ROOT_PASSWORD as an environment variable. After re doing the above with the correct variables in  place I still got the same DatabaseError: connect ECONNREFUSED <IP_ADDRESS>:3306 error.

Turns out that just waiting for a few minutes made the database finish initializing and it would connects and starts working as expected.


Fix FollowPath bug

Fix follow path bug
Fixes

Setting number_lap to -1 is supposed to not limit the number of laps when using FollowPath, but the bot starts to rest after 1 round.

@solderzzc Ping!

still, the CI build failed. ...

how to trigger the CI again, seems I just fixed it

@solderzzc Seems like the server was busy earlier. I see that you've already submitted the fix. I've triggered a new CI build by reopening the PR.

Great, time to merge :) thanks @alexyaoyang

That's great! 👍


Eliminate scala.Some overhead in Enso IR

IR is a deep tree structure that is generated by TreeToIr and processed in subsequent steps by IRPasses producing new copy of the IR in every run. While the copying has been improved by #10718 all the IR structure objects need to be allocated on the heap at the end to be passed to IrToTruffle and held in the memory indefinitely.
Many of the IR (case) classes are using scala.Option which adds overhead of at least 16 bytes (empty object takes 8 bytes, object with a field at least 16 bytes), yet another pointer indirection when accessing the object wrapped by Some(object) and defies cache locality as Some and object needn't be allocated next to each other and may cause processor cache misses.
enso$ javap -cp runtime.jar -private org/enso/compiler/core/ir/Expression.Block
public class org.enso.compiler.core.ir.Expression$Block implements  ... {
  private final scala.Option<org.enso.compiler.core.ir.IdentifiedLocation> location;

Whenever there is an IdentifiedLocation associated with an IR (and there are thousands), the pointer isn't from the IR to the IdentifiedLocation object, but to scala.Some and then to the IdentifiedLocation object. That's wasting 16 bytes of memory per instance. Rather than that we should change the location field to point directly to IdentifiedLocation or null.
There is no need to change the IR API. We only need to change the internal implementation. Only that enough to effectively use the allocated memory. Moreover, because of JVM's escape analysis (see this paper for more details), there is no overhead in wrapping/unwrapping such a location & co. fields by scala.Option when needed - the allocation will be virtualized when it is known the allocated scala.Option doesn't escape:

let's store objects on the heap without scala.Option wrapper - e.g. private final org.enso.compiler.core.ir.IdentifiedLocation location
let's wrap them in a getter: scala.Option<IdentifiedLocation> location() { return scala.Option.apply(location); }
let's keep the API intact - one can still write ir.location().map(l -> l.start()) - the scala.Option instance won't even be allocated thanks to escape analysis

All we need to do is to find a way to rewrite our IR subclasses to avoid these scala.Option fields. An example how such API should look like is the IdentifiedLocation record (already written in Java) and its id field:
enso$ javap -cp runtime.jar -private org.enso.compiler.core.ir.IdentifiedLocation
public final class org.enso.compiler.core.ir.IdentifiedLocation extends java.lang.Record {
  private final java.util.UUID uuid;

  public org.enso.compiler.core.ir.IdentifiedLocation(org.enso.compiler.core.ir.Location, java.util.UUID);
  public static org.enso.compiler.core.ir.IdentifiedLocation create(org.enso.compiler.core.ir.Location, scala.Option<java.util.UUID>);

  public scala.Option<java.util.UUID> id();
}

The constructor still accepts scala.Option<java.util.UUID> and the getter returns scala.Option<java.util.UUID>. The id() getter does the wrapping when needed, but the reference to uuid is stored effectively in the memory.

The measure the potential benefit, let's run following experiment:
diff --git engine/runtime-parser/src/test/java/org/enso/compiler/core/EnsoParserTest.java engine/runtime-parser/src/test/java/org/enso/compiler/core/EnsoParserTest.java
index 249151bec5..0f54b5380a 100644
--- engine/runtime-parser/src/test/java/org/enso/compiler/core/EnsoParserTest.java
+++ engine/runtime-parser/src/test/java/org/enso/compiler/core/EnsoParserTest.java
@@ -41,6 +41,26 @@ public class EnsoParserTest {
     if (ensoCompiler != null) ensoCompiler.close();
   }

+  @Test
+  public void parseHugeFile() throws Exception {
+    var f = new File("../../distribution/lib/Standard/Table/0.0.0-dev/src/Table.enso");
+    assertTrue("File should exist " + f.getCanonicalPath(), f.exists());
+    var src = Files.readString(f.getCanonicalFile().toPath());
+    var ir = ensoCompiler.compile(src);
+    var cnt = new int[2];
+    ir.preorder().foreach(e -> {
+      if (e.location().isDefined()) {
+        cnt[0]++;
+      } else {
+        cnt[1]++;
+      }
+      return null;
+    });
+    System.err.println("Take heap dump now!");
+    Thread.sleep(10000);
+    fail("With location " + cnt[0] + " without location " + cnt[1]);
+  }
+
   @Test
   public void testParseMain7Foo() {
     parseTest("""

execute as
sbt:enso> runtime-parser/testOnly *EnsoParserTest -- -z org.enso.compiler.core.EnsoParserTest.parseHugeFile

the test parses the largest Enso file in our distribution, creates IR for it and stops for 10s - take a heap dump and look for scala.Option in it:

org.enso.compiler.core.ir.IdentifiedLocation has 11506 instances that occupy	276144 bytes
org.enso.compiler.core.ir.Location has 11506 instances	with 276144 bytes
scala.Some there is 12005 instances occupying 192080 bytes

There is a single org.enso.compiler.core.ir.Module instance (as expected) and its retained size (e.g. the whole tree) occupies 1 890 808 bytes. E.g. scala.Some adds 10% overhead to the whole IR size!

Based on previous investigation I run a little inlining experiment trying to eliminate overhead of Location reference in IdentifiedLocation. With following change:
enso$ git diff engine/runtime-parser/src/main/java/org/enso/compiler/core/ir/IdentifiedLocation.java
diff --git engine/runtime-parser/src/main/java/org/enso/compiler/core/ir/IdentifiedLocation.java engine/runtime-parser/src/main/java/org/enso/compiler/core/ir/IdentifiedLocation.java
index 14d790adba..5a2e79d779 100644
--- engine/runtime-parser/src/main/java/org/enso/compiler/core/ir/IdentifiedLocation.java
+++ engine/runtime-parser/src/main/java/org/enso/compiler/core/ir/IdentifiedLocation.java
@@ -3,35 +3,46 @@ package org.enso.compiler.core.ir;
 import java.util.UUID;
 import scala.Option;

-public record IdentifiedLocation(Location location, UUID uuid) {
+public record IdentifiedLocation(int start, int end, UUID uuid) {
   public IdentifiedLocation(Location location) {
-    this(location, (UUID) null);
+    this(location.start(), location.end(), null);
+  }
+
+  public IdentifiedLocation(Location location, UUID uuid) {
+    this(location.start(), location.end(), uuid);
   }

   /** Creates new location from an optional UUID. */
   public static IdentifiedLocation create(Location location, Option<UUID> uuid) {
-    return new IdentifiedLocation(location, uuid.isEmpty() ? null : uuid.get());
+    return new IdentifiedLocation(location.start(), location.end(), uuid.isEmpty() ? null : uuid.get());
+  }
+
+  /**
+   * @return location of this identified location
+   */
+  public Location location() {
+    return new Location(start, end);
   }

   /**
    * @return the character index of the start of this source location.
    */
   public int start() {
-    return location().start();
+    return start;
   }

   /**
    * @return the character index of the end of this source location.
    */
   public int end() {
-    return location().end();
+    return end;
   }

   /**
    * @return the length in characters of this location.
    */
   public int length() {
-    return location().length();
+    return end - start;
   }

   /**

the retained size of org.enso.compiler.core.ir.Module goes down by 15% to 1 614 664 bytes

as the Location instances completely disappear from the heap and the IdentifiedLocation instances keep occupying the same about of space.
Looks like there is potential in inlining objects into each other - and not only of scala.Some!


Extend 'X-ClickHouse-Progress' header in HTTP protocol

HTTP protocol allows to track query execution process. Currently it returns following data:
< X-ClickHouse-Progress: {"read_rows":"88089","read_bytes":"1884341","total_rows":"122925998080"}
Would be more useful for the client to have 'percent_complete' or something similar to display the progress bar, similar to clickhouse-client

+1

Console client basically just receives the same numbers and divides read_rows by total_rows with couple additional tricks.
See https://github.com/yandex/ClickHouse/blob/master/dbms/programs/client/Client.cpp#L1382

If it is used from some JS, for example, it may be complicated, since JS does not support Int64, while total rows may easily exceed that.

Javascript uses Floats for big numbers. And it's completely ok to use floats for percent calculations:
x = JSON.parse('{"read_rows":"12292599800000000000000","read_bytes":"1884341","total_rows":"122925998000000000000000"}');
percent = 100 * x.read_rows / x.total_rows;

There is always a workaround. This is a usability question.

Actually, there are different problems with progress in HTTP.
For example when you get a streaming output  (like 'SELECT columns FROM big_table PREWHERE condition_meet_thousands_of_rows')  and server buffer is full server start sending data and stop sending progress.
Or there are often moment when progress came to 'almost' finish - and process something which can't estimate and it you will output speed of processing (in rows/s, Gb/s) you see that progress came to 100% stop there for minutes, while speed continuously degrades.
Or you don't get the final stats in HTTP.
But if calculating percents makes some problems for some scenarios - it's really easy to add.

Please check that: https://github.com/yandex/ClickHouse/issues/208

The response would be larger in data size and those % are implicit and can be calculated everywhere. I think that would be a step back as there's no really need for that.


Scripts don't work while running ironbar  under a systemd service

Describe the bug
Scripts don't work while running ironbar  under a systemd service
To Reproduce
Steps to reproduce the behavior:

Use script
See error

Expected behavior
Scripts work
System information:

Distro: NixOS unstable (aka 23.05)
Compositor: Hyprland
Ironbar version: 0.11 (latest commit on master)

Configuration
Config (generated json from nix)
{
  "anchor_to_edges": true,
  "center": [
    {
      "bar": [
        {
          "class": "music-img-img",
          "src": "{{poll:5000:playerctl metadata mpris:artUrl}}",
          "type": "image"
        }
      ],
      "class": "music-img",
      "name": "music-img",
      "on_mouse_enter": {
        "cmd": "/nix/store/7kaxirybmsqz1wnq290aj9fifwgdisjx-mctrls-shower/bin/mctrls-shower"
      },
      "on_mouse_exit": {
        "cmd": "/nix/store/c6swqv9dw2sbiw5zsxar5f3s5zl1m0ix-mctrls-hider/bin/mctrls-hider"
      },
      "type": "custom"
    },
    {
      "format": "{title}",
      "icons": {
        "pause": "icon:media-playback-pause",
        "play": "icon:media-playback-start"
      },
      "music_dir": "/home/yavor/Music",
      "on_mouse_enter": {
        "cmd": "/nix/store/7kaxirybmsqz1wnq290aj9fifwgdisjx-mctrls-shower/bin/mctrls-shower"
      },
      "on_mouse_exit": {
        "cmd": "/nix/store/c6swqv9dw2sbiw5zsxar5f3s5zl1m0ix-mctrls-hider/bin/mctrls-hider"
      },
      "player_type": "mpris",
      "show_status_icon": false,
      "truncate": "end",
      "type": "music"
    },
    {
      "bar": [
        {
          "class": "music-ctrls-prev",
          "label": "玲",
          "on_click": "!/nix/store/nxip05k3sg7m937vvavjpfdak6k3ica0-iron-previous/bin/iron-previous",
          "type": "button"
        },
        {
          "class": "music-ctrls-pp",
          "label": "{{poll:100:/nix/store/jha3arpk5fpcspbam0nkdrrdsvx79l09-pp-icon/bin/pp-icon}}",
          "on_click": "!/nix/store/rqacrmlhvwnmp90i88wmmx14bismvjp2-iron-play-pause/bin/iron-play-pause",
          "type": "button"
        },
        {
          "class": "music-ctrls-next",
          "label": "怜",
          "on_click": "!/nix/store/qf2246j28xybmz7h9yb23nxi4b0yhm13-iron-next/bin/iron-next",
          "type": "button"
        }
      ],
      "class": "music-ctrls",
      "name": "music-ctrls",
      "on_mouse_enter": {
        "cmd": "/nix/store/7kaxirybmsqz1wnq290aj9fifwgdisjx-mctrls-shower/bin/mctrls-shower"
      },
      "on_mouse_exit": {
        "cmd": "/nix/store/c6swqv9dw2sbiw5zsxar5f3s5zl1m0ix-mctrls-hider/bin/mctrls-hider"
      },
      "show_if": {
        "cmd": "/nix/store/i9mm84k7myf1fis2xgnbsqxj3jd1ygn6-mctrls-checker/bin/mctrls-checker",
        "interval": 500,
        "mode": "poll"
      },
      "transition_type": "slide_start",
      "type": "custom"
    }
  ],
  "end": [
    {
      "type": "tray"
    },
    {
      "format": [
        " {cpu_percent}%",
        " {memory_percent}%"
      ],
      "type": "sys_info"
    },
    {
      "bar": [
        {
          "label": "",
          "name": "power-btn",
          "on_click": "popup:toggle",
          "type": "button"
        }
      ],
      "class": "power-menu",
      "popup": [
        {
          "orientation": "vertical",
          "type": "box",
          "widgets": [
            {
              "label": "Power menu",
              "name": "header",
              "type": "label"
            },
            {
              "type": "box",
              "widgets": [
                {
                  "class": "power-btn",
                  "label": "",
                  "on_click": "!shutdown now",
                  "type": "button"
                },
                {
                  "class": "power-btn",
                  "label": "",
                  "on_click": "!reboot",
                  "type": "button"
                }
              ]
            },
            {
              "label": "Uptime: {{30000:/nix/store/ws0j8bcqxcy08d664cbbbg6j0xlg1vg6-fmt-uptime/bin/fmt-uptime}}",
              "name": "uptime",
              "type": "label"
            }
          ]
        }
      ],
      "tooltip": "Up: {{30000:/nix/store/ws0j8bcqxcy08d664cbbbg6j0xlg1vg6-fmt-uptime/bin/fmt-uptime}}",
      "type": "custom"
    },
    {
      "format": "%l:%M %P",
      "type": "clock"
    }
  ],
  "height": 32,
  "icon_theme": "Papirus-Dark",
  "position": "top",
  "start": [
    {
      "bar": [
        {
          "label": "",
          "on_click": "!hyprctl dispatch exec \"zsh -c 'rofi -show drun'\"",
          "type": "button"
        }
      ],
      "class": "nix-launcher",
      "name": "nix-launcher",
      "type": "custom"
    },
    {
      "all_monitors": false,
      "name_map": {
        "1": "●",
        "10": "●",
        "2": "●",
        "3": "●",
        "4": "●",
        "5": "●",
        "6": "●",
        "7": "●",
        "8": "●",
        "9": "●"
      },
      "type": "workspaces"
    }
  ]
}

Additional context
Works while executing normally, and not under systemd service
Screenshots

From journalctl and is same for like every occurance of the issue, which is like a lot cuz its polling (and retrying? I presume)

Also forgot to mention, it only doesn't havent with on_click scripts/cmd, example: !do_something

I think I know why - the program is spawned as just sh but systemd won't check PATH. Before I fix, can I double check /bin/sh exists on NixOS?

I think I know why - the program is spawned as just sh but systemd won't check PATH. Before I fix, can you confirm /bin/sh exists on NixOS pls?

It does

Why not just eliminate all usage of sh? Also it's even more broken since I last tried, works fine in terminal, but here is the error i get in systemd

not sure if its from this PR, or just from other master changes

Why not just eliminate all usage of sh?

Firstly because it would make this issue worse - systemd units do not read PATH because they have no idea of it, so it would add the requirement for every script to use the absolute path. Also because being able to do bash scripting in your config is neat, and I'd rather not lose that.

Also it's even more broken since I last tried, works fine in terminal, but here is the error i get in systemd

I'm not sure what's causing that, but it sure ain't the PR as that literally changes the sh calls to /bin/sh. I've also not merged any changes to the Wayland code yet. I'd imagine this is likely another issue with running under systemd you'd not encountered up until now (possibly again because it won't read any env vars unless you specifically write them into the unit, or maybe some nix things)

I'm not sure what's causing that, but it sure ain't the PR as that literally changes the sh calls to /bin/sh. I've also not merged any changes to the Wayland code yet. I'd imagine this is likely another issue with running under systemd you'd not encountered up until now (possibly again because it won't read any env vars unless you specifically write them into the unit, or maybe some nix things).

Not sure either but it went away

Have just tested this myself, and it works fine on Arch using the 0.12.0 release build (without the PR).
I used the following systemd user service file:
❯ cat .config/systemd/user/ironbar.service
[Unit]
Description=Ironbar

[Service]
ExecStart=/usr/bin/ironbar
Restart=on-failure
RestartSec=1s

[Install]
WantedBy=default.target

With a very basic script:
let {
  $test = { type = "script" cmd = "echo 'hello world'"}
} in {
  start = [ $test ]
}

I suspect there's something weird going on with Nix.

I suspect there's something weird going on with Nix.

It probably is, mine is super similar

its the one I have autogenerated with the flake

[Install]
WantedBy=hyprland-session.target
WantedBy=sway-session.target

[Service]
ExecStart=/nix/store/z8a1n04wsrmylr683rd75z9w8n6ryy88-ironbar-0.11.0+date=2023-05-01_31a57ae/bin/ironbar
Type=simple

[Unit]
Description=Systemd service for Ironbar
Requires=graphical-session.target

Could you try running the nix version under arch?
sudo pacman -S nix
mkdir -p ~/.config/nix
echo "experimental-features = nix-command flakes" >> ~/.config/nix/nix.conf
nix build

then the built thing from a service?

Just tested using the same config as I used before (not managed through nix) and your systemd user unit, and it works fine

Just tested using the same config as I used before (not managed through nix) and your systemd user unit, and it works fine

Weird, let me get back trace working to see why it wont get output

Before even adding the environment variable, it seems path is just gone, not only that, the mpris widget doesn't work anymore ?!?

I've just tried { type = "script" cmd = "echo $PATH"}   with the Nix build/systemd unit on Arch and that does work as expected with the full path.

Just tested using the same config as I used before (not managed through nix) and your systemd user unit, and it works fine

Figured it out, so typical nix thing, in my scripts I didn't add runtime deps, so yeah ..., I am sorry for wasting your time on this, but actually you did fix it with that PR, since it's referenced directly, and sh isn't in path, maybe? So yeah just merge it. This issue has been weird as it kept having different error messages, once again I am sorry.

No need to apologise, issues like this are commonplace in software dev and sometimes you just need to whittle your way down to find the actual issue, and often it's hidden in plain sight. I'm glad we managed to get there :)


Add feature to manage plugins without using the command line

See: Design and build a means to automatically install, upgrade, and uninstall a plugin with a push of a button

Links in Plugins page changed to buttons for accessability
No client-side javascript maintains previous functionality including instructions of how to install a plugin
Additional client-side javascript asset:

initiate install via ajax and display processing panel
polls status of install until complete or failure
display appropriate confirmation or failure panel
account for user refreshing page after confirmation panel is displayed

Server-side changes to initiate install mode via ajax and return status of install
Make sure the above works for uninstall, and upgrade modes as well as install
Add Acceptance tests to test all the above in Linux, Windows, and MacOS for both node 16 and 18

@BenSurgisonGDS thanks for the demo earlier, this is coming along well. I've been thinking a bit more about the security of this. I'm thinking about the fact that currently this feature lets the user install code onto their machine, and could be used by third parties to install code on to the users' machine. I know that at the moment we have some protection because we only install packages from our list, but that might not stay the case in future, and it's generally better to be prudent now.
I’d like to see this feature to have the following restriction: if someone clicks a link to install a plugin from outside the kit, or types a URL into the browser, they will see an ‘are you sure?’ page to confirm (I think we can keep it so that if they click the existing button from the kit website they don’t see that page).
I think we can do this by a) requiring that a POST is made and b) use some kind of CSRF mitigation to protect the POST request (preferable synchronizer token pattern) [1].

We just met about this, I think the outcome was it was easier to show everyone the page with the button on it, regardless of how they got there. I've added to the Figma:
https://www.figma.com/file/sWnqT4u4uup91TuTe9T3tp/GOV.UK-Prototype-Kit?node-id=325%3A68&t=fRtdngYlr08O97Mu-0

found a tutorial to do csrf by hand, might be helpful:
https://levelup.gitconnected.com/how-to-implement-csrf-tokens-in-express-f867c9e95af0

We've decided to go with the csurf package for our CSRF token middleware. Although it is deprecated [1], we could not find any well-supported alternative or single fork that has been chosen by the community (a large number of projects, including new projects, still rely on csurf).
Because the code is stable and has been for 3 years, and there are no known vulnerabilities that would affect our use case, the risk of using the deprecated library is in my opinion lower than the risk of picking an alternative or a fork that is more liable to change. There is possibly a greater risk of hijacking of the csurf npm package than of a well maintained package, however with the npm shrinkwrap mitigation we have in place (see #1654) that would be unlikely to affect our users.
Note: there is a known vulnerability in the implementation of the double submit cookie pattern implementation in csurf [3], however as we are using cookie: false we are using the synchronization token pattern implementation [4], which is not vulnerable.

@BenSurgisonGDS thanks for the demo earlier, this is coming along well. I've been thinking a bit more about the security of this. I'm thinking about the fact that this feature lets the user install code onto their machine, and could be used by third parties to install code on to the users' machine.
At the moment, a user could click a link on an external website, and the kit would immediately start installing the plugin. The user may not necessarily know or expect that based on the text of the link, and would have no way to cancel the action if they weren't expecting or didn't want the change. I know that at the moment we have some protection because we only install packages from our list, but that might not stay the case in future, and it's generally better to be prudent now.
I’d like to see this feature to have the following restriction: if someone clicks a link to install a plugin from outside the kit, or types a URL into the browser, they will see an ‘are you sure?’ page to confirm (I think we can keep it so that if they click the existing button from the kit website they don’t see that page).
I think we can do this by a) requiring that a POST is made before the JavaScript kicks in and b) use some kind of CSRF mitigation to ensure the POST request is coming from the kit website (preferably the synchronizer token pattern) [1].
We should also have tests that check we can't install a plugin with a link without first getting user authorization.

All the above now implemented.

Seems to be a problem in Safari on Mac, the process does not start automatically but instead shows the install button. Thats not a blocking issue so happy to raise it separately if you prefer.

Seems to be a problem in Safari on Mac, the process does not start automatically but instead shows the install button. Thats not a blocking issue so happy to raise it separately if you prefer.

Yes, please raise separately.


named imports/exports

@Rich-Harris mentioned this already in https://github.com/rollup/rollup-plugin-commonjs/issues/2#issuecomment-150665034 that named imports/exports are really difficult or even completely unfeasible.
The problem is that the current pattern is extremely popular, and established tools (even rollup) work well with it.
So far I use rollup with external to link to npm modules and it works like expected with named imports. Switching to rollup-plugin-commonjs however breaks because it now complains about missing exports (since the converted modules only have a default export and nothing else).

What you mean is that you can no longer write
import { relative } from 'path';

since there doesn't exist any named exports from modules written in CommonJS?

Yes exactly.
That pattern works fine when using the external option to just skip analyzing the import altogether. And it works with other tools such as webpack.

Have been thinking about this. We could add named exports in unambiguous exports.foo = 'bar' cases, alongside the default export:
var x = (function (module) {
  var exports = module.exports;

  /* the original code starts here */
  exports.foo = 'bar';
  module.exports.baz = 'qux'; // we detect both of these

  return module.exports;
});

export default x;
export const foo = x.foo;
export const baz = x.baz;

That would reduce friction a bit. It wouldn't completely remove the inconsistency between internal vs external modules, because with external modules we have the luxury of waiting till runtime – so cases like this would fail...
function augment ( x ) {
  x.foo = 'bar';
  x.baz = 'qux';
}

augment( module.exports );

...but maybe that's okay.

I would argue that the dynamic case (your augment) is not that common. A more common case is probably dynamic imports in commonjs (such as require(a ? './a' : './b') or even a = b.map(require))
So yes, it would be a good start to support the "static" use case and just error when there is some "dynamic" usage involved. Maybe prompting the user to switch to external to avoid such issues.

Just released 1.2.0 which implements this. Let me know if you run into problems with it

Works like a charm! But now the need for caching/incremental compilation is even more evident, since now the rollup bundle time went from ~300ms to ~2s which is unbearable in production.
But very good job so far :-)

the rollup bundle time went from ~300ms to ~2s

woah – is that a like-for-like comparison, or is it just bundling a lot more files now?

It is pulling in the whole of core-js and some other libs now.
Also I’m not using rollup-plugin-babel right now because with that the rebuild time would be more like 6s instead of ~300-600ms as it is now.
also the bundle size went doubled from 300k to 600k+ with all our external dependencies bundled in.

Also I’m not using rollup-plugin-babel right now because with that the rebuild time would be more like 6s

Is that true if you restrict it to your source files? (i.e. exclude: 'node_modules/**')?

That is with only source files (before i tried the npm+commonjs plugins)
So the timing for our project is roughly:

~300-600ms without plugins
~4-6s with only babel (using external to only work on local source files)
~2s with only npm/commonjs (without babel, which is done via gobble in a separate step)


Add Privacy modes and other basic stuff

a) yes, will add check for valid mode
b) will add additional patch on top with ble_ll_is_controller_busy(), I think that should work. please review.
c) yes, I forgot to do the check if resolving list is enabled. Actually it does not break anything because if it is not enabled we will basically not find the IRK on the list. However I will add it


[incubator/zookeeper] Allow quorumListenOnAllIPs to be configurable

Is your feature request related to a problem? Please describe.
When quorumListenOnAllIPs is set to "false" (the default value), Zookeeper binds to only the pod IP address and not <IP_ADDRESS>.  This causes problems when using Istio as the proxy sidecar wants to forward packets to the localhost address which is not listening on port 3888, resulting in "connection refused" errors.  The end result is that the Zookeeper nodes are unable to elect a leader and the ensemble never starts.
When quorumListenOnAllIPs is set to "true", Zookeeper binds to <IP_ADDRESS> and the Istio proxy sidecar is able to forward packets to Zookeeper.
Describe the solution you'd like
I would like a new environment value (env.ZK_QUORUM_LISTEN_ALL_IP) added to the values file that would allow quorumListenOnAllIPs to be enabled.  The config-script.yaml file would append the directive to the zoo.cfg file.  The default value would be "false" to maintain compatibility with previous behavior.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
I've already made and tested this change locally

/remove-lifecycle stale

/remove-lifecycle stale

/remove-lifecycle stale


New busDayCalendars and tests

What is this PR for?
New busDayCalendars and tests:

Added London and NYC busDay calendars
Added units tests for calendars

What type of PR is it?
Improvement

This code does not work in (at least in all situations). You are checking if a date is a holiday via String comparison using LocalDate.toString(). The standard output LocalDate.toString() is ISO-8601 (2016-08-16) - see LocalDate. The list of dates used dd/MM/yyyy (16/08/2016). In addition: I would not rely on toString() having a specific format. It may depend on the users Locale settings.
I changed that buy using a static initialised to a Set of LocalDates and using compare on them.
In addition the code has come performance issues. Scanning ArrayList is O(n). It is better to use a TreeSet here which is O(log(n)).

Error was introduced when converting my java6 code to Java8. It used to be org.joda.time.LocalDate.toString("dd/MM/yyyy"). Thanks for Fixing that

Ah. I was wondering why the format could be different on your system.... now I see.


fixed typo

looks like a neat project, just reading through the code before i play with it.  grammar stood out. NBD

Thanks a lot


Add VIP: Title of the New VIP

Create a new proposal

File vips/title_of_new_vip.md
Requires 1 more reviewers from @editor1, @editor2, @editor3, @editor4, @editor5


get_model throws error when used before task initilization

When running evals with a scorer (e.g. the built in model_graded_fact scorer) with unspecified model I keep getting ValueError: No model specified (and no INSPECT_EVAL_MODEL defined). It would be nice if there was either a param you could pass to the cli to specify the eval model or to default to the provided model (perhaps with a warning).
Currently the call to get_model tries to get the active_model, but the active model is only set later in task_run.
Command to reproduce error:
inspect eval examples/theory_of_mind.py --model hf/meta-llama/Meta-Llama-3-8B-Instruct -M device='cuda:0'

Thanks for reporting this! Should be resolved here: https://github.com/UKGovernmentBEIS/inspect_ai/pull/99


create npm build script

resolves #23

Can one of the admins verify this patch?

ok to test

ok to test

ok to test

ok to test


:bug: OP Rewards Estimates Not Updating

Subject of the issue
Kwenta displays OP rewards in real-time as they are accrued, this seems broken currently.
Your environment
Chrome/Brave
Expected behavior
Update Rewards after a trade is placed
Actual behaviour
Static number of 0
Steps to reproduce
Trade on Kwenta

#2641


API Server panic on GET /apis/custom.metrics/k8s.io/v1beta1

Hi, this is essentially the same issue as in #331 but that issue was closed because it should have been fixed by #335. However, that doesn't appear to be the case and I am still seeing this issue in v0.8.2:
I1222 23:51:37.570679       1 httplog.go:89] "HTTP" verb="GET" URI="/apis/custom.metrics.k8s.io/v1beta1" latency="17.696874ms" userAgent="Go-http-client/2.0" srcIP="<IP_ADDRESS>:45980" resp=200
E1222 23:51:37.570761       1 runtime.go:76] Observed a panic: runtime error: invalid memory address or nil pointer dereference
goroutine 487 [running]:
github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/server/filters.(*timeoutHandler).ServeHTTP.func1.1(0xc004476660)
        /go/src/github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/server/filters/timeout.go:108 +0x107
panic(0x1aa1b60, 0x3189600)
        /usr/local/go/src/runtime/panic.go:679 +0x1b2
compress/gzip.(*Writer).Write(0xc000271810, 0xc004687200, 0x71, 0x71, 0x30, 0x1b79a80, 0xc005360c01)
        /usr/local/go/src/compress/gzip/gzip.go:168 +0x237
github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/endpoints/handlers/responsewriters.(*deferredResponseWriter).Write(0xc0005ac640, 0xc004687200, 0x71, 0x71, 0xc004687200, 0x71, 0x71)
        /go/src/github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/endpoints/handlers/responsewriters/writers.go:182 +0x54e
github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/endpoints/handlers/responsewriters.SerializeObject(0x1d35b02, 0x10, 0x7fdf29c8af98, 0xc00030fc20, 0x7fdf2a1f8198, 0xc00000e0e8, 0xc000574d00, 0xc8, 0x2096640, 0xc0005ac5f0)
        /go/src/github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/endpoints/handlers/responsewriters/writers.go:117 +0x389
github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/endpoints/handlers/responsewriters.WriteObjectNegotiated(0x20b9640, 0xc0003f6300, 0x20b98c0, 0x31ce040, 0x0, 0x0, 0x0, 0x0, 0x7fdf2a1f8198, 0xc00000e0e8, ...)
        /go/src/github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/endpoints/handlers/responsewriters/writers.go:251 +0x555
github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/endpoints/discovery.(*APIVersionHandler).ServeHTTP(0xc000706440, 0x7fdf2a1f8198, 0xc00000e0e8, 0xc000574d00)
        /go/src/github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/endpoints/discovery/version.go:81 +0x17b
github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/endpoints/discovery.(*APIVersionHandler).handle(...)
        /go/src/github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/endpoints/discovery/version.go:77
github.com/directxman12/k8s-prometheus-adapter/vendor/github.com/emicklei/go-restful.(*Container).dispatch(0xc00064a480, 0x7fdf2a1f8198, 0xc00000e0e8, 0xc000574d00)
        /go/src/github.com/directxman12/k8s-prometheus-adapter/vendor/github.com/emicklei/go-restful/container.go:294 +0x635
github.com/directxman12/k8s-prometheus-adapter/vendor/github.com/emicklei/go-restful.(*Container).Dispatch(...)
        /go/src/github.com/directxman12/k8s-prometheus-adapter/vendor/github.com/emicklei/go-restful/container.go:204
github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/server.director.ServeHTTP(0x1d44a83, 0x1a, 0xc00064a480, 0xc00036e930, 0x7fdf2a1f8198, 0xc00000e0e8, 0xc000574d00)
        /go/src/github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/server/handler.go:146 +0x531
github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/endpoints/filters.WithAuthorization.func1(0x7fdf2a1f8198, 0xc00000e0e8, 0xc000574d00)
        /go/src/github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/endpoints/filters/authorization.go:64 +0x548
net/http.HandlerFunc.ServeHTTP(0xc0000a1840, 0x7fdf2a1f8198, 0xc00000e0e8, 0xc000574d00)
        /usr/local/go/src/net/http/server.go:2036 +0x44
github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/server/filters.WithMaxInFlightLimit.func2(0x7fdf2a1f8198, 0xc00000e0e8, 0xc000574d00)
        /go/src/github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/server/filters/maxinflight.go:175 +0x4f6
net/http.HandlerFunc.ServeHTTP(0xc001a0f3b0, 0x7fdf2a1f8198, 0xc00000e0e8, 0xc000574d00)
        /usr/local/go/src/net/http/server.go:2036 +0x44
github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/endpoints/filters.WithImpersonation.func1(0x7fdf2a1f8198, 0xc00000e0e8, 0xc000574d00)
        /go/src/github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/endpoints/filters/impersonation.go:50 +0x22f3
net/http.HandlerFunc.ServeHTTP(0xc0000a1880, 0x7fdf2a1f8198, 0xc00000e0e8, 0xc000574d00)
        /usr/local/go/src/net/http/server.go:2036 +0x44
github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/endpoints/filters.WithAuthentication.func1(0x7fdf2a1f8198, 0xc00000e0e8, 0xc000574c00)
        /go/src/github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/endpoints/filters/authentication.go:70 +0x5ea
net/http.HandlerFunc.ServeHTTP(0xc0000ebe00, 0x7fdf2a1f8198, 0xc00000e0e8, 0xc000574c00)
        /usr/local/go/src/net/http/server.go:2036 +0x44
github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/server/filters.(*timeoutHandler).ServeHTTP.func1(0xc004476660, 0xc001a135c0, 0x20c3800, 0xc00000e0e8, 0xc000574c00)
        /go/src/github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/server/filters/timeout.go:113 +0xd0
created by github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/server/filters.(*timeoutHandler).ServeHTTP
        /go/src/github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/server/filters/timeout.go:99 +0x1cb

Hi @fractos, this sounds like #292, which should be fixed by #352. However, this PR is not part of any release yet, we might need to cut v0.8.3 to include this bugfix.

Hi @fractos, this sounds like #292, which should be fixed by #352. However, this PR is not part of any release yet, we might need to cut v0.8.3 to include this bugfix.

Hi @dgrisonnet, I've tried this with v0.8.3 on k8s 1.15 and the panic is no longer there. However, it is still not responding in a way that is letting the custom metrics work.
i.e.
kubectl get --raw /apis/custom.metrics.k8s.io/v1beta1

and shows this in the prometheus-adapter logs:

Hmm, it seems that prometheus-adapter isn't attached to the custom metrics aggregated API. On top of my head, the reasons could be either that the apiservice doesn't exist or that you didn't defined any custom metrics rules in your prometheus-adapter configuration.
Could you perhaps share the output of the following command: kubectl get apiservices

@dgrisonnet Ok I'll take you through it.
We are installing Helm v3 and setting it up with the prometheus-community repos like so:
curl https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3 > get_helm.sh
chmod 700 get_helm.sh
./get_helm.sh

helm repo add stable https://charts.helm.sh/stable
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update

helm upgrade --install --namespace monitoring --create-namespace prometheus stable/prometheus-operator -f ./application/prometheus/prometheus-config.yaml

kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/download/v0.3.7/components.yaml

where application/prometheus/prometheus-config.yaml is:
grafana:
  enabled: false
alertmanager:
  enabled: false

Then adding the prometheus-adapter with:
helm upgrade --install prometheus-adapter prometheus-community/prometheus-adapter --namespace staging -f ./application/prometheus/prometheus-adapter.yaml

where application/prometheus/prometheus-adapater.yaml is:
prometheus:
  url: http://prometheus-prometheus-oper-prometheus.monitoring
rules:
  default: false
  custom:
  - metricsQuery: sum(sum_over_time(vuln_find_running{service="phoenix", service_status="ready"}[1m]))
      by (<<.GroupBy>>)
    name:
      as: phoenix_vuln_find_running
    resources:
      template: <<.Resource>>
    seriesQuery: vuln_find_running{service="phoenix"}

(plus some similar other rules)
Output from kubectl get apiservices is:
NAME                                   SERVICE                      AVAILABLE   AGE
v1.                                     Thanks for any thoughts on this!

Thank you providing all these information :). As I suspected it seems that the custom-metrics apiservices aren't created. For custom-metrics to be served properly, you should be able to see the following via kubectl get apiservices:
v1beta1.custom.metrics.k8s.io          monitoring/prometheus-adapter   True        94s
v1beta2.custom.metrics.k8s.io          monitoring/prometheus-adapter   True        94s

I don't know where the helm community is hosting the manifests, but you can always use the one we provide here:
https://github.com/kubernetes-sigs/prometheus-adapter/blob/master/deploy/manifests/custom-metrics-apiservice.yaml#L1-L27

@dgrisonnet Thanks very much, I'll try that.

@dgrisonnet Hi, thanks for the help earlier. I've managed to get the custom metrics apiservices available now (using metrics-server 0.3.7) and I can see from the prometheus-adapter logs that it is accepting queries about the custom metrics, but it is responding with a 404. I know it's off-topic, but here's what I'm installing; any ideas why it can't find the custom metrics?
helm upgrade --install prometheus-adapter prometheus-community/prometheus-adapter --namespace myapplication -f ./prometheus-adapter.yaml

Where ./prometheus-adapter.yaml is:
affinity: {}
image:
  repository: directxman12/k8s-prometheus-adapter-amd64
  tag: v0.8.3
  pullPolicy: IfNotPresent
logLevel: 4
metricsRelistInterval: 1m
listenPort: 6443
nodeSelector: {}
priorityClassName: ""
prometheus:
  url: http://prometheus-prometheus-oper-prometheus.monitoring
  port: 9090
  path: ""
replicas: 1
rbac:
  create: true
psp:
  create: false
serviceAccount:
  create: true
  name:
dnsConfig: {}
resources: {}
rules:
  default: false
  custom:
  - metricsQuery: sum(sum_over_time(vuln_find_running{service="phoenix", service_status="ready"}[1m]))
      by (<<.GroupBy>>)
    name:
      as: phoenix_vuln_find_running
    resources:
      template: <<.Resource>>
    seriesQuery: vuln_find_running{service="phoenix"}
  - metricsQuery: sum(sum_over_time(vuln_find_waiting{service="phoenix", service_status="ready"}[1m]))
      by (<<.GroupBy>>)
    name:
      as: phoenix_vuln_find_waiting
    resources:
      template: <<.Resource>>
    seriesQuery: vuln_find_waiting{service="phoenix"}
  - metricsQuery: sum(sum_over_time(vuln_find_running{service="phoenix-worker", service_status="ready"}[1m]))
      by (<<.GroupBy>>)
    name:
      as: phoenix_worker_vuln_find_running
    resources:
      template: <<.Resource>>
    seriesQuery: vuln_find_running{service="phoenix-worker"}
  - metricsQuery: sum(sum_over_time(vuln_find_waiting{service="phoenix-worker", service_status="ready"}[1m]))
      by (<<.GroupBy>>)
    name:
      as: phoenix_worker_vuln_find_waiting
    resources:
      template: <<.Resource>>
    seriesQuery: vuln_find_waiting{service="phoenix-worker"}
  existing:
  external: []
  resource: {}
service:
  annotations: {}
  port: 443
  type: ClusterIP
tls:
  enable: false
  ca: |-
    # Public CA file that signed the APIService
  key: |-
    # Private key of the APIService
  certificate: |-
    # Public key of the APIService
extraArguments: []
extraVolumes: []
extraVolumeMounts: []
tolerations: []
podLabels: {}
podAnnotations: {}
hostNetwork:
  enabled: true
dnsPolicy: ClusterFirstWithHostNet
podDisruptionBudget:
  enabled: false
  minAvailable:
  maxUnavailable: 1
certManager:
  enabled: false
  caCertDuration: 43800h
  certDuration: 8760h

I am also seeing this issue:

ApiServices:


Langchain Invalid Token Error

Getting a invalid token error when trying to run autogpt

@IntelligenzaArtificiale Yes I set up the .env file and prior to the latest pull it was all working just fine.

@IntelligenzaArtificiale Ok I figured out what I was doing wrong, I was getting the Open Ai Token from an old method using the link https://chat.openai.com/api/auth/session which I was grabbing from Firefox, not sure if that had anything to do with it. But I did the other method thru Chrome and was able to grab the proper token.


Added secure and samesite params to cookies

I also added some minimal build tooling (that you can leave out of course)

In line 114 of jquery.ihavecookies.js the variables settings.secure and settings.samesite are not used for cookieControlPrefs.


Leading Icon all inputs

Leading icon for all inputs (Input, InputNumber, Currency, Datepicker, etc) not showing on mobile.

Fixed with 0.0.26-next


MicroWebSrv.Start never terminates when threaded=False and Stop() is called

When Stop is called from a route handler in un-threaded mode, Start() never terminates. An important thing to note is that I'm running the module on ESP8266 (precompiled using the cross compiler) and removed the import of _thread and replaced the content of _startThread() with return True, since it isn't needed and was using _thread.
I fixed this in my own copy by changing the following line of _serverProcess() from
while True: to while self._started:
and adding self._started = False to Stop().

👍🏻 @serious-scribbler


inspect message test still passed even though dfx was not running

While I was updating the test.yml I found a weird thing, I'm hoping its not indicative of a larger underlying problem, but I wanted to document it just in case so we can look into it.
My conditions got messed up and as a result I had a run where dfx never started for any of the tests. Most of the tests failed (I think on pretest) but inspect message somehow powered through and was able to pass even though it didn't run any of it's tests and dfx wasn't even running.
This issue would be easy to replicate by simply making the conditions under which dfx starts to be such that it never starts.
I would expect everything to fail if none of the tests run. We may want to add some sort of check to make sure that happens. At the very least we should understand why inspect message passed even thought it ran into an error.

I double checked everything, I looked over the tests to make sure that everything looked right, I ran the tests locally without dfx, I ran the tests on CICD without dfx, and all of them failed in the same way we would expect; in pretest it fails to deploy and that ends the whole process. To be sure I by passed pretest and the tests fail just fine, and everything looks like it should.
In short nothing looks wrong and I can't reproduce the error.
My guess is that it was indicative of a deeper problem, but that deeper problem was a very messed up workflow yml that is now in ship shape so we can't reproduce it. This makes the most sense to me, since we were doing a lot of changes in the yml at the time we saw this error and we hadn't yet finished modifying the workflows.


Bug: when spatial_editor-camera-origin rests on worlds y-axis

Operating system or device - Godot version:
2.1
Issue description:
When the origin of the "spatial_editor-camera" is on the y axis of 3dWorld,
SpatialEditorViewport::update_transform_gizmo_view() is not workin properly.
this should fix it:
void SpatialEditorViewport::update_transform_gizmo_view() {
        ......
	float d0 = camera->unproject_position(camera_xform.origin+camz*gizmo_d).y;
	float d1 = camera->unproject_position(camera_xform.origin+camz*gizmo_d+camy).y;
	float dd = Math::abs(d0-d1);

//	if (dd==0)
//////////////////////////////////////////////////
	if (dd==0 || Math::is_nan(dd) )
//////////////////////////////////////////////////
		dd=0.0001;

	float gsize = EditorSettings::get_singleton()->get("3d_editor/manipulator_gizmo_size");
	gizmo_scale=(gsize/Math::abs(dd));
	Vector3 scale = Vector3(1,1,1) * gizmo_scale;
        ......
}

on a 2nd thought, maybe its better to fix unproject_position
to not return nan values.

I couldn't reproduce this by simply creating a camera and moving it to (0,1,0). Does it have to the pointing toward a particular direction?
Or maybe can you post a minimal example project?


How to contribute

Save the solution in print_hello_world.go file
Add print_hello_world.go file in print-hello-world folder

!assign


fixed：destory action hold onto offers

Do not hold onto offers: If an offer cannot be used, decline it immediately. Otherwise the resources cannot be offered to other schedulers and the scheduler itself will receive fewer additional offers
https://mesos.apache.org/documentation/latest/app-framework-development-guide/#multi-scheduler-scalability

after destory action, scheduler can't create new cluster because of holding onto offers.

Codecov Report

Merging #100 into master will decrease coverage by 0.02%.
The diff coverage is 0%.

@@            Coverage Diff             @@
##           master     #100      +/-   ##
==========================================
- Coverage   50.08%   50.05%   -0.03%
==========================================
  Files          62       62
  Lines        6168     6171       +3
==========================================
  Hits         3089     3089
- Misses       2733     2736       +3
  Partials      346      346

Impacted Files
Coverage Δ

platform/mesos/scheduler.go
0% <0%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c540009...cd6cf1c. Read the comment docs.


Bugged images after binding a Sprite in Image Component using OneWayPropertyBinding

When using One Way Property Binding to bind a public bindable Sprite property from my ViewModel it seems Weld is overriding something more than just the sprite itself, as I it stops resizing to RectTransform properly as it should be and also the image is stretched despite being set to constrain original proportions.
I'm using Unity Beta 2019.1.0b4, which also can generate problems, but all the images without binding act properly.

The funny thing is that binding an overrideSprite property works well. It's probably the best way to bypass the problem for now

The problem probably derives from a Collection Binding to a template using a child View-Model, as it works well with the original sprite in the gameobject. As soon as we do a one-way property binding of the sprite in the Image component, the things get buggy. You cannot event set the color of the sprite anymore

I'm nevertheless caught due to the fact the drop down listing, as you point out to your initial comment, does not bind to an ObservableList to make the program for welding works.


Signatures for Commits & the Author model

In Atomic Commits, every change is signed by some author. This makes Commits truly atomic, and means that they can be shared as fully verifiable pieces of information, similar to W3C Verifiable Credentials (although Atomic Commits are specifically made to describe changes instead of current state).
However, this requires some implementation. Here's what I'm thinking:

The Author creates a keypair. The public key is stored publicly at the Author's URL, the  private key stores somewhere safe
The Commit should be serialized to some format (I'm thinking JSON-LD). This process should be deterministic - serializing some commit should always result in the exact same string, so perhaps the keys should be sorted alphabetically. Note that the Commit can only be serialized as some partial struct, since the signature is of course missing.
The serialized Commit should be signed by the private key. This signature is included in the Commit that's sent to the server.
The server receives the Commit and checks if the Author has the correct rights.
If it does, it gets the Author resource and its public key.
It generates the same serialized Commit representation from step 2, and it checks the signature using the public key.

I've implemented a version of this process in atomic. Perhaps the most important step of this process is the serialization of the commit, since this process needs to be deterministic. This is what I'm doing now:
 /// Generates a deterministic serialized JSON representation of the Commit.
    /// Does not contain the signature, since this function is used to check if the signature is correct.
    pub fn serialize_deterministically(&self) -> AtomicResult<String> {
        let mut obj = serde_json::Map::new();
        obj.insert(
            "subject".into(),
            serde_json::Value::String(self.subject.clone()),
        );
        obj.insert(
            "createdAt".into(),
            serde_json::Value::Number(self.created_at.into()),
        );
        obj.insert(
            "signer".into(),
            serde_json::Value::String(self.signer.clone()),
        );
        if let Some(set) = self.set.clone() {
            if !set.is_empty() {
                let mut collect: Vec<(String, String)> = set.into_iter().collect();
                // All keys should be ordered alphabetically
                collect.sort();
                // Make sure that the serializer does not mess up the order!
                let mut set_map = serde_json::Map::new();
                for (k, v) in collect.iter() {
                    set_map.insert(k.into(), serde_json::Value::String(v.into()));
                }
                obj.insert("set".into(), serde_json::Value::Object(set_map));
            }
        }
        if let Some(mut remove) = self.remove.clone() {
            if !remove.is_empty() {
                // These, too, should be sorted alphabetically
                remove.sort();
                obj.insert("remove".into(), remove.into());
            }
        }
        if let Some(destroy) = self.destroy {
            // Only include this key if it is true
            if destroy {
                obj.insert("destroy".into(), serde_json::Value::Bool(true));
            }
        }
        let string = serde_json::to_string(&obj)?;
        Ok(string)
    }

Couple of thoughts / dilemmas:

Should the deterministically serialized representation be a valid JSON-LD resource? If that is the case, we'll need to include @context in all responses. Seems a bit much.
Should it use URLS as keys, or shortnames? For now I've gone with shortnames, since that is easier to read, less prone to typos and more compact. However, this could also be confusing, as all resources in Atomic Data actually use URLs for keys. This perhaps is more of a fundamental question for how JSON + Atomic Data interoperability will work.
How to deal with extensions to Commits? Let's assume some developer wants to introduce a new method, e.g. line-based changes to some specific atom. This would probably be a new property on a Commit, which is perfectly fine in

I've updated the docs about this discussion. Long story short, I've opted for deterministic JSON-AD serialization and made two implementations (one in rust and one in typescript)


[Feature request] Python Support

Please consider adding support for Python
Syntax example:
import re

regex1 = r"^foo"

re.search('bar foo', regex1)
<re.Match object; span=(0, 7), match='bar foo'>

Thank you

Tracking as https://github.com/chrmarti/vscode-regex/issues/35.


Merge from main

So it adds await which is pretty cool

wait wait wait there are problems

wait wait wait there are problems


cannot add text in image

I think use other TextFont

Please edit the issue's description to include the code, I cannot read all of it as an image, and it's important for me to be able to read it fully.

I will close this issue soon if no more information is given, I cannot help without being able to see the entire code and without having reproducible code.


[Feature] Display the model-level constraints in the documentation generated by dbt

Is this your first time submitting a feature request?

[X] I have read the expectations for open source contributors
[X] I have searched the existing issues, and I could not find an existing issue for this feature
[X] I am requesting a straightforward extension of existing dbt functionality, rather than a Big Idea better suited to a discussion

Describe the feature
When I run dbt docs generate, it fails to include constraints defined at the model level in the documentation. For instance, if I establish a primary key at the model level, this information isn't reflected in the documentation. Instead, it only appears when I define the primary key at the column constraint level. However, because I'm utilizing dbt-bigquery, I'm restricted to declaring only one primary key field at the column level. This necessitates the declaration of primary keys at the model level, thereby creating a discrepancy in the documentation.
Describe alternatives you've considered
To enhance the documentation process in dbt for dbt-bigquery, consider implementing the ability to define multiple primary key constraints at the column level while also ensuring that primary key constraints established at the model level are accurately reported in the documentation. This improvement would streamline the documentation process and provide comprehensive information about the primary keys used in the data models.
Who will this benefit?
No response
Are you interested in contributing this feature?
No response
Anything else?
No response

For the initial request:
As of now, when specified at the column level, it displays accordingly. However, if declared at the model level constraint, it doesn't appear at either level. I wouldn't oppose consistently displaying it at the column level. What's essential to me is ensuring visibility of the primary key, especially when handling multiple primary key fields in BigQuery, which currently doesn't function when declared in the column level constraint.

Currently, the docs have five main sections:

Details
Description
Columns (which contains column-level constraints)
Depends on
Code

And none of those sections includes model-level constraints like these:

Does that sound right? If so, would adding a new section for model-level constraints solve this for you?

Here's the files + commands I'm using to see how this is behaving currently:

Reprex

models/dual.sql
{{ config(materialized="ephemeral") }}

select 'X' as dummy

models/my_other_model.sql
select
    3 as other_pk_1,
    4 as other_pk_2

from {{ ref("dual") }}

models/my_model.sql
select
    1 as pk_1,
    2 as pk_2,
    3 as fk_1,
    4 as fk_2,
    5 as check_1,
    6 as check_2

from {{ ref("dual") }}

models/_models.yaml
models:
  - name: my_other_model
    config:
      materialized: table
      contract:
        enforced: true

    # model-level
    constraints:
      - type: primary_key
        columns: [other_pk_1, other_pk_2]
      - type: unique
        columns: [other_pk_1, other_pk_2]

    # column-level
    columns:
      - name: other_pk_1
        data_type: int
      - name: other_pk_2
        data_type: int

  - name: my_model

    config:
      materialized: table
      contract:
        enforced: true

    # model-level
    constraints:
      - type: primary_key
        columns: [pk_1, pk_2]
      - type: unique
        columns: [pk_1, pk_2]
      - type: foreign_key
        columns: [fk_1, fk_2]
        expression: "YOUR_SCHEMA_HERE.my_other_model (other_pk_1, other_pk_2)"
      - type: check
        columns: [check_1, check_2]
        expression: "check_1 != check_2"
        name: human_friendly_name

    # column-level
    columns:
      - name: pk_1
        data_type: int
      - name: pk_2
        data_type: int
      - name: fk_1
        data_type: int
      - name: fk_2
        data_type: int
      - name: check_1
        data_type: int
      - name: check_2
        data_type: int

dbt build --full-refresh
dbt docs generate && dbt docs serve

sorry for the late answer, i Think yes, that would be perfect, thank you.

Do you believe this is a challenge we can resolve in the coming days? This matter pertains to a proof of concept (POC) that, if successful, will guide several teams within our company toward adopting dbt.

Summary
We'd be open to adding model-level constraints to the generated documentation website.
I've labeled this as refinement for us to determine how exactly the user experience would look for this and to determine acceptance criteria.
Timeline
This isn't a time-sensitive priority for us. So no, this is not something that would be available on the timeline you mentioned.
Workaround
However, we do have other capabilities that would allow you to add your own free-form content to the documentation website.

Adding model descriptions to your project
Using docs blocks

The description field can contain Markdown which provides some flexibility for how you want to format things.
Example
models:
  - name: my_other_model
    config:
      materialized: table
      contract:
        enforced: true

    description: >
      ### Model-level constraints

      - primary_key: [`other_pk_1`, `other_pk_2`]

      - unique: [`other_pk_1`, `other_pk_2`]

    # model-level
    constraints:
      - type: primary_key
        columns: [other_pk_1, other_pk_2]
      - type: unique
        columns: [other_pk_1, other_pk_2]

dbt docs generate && dbt docs serve

Screenshot

Workaround
However, we do have other capabilities that would allow you to add your own free-form content to the documentation website.
* [Adding model descriptions to your project](https://docs.getdbt.com/docs/collaborate/documentation#adding-descriptions-to-your-project)

* [Using `docs` blocks](https://docs.getdbt.com/docs/collaborate/documentation#using-docs-blocks)

The description field can contain Markdown which provides some flexibility for how you want to format things.
Example
models:
  - name: my_other_model
    config:
      materialized: table
      contract:
        enforced: true

    description: >
      ### Model-level constraints

      - primary_key: [`other_pk_1`, `other_pk_2`]

      - unique: [`other_pk_1`, `other_pk_2`]

    # model-level
    constraints:
      - type: primary_key
        columns: [other_pk_1, other_pk_2]
      - type: unique
        columns: [other_pk_1, other_pk_2]

dbt docs generate && dbt docs serve

Adding primary key and foreign key in dbt docs is needed by a lot of users.
Duplicate this information in model description is a workaround, but it increases risk of incoherence between contract model and documentation.


Cannot use Client::connect in a background task

Hi,
I cannot seem to use Client::connect in a background tokio task:
#[tokio::main]
async fn main() {
  tokio::spawn(async move {
    let mut client = Client::connect(Duration::from_secs(1)).await;
    // do some iRacing stuff
  });

  // do something else
}

The error is: future is not Send as this value is used across an await, caused by `src\iracing\client.rs:267``:

Since the Windows API doesn't give us Send safety for shared memory handles, I'm not forcing the trait either.
As a result the iRacing client has to live on a single thread (unless someone wants to do the work of guaranteeing that handles can be sendable)

Well that's annoying.
I won't have a chance in the next few days to investigate, but once I do I'll try to get things to implement Send/Sync where possible.

I've added a compiles_if_send_is_valid test to prevent a regression like this from happening in the future.
I've released version 0.2.1 that has this issue fixed.
For future reference, it's the usual footgun of the scope of an unsendable value going through an await, even though its actual lifetime is shorter.

Once you verify that the new version works, please close the issue.

Works like a charm now. Thanks a bunch


chore(deps): upgrade configuration

Upgrades project dependencies. See details in workflow run.

Automatically created by projen via the "upgrade-configuration-k8s-25-main" workflow

⚪ Backport skipped
The pull request was not backported as there were no branches to backport to. If this is a mistake, please apply the desired version labels or run the backport tool manually.
Manual backport
To create the backport manually run:
backport --pr 3258

Questions ?
Please refer to the Backport tool documentation


Doesn't support abstract UNIX sockets

To use a socket in the abstract namespace, one bind()s a socket to a path beginning with a NUL byte. Unfortunately, there's an explicit test that fails on a NUL encountered in a path. Additionally, functions like strcpy() are called in parts of the library, and won't work if there's a NUL that's not the last byte in the string.

I discovered one other related issue prohibiting this kind of use.
When bind() is called, a length is passed. This length specifies the size of the sockaddr struct passed. In the case where sun_path is a filesystem path, it's fine to pass sizeof(sockaddr_un) since the path is terminated with a NUL byte. In the abstract namespace case, the length passed must be the size of actual path intended (plus the two bytes for sun_family) — NULs have no special meaning and are ignored.

Hmmm... thanks for the report. I am going to have to read up on abstract UDS.

They're (conceptually) pretty simple. If you provide a path that's prefixed with a "\0", the address provided won't refer to a location on the filesystem. Additionally, the path will not be treated as a NUL-terminated string. This last bit is the only semi tricky bit to implement in nix — you will need to remember somewhere the length of the path given, since something like \0foo\0\0 with length 4 could be a perfectly valid abstract path which would be distinct from \0foo\0\0\0 with length 5.

I think the way to fix this would be to take an Option path to represent the abstract socket. I would take a PR for this. It would be a breaking change so the nix version would need to be bumped.

Happy to provide. I'm getting back up to speed w/ Rust after several months away, so apologies if I'm a bit slow. I assume you mean an Option<NixPath> to UnixAddr::new()? I'm not sure how this distinguishes between an abstract path and a filesystem path.

Ok, I took a look.
SocketAddr is an Inet or Unix addr. UnixAddr contains a libc::sockaddr_un, which would have a null path to indicate an abstract socket.
I would think that the best way to handle this would be to add UnixAddr::abstract() that returns an abstract unix address. However, existing addr related code would need to be audited to ensure that the abstract socket case is handled internally.

A NULL sun_path doesn't indicate an abstract socket. A sun_path starting with a NULL byte does. Unfortunately, this means we also have to store the length of the path provided, as abstract paths are not NULL-terminated strings (whereas filesystem paths are).

Let me know if the changes in #198 are sufficient to fix this in your opinion. I've added a new field to UnixAddr, removed the calls to strcpy and strlen, and changed the function returning &Path to instead return Option<&Path>, which bypasses the embedded-NUL check on non-filesystem socket addresses. And added a constructor new_abstract, for convenience.
Maybe .sun_path() should be public?

I'll take a look, thanks!

It looks sufficient, but I won't have a chance to play around with it for a bit.


Merging Hax and Charon

This is a meta-issue tracking the roadmap to merge parts of Charon into the Hax frontend (written with @W95Psp and @sonmarcho). PRs related to this merge should reference this issue.
Current Workflow
The hax frontend extracts information from rustc's MIR and exposes an hax AST, which is not sufficient for Charon to build ULLBC, requiring additional rustc queries in Charon. The hax AST is intended as an unopinionated version of MIR; ULLBC is a clean-up, simplified version that preserves the general structure, but makes opinionated choices about some representations (e.g., a unique representation for ADTs). Finally, LLBC is a transformation from ULLBC that performs control-flow reconstruction, leading to a "structured" MIR.
Goal
We aim for a clear separation between the different phases (rustc MIR, hax AST, ULLBC, LLBC). To do this, we propose a refactor of hax and charon so that Charon does not require calls into rustc, instead purely consuming the hax AST. The advantages are the following:

This will simplify maintenance of both projects wrt rustc: only hax will have to track changes to rustc (while Charon will track changes to hax)
This will reduce the number of needed entrypoints into hax, simplifying testing hax code paths: currently, Charon requires calls into hax on subexpressions and sub-ASTs, thus requiring fine-grained hooks into hax; as a consequence, testing these hooks in isolation is currently tricky.

Proposed Roadmap
Achieving this goal requires the following steps:

Add more information to the hax AST to provide all required constructs to Charon to remove the need for "localized" rustc queries on items (e.g., the discriminant of an enum variant).
Remaining rustc queries in Charon will relate to traversing crates. We will refactor Charon to move these to a first stage of Charon, which will be followed by the hax AST to ULLBC translation.
Once separated from the translation, these queries will then be moved to hax, removing rustc queries from Charon. hax currently relies on similar queries for crate traversal, but with different options (e.g., ignoring external declarations). Supporting hax- and charon-compatible versions of queries will require the parametrization by "options", which will facilitate the development of future extensions and use cases that might arise.
ULLBC and LLBC (and their translations from the hax AST) will remain in Charon at the end of this work.

Before this, we intend to upgrade hax and Charon to the latest rustc (https://github.com/hacspec/hax/issues/224, https://github.com/AeneasVerif/charon/issues/182)

There is also the https://github.com/formal-land/coq-of-rust project in this space that could be useful to merge, too, as suggested by @spitters . It uses THIR as input, so this seems to be a major difference from Hax and Charon, which use MIR, as I understand. Also, we do not follow information from the borrow checker and put everything in a generic monad. While doing the proofs, we basically make and prove the transformation that Aeneas is doing by hand in Coq. So, eventually, coq-of-rust should be completely replaced by Aeneas, at least for the safe parts of Rust.
Maybe coq-of-rust is too different to share anything for now.

Hi @clarus!
Actually Hax also uses THIR, so it might indeed be interesting to chat :)
The "frontend" of Hax is the part that links into Rustc. It mirrors Rust internal ASTs to hax variants of those ASTs in which all the information you might want is there already.
For instance, on a method call, you get an expression that constructs the precise impl you call onto. Another example: instead of opaque IDs for sub-expression, the mirrored THIR/MIR contains directly sub-expressions. The aim is that when you get the hax AST, you can just work with it with no further interaction with rustc.
The frontend of hax supports both THIR and MIR (this issue is partially about improving the MIR output so that Charon can rely on it and stop interacting with Rustc)


How to update Intel Graphics Driver to the latest version?

My laptop is currently running the version <IP_ADDRESS>18 from 09-Jan-19.
I would like to upgrade my Intel Graphics Driver to the latest version or to the modern DCH version, because Chrome sometimes flickers on this version while playing videos.
Tried installing the drivers downloaded from the Intel website, doesn't work. It says OEM should certify the drivers for the device model.
Are you guys running the same version?

@ni554n , Hi!
I'm using the last intel UHD DCH drivers. To achieve that (sorry some mistake):

uninstall the OEM drivers with DDU;
install the latest DCH intel 620 UHD drivers from intel website;
use Intel driver updater to regularly update them.

Try it and give some feedback ;)


Regarding Teams users App and graph Permission

Our User case is to send message to DL / Security group in communicator app . In this use case should we deploy the teams user app compulsory  or when sending to DL or security group users will receive the message as 1:1 chat where there is no need to deploy the user app to all users

If we don't enable this permission TeamsAppInstallation.ReadWriteForUser.All  what are the impact which function will not work

Hi @sponlinedev ,
Thank you for raising the query.

User app must be present at the tenant level so that DL selected through 4th sending option where each user within the DL/M365 group will be installed with user app otherwise messages won't be delivered.
If we don't enable the permission-TeamsAppInstallation.ReadWriteForUser.All, 4th sending option won't work.

If the message to be viewed teams user app should be deployed and made available for all the users so that they can see the message broadcasted.
If we deploy and made the app available for anyone so we still need the permission mentioned above?

would like to clarify  ,  let say from the teams admin center we install the user app and made available for all users .  Do we still need the grant permission for TeamsAppInstallation.ReadWriteForUser.All

Hi @sponlinedev ,
User app must be added at the tenant level and the permission need to be kept because members in the team is not fixed(may get added/removed).

in one of the issue its mentioned auto install when message is sent or install and made available via teams admin center
if we install install and made available can proactive install can be disabled thus the graph permission is not required.
From other Issue
This is the permission that allows for the proactive app installation feature that was part of the V3 release. Before this feature was available, only users that had the Company Communicator app installed would receive a message sent to them. Meaning you would either need to install the app for all users via the Teams admin center or ask everyone to side-load the app.

https://github.com/OfficeDev/microsoft-teams-apps-company-communicator/issues/240

@gsv022  Any updates on this query

Hi @sponlinedev ,

When the message is about to deliver to the user, there will be a check whether user app is installed at the tenant level or not. If proactive installation is set to true and permission is granted within API permission then user will get installed with the user app.

If the members count in team is same , this process is a one time activity.  But practically this wont be the scenario, members in the team will be dynamic(increase/decrease). When new user joins, CC will install the user app to new user with the help of proactive installation and then will send the message.


pages not getting connected to Nav Bar

error using  tag is popping up

#59 use id tag in pages to link the nav bar to the pages


Review duplicate processing in pod watcher callbacks that can cause increased resources utilization

I observed that when a pod is deleted, cilium Kubernetes pod watcher triggers updateK8sPodV1 [1], which in turn calls add addK8sPodV1[2].
Both these functions do parsing of various pod related data. For example, updateK8sPodV1 compares annotations and labels for the received old and new pod objects, while addK8sPodV1 upserts host to port mappings in the service maps, upserts an IP entry in ipcache, parses named ports for updates, etc. These events are then immediately followed by the actual deleteK8sPodV1 event, which would trigger deletion of all the saved internal state associated with the pod.
The duplicated and redundant processing can lead to increased CPU utilization. I haven't measured the actual CPU cycles consumed during the processing that happens, but it's good to collect that data in high scale and dynamic clusters.
I added a comparator.Compare [3] (thanks to @joestringer for the pointer) call in the updateK8sPodV1 to see why the update callback function was being called [4].
We should review these call chains to see if they are intended to be this way, and if not, can the processing during pod update callbacks be optimized? Additionally, we should check if any of the unused fields in the slim pod object need be trimmed further in order to avoid triggering the update events in such cases?
Here is the diff printed by the compare method during one of the iterations -
Unified diff:
--- a
+++ b
@@ -1,83 +1,62 @@
 &v1.Pod{
     TypeMeta:   v1.TypeMeta{},
     ObjectMeta: v1.ObjectMeta{
         Name:            "lrp-pod",
         Namespace:       "default",
         UID:             "24f7b0bb-1f03-4db6-a3f0-6a9c60a86000",
-        ResourceVersion: "3348474",
+        ResourceVersion: "3348479",
         Labels:          {"app":"proxy"},
         Annotations:     {"kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"v1\",\"kind\":\"Pod\",\"metadata\":{\"annotations\":{},\"labels\":{\"app\":\"proxy\"},\"name\":\"lrp-pod\",\"namespace\":\"default\"},\"spec\":{\"containers\":[{\"image\":\"nginx\",\"name\":\"lrp-pod\",\"ports\":[{\"containerPort\":80,\"name\":\"tcp\",\"protocol\":\"TCP\"}]}]}}\n"},
     },
     Spec: v1.PodSpec{
         InitContainers: nil,
         Containers:     {
             {
                 Name:  "lrp-pod",
                 Image: "nginx",
                 Ports: {
                     {Name:"tcp", HostPort:0, ContainerPort:80, Protocol:"TCP", HostIP:""},
                 },
                 VolumeMounts: {
                     {MountPath:"/var/run/secrets/kubernetes.io/serviceaccount"},
                 },
             },
         },
         ServiceAccountName: "default",
         HostNetwork:        false,
     },
     Status: v1.PodStatus{
         HostIP: "<IP_ADDRESS>",
         PodIP:  "<IP_ADDRESS>",
         PodIPs: {
             {IP:"<IP_ADDRESS>"},
         },
         StartTime: &v1.Time{
             Time: time.Time{
                 wall: 0x0,
                 ext:<PHONE_NUMBER>6,
                 loc:  &time.Location{
                     name: "Local",
                     zone: {
                         {name:"UTC", offset:0, isDST:false},
                     },
                     tx: {
                         {when:-576460752303423488, index:0x0, isstd:false, isutc:false},
                     },
                     cacheStart: -576460752303423488,
                     cacheEnd:<PHONE_NUMBER>854775807,
                     cacheZone:  &time.zone{(CYCLIC REFERENCE)},
                 },
             },
         },
         ContainerStatuses: {
             {
-                Name:  "lrp-pod",
-                State: v1.ContainerState{
-                    Running: &v1.ContainerStateRunning{
-                        StartedAt: v1.Time{
-                            Time: time.Time{
-                                wall: 0x0,
-                                ext:<PHONE_NUMBER>9,
-                                loc:  &time.Location{
-                                    name: "Local",
-                                    zone: {
-                                        {name:"UTC", offset:0, isDST:false},
-                                    },
-                                    tx: {
-                                        {when:-576460752303423488, index:0x0, isstd:false, isutc:false},
-                                    },
-                                    cacheStart: -576460752303423488,
-                                    cacheEnd:<PHONE_NUMBER>854775807,
-                                    cacheZone:  &time.zone{(CYCLIC REFERENCE)},
-                                },
-                            },
-                        },
-                    },
-                },
+                Name:        "lrp-pod",
+                State:       v1.ContainerState{},
                 Image:       "nginx:latest",
                 ImageID:     "docker-pullable://nginx@sha256:5dc39e6ea65a5bf52d0cb6cbe687445a03ab7fdf1382cf51d6af7f2e0c2f464b",
                 ContainerID: "docker://240f6735becc0e6eca11de7c9f2f545135d66348d5339c931d789c6c648dbab4",
             },
         },
     },
 }

[1] https://github.com/cilium/cilium/blob/d7433cff08bfd29b3aa3b15769bd01fdfcb115ca/pkg/k8s/watchers/pod.go#L215
[2] https://github.com/cilium/cilium/blob/d7433cff08bfd29b3aa3b15769bd01fdfcb115ca/pkg/k8s/watchers/pod.go#L188
[3] https://github.com/cilium/cilium/blob/d7433cff08bfd29b3aa3b15769bd01fdfcb115ca/pkg/comparator/comparator.go#L23
[4] https://github.com/cilium/cilium/blob/d7433cff08bfd29b3aa3b15769bd01fdfcb115ca/pkg/k8s/watchers/pod.go#L86

We should review these call chains to see if they are intended to be this way, and if not, can the processing during pod update callbacks be optimized?

Reopening the issue, and parking it in my queue to review this further.

@aditighag is the issue still happening with #13179

@aditighag is the issue still happening with #13179?

I haven't checked yet. But based on the diff I posted in the description, Update callback shouldn't get called after #13179. I reopened the bug to review whether the Add callback needs to be called on every update. The reasons being, label/annotations updates are already handled in the Update callback and namedPorts can't change.

Check if duplicate events are generated during pod delete callback after PR #13394 added a new field in the pod spec.

@aditighag can we close this issue? is it still happening?

https://github.com/cilium/cilium/issues/13136#issuecomment-694357756
Yes, this code is still unchanged, no? Hoping to fix this in the next release.


Templates are not added in pip package

I installed using 'pip install pinax-stripe'. The default template directory is empty, except for some emails. This causes TemplateDoesNotExist errors when trying the default views.
Only templates succesfully included are e-mail templates:
find (venv folder)/local/lib/python2.7/site-packages/pinax/stripe/templates -type f
(venv folder)/local/lib/python2.7/site-packages/pinax/stripe/templates/pinax/stripe/email/body_base.txt
(venv folder)/local/lib/python2.7/site-packages/pinax/stripe/templates/pinax/stripe/email/subject.txt
(venv folder)/local/lib/python2.7/site-packages/pinax/stripe/templates/pinax/stripe/email/body.txt

@JosPolfliet Take a look at https://github.com/pinax/pinax-theme-bootstrap
This is a bit of a dupe of #242 - one of these should probably stay open until the docs can be updated.

Thanks. Should have done more research. Indeed the documentation should be updated. My personal opinion is that standard templates are a good thing, even if they are crappy. It gives you a minimum working example which you then can start to extend..

@JosPolfliet We would appreciate if you'd make a PR for adding this to the docs :)