[RFC] Trusted storage proposal for CoCo

Motivation
The CoCo stack utilizes hardware-based TEE technology for runtime protection which provides the confidential memory. Current CoCo uses confidential memory for data storage like container images, container RW layers. Memory is an expensive resource and we need a low cost solution to provide trusted storage as part of CoCo stack. Thanks @jiangliu and @liangzhou121, this proposal is based on https://github.com/confidential-containers/documentation/pull/23 and https://github.com/confidential-containers/documentation/issues/20 .
Short Term Goals

Provide Confidential Ephemeral Storage

Auto crypt setup(encrypt) for the storage, use dm-crypt to protect the block volume
Host downloaded container images (Pod level)
Host RW layer of container rootfs
Provide ephemeral volume for containers

Provide Confidential Persistent Storage

Auto crypt setup(encrypt/decrypt) for the storage

Compatible with current Kubernetes PodSpec

Use pause sidecar container to host the block volume

Integrate with at least one Kubernetes CSI driver

open-local: https://github.com/alibaba/open-local
topolvm: https://github.com/topolvm/topolvm

Long Term Goals

Transparent for the pod spec of enduser

Utilize Kubernetes MutatingAdmissionWebhook to auto inject the config

Kubernetes support Pod level storage resource definition and allocation

High Level Architecture

This is the high level architecture for trusted storage for CoCo. The CSI driver will dynamically provision and allocate a block volume for the pod. We will associate this block volume with a sidecar container with pause image. Since the block volume will be used to store container images, to avoid the bootstrap issue, the pause image will be in the rootfs by default to avoid image downloading. After the block volume is attached to guest through virtio-blk, the kata-agent will do the initialization and crypt setup work, if the block device is encrypted, kata-agent will get the decrypt key from trusted party like attestation-agent.
Implementation Details

PodSpec Definition
Since the volumes consumed by a pod need associate with one container, we choose use a sidecar container with pause image to host the block volume:

apiVersion: v1
kind: Pod
metadata:
  name: trusted-lvm-block
spec:
  runtimeClassName: kata-cc
  containers:
   - name: sidecar-trusted-store
     image: pause
     volumeDevices:
     - devicePath: "/dev/trusted_store"
       name: trusted-store

  volumes:
   - name: trusted-store
     persistentVolumeClaim:
       claimName: trusted-store-block-pvc

Kata Guest Kernel
Add dm-crypt required kernel config
Kata rootfs
Add pause image, cryptsetup and filesystem utilities related packages into the rootfs
Kata-agent
Detect the block volume with device path: /dev/trusted_store, start the crypt setup and file system initialization for this volume and mount the encrypted disk to the container image store path: /run/image. The following containers images and container RW layers will saved in this directory.

This is the demo link:
https://asciinema.org/a/W90GYRZmFoCb1nEIKS3BjAToH
Potential GAPs

Container launch sequence
The initContainer will launch sequencely and will wait previous container finished successfully, but normal container for runc runtime will launch in paralle, but for kata runtime I found they also run sequencely, but we need define the sidecar container before normal application container.

Storage resouce restriction for each container
After kata-agent setup the block volume and export as directory for other containers to use, currently there don't have resource restriction for each container

Container defined mount path support
In pod yaml file, the volumes can be used as shared volume to allow the communication between containers, after kata-agent manage the block volume, the definition for each containers to define the mount path also need support.

Block volume sharing between pods
Since the volume is encrypted and can not mount by other pods, sharing between pods is not supported

Devel Plan for CoCo first release
For CoCo September first release, we will support the ephemeral trusted storage to host container images and RW layers.

[ ] Guest kernel support dm-verify, dm-crypt
[ ] Integrate cryptsetup and filesystem utilities into Kata rootfs
[ ] Package pause image inside Kata rootfs
[ ] Kata agent crypt setup raw block volume with shell scripts
[ ] Kata agent crypt setup raw block volume with rust native implementation(one GAP is libcryptsetup-rs don't support musl build)
[ ] Mount the encrypted disk for image/container RW layer store

The next step will be support the persistent trusted storage with Kata direct volume features.

[ ] Kata agent retrive disk encrypted key from attestation-agent
[ ] Kata agent crypt setup persistent encrypted raw block vloume

@arronwy Nice write-up, thanks. A few remarks:

"Provide Confidential Non-Persistent Storage" – I think we settled for the "ephemeral' terminology (as in your iv) rather than "non-persistent", to emphasize that the volume's data is lost when you shut down the pod.
Do we want to mix ephemeral storage for images with persistent storage in the same proposal?
One point of confusion for me because of 2. is: wouldn't persistent volumes be associated to the actual container using it, rather than the pause container? Or are you saying that the encrypted volume is attached to the pause container and the mutating webhook provides the decrypted one to the containers? If so, that seems to contradict the idea that the agent would do the crypto setup and expose the volume (in which case it could as well do the relevant device name mapping if necessary)
"Utilize Kubernetes MutatingAdmissionWebhook to auto inject the config": not clear what mutation would happen there. Is it the mapping from encrypted block volume to decrypted one?
"will dynamic provisioning and allocating " -> will dynamically provision and allocate"
"if the block device is encrypted, kata-agent will get the decrypt key from trusted party like attestation-agent." I would be a bit more forceful here, and state that the device must be encrypted, and that the decryption key will come from a trusted party through the attestation agent, Unless you want to mention OPA policies and the ability to have a policy allowing secrets to be in initrd for development purpose?

I also did the following changes to the text:
4. "Transpant" -> "Transparent"
7. "pasue" -> "pause"
8. "vloume" -> "volume"
9. "initializaiton" -> "initialization"

Thanks @c3d, my comments inline:

"Provide Confidential Non-Persistent Storage" – I think we settled for the "ephemeral' terminology (as in your iv) rather than "non-persistent", to emphasize that the volume's data is lost when you shut down the pod.

Yes, I'll use ephemeral to avoid confusing.

Do we want to mix ephemeral storage for images with persistent storage in the same proposal?

ephemeral or persistent is defind by the reclaim policy of the PV, the default policy is Delete, for persistent, we will define as Retain. For our runtime side, persistent storage may encrypted by default which need decrypt before to use.

One point of confusion for me because of 2. is: wouldn't persistent volumes be associated to the actual container using it, rather than the pause container? Or are you saying that the encrypted volume is attached to the pause container and the mutating webhook provides the decrypted one to the containers? If so, that seems to contradict the idea that the agent would do the crypto setup and expose the volume (in which case it could as well do the relevant device name mapping if necessary)

Yes, if the volume is associated with pause container with our defined device path /dev/trusted_store, we will use it as pod level resource to host the container images, if the volume is associated with actual workload container, it will be container level resource only used by workload container.

"Utilize Kubernetes MutatingAdmissionWebhook to auto inject the config": not clear what mutation would happen there. Is it the mapping from encrypted block volume to decrypted one?

Mutating Webhook will patch the pod yaml file to avoid the end user to define the sidecar container to associate the trusted storage.

"will dynamic provisioning and allocating " -> will dynamically provision and allocate"

Will fix

"if the block device is encrypted, kata-agent will get the decrypt key from trusted party like attestation-agent." I would be a bit more forceful here, and state that the device must be encrypted, and that the decryption key will come from a trusted party through the attestation agent, Unless you want to mention OPA policies and the ability to have a policy allowing secrets to be in initrd for development purpose?

If the block device is ephemeral which means we can format and encrypt it directly. For persisten block device, we can ensure the device must be encrypted before use, but we can not enforce the device is encrypted when it is detected by kata-agent, because when CSI driver allocating the block device, it will not do the crypt setup operations by default.

I also did the following changes to the text:

"Transpant" -> "Transparent"
"pasue" -> "pause"
"vloume" -> "volume"
"initializaiton" -> "initialization"

Fixed, thanks a lot!

Hi @arronwy, thank you for opening this issue. I was wondering what you had in mind for the following.

Provide ephemeral volume for containers

The current Kata CC ephemerality solution is provided by one of these (I am not sure, which):

ramfs - completely contained in RAM
tmpfs - RAM-based but backed by swap storage

Here are some possible solutions:

dm-crypt - vulnerable to replay attack.
dm-crypt + dm-integrity - still vulnerable to replay attack where the verity-tag/block pairs are replayed together.
dm-crypt + dm-integrity with bitmap mode - might not be vulnerable to replay attacks since the verity tags are not synchronized to disk, but I have not verified this myself. But with dm-integrity the kernel initializes the device on first use, which can be extremely slow, but this might not be a problem for bitmap mode.
develop a new device mapper target for the problem of ephemerality specifically.

Do you have any solutions to add that are not on this list?

The current Kata CC ephemerality solution is provided by one of these (I am not sure, which):

Hi, kata pod should use tmpfs to store the downloaded container image currently (/run/kata-containers/xxxxx).
But the pod's rootfs is mounted as readonly permission, so I think its tmpfs's swap storage should be disabled. And I also tried to execute swapon -s inside pod, its output is empty.

Provide Confidential Ephemeral Storage

Auto crypt setup(encrypt) for the storage, use dm-crypt to protect the block volume
Host downloaded container images (Pod level)
Host RW layer of container rootfs
Provide ephemeral volume for containers

The downloaded container images would need integrity protection. The read-only layers can be protected with dm-crypt/dm-integrity.
Even with dm-crypt/dm-integrity, the RW layer of the container rootfs would still be vulnerable to replay attack (attacker captures an older version of the cipertext and the integrity tag, after the guest VM writes a new version, replace the new version with the captured old version). Dm-verity/dm-crypt would provide protection against the replay attacks, but it's not a standard mode Linux distros support.
Another concern I have on the current Kata-CC design for confidential container rootfs is its inability to share common unencrypted layers across pods. Each pod would need to download the unencrypted layers, write out to its dm-crypt/dm-integrity block device, even if the layer was already downloaded and processed on the K8S node. The extra latency would jeopardize adoption by Cloud Service Providers. It's a very tough problem to solve. I'm thinking in the direction of dm-verity for unencrypted layers, and dm-crypt/dm-integrity for the encrypted layers, but a lot of complication still remains. We probably should have a separate issue to discuss image layers sharing.

Yes, current CC image pull inside guest have GAP for trusted image sharing between pods, upstream also have some discussing, and @c3d show a proposal on the weekly meeting at May 19th: https://docs.google.com/document/d/1E3GLCzNgrcigUlgWAZYlgqNTdVwiMwCRTJ0QnJhLZGA/edit#heading=h.wptyafxr099x
Incremental image download using device mapper in the guest + nbdkit on the host

Thanks for the detailed proposal @arronwy, this is well aligned with the multiple discussions we've had in the CC meetings and in the previous issues (https://github.com/confidential-containers/documentation/pull/23 and https://github.com/confidential-containers/documentation/issues/20).
On the ephemeral storage support, I think we should add a section describing the volume lifecycle. For example, your proposal does not explain what happens to the volume once the pod terminates. Describing how we initialize it, which crypto material we use for the initial dm-crypt setup, and how we clean it up when the pod terminates would be useful.
Another comment: I would remove the confidential persistent storage support as short term goals. As @mikbras and @bodzhang highlighted, there are many open issues with that use case that we need to clarify, I think.

Hey @mikbras. Thanks for opening that discussion.

Provide Confidential Ephemeral Storage

Auto crypt setup(encrypt) for the storage, use dm-crypt to protect the block volume
Host downloaded container images (Pod level)
Host RW layer of container rootfs
Provide ephemeral volume for containers

The downloaded container images would need integrity protection.

Could  you elaborate on the need for integrity protection need for the ephemeral storage use case? Are you thinking about pods being able to re-use ephemeral volumes between restarts?

Even with dm-crypt/dm-integrity, the RW layer of the container rootfs would still be vulnerable to replay attack (attacker captures an older version of the cipertext and the integrity tag, after the guest VM writes a new version, replace the new version with the captured old version). Dm-verity/dm-crypt would provide protection against the replay attacks, but it's not a standard mode Linux distros support.

For confidential persistent storage support, this makes sense to me. FWIW, we control the CC guest and could potentially add dm-verity support to it if we thing this is the right solution.

@sameo, if the ephemeral storage holding the downloaded container rootfs only has dm-crypt protection, alteration to the cipertext of the container rootfs will not be explicitly detected.  After decryption, the resulting plaintext code might be invalid CPU instruction or high level language code with syntax error, leading to crashes/exceptions when executed, but might also contain executable/interpretable code. Through trial and error, an attacker might be able to trigger execution flow unintended by the container developer. Similarly, corrupted plaintext data after decryption (due to alteration of the cipertext) can also lead to behavior/result unintended by the container developer. With portions of the plaintext content known (unencrypted container image layers), depending on the crypto algorithm used, more targeted attack might be possible. Even with the XTS-AES algorithm (newest mode added to dm-crypt), code/data integrity is still a significant concern.

Yes, to protect container rootfs R/W layers and scratch storage used by the guest VM, dm-verity/dm-crypt, with the verity hash tree kept inside the confidential memory, will meet all the security requirements. Before we have a good implementation for that, I propose we use Linux kernel's native support for dm-integrity/dm-crypt configuration for R/W layers.

Agreed, we can have a 2 steps implementation plan and start with the current kernel native support. We should document the identified security gaps and also our plans to fill them with dm-verity.

I would like to bring up the other concern where dm-verity might have a role to play.

Another concern I have on the current Kata-CC design for confidential container rootfs is its inability to share common unencrypted layers across pods. Each pod would need to download the unencrypted layers, write out to its dm-crypt/dm-integrity block device, even if the layer was already downloaded and processed on the K8S node. The extra latency would jeopardize adoption by Cloud Service Providers. It's a very tough problem to solve. I'm thinking in the direction of dm-verity for unencrypted layers, and dm-crypt/dm-integrity for the encrypted layers, but a lot of complication still remains.

@c3d proposed last month Incremental image download using device mapper in the guest + nbdkit on the host to support shared layers. But I believe the forum expressed concerns on the requirement of a new container image format.
I know @mikbras is working on a write-up with more details on the challenges and options to support shared layers. Hopefully, it will be posted soon and can facilitate further discussion.

Sounds very good. We can also bring that topic to the weekly CC meeting. It's related to what @c3d presented and is an important topic to bring to the rest of the community. @mikbras let us know if you're up for it.

@arronwy Could you please amend the proposal with the dm-integrity requirement?

@bodzhang Thanks for the explanation. I think I get your point now. A malicious host could tamper with the cipher text on the encrypted block at any point during the pod lifecycle, and at best trigger a DoS. At worst it could trigger an unintended code execution flow in the guest (and e.g. leak data out of the guest).

I was initially under the impression that we would still validate the container's image checksums from the metadata. In other words, if it decrypts, but say the SHA256 does not match, we would know it's been tampered with.
But I think you are talking about tampering with it after we did the check, or tempering with persistent volumes. In that case, yes, dm-verity or dm-integrity could matter. The dm-integrity case is a bit more expensive though, so that's why at least for images, we were considering using only dm-verity (meaning the image is read-only for the guest).

@bodzhang Thanks for the explanation. I think I get your point now. A malicious host could tamper with the cipher text on the encrypted block at any point during the pod lifecycle, and at best trigger a DoS. At worst it could trigger an unintended code execution flow in the guest (and e.g. leak data out of the guest).

I was initially under the impression that we would still validate the container's image checksums from the metadata. In other words, if it decrypts, but say the SHA256 does not match, we would know it's been tampered with.
But I think you are talking about tampering with it after we did the check, or tempering with persistent volumes.

We're trying to protect against a host that can tamper with the disk ciphered sectors, because it does have access to them. Images integrity are checked at download time, but once they're verified, decrypted and mounted, the host is free to mess with the backend disk sectors at any point in the pod lifecycle if we don't HMAC them with a key that's only known to the guest kernel.

In that case, yes, dm-verity or dm-integrity could matter. The dm-integrity case is a bit more expensive though, so that's why at least for images, we were considering using only dm-verity (meaning the image is read-only for the guest).

Thanks for the detailed proposal @arronwy, this is well aligned with the multiple discussions we've had in the CC meetings and in the previous issues (#23 and #20).
On the ephemeral storage support, I think we should add a section describing the volume lifecycle. For example, your proposal does not explain what happens to the volume once the pod terminates. Describing how we initialize it, which crypto material we use for the initial dm-crypt setup, and how we clean it up when the pod terminates would be useful.

Yes, I'll add this volume lifecycle part.

Another comment: I would remove the confidential persistent storage support as short term goals. As @mikbras and @bodzhang highlighted, there are many open issues with that use case that we need to clarify, I think.

Make sense to me, persistent storage still have many open issues, we can remove from short term goals.

@arronwy Could you please amend the proposal with the dm-integrity requirement?

Sure, we shoud provide the integrity protection for the encrypted volume.

Sounds very good. We can also bring that topic to the weekly CC meeting. It's related to what @c3d presented and is an important topic to bring to the rest of the community. @mikbras let us know if you're up for it.

@sameo: I would be glad to discuss some of our thinking around dm-verity protection. Would this week's CC meeting work?

Hey @mikbras. This week's agenda is full, but I'd be glad to add you to the following week's one (July 28th). Is that ok with you?

Hey @mikbras. This week's agenda is full, but I'd be glad to add you to the following week's one (July 28th). Is that ok with you?

Sounds good. Thank you @sameo.

Hey @mikbras. This week's agenda is full, but I'd be glad to add you to the following week's one (July 28th). Is that ok with you?

Sounds good. Thank you @sameo.

Done, you're set for a presentation on 07/28. Thanks @mikbras.

Done, you're set for a presentation on 07/28. Thanks @mikbras.

Thank you @sameo, looking forward to it.

Hi @sameo @c3d @mikbras @bodzhang @liangzhou121  I submitted a proposal to protect the integrity of boot image https://github.com/confidential-containers/documentation/issues/40  please review.

Hi @sameo @c3d @mikbras @bodzhang @liangzhou121 I submitted a proposal to protect the integrity of boot image #40 please review.

I've added it to today's calendar if time permits. since this seems fitting with the dm-{verity,integrity} discussion. However, I feel like the agenda is pretty full, and we probably need to dedicate more time to the discussion. Also, I'm just back, and have not looked at the proposal in detail yet. Added to my to-do list ;-)

@arronwy when use pause sidecar container to initialize the encrypted disk and volume to host the signed images as ephemeral storage. I know kata-agent will launch the containers in sequence according to the containers order in Pod descriptor. However, as far as I know (I tested also), it can not guarantee the long run job can be completed in sidecar before 2nd container get launched. Which means when pull images for 2nd container, it's possible the ephemeral disk is not ready yet. Is there a solution for this case now?

Hi @huoqifeng , yes, we need ensure the ephemeral disk is initialized before launch the second container, current implementation is do the initialization and setup work in  do_create_container for the sidecar pause container, then everything will be ready when pause container is started.

@arronwy is this issue still relevant or can be closed?
If it's still relevant to what release do you think we should map it to (mid-November, end-December, mid-February etc...)?


healthcare  member schema Changes

Updated healthcare member field group as per the feedback.

Moved this to
https://github.com/adobe/xdm/pull/1563


Feat: Welcome Page

Closes #17138
The welcome message uses the users name if set, defaults to Camper.

Just a reminder, to update the migration scripts to add/remove the keys that are being changed on the User object.


Update rebar/rebar3 to latest version

please have a look at CI build failure log, could you update the SHA256SUM as well?
https://travis-ci.org/c0b/docker-erlang-otp/jobs/156210183#L8068

My bad... Done!
PS: I rebased the commit to have only one. I find it much cleaner. YMMV

it's interesting that when I want to test this today, just found rebar released 2.6.4 a few hours ago, do you want to update to that instead? we can skip 2.6.3
https://github.com/rebar/rebar/releases

Ahah. No problem I'm doing it.

Done ;)

I have built it locally and it runs look like fine; so I merged it, while the image at http://hub.docker.com/_/erlang/ is still the old one because that's managed by the official-images; if you like you can make a PR to there (examples: https://github.com/docker-library/official-images/issues?q=label:library/erlang ) or I think it's good to be staged for next erlang releases
root@98c790ea9877:/# rebar version
rebar 2.6.4 20 20160901_221634 No VCS info available.
root@98c790ea9877:/# rebar3 version
rebar 3.3.0 on Erlang/OTP 20 Erts 8.0.3

Yeah, it's OK for the next erlang release. Thanks!


Problem with ASP.NET MVC Bundling

Module 'pascalprecht.translate' is not available! You either misspelled the module name or forgot to load it. If registering a module ensure that you specify the dependencies as the second argument.
the problem solved when i disabled the bundling optimiazation in my MVC website

I have actually no clue what about you are talking. We don't use ASP.NET, but I'm sure that the language will not have such a feature. Perhaps you tool chain is the problem.. but not this plugin.

I will be more specific, the ASP.NET MVC bundling is just a tool for CSS and JS minification, when i enable bundling i got this error in my AngilarJS application
Module 'pascalprecht.translate' is not available! You either misspelled the module name or forgot to load it. If registering a module ensure that you specify the dependencies as the second argument.
i faced this problem before in my AngularJS controller, the reason is common with JS Minifiers, Minifiers will shorten identifiers where possible, they won't modify any string literals in your code. The idea is to provide the dependency names as a separate array of strings which will survive the minification process. My favorite approach to pass this array to the constructor function is through the $inject property.

But - how can we help or support here in any issues regarding angular-translate itself?

I'm not sure, maybe the plugin should use dependency as mentioned in my previous comment in its inner code.

Dear,
i was using the plugin file that is already minified "angular-translate-min.js" i replaced it with the original one because some people reported that minification of a minified file may cause issues , and the problem is gone, and the plugin is working fine.
But, after adding "storage-cookie.js" to my application i faced a new problem
Unknown provider: nProvider <- n <- $translateCookieStorage <- $translate

The problem solved by applying the concept i mentioned in previous comment

Provide the dependency names as a separate array of strings which will survive the minification process

i changed the way the factory is defined in "storage-cookie.js" file.
angular.module('pascalprecht.translate')

/**
 * @ngdoc object
 * @name pascalprecht.translate.$translateCookieStorage
 * @requires $cookieStore
 *
 * @description
 * Abstraction layer for cookieStore. This service is used when telling angular-translate
 * to use cookieStore as storage.
 *
 */
  .factory('$translateCookieStorage', ['$cookieStore', function ($cookieStore) {

      'use strict';

      var $translateCookieStorage = {

          /**
           * @ngdoc function
           * @name pascalprecht.translate.$translateCookieStorage#get
           * @methodOf pascalprecht.translate.$translateCookieStorage
           *
           * @description
           * Returns an item from cookieStorage by given name.
           *
           * @param {string} name Item name
           * @return {string} Value of item name
           */
          get: function (name) {
              return $cookieStore.get(name);
          },

          /**
           * @ngdoc function
           * @name pascalprecht.translate.$translateCookieStorage#set
           * @methodOf pascalprecht.translate.$translateCookieStorage
           *
           * @description
           * Sets an item in cookieStorage by given name.
           *
           * @deprecated use #put
           *
           * @param {string} name Item name
           * @param {string} value Item value
           */
          set: function (name, value) {
              $cookieStore.put(name, value);
          },

          /**
           * @ngdoc function
           * @name pascalprecht.translate.$translateCookieStorage#put
           * @methodOf pascalprecht.translate.$translateCookieStorage
           *
           * @description
           * Sets an item in cookieStorage by given name.
           *
           * @param {string} name Item name
           * @param {string} value Item value
           */
          put: function (name, value) {
              $cookieStore.put(name, value);
          }
      };

      return $translateCookieStorage;
  }]);

//$translateCookieStorageFactory.displayName = '$translateCookieStorage';

I think this way of definition should be used in every where in the plugin code.
What do you think ?

We use ng-annotate in the build chain to avoid such  boiler code. You can look into our Gruntfile.
I think we are done, I'll close in forward.


Handle "line too long"

It would be nice for docformatter to also handle lines that are too long in docstrings

It currently wraps long lines in some cases. Were you thinking of doing something specific to long lines?

I would expect to have a tool that wraps them at the max line length setting (usually 80 chars).
Wrapping bullet point lists is a little more tricky that normal paragraphs (indent needs to be right).

Sorry, I'm not following. It should be wrapping simple cases as described. Could you provide an example?


Provide manylinux builds on PyPi

Loosely adapted from Dedupe.

Not sure why this didn't trigger a Travis build. Closing and reopening to try again.

OK to test


Analysis page: Add progress bar or something similar to indicate progress

Page 3: Add progress bar or something similar to indicate progress

The log is now shown as part of the output, possibly more enhancement needed, but basic functionality is there now


Refresh updated_at timestamp on soft delete

For issue https://github.com/laravel/framework/issues/19529
Performing a soft delete updates the updated_at timestamp in the database but not on the model. Assumed the fix would be in line with the desired behaviour?
The changes to SoftDeletes makes the testDeleteSetsSoftDeletedColumn method on DatabaseSoftDeletingTraitTest fail, have written the code to get it passing again, but unsure if my thinking is in the right direction for the test.
Would appreciate any feedback as still pretty new to contributing. Thanks!

Thank you for the fix! Unfortunately, I think your change forces an updated_at attribute on the model, even if public $timestamps = false; is set on it. I doubt there are many models with SoftDeletes enabled while not having normal timestamps. But to be sure it always works, something along the lines
protected function runSoftDelete()
{
    $query = $this->newQueryWithoutScopes()->where($this->getKeyName(), $this->getKey());

    $time = $this->freshTimestamp();

    $this->{$this->getDeletedAtColumn()} = $time;
    $attributes = [$this->getDeletedAtColumn() => $this->fromDateTime($time)];

    // first check of the if is pretty much optional but a safe bet
    if ($this instanceof Concerns\HasAttributes && $this->timestamps === true) {
        $this->{$this->getUpdatedAtColumn()} = $time;
        $attributes[$this->getUpdatedAtColumn()] = $this->fromDateTime($time);
    }

    $query->update($attributes);
}

seems more realistic to me. I'm also only a new Laravel contributor though and still trying to understand everything behind the scenes... :)
For full coverage, this obviously requires two tests thanks to the if statement.


add Collective.expensePolicy

Required for https://github.com/opencollective/frontend/pull/368

Coverage decreased (-0.01%) to 72.067% when pulling ed31c6cc634e67cf0395796e753733a7c2bdff58 on expensePolicy into 1eecf2b90113f2d0709fea52bc670a765248af1e on master.


Environment Manager can't see New VENV despite being created by the extension.

Type: Bug

Behaviour
Expected vs. Actual
I have a small VSCode workspace that I created solely for Python development. I have two virtual environments (venv). One of them (Font Tools) was created through the plugin itself.
My problem is that the environment manager can only see one of the venv environments.
Expected: Both environments to be listed.
Actual: Only one environment is listed.
Steps to reproduce:
I'm not exactly sure how to reproduce, other than re-creating the structure of my workspace in VSCode and then creating the virtual environments.

Diagnostic data

Python version (& distribution if applicable, e.g. Anaconda):
Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Unknown
Value of the python.languageServer setting: Default

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

XXX

User Settings

languageServer: "Pylance"

Extension version: 1.0.4
VS Code version: Code 1.70.1 (6d9b74a70ca9c7733b29f0456fd8195364076dda, 2022-08-10T06:08:33.642Z)
OS version: Windows_NT x64 10.0.19044
Modes:

System Info

Item
Value

CPUs
Intel(R) Core(TM) i9-9900KF CPU @ 3.60GHz (16 x 3600)

GPU Status
2d_canvas: enabledcanvas_oop_rasterization: disabled_offdirect_rendering_display_compositor: disabled_off_okgpu_compositing: enabledmultiple_raster_threads: enabled_onopengl: enabled_onrasterization: enabledraw_draw: disabled_off_okskia_renderer: enabled_onvideo_decode: enabledvideo_encode: enabledvulkan: disabled_offwebgl: enabledwebgl2: enabled

Load (avg)
undefined

Memory (System)
63.92GB (52.62GB free)

Process Argv
D:\Dev\Python\python.code-workspace --crash-reporter-id 980edeb0-3f45-42f4-8995-c0d13015b889

Screen Reader
no

VM
0%

Here are screenshots to further clarify what is happening. This is my workspace:

And this is the environment manager (only showing one environment):

And yes, I've tried clicking the "Refresh Envrionments" button.
I think it would be helpful if someone could elaborate on how the plugin detects VENVs. Maybe that would help me resolve this issue.

I can't see all my venv, do you know how to fix it ?

Possible related, but the "Refresh Environments" button doesn't seem to work. I install virtualenv in the global packages and hitting the refresh button doesn't load the new package.
The only way I'm able to refresh the view is to close Visual Studio Code, then reopen. This will now show the new packages in the global packages list.


version clarification

I didn't read in the instructions for what version this should be applied too.
If I just get the game on steam and install it, I can just get the 3rd version from here, install the silent patch and I'm good to go?

That's correct.


Restore default target output path on Windows CI builds

Motivation

When Tokio was upgraded (#2994), the CI Windows builds started failing due to insufficient disk space. As a temporary workaround, the target output directory was changed to use a separate larger drive (C: instead of D:). However, it would be best to not have this workaround, and to just optimize the Zebra build to use less disk space.
Related Work

#2994 introduced the workaround.

Hey team! Please add your planning poker estimate with ZenHub @conradoplg @dconnolly @jvff @oxarbitrage @teor2345 @upbqdn

Let's do this after we've resolved most of our duplicate dependencies.
If we do it too soon, the build directory could get big again, and we might have to revert it.

Not strictly required for release candidate


update backup cli output

changes a couple things in the backup cli output:

appends the backup id after completion
prints an error if the backup fails
replaces multierr with a clues error
logs each backup failure individually

Does this PR need a docs update or release note?

[x] :no_entry: No

Type of change

[x] :broom: Tech Debt/Cleanup

Issue(s)

#1970

Test Plan

[x] :muscle: Manual

Some comparisons for a two-user backup:
Both successful
# before
Connecting to repository:      4s done
Backing Up ∙<EMAIL_ADDRESS>Connecting to M365:         done
Discovering items to backup:      2s done
∙ contacts:      1s done
Backing up data:      1s done
Backing Up ∙<EMAIL_ADDRESS>Connecting to M365:         done
Discovering items to backup:      2s done
∙ contacts:      1s done
Backing up data:      0s done
  Started At            ID                                    Status     Resource Owner
  2023-03-21T23:42:14Z  be0fab6d-edb0-4c36-b353-910d366804e6  Completed<EMAIL_ADDRESS>  2023-03-21T23:42:19Z  dcc40af7-0f19-45f7-a481-5a995b64e61f  Completed<EMAIL_ADDRESS>
# after
Connecting to repository:      5s done
Backing Up ∙<EMAIL_ADDRESS>Connecting to M365:         done
Discovering items to backup:      2s done
∙ contacts:      1s done
Backing up data:      1s done
Done - ID: 2f841c98-4fd7-4e8b-b26d-9f0afe0c163c

Backing Up ∙<EMAIL_ADDRESS>Connecting to M365:         done
Discovering items to backup:      2s done
∙ contacts:      1s done
Backing up data:      0s done
Done - ID: 84ad82d1-9633-45be-8b8b-4fbfcf7ef239

All Backups Complete:
  Started At            ID                                    Status     Resource Owner
  2023-03-21T23:48:55Z  2f841c98-4fd7-4e8b-b26d-9f0afe0c163c  Completed<EMAIL_ADDRESS>  2023-03-21T23:49:01Z  84ad82d1-9633-45be-8b8b-4fbfcf7ef239  Completed<EMAIL_ADDRESS>
Both Failed
# Before
Connecting to repository:      4s done
Backing Up ∙<EMAIL_ADDRESS>Connecting to M365:         done
Discovering items to backup:      3s done
∙ contacts:      2s done
Backing Up ∙<EMAIL_ADDRESS>Connecting to M365:         done
Discovering items to backup:      2s done
∙ contacts:      1s done
No backups available
Error: 2 errors occurred:
	* Failed to run Exchange backup for user<EMAIL_ADDRESS>doing backup: producing backup data collections: forced fail
	* Failed to run Exchange backup for user<EMAIL_ADDRESS>doing backup: producing backup data collections: forced fail

# After
Connecting to M365:         done
Discovering items to backup:      2s done
∙ contacts:      1s done
Error: running backup: producing backup data collections: forced fail

Backing Up ∙<EMAIL_ADDRESS>Connecting to M365:         done
Discovering items to backup:      2s done
∙ contacts:      1s done
Error: running backup: producing backup data collections: forced fail

Error: 2 of 2 backups failed:
∙<EMAIL_ADDRESS>running backup: producing backup data collections: forced fail
∙<EMAIL_ADDRESS>running backup: producing backup data collections: forced fail

End-of-run with a mixed failure looks as follows.
All Backups Complete:
  Started At            ID                                    Status     Resource Owner
  2023-03-22T00:13:01Z  82c25235-59c9-41c3-b66b-8fa4a3a90bcb  Completed<EMAIL_ADDRESS>Error: 1 of 2 backups failed:
∙<EMAIL_ADDRESS>running backup: producing backup data collections: forced fail


Option to make the progress bar red (or any preferred hex colour)

YouTube has been tweaking the progress bar to a red-pink gradient, which is in my opinion horrible.
Could you perhaps add a mod, under the video section, to prevent this from happening?
More info about the subject: https://www.androidpolice.com/youtube-progress-bar-ui-tweak-red-to-pink/

I second this! I would love to make my playbar orange!


@releng - trying to get 'dev' in sync with 'main' changes missing from 'dev'

Sync was done in an other way, closing this PR.


Very slow copy

Running a HP Z820 workstation with unraid and an LG sata blu-ray drive.
MakeMKV is running in a docker. Identified and set the 2 devices for the optical drive.
I am able to rip both DVDs and Blu-rays with no obvious issues but they take absurdly long to copy to a file. A DVD at around 6gb takes 2hours, and only shows a read rate of around 600kb/s.
A blu-ray of roughly 35gb take 12+hours at around the same speed. The log shows a series of errors on startup all containing references to "fontconfig" and the last line:
Fontconfig error: Cannot load default config file
I'm a bit new to unraid and dockers so i'm hoping im missing something obvious. Any help is appreciated.

Do you have the container's log?  On the Docker page, for the MakeMKV container, click the icon in the last column.

Yes I have the log file that's were the image from the original post came from.

Registration key already up-to-date.
[cont-init.d] 90-makemkv.sh: exited 0.
[cont-init.d] 95-check-optical-drive.sh: executing...
[cont-init.d] 95-check-optical-drive.sh: looking for usable optical drives...
[cont-init.d] 95-check-optical-drive.sh: found optical drive [/dev/sr0, /dev/sg1], group 19.
[cont-init.d] 95-check-optical-drive.sh: exited 0.
[cont-init.d] 96-install-autodiscripper-srv.sh: executing...
[cont-init.d] 96-install-autodiscripper-srv.sh: installing automatic disc ripper service...
[cont-init.d] 96-install-autodiscripper-srv.sh: exited 0.
[cont-init.d] done.
[services.d] starting services
[services.d] starting s6-fdholderd...
[services.d] starting certsmonitor...
[services.d] starting nginx...
[services.d] starting autodiscripper...
[nginx] starting...
[services.d] starting xvfb...
[certsmonitor] disabling service: secure connection not enabled.
[xvfb] starting...
[autodiscripper] starting...
[autodiscripper] Ready.
[services.d] starting logmonitor...
[services.d] starting statusmonitor...
[logmonitor] no file to monitor: disabling service...
[services.d] starting x11vnc...
[statusmonitor] no file to monitor: disabling service...
[services.d] starting openbox...
[x11vnc] starting...
[openbox] starting...
09/04/2019 15:06:40 passing arg to libvncserver: -rfbport
09/04/2019 15:06:40 passing arg to libvncserver: 5900
09/04/2019 15:06:40 passing arg to libvncserver: -rfbportv6
09/04/2019 15:06:40 passing arg to libvncserver: -1
09/04/2019 15:06:40 passing arg to libvncserver: -httpportv6
09/04/2019 15:06:40 passing arg to libvncserver: -1
09/04/2019 15:06:40 passing arg to libvncserver: -desktop
09/04/2019 15:06:40 passing arg to libvncserver: MakeMKV
09/04/2019 15:06:40 x11vnc version: 0.9.14 lastmod: 2015-11-14 pid: 1242
09/04/2019 15:06:40 Using X display :0
09/04/2019 15:06:40 rootwin: 0x43 reswin: 0x200001 dpy: 0x12e129e0
09/04/2019 15:06:40
09/04/2019 15:06:40 ------------------ USEFUL INFORMATION ------------------
09/04/2019 15:06:40 X DAMAGE available on display, using it for polling hints.
09/04/2019 15:06:40 To disable this behavior use: '-noxdamage'
09/04/2019 15:06:40
09/04/2019 15:06:40 Most compositing window managers like 'compiz' or 'beryl'
09/04/2019 15:06:40 cause X DAMAGE to fail, and so you may not see any screen
09/04/2019 15:06:40 updates via VNC. Either disable 'compiz' (recommended) or
09/04/2019 15:06:40 supply the x11vnc '-noxdamage' command line option.
09/04/2019 15:06:40 X COMPOSITE available on display, using it for window polling.
09/04/2019 15:06:40 To disable this behavior use: '-noxcomposite'
09/04/2019 15:06:40
09/04/2019 15:06:40 Wireframing: -wireframe mode is in effect for window moves.
09/04/2019 15:06:40 If this yields undesired behavior (poor response, painting
09/04/2019 15:06:40 errors, etc) it may be disabled:
09/04/2019 15:06:40 - use '-nowf' to disable wireframing completely.
09/04/2019 15:06:40 - use '-nowcr' to disable the Copy Rectangle after the
09/04/2019 15:06:40 moved window is released in the new position.
09/04/2019 15:06:40 Also see the -help entry for tuning parameters.
09/04/2019 15:06:40 You can press 3 Alt_L's (Left "Alt" key) in a row to
09/04/2019 15:06:40 repaint the screen, also see the -fixscreen option for
09/04/2019 15:06:40 periodic repaints.
09/04/2019 15:06:40 GrabServer control via XTEST.
09/04/2019 15:06:40
09/04/2019 15:06:40 Scroll Detection: -scrollcopyrect mode is in effect to
09/04/2019 15:06:40 use RECORD extension to try to detect scrolling windows
09/04/2019 15:06:40 (induced by either user keystroke or mouse input).
09/04/2019 15:06:40 If this yields undesired behavior (poor response, painting
09/04/2019 15:06:40 errors, etc) it may be disabled via: '-noscr'
09/04/2019 15:06:40 Also see the -help entry for tuning parameters.
09/04/2019 15:06:40 You can press 3 Alt_L's (Left "Alt" key) in a row to
09/04/2019 15:06:40 repaint the screen, also see the -fixscreen option for
09/04/2019 15:06:40 periodic repaints.
09/04/2019 15:06:40
09/04/2019 15:06:40 XKEYBOARD: number of keysyms per keycode 7 is greater
09/04/2019 15:06:40 than 4 and 51 keysyms are mapped above 4.
09/04/2019 15:06:40 Automatically switching to -xkb mode.
09/04/2019 15:06:40 If this makes the key mapping worse you can
09/04/2019 15:06:40 disable it with the "-noxkb" option.
09/04/2019 15:06:40 Also, remember "-remap DEAD" for accenting characters.
09/04/2019 15:06:40
09/04/2019 15:06:40 X FBPM extension not supported.
Xlib: extension "DPMS" missing on display ":0".
09/04/2019 15:06:40 X display is not capable of DPMS.
09/04/2019 15:06:40 --------------------------------------------------------
09/04/2019 15:06:40
09/04/2019 15:06:40 Default visual ID: 0x21
09/04/2019 15:06:40 Read initial data from X display into framebuffer.
09/04/2019 15:06:40 initialize_screen: fb_depth/fb_bpp/fb_Bpl 24/32/5120
09/04/2019 15:06:40
09/04/2019 15:06:40 X display :0 is 32bpp depth=24 true color
09/04/2019 15:06:40
09/04/2019 15:06:40 Listening for VNC connections on TCP port 5900
09/04/2019 15:06:40
09/04/2019 15:06:40 Xinerama is present and active (e.g. multi-head).
09/04/2019 15:06:40 Xinerama: number of sub-screens: 1
09/04/2019 15:06:40 Xinerama: no blackouts needed (only one sub-screen)
09/04/2019 15:06:40
09/04/2019 15:06:40 fb read rate: 1631 MB/sec
09/04/2019 15:06:40 fast read: reset -wait ms to: 10
09/04/2019 15:06:40 fast read: reset -defer ms to: 10
09/04/2019 15:06:40 The X server says there are 10 mouse buttons.
09/04/2019 15:06:40 screen setup finished.
09/04/2019 15:06:40
The VNC desktop is: a507a8a5e8b4:0
0

Have you tried the x11vnc '-ncache' VNC client-side pixel caching feature yet?
The scheme stores pixel data offscreen on the VNC viewer side for faster
retrieval. It should work with any VNC viewer. Try it by running:
x11vnc -ncache 10 ...
One can also add -ncache_cr for smooth 'copyrect' window motion.
More info: http://www.karlrunge.com/x11vnc/faq.html#faq-client-caching
[services.d] starting app...
[app] starting MakeMKV...
[services.d] done.
Fontconfig warning: line 5: unknown element "its:rules"
Fontconfig warning: line 6: unknown element "its:translateRule"
Fontconfig error: line 6: invalid attribute 'translate'
Fontconfig error: line 6: invalid attribute 'selector'
Fontconfig error: line 7: invalid attribute 'xmlns:its'
Fontconfig error: line 7: invalid attribute 'version'
Fontconfig warning: line 9: unknown element "description"
Fontconfig warning: "/etc/fonts/conf.d/10-hinting-slight.conf", line 4: unknown element "its:rules"
Fontconfig warning: "/etc/fonts/conf.d/10-hinting-slight.conf", line 5: unknown element "its:translateRule"
Fontconfig error: "/etc/fonts/conf.d/10-hinting-slight.conf", line 5: invalid attribute 'translate'
Fontconfig error: "/etc/fonts/conf.d/10-hinting-slight.conf", line 5: invalid attribute 'selector'
Fontconfig error: "/etc/fonts/conf.d/10-hinting-slight.conf", line 6: invalid attribute 'xmlns:its'
Fontconfig error: "/etc/fonts/conf.d/10-hinting-slight.conf", line 6: invalid attribute 'version'
Fontconfig warning: "/etc/fonts/conf.d/10-hinting-slight.conf", line 8: unknown element "description"
Fontconfig warning: "/etc/fonts/conf.d/10-scale-bitmap-fonts.conf", line 4: unknown element "its:rules"
Fontconfig warning: "/etc/fonts/conf.d/10-scale-bitmap-fonts.conf", line 5: unknown element "its:translateRule"
Fontconfig error: "/etc/fonts/conf.d/10-scale-bitmap-fonts.conf", line 5: invalid attribute 'translate'
Fontconfig error: "/etc/fonts/conf.d/10-scale-bitmap-fonts.conf", line 5: invalid attribute 'selector'
Fontconfig error: "/etc/fonts/conf.d/10-scale-bitmap-fonts.conf", line 6: invalid attribute 'xmlns:its'
Fontconfig error: "/etc/fonts/conf.d/10-scale-bitmap-fonts.conf", line 6: invalid attribute 'version'
Fontconfig warning: "/etc/fonts/conf.d/10-scale-bitmap-fonts.conf", line 8: unknown element "description"
Fontconfig warning: "/etc/fonts/conf.d/20-unhint-small-vera.conf", line 4: unknown element "its:rules"
Fontconfig warning: "/etc/fonts/conf.d/20-unhint-small-vera.conf", line 5: unknown element "its:translateRule"
Fontconfig error: "/etc/fonts/conf.d/20-unhint-small-vera.conf", line 5: invalid attribute 'translate'
Fontconfig error: "/etc/fonts/conf.d/20-unhint-small-vera.conf", line 5: invalid attribute 'selector'
Fontconfig error: "/etc/fonts/conf.d/20-unhint-small-vera.conf", line 6: invalid attribute 'xmlns:its'
Fontconfig error: "/etc/fonts/conf.d/20-unhint-small-vera.conf", line 6: invalid attribute 'version'
Fontconfig warning: "/etc/fonts/conf.d/20-unhint-small-vera.conf", line 8: unknown element "description"
Fontconfig warning: "/etc/fonts/conf.d/30-metric-aliases.conf", line 4: unknown element "its:rules"
Fontconfig warning: "/etc/fonts/conf.d/30-metric-aliases.conf", line 5: unknown element "its:translateRule"
Fontconfig error: "/etc/fonts/conf.d/30-metric-aliases.conf", line 5: invalid attribute 'translate'
Fontconfig error: "/etc/fonts/conf.d/30-metric-aliases.conf", line 5: invalid attribute 'selector'
Fontconfig error: "/etc/fonts/conf.d/30-metric-aliases.conf", line 6: invalid attribute 'xmlns:its'
Fontconfig error: "/etc/fonts/conf.d/30-metric-aliases.conf", line 6: invalid attribute 'version'
Fontconfig warning: "/etc/fonts/conf.d/30-metric-aliases.conf", line 8: unknown element "description"
Fontconfig warning: "/etc/fonts/conf.d/40-nonlatin.conf", line 4: unknown element "its:rules"
Fontconfig warning: "/etc/fonts/conf.d/40-nonlatin.conf", line 5: unknown element "its:translateRule"
Fontconfig error: "/etc/fonts/conf.d/40-nonlatin.conf", line 5: invalid attribute 'translate'
Fontconfig error: "/etc/fonts/conf.d/40-nonlatin.conf", line 5: invalid attribute 'selector'
Fontconfig error: "/etc/fonts/conf.d/40-nonlatin.conf", line 6: invalid attribute 'xmlns:its'
Fontconfig error: "/etc/fonts/conf.d/40-nonlatin.conf", line 6: invalid attribute 'version'
Fontconfig warning: "/etc/fonts/conf.d/40-nonlatin.conf", line 8: unknown element "description"
Fontconfig warning: "/etc/fonts/conf.d/45-generic.conf", line 4: unknown element "its:rules"
Fontconfig warning: "/etc/fonts/conf.d/45-generic.conf", line 5: unknown element "its:translateRule"
Fontconfig error: "/etc/fonts/conf.d/45-generic.conf", line 5: invalid attribute 'translate'
Fontconfig error: "/etc/fonts/conf.d/45-generic.conf", line 5: invalid attribute 'selector'
Fontconfig error: "/etc/fonts/conf.d/45-generic.conf", line 6: invalid attribute 'xmlns:its'
Fontconfig error: "/etc/fonts/conf.d/45-generic.conf", line 6: invalid attribute 'version'
Fontconfig warning: "/etc/fonts/conf.d/45-generic.conf", line 8: unknown element "description"
Fontconfig warning: "/etc/fonts/conf.d/45-latin.conf", line 4: unknown element "its:rules"
Fontconfig warning: "/etc/fonts/conf.d/45-latin.conf", line 5: unknown element "its:translateRule"
Fontconfig error: "/etc/fonts/conf.d/45-latin.conf", line 5: invalid attribute 'translate'
Fontconfig error: "/etc/fonts/conf.d/45-latin.conf", line 5: invalid attribute 'selector'
Fontconfig error: "/etc/fonts/conf.d/45-latin.conf", line 6: invalid attribute 'xmlns:its'
Fontconfig error: "/etc/fonts/conf.d/45-latin.conf", line 6: invalid attribute 'version'
Fontconfig warning: "/etc/fonts/conf.d/45-latin.conf", line 8: unknown element "description"
Fontconfig warning: "/etc/fonts/conf.d/49-sansserif.conf", line 4: unknown element "its:rules"
Fontconfig warning: "/etc/fonts/conf.d/49-sansserif.conf", line 5: unknown element "its:translateRule"
Fontconfig error: "/etc/fonts/conf.d/49-sansserif.conf", line 5: invalid attribute 'translate'
Fontconfig error: "/etc/fonts/conf.d/49-sansserif.conf", line 5: invalid attribute 'selector'
Fontconfig error: "/etc/fonts/conf.d/49-sansserif.conf", line 6: invalid attribute 'xmlns:its'
Fontconfig error: "/etc/fonts/conf.d/49-sansserif.conf", line 6: invalid attribute 'version'
Fontconfig warning: "/etc/fonts/conf.d/49-sansserif.conf", line 8: unknown element "description"
Fontconfig warning: "/etc/fonts/conf.d/50-user.conf", line 4: unknown element "its:rules"
Fontconfig warning: "/etc/fonts/conf.d/50-user.conf", line 5: unknown element "its:translateRule"
Fontconfig error: "/etc/fonts/conf.d/50-user.conf", line 5: invalid attribute 'translate'
Fontconfig error: "/etc/fonts/conf.d/50-user.conf", line 5: invalid attribute 'selector'
Fontconfig error: "/etc/fonts/conf.d/50-user.conf", line 6: invalid attribute 'xmlns:its'
Fontconfig error: "/etc/fonts/conf.d/50-user.conf", line 6: invalid attribute 'version'
Fontconfig warning: "/etc/fonts/conf.d/50-user.conf", line 8: unknown element "description"
Fontconfig warning: "/etc/fonts/conf.d/51-local.conf", line 4: unknown element "its:rules"
Fontconfig warning: "/etc/fonts/conf.d/51-local.conf", line 5: unknown element "its:translateRule"
Fontconfig error: "/etc/fonts/conf.d/51-local.conf", line 5: invalid attribute 'translate'
Fontconfig error: "/etc/fonts/conf.d/51-local.conf", line 5: invalid attribute 'selector'
Fontconfig error: "/etc/fonts/conf.d/51-local.conf", line 6: invalid attribute 'xmlns:its'
Fontconfig error: "/etc/fonts/conf.d/51-local.conf", line 6: invalid attribute 'version'
Fontconfig warning: "/etc/fonts/conf.d/51-local.conf", line 8: unknown element "description"
Fontconfig warning: "/etc/fonts/conf.d/60-generic.conf", line 4: unknown element "its:rules"
Fontconfig warning: "/etc/fonts/conf.d/60-generic.conf", line 5: unknown element "its:translateRule"
Fontconfig error: "/etc/fonts/conf.d/60-generic.conf", line 5: invalid attribute 'translate'
Fontconfig error: "/etc/fonts/conf.d/60-generic.conf", line 5: invalid attribute 'selector'
Fontconfig error: "/etc/fonts/conf.d/60-generic.conf", line 6: invalid attribute 'xmlns:its'
Fontconfig error: "/etc/fonts/conf.d/60-generic.conf", line 6: invalid attribute 'version'
Fontconfig warning: "/etc/fonts/conf.d/60-generic.conf", line 8: unknown element "description"
Fontconfig warning: "/etc/fonts/conf.d/60-latin.conf", line 4: unknown element "its:rules"
Fontconfig warning: "/etc/fonts/conf.d/60-latin.conf", line 5: unknown element "its:translateRule"
Fontconfig error: "/etc/fonts/conf.d/60-latin.conf", line 5: invalid attribute 'translate'
Fontconfig error: "/etc/fonts/conf.d/60-latin.conf", line 5: invalid attribute 'selector'
Fontconfig error: "/etc/fonts/conf.d/60-latin.conf", line 6: invalid attribute 'xmlns:its'
Fontconfig error: "/etc/fonts/conf.d/60-latin.conf", line 6: invalid attribute 'version'
Fontconfig warning: "/etc/fonts/conf.d/60-latin.conf", line 8: unknown element "description"
Fontconfig warning: "/etc/fonts/conf.d/65-fonts-persian.conf", line 34: unknown element "its:rules"
Fontconfig warning: "/etc/fonts/conf.d/65-fonts-persian.conf", line 35: unknown element "its:translateRule"
Fontconfig error: "/etc/fonts/conf.d/65-fonts-persian.conf", line 35: invalid attribute 'translate'
Fontconfig error: "/etc/fonts/conf.d/65-fonts-persian.conf", line 35: invalid attribute 'selector'
Fontconfig error: "/etc/fonts/conf.d/65-fonts-persian.conf", line 36: invalid attribute 'xmlns:its'
Fontconfig error: "/etc/fonts/conf.d/65-fonts-persian.conf", line 36: invalid attribute 'version'
Fontconfig warning: "/etc/fonts/conf.d/65-nonlatin.conf", line 4: unknown element "its:rules"
Fontconfig warning: "/etc/fonts/conf.d/65-nonlatin.conf", line 5: unknown element "its:translateRule"
Fontconfig error: "/etc/fonts/conf.d/65-nonlatin.conf", line 5: invalid attribute 'translate'
Fontconfig error: "/etc/fonts/conf.d/65-nonlatin.conf", line 5: invalid attribute 'selector'
Fontconfig error: "/etc/fonts/conf.d/65-nonlatin.conf", line 6: invalid attribute 'xmlns:its'
Fontconfig error: "/etc/fonts/conf.d/65-nonlatin.conf", line 6: invalid attribute 'version'
Fontconfig warning: "/etc/fonts/conf.d/65-nonlatin.conf", line 8: unknown element "description"
Fontconfig warning: "/etc/fonts/conf.d/69-unifont.conf", line 4: unknown element "its:rules"
Fontconfig warning: "/etc/fonts/conf.d/69-unifont.conf", line 5: unknown element "its:translateRule"
Fontconfig error: "/etc/fonts/conf.d/69-unifont.conf", line 5: invalid attribute 'translate'
Fontconfig error: "/etc/fonts/conf.d/69-unifont.conf", line 5: invalid attribute 'selector'
Fontconfig error: "/etc/fonts/conf.d/69-unifont.conf", line 6: invalid attribute 'xmlns:its'
Fontconfig error: "/etc/fonts/conf.d/69-unifont.conf", line 6: invalid attribute 'version'
Fontconfig warning: "/etc/fonts/conf.d/80-delicious.conf", line 4: unknown element "its:rules"
Fontconfig warning: "/etc/fonts/conf.d/80-delicious.conf", line 5: unknown element "its:translateRule"
Fontconfig error: "/etc/fonts/conf.d/80-delicious.conf", line 5: invalid attribute 'translate'
Fontconfig error: "/etc/fonts/conf.d/80-delicious.conf", line 5: invalid attribute 'selector'
Fontconfig error: "/etc/fonts/conf.d/80-delicious.conf", line 6: invalid attribute 'xmlns:its'
Fontconfig error: "/etc/fonts/conf.d/80-delicious.conf", line 6: invalid attribute 'version'
Fontconfig warning: "/etc/fonts/conf.d/90-synthetic.conf", line 4: unknown element "its:rules"
Fontconfig warning: "/etc/fonts/conf.d/90-synthetic.conf", line 5: unknown element "its:translateRule"
Fontconfig error: "/etc/fonts/conf.d/90-synthetic.conf", line 5: invalid attribute 'translate'
Fontconfig error: "/etc/fonts/conf.d/90-synthetic.conf", line 5: invalid attribute 'selector'
Fontconfig error: "/etc/fonts/conf.d/90-synthetic.conf", line 6: invalid attribute 'xmlns:its'
Fontconfig error: "/etc/fonts/conf.d/90-synthetic.conf", line 6: invalid attribute 'version'
Fontconfig error: Cannot load default config file
info string reallocated, size=560
09/04/2019 15:25:20 Got connection from client <IP_ADDRESS>
09/04/2019 15:25:20 other clients:
09/04/2019 15:25:20 Got 'ws' WebSockets handshake
09/04/2019 15:25:20 Got protocol: binary
09/04/2019 15:25:20 - webSocketsHandshake: using binary/raw encoding
09/04/2019 15:25:20 - WebSockets client version hybi-13
09/04/2019 15:25:20 Disabled X server key autorepeat.
09/04/2019 15:25:20 to force back on run: 'xset r on' (3 times)
09/04/2019 15:25:20 incr accepted_client=1 for <IP_ADDRESS>:53104 sock=10
09/04/2019 15:25:20 Client Protocol Version 3.8
09/04/2019 15:25:20 Protocol version sent 3.8, using 3.8
09/04/2019 15:25:20 rfbProcessClientSecurityType: executing handler for type 1
09/04/2019 15:25:20 rfbProcessClientSecurityType: returning securityResult for client rfb version >= 3.8
09/04/2019 15:25:21 Pixel format for client <IP_ADDRESS>:
09/04/2019 15:25:21 32 bpp, depth 24, little endian
09/04/2019 15:25:21 true colour: max r 255 g 255 b 255, shift r 16 g 8 b 0
09/04/2019 15:25:21 no translation needed
09/04/2019 15:25:21 Enabling NewFBSize protocol extension for client <IP_ADDRESS>
09/04/2019 15:25:21 Enabling full-color cursor updates for client <IP_ADDRESS>
09/04/2019 15:25:21 Using image quality level 6 for client <IP_ADDRESS>
09/04/2019 15:25:21 Using JPEG subsampling 0, Q79 for client <IP_ADDRESS>
09/04/2019 15:25:21 Using compression level 9 for client <IP_ADDRESS>
09/04/2019 15:25:21 Enabling LastRect protocol extension for client <IP_ADDRESS>
09/04/2019 15:25:21 rfbProcessClientNormalMessage: ignoring unsupported encoding type Enc(0xFFFFFECC)
09/04/2019 15:25:21 Using tight encoding for client <IP_ADDRESS>
09/04/2019 15:25:21 client_set_net: <IP_ADDRESS> 0.0000
09/04/2019 15:25:21 created xdamage object: 0x20002c
09/04/2019 15:25:30 created selwin: 0x20002d
09/04/2019 15:25:30 called initialize_xfixes()
09/04/2019 15:25:33 copy_tiles: allocating first_line at size 41
09/04/2019 15:25:40 client 1 network rate 1734.9 KB/sec (43837.0 eff KB/sec)
09/04/2019 15:25:40 client 1 latency: 1.2 ms
09/04/2019 15:25:40 dt1: 0.0041, dt2: 0.0129 dt3: 0.0012 bytes: 28586
09/04/2019 15:25:40 link_rate: LR_LAN - 1 ms, 1734 KB/s
09/04/2019 15:25:52 got closure, reason 1001
09/04/2019 15:25:52 rfbProcessClientNormalMessage: read: Connection reset by peer
09/04/2019 15:25:52 client_count: 0
09/04/2019 15:25:52 Restored X server key autorepeat to: 1
09/04/2019 15:25:52 Client <IP_ADDRESS> gone
09/04/2019 15:25:52 Statistics events Transmit/ RawEquiv ( saved)
09/04/2019 15:25:52 ServerCutText : 1 | 8/ 8 ( 0.0%)
09/04/2019 15:25:52 FramebufferUpdate : 28 | 0/ 0 ( 0.0%)
09/04/2019 15:25:52 LastRect : 16 | 192/ 192 ( 0.0%)
09/04/2019 15:25:52 tight : 212 | 182536/ 7325172 ( 97.5%)
09/04/2019 15:25:52 RichCursor : 1 | 1374/ 1374 ( 0.0%)
09/04/2019 15:25:52 TOTALS : 258 | 184110/ 7326746 ( 97.5%)
09/04/2019 15:25:52 Statistics events Received/ RawEquiv ( saved)
09/04/2019 15:25:52 PointerEvent : 693 | 4158/ 4158 ( 0.0%)
09/04/2019 15:25:52 FramebufferUpdate : 29 | 290/ 290 ( 0.0%)
09/04/2019 15:25:52 SetEncodings : 1 | 56/ 56 ( 0.0%)
09/04/2019 15:25:52 SetPixelFormat : 1 | 20/ 20 ( 0.0%)
09/04/2019 15:25:52 TOTALS : 724 | 4524/ 4524 ( 0.0%)
09/04/2019 15:25:52 destroyed xdamage object: 0x20002c

Well it turns out it was something obvious. The auto-rip feature was enabled and I was trying to start the rip via the webui at the same time. If I simply insert a disk and do nothing it works quickly, finishing a blu-ray in around 10min.  I noticed after inserting a disk a folder showed up before I selected anything and there was drive activity. I just wasn't paying attention previously.
I'm still curious when the fontconfig error is if anyone has any information.

The fontconfig errors are now fixed with recent image versions.


What happens when the world gets upgraded?

Is this the same datapack that Hermitcraft is using?
What happens when 1.21 comes out?  Will the world need to be re-created, or should things upgrade, or do you need to manually remove the datapack?

It should work without problems. When 1.21 is released you can remove the data pack, and the world will upgrade without problems. But of course take a backup of your world first.


fix: recalculate total leave on all FormR Part Bs

The totalLeave field was not correctly populated for some forms,
create a migrator to recalculate the field any record where it is zero.
TIS21-2791

Converted to draft while we consider how to update the S3 copies of the forms.


feat: Adding more details at footer

Adding more details at footer #398
Issue No. :#398
I have provided specific  domains  in footer so the person who wants to know about a specific domain will go to that section and will easily get to know about it.
so i have added these extra details:

So like if a person goes to domain like adventure places he will see this page

and after clicking book now will see this:

i have keep the styling same so format remains same as it will be proffesional
Type of PR
->its a feature enhancement
__
Checklist:

[X] I have performed a self-review of my code.
[X] I have commented my code, particularly in hard-to-understand areas.
[X] I have tested the changes thoroughly before submitting this pull request.
[X] I have provided relevant issue numbers, screenshots, and videos after making the changes.
[X] I have gone through the  contributing.md file before contributing

i have added some new files like:

##Are you contributing under any Open-source programme?
Yes I am Contributing in Gssoc'24
I hope you like it. If you need any changes do let me know..

@apu52 Plz review it. Hope you like it. If you need any changes do let me know.


Sink linker directive computation into IRGen

With an inverted pipeline, IRGen needs to be able to compute the linker directives itself, so sink it down such that it can be computed by the IRGenDescriptor.

https://github.com/apple/llvm-project/pull/1447
@swift-ci please test

https://github.com/apple/llvm-project/pull/1447
@swift-ci please test


Swift 2.3 tag

Please add tag 1.0.8 to the commit d9367fbb83847749bfb7137c225e9b362e9d690b, so users may use your framework with Swift 2.3 via that line in podfile:
pod 'EPContactsPicker', '~> 1.0.8'

In that commit I've already added version 1.0.8 to a podspec.

Can we please do this? Not all pods are updated for swift 3 but almost all of them are 2.3 compliant. This would unblock me.

@yashh, you can use as temporary solution:
pod 'EPContactsPicker', :git => 'https://github.com/ipraba/EPContactsPicker/', :commit => 'd9367fbb83847749bfb7137c225e9b362e9d690b'

This project will support swift 2.


Update forwarding-refs-fancy-button.js

This should be FancyButton or else it wouldn't make sense

Great 👍🏽


Make it more obvious which tasks to run in which order

There are a lots of different Ansible tasks in this repository. I propose that the major tasks should be prepended with their order of execution number. e.g provision_machine.yaml should be renamed to 01_provision_machine.yaml.
For all other yaml files where the execution order does not  matter could potentially receive the same prepended index number.

https://github.com/ethereum/consensus-deployment-ansible/issues/11


heart color selector

user has a color picker to pick the primary heart color
hover color is logically computed from that automatically

make the heart itself a color picker, so it has a cursor over it
or, make the heart itself a "random color toggler" not a picker


chore: lets reduce the bot token health period to reduce api calls an…

…d risk of being rate limitted

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To complete the pull request process, please assign rawlingsj
You can assign the PR to them by writing /assign @rawlingsj in a comment when ready.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

/approve

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: rawlingsj
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [rawlingsj]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


Fresh install of Lens has blank screen

Describe the bug
Lens displays a blank screen upon starting up.
To Reproduce
Steps to reproduce the behavior:

Install the latest Lens DMG file
Attempt to start Lens

Expected behavior
I expect Lens to work.
Screenshots

Environment (please complete the following information):

Lens Version: 5.1.3-latest.20210722.1
OS: OSX
Installation method: DMG

Logs:
When you run the application executable from command line you will see some logging output. Please paste them here:
Running 2.1.0-beta.4 migration for /Users/username/Library/Application Support/Lens/lens-user-store.json
Lens.js:47437 Running 5.0.0-alpha.3 migration for /Users/username/Library/Application Support/Lens/lens-user-store.json
Lens.js:47437 Running 5.0.3-beta.1 migration for /Users/username/Library/Application Support/Lens/lens-user-store.json
Lens.js:359583 Uncaught (in promise) Error: Something went wrong during the migration! Changes applied to the store until this failed migration will be restored. TypeError: Cannot read property 'getPath' of undefined
    at Conf._migrate (Lens.js:359583)
    at new Conf (Lens.js:359320)
    at user_store_UserStore.load (Lens.js:59009)
    at new user_store_UserStore (Lens.js:39761)
    at Function.createInstance (Lens.js:128609)
    at bootstrap (Lens.js:578488)

Additional context
N/A

I was able to resolve this by uninstalling Lens, deleting /Users/username/Library/Application Support/Lens/ and then re-installing Lens. The first time I started the app it had the same issue but upon restarting, the issue was resolved.


Add 'transparent' and 'inherit' to background-color

@renatoi @src-code I just added missing values for background-color.

CLA is valid!

Coverage remained the same at 100.0% when pulling 94c571fec3696e584a15dfa4966448f59d9c80a3 on add-values-for-background-color into cadfcb8dc1c9f9aeb0d21e81691538a9e3d2c95b on master.

Coverage remained the same at 100.0% when pulling 94c571fec3696e584a15dfa4966448f59d9c80a3 on add-values-for-background-color into cadfcb8dc1c9f9aeb0d21e81691538a9e3d2c95b on master.


CPU temp increase while enabled

CPU Temp Increases
With this plugin enabled, inside and outside of deep dungeon causes a huge temperature spike on my system. I have a 7700x and enabled cpu temp is 72-80c, once I disable the plugin the cpu temp drops to 54c.
Not a developer so not entirely sure but it's entirely caused by this plugin. I do like the concept of this plugin though. Currently cannot use it in it's current state.
Where?
Everywhere inside and outside deep dungeons, as long as the plugin is enabled.

You got me with a nice beginner-style oopsie :D
Pushed a fix - it should run with much less cpu load now


Unsupported git flow version

On mac

[ ] bug report

Did you check this guide? https://github.com/OpherV/gitflow4idea/blob/develop/GITFLOW_VERSION.md
Ran into the same problem a few minutes ago but the plugin hints you into the right direction

gitflow-avh  is archived repository
https://github.com/CJ-Systems/gitflow-cjs is maintained, but version is incompatible with this plugin
> git flow version
2.2.0 (CJS Edition)


[staging] {cargo-c,rav1e}: switch to fetchCrate

Motivation for this change
this avoids unnecessary vendors of Cargo.lock and just overall looks cleaner
might be able to target master if there isn't too much rebuilds
Probably the end of the triology: #140864, #140913
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
21.11 Release Notes (or backporting 21.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

@ofborg build gimp

I targetted the wrong branch ....


Possible documentation problem about Visual Studio Tools for Unity maintaining the project reference

Sorry for asking the question here but I don't know where to ask it.
In this page, it says that we can add a new managed DLL project to the generated Unity solution and Visual Studio Tools for Unity maintains the project reference, even if it has to regenerate the project and solution files again.
How can it maintain the project reference when the solution file doesn't exist or gets deleted?
Because, normally, the solution file generated by Unity or Visual Studio Tools for Unity is not added to the source control and it gets generated from scratch every time the project is downloaded to a new computer or when the developer decides to delete the existing project folder and re-download it for any reason.
Is this document promising something which Visual Studio Tools for Unity can't actually do?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: f5c0874b-2fb8-85a2-6e3a-2dc3c0ae82f2
Version Independent ID: a02a434f-a213-ee56-5dd8-ecaf60e2dac3
Content: Using Visual Studio Tools for Unity
Content Source: gamedev/unity/get-started/using-visual-studio-tools-for-unity.md
Product: visual-studio-dev16
Technology: vs-unity-tools
GitHub Login: @therealjohn
Microsoft Alias: johmil

@XtroTheArctic -- Onur, thank you for your thorough feedback.  Please consider these resources:

Visual Studio on Microsoft Q&A
Visual Studio MSDN Forum
Stack Overflow
Reddit

@therealjohn -- John, please look into this potential documentation issue.

Is this document promising something which Visual Studio Tools for Unity can't actually do?

No, the document is accurate. Our tooling helps make this possible, regardless of the regeneration of these files. If you run into an issue with it working, it's a bug. In that case, please let us know and we'll fix it!

@WilliamAntonRohm This can be closed.

#please-close


[SPARK-9165][SQL] codegen for CreateArray, CreateStruct and CreateNamedStruct

JIRA: https://issues.apache.org/jira/browse/SPARK-9165

Can one of the admins verify this patch?

Can one of the admins verify this patch?

/cc @rxin

Test build #1133 has started for   PR 7537 at commit 39cefb8.

Test build #1133 has finished for   PR 7537 at commit 39cefb8.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Thanks @marmbrus , I would update my code soon.

Test build #1142 has started for   PR 7537 at commit 39cefb8.

Test build #1142 has finished for   PR 7537 at commit 39cefb8.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
case class CreateArray(children: Seq[Expression]) extends Expression
case class CreateStruct(children: Seq[Expression]) extends Expression
case class CreateNamedStruct(children: Seq[Expression]) extends Expression

Thanks!  Merged to master.


A request to review "list-t-libcurl"

I've released a library, which abstracts over "curlhs" with a streaming interface. Could you please review it on the subject of whether I handle and release the resources properly?
https://github.com/nikita-volkov/list-t-libcurl/blob/master/library/ListT/Libcurl.hs

Firstly, I'm glad to see my library in use. Secondly, I'm not as advanced Haskeller, as I would like to be. My understanding of all those functional abstractions, interfaces etc. is far from great. But if I read your code correctly, the use of 'curlhs' seems correct. Despite this, I would like to draw your attention to two things.
First in the code:
acquire = do
  h <- C.curl_easy_init
  C.curl_easy_setopt h [C.CURLOPT_FAILONERROR True]
  return h

curl_easy_setopt may throw an exception. It should never happen in the above case (it's only a flag set), but with some other libcurl options the exceptions are possible (not builtin feature / not enabled etc.) and acquired handle will leak unreleased then. With more advanced settings consider this:
acquire = do
  bracketOnError (C.curl_easy_init) (C.curl_easy_cleanup) $ \h -> do
    C.curl_easy_setopt h [ ... ]
    return h

Second, more important remark concerns the withlib placement. The short explanation is this: withlib should be placed into main, directly or as close as possible, wrapping all uses of libcurl.
But why? withlib performs two things:

It loads/unloads libcurl shared object (libcurl.dll etc.). I don't know what is the performance cost of that operation, but I think it's not cheap enough, to loose it for every session. If that aspect is not important then of course it's possible to load/unload on demand, but look at the second point.

It does global initialization and cleanup of the library (curl_global_init / curl_global_cleanup). As stated in the documentation at http://curl.haxx.se/libcurl/c/libcurl.html and http://curl.haxx.se/libcurl/c/curl_global_init.html this operation is mandatory and it is not thread safe in addition. It probably concerns not the libcurl itself, but something deeper like ssl engine. I don't know for sure. But I think that in multithreaded Haskell it is highly possible that something may go wrong.

This defers the responsibility of using withlib to the user of your library or even later up to the application programmer. I know that it is not in the Haskell style, but it is the cost of using the explicit linking method and probably more the cost of using the foreign library at all. Haskell 'network' package has its 'withSocketsDo', 'curl' package its 'withCurlDo'. Many C libraries requires some form of global init/exit. I've been experimenting with a few (sdl, glfw, glut, fmod), and I don't have a good solution now.

Thanks for your review!
Concerning withlib, forcing the user to place it in main really does a great damage to the usability of the library. Even in the mentioned "network" package they're finally getting rid of that annoying withSocketsDo AFAIK.
I think I have an idea of what we can do about it here. One only needs to load the "libcurl" shared object once per the whole application, right? So how about doing it globally with unsafePerformIO the same way I'm building a global pool in the referred module?
I can even do the same in my library: https://github.com/nikita-volkov/list-t-libcurl/commit/ed0ce643762d0fd81262df9cd95aaa180ed1dc63. However, I suspect that unless loadlib is multicall-checked, this might cause conflicts with other libraries, which use "curlhs". It'd be trivial to update loadlib to fix such issue however.

Your solution with placing loadlib behind unsafePerformIO should be sufficient for most use cases. loadlib is multicall-checked, so it's not an issue to call it multiple times (even concurrently). And in case when there is no corresponding freelib, the library is freed by the operating system on program exit. Considering only loading/unloading of the library everything should be fine. So far so good.
My doubts comes rather from the libcurl initialization specifics. From the docs that I linked before:

It isn't actually required that the functions be called at the beginning and end of the program -- that's just usually the easiest way to do it. It is required that the functions be called when no other thread in the program is running.

Computation wrapped in unsafePerformIO may be performed at any time. Is that such relevant in our case? I think it may be, but please read carefully the libcurl docs on the subject and decide for yourself.

Okay. Thanks! You've been very helpful!

BTW, what do you think about using unsafePerformIO to loadlib behind the scenes in the "curlhs" itself?

I was thinking about this some time ago, and I decided finally that I want to give some control of the loading process to users of the library. For now, I don't want to change this. Maybe someday.
I think that by making the loader implicit (load on first use), the curl_global_init / curl_global_cleanup API should be exposed. So the situation will not change.

A problem with the current API is that it makes the dependencies of "curlhs" likely to conflict. E.g., if my library gets used with some other library, which calls "freelib" or "withlib" on a concurrent thread, I expect some nasty and hardly trackable failures to appear.

The "curlhs" loader uses reference counting method, with a counter protected by an MVar. There will be no conflicts as long as anyone uses the loader correctly, and by the correct use I mean just that no one calls freelib without using loadlib at first. Try this example:
module Main where

import Control.Concurrent
import Control.Exception
import Network.CURL720

curlGET xs =
  bracket (curl_easy_init) (curl_easy_cleanup) $ \curl -> do
    curl_easy_setopt curl [ CURLOPT_URL ("http://httpbin.org/get?id=" ++ xs) ]
    putStrLn ("Start #" ++ xs) >> curl_easy_perform curl

main = do
  withlib CURL720 (curlGET "1")
  withlib CURL720 $ do
    forkIO $ withlib CURL730 (curlGET "2")
    forkIO $ withlib CURL720 (curlGET "3")
  withlib CURL730 $ do
    forkIO $ withlib CURL730 (curlGET "4")
    forkIO $ withlib CURL720 (curlGET "5")
  threadDelay 5000000

But ok, let's forget about loadlib, freelib and withlib trio. Let's assume that there are only plain, ready to use "libcurl" calls, and among them mentioned curl_global_init and curl_global_cleanup with their requirements about thread safety. Does it really change something? Does it really help you with your library? Correct me if I'm wrong, but I don't think so.
The "libcurl" team introduced this global initialization for some reason, and there is nothing we can do about that. One more citation from the "libcurl" docs:

There is a failsafe in libcurl that makes it usable in simple situations without you having to worry about the global constant environment at all: curl_easy_init sets up the environment itself if it hasn't been done yet. The resources it acquires to do so get released by the operating system automatically when the program exits.
This failsafe feature exists mainly for backward compatibility because there was a time when the global functions didn't exist. Because it is sufficient only in the simplest of programs, it is not recommended for any program to rely on it.

How can I help you more?

The "curlhs" loader uses reference counting method, with a counter protected by an MVar.

Good. I was going to suggest that as an alternative.

Let's assume that there are only plain, ready to use "libcurl" calls, and among them mentioned curl_global_init and curl_global_cleanup with their requirements about thread safety. Does it really change something? Does it really help you with your library? Correct me if I'm wrong, but I don't think so.

Well, I hope you already understand my concern about the possible conflicts triggered by the shared state. I'm worried about some code absolutely unrelated to my library being able to bring it into a state of failure. This is a common problem in imperative languages, so their library authors don't worry about that much, but in Haskell such a behaviour of a library will simply catch users off guard, causing an extreme frustration.
With that said, I'm now concerned about what's gonna happen if somebody calls curl_global_cleanup concurrently, while my library is still working?

The "libcurl" team introduced this global initialization for some reason, and there is nothing we can do about that.

Well, yes, but I doubt they had Haskell dependencies in mind. The library is written for the C language, which generally practices very different approaches, which most of us consider suboptimal at very least. So, IMO, even the binding libraries need to provide the bare minimum abstractions, which ensure a certain safety level. Don't get me wrong, I'm not criticising, I'm just making an argument that it's Okay for a binding library to introduce certain tweaks over the wrapped library.

So, to make it more precise, I'd like to rephrase my suggestion to the following: hide all the functions, which deal with the shared state. E.g., perform loadlib and curl_global_init behind the scenes and never use curl_global_cleanup and freelib at all.

I think you know how reference counting works, and therefore, that if there are no buggy parts in the program, then in case of your library, the "libcurl" will never be unloaded, even if another library will concurrently perform many load/free calls. I guess that your concerns comes down rather to a buggy situations, and that you suggest to try to eliminate the possibility of writing bad code by the users of "curlhs". For example the situation where in some part of the program someone calls freelib (and therefore curl_global_cleanup) to many times (more frees than loads). This is possible of course. In such case your library may fail unexpected (like the whole program). It's true.
Global shared state is always dangerous. And in the environment like Haskell, where users usually do not have to worry about such things, it is especially touchy subject. I've tried to make this part of the library as safe as possible, but not more.
It is possible to perform loadlib and curl_global_init behind the scenes like you sugest. The process of load/init will be performed on first use then and it will be irreversible, so no one can do something wrong. Almost... what about the "libcurl" requirement that this should be done on the main thread and when there is only that one thread in the program? From docs about curl_global_init:

This function is not thread safe. You must not call it when any other thread in the program (i.e. a thread sharing the same memory) is running. This doesn't just mean no other thread that is using libcurl. Because curl_global_init calls functions of other libraries that are similarly thread unsafe, it could conflict with any other thread that uses these other libraries.

Should I ignore that? I wouldn't like to give to users a sense of safety, when I can not guarantee it.

Should I ignore that? I wouldn't like to give to users a sense of safety, when I can not guarantee it.

Okay. I get your concern. But even then it's still possible to protect from improper freelib usage by only exposing loadlib, which instead of reference counting will only just whether it's been called before to ensure that it's only called once. You can suggest to users to call it in the main thread in the docs then.

Yes, this may be the solution. Thanks for the discussion.

Thank you too.

I think that this patch may interest you: 3babfea992712ec92df5b7fcd4b62147ad30bab5. Maybe not the details, but the overall idea: linking and initializing is performed on the lower level now. loadlib/freelib still exists for compatibility reasons, but they does nothing and I'll probably remove them entirely in the future versions.
All on hackage now (0.1.6).

@kkardzis Thanks for the update Krzysztof!


v0.1.5 Deployment

use tofile instead of tobytes for numpy 1.7 compatibility
fix minor bug in array errors
remove 'inc' from '3point Science'

Coverage remained the same at 87.545% when pulling a7a7422caf6c9c93c0149979df62291426777fb8 on dev into 9c9a5b0b7046029081fc4488b9eed2ea1cb6207d on master.


refactor LDAP driver in userprovider

Move some common code into helper functions. Also don't assume that the
userid in reva matches the user's username. So add an additional LDAP
lookup to get the username before looking up groupmembership.

This has landed in edge a while ago. I am going to close this for now. (Trying to bring back the edge an master branch in sync with another PR)


Fix bug related to calculation of flatten op output dim

Fixes #646
Pull Request Template
Checklist

[x] Confirm that run-checks script has been executed.

Related Issues/PRs
https://github.com/burn-rs/burn/issues/646
Changes

Identified the bug related to ONNX flatten operation. The output dimension was incorrectly calculated.
Added ONNX file tests.
Added a new case for flatten tensor check.

Testing
Added new tests (onnx file and tensor test)

Hi @antimora,
When making this fix, did you test against the onnx model I had uploaded or just the test case?
For me the onnx model I was using now throughs the new error message you wrote:
thread 'predict_m6a::onnx::my_test' panicked at '=== Tensor Operation Error ===
  Operation: 'Flatten'
  Reason:
    1. The destination dimension (1) must be large enough to accommodate the flattening operation.

This is an improvement but I was hoping that the onnx model would be valid since it is in pytroch.
Thanks in advance!
Mitchell

Nevermind, ignore my previous comment! I forgot to remove my Cargo.lock, so I was using something that was out of date.

However, I do get an error now at the next operation in my onnx model:
thread 'predict_m6a::onnx::my_test' panicked at '=== Tensor Operation Error ===
  Operation: 'Matmul'
  Reason:
    1. The inner dimension of matmul should be the same, but got 75 and 25. Lhs shape [1, 75], rhs shape [25, 5].
', /Users/mrvollger/Desktop/repos/burn/burn-tensor/src/tensor/api/float.rs:196:9
stack backtrace:
   0: rust_begin_unwind
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/std/src/panicking.rs:593:5
   1: core::panicking::panic_fmt
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/core/src/panicking.rs:67:14
   2: core::panicking::panic_display
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/core/src/panicking.rs:150:5
   3: burn_tensor::tensor::api::float::<impl burn_tensor::tensor::api::base::Tensor<B,_>>::matmul
             at /Users/mrvollger/Desktop/repos/burn/burn-tensor/src/tensor/api/float.rs:196:9
   4: burn_core::nn::linear::Linear<B>::forward
             at /Users/mrvollger/Desktop/repos/burn/burn-core/src/nn/linear.rs:83:22
   5: fibertools_rs::model::revio::Model<B>::forward
             at ./target/debug/build/fibertools-rs-531f3735a2cf7cfe/out/model/revio.rs:115:28
   6: fibertools_rs::predict_m6a::onnx::my_model
             at ./src/predict_m6a/onnx.rs:18:18
   7: fibertools_rs::predict_m6a::onnx::my_test
             at ./src/predict_m6a/onnx.rs:25:13
   8: fibertools_rs::predict_m6a::onnx::my_test::{{closure}}
             at ./src/predict_m6a/onnx.rs:24:14
   9: core::ops::function::FnOnce::call_once
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/core/src/ops/function.rs:250:5
  10: core::ops::function::FnOnce::call_once
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

And here is the generated model code:
// Generated from ONNX "src/model/revio.onnx" by burn-import
use burn::nn::conv::Conv1d;
use burn::nn::conv::Conv1dConfig;
use burn::nn::Linear;
use burn::nn::LinearConfig;
use burn::nn::PaddingConfig1d;
use burn::record::Recorder;
use burn::{
    module::Module,
    tensor::{backend::Backend, Tensor},
};

#[derive(Module, Debug)]
pub struct Model<B: Backend> {
    conv1d1: Conv1d<B>,
    conv1d2: Conv1d<B>,
    conv1d3: Conv1d<B>,
    linear1: Linear<B>,
    linear2: Linear<B>,
    phantom: core::marker::PhantomData<B>,
}

impl<B: Backend> Default for Model<B> {
    fn default() -> Self {
        let record = burn :: record :: DefaultFileRecorder :: < burn :: record :: FullPrecisionSettings > :: new () . load ("/Users/mrvollger/Desktop/repos/fibertools-rs/target/debug/build/fibertools-rs-531f3735a2cf7cfe/out/model/revio" . into ()) . expect ("Record file to exist.") ;
        Self::new_with(record)
    }
}

impl<B: Backend> Model<B> {
    #[allow(unused_variables)]
    pub fn new_with(record: ModelRecord<B>) -> Self {
        let conv1d1 = Conv1dConfig::new(6, 30, 1)
            .with_stride(1)
            .with_padding(PaddingConfig1d::Valid)
            .with_dilation(1)
            .with_groups(1)
            .with_bias(true)
            .init_with(record.conv1d1);
        let conv1d2 = Conv1dConfig::new(30, 10, 1)
            .with_stride(1)
            .with_padding(PaddingConfig1d::Valid)
            .with_dilation(1)
            .with_groups(1)
            .with_bias(true)
            .init_with(record.conv1d2);
        let conv1d3 = Conv1dConfig::new(10, 5, 1)
            .with_stride(1)
            .with_padding(PaddingConfig1d::Valid)
            .with_dilation(1)
            .with_groups(1)
            .with_bias(true)
            .init_with(record.conv1d3);
        let linear1 = LinearConfig::new(25, 5)
            .with_bias(true)
            .init_with(record.linear1);
        let linear2 = LinearConfig::new(5, 2)
            .with_bias(true)
            .init_with(record.linear2);
        Self {
            conv1d1,
            conv1d2,
            conv1d3,
            linear1,
            linear2,
            phantom: core::marker::PhantomData,
        }
    }

    #[allow(dead_code)]
    pub fn new() -> Self {
        let conv1d1 = Conv1dConfig::new(6, 30, 1)
            .with_stride(1)
            .with_padding(PaddingConfig1d::Valid)
            .with_dilation(1)
            .with_groups(1)
            .with_bias(true)
            .init();
        let conv1d2 = Conv1dConfig::new(30, 10, 1)
            .with_stride(1)
            .with_padding(PaddingConfig1d::Valid)
            .with_dilation(1)
            .with_groups(1)
            .with_bias(true)
            .init();
        let conv1d3 = Conv1dConfig::new(10, 5, 1)
            .with_stride(1)
            .with_padding(PaddingConfig1d::Valid)
            .with_dilation(1)
            .with_groups(1)
            .with_bias(true)
            .init();
        let linear1 = LinearConfig::new(25, 5).with_bias(true).init();
        let linear2 = LinearConfig::new(5, 2).with_bias(true).init();
        Self {
            conv1d1,
            conv1d2,
            conv1d3,
            linear1,
            linear2,
            phantom: core::marker::PhantomData,
        }
    }

    #[allow(clippy::let_and_return)]
    pub fn forward(&self, input1: Tensor<B, 3>) -> Tensor<B, 2> {
        let conv1d1_out1 = self.conv1d1.forward(input1);
        let relu1_out1 = burn::tensor::activation::relu(conv1d1_out1);
        let conv1d2_out1 = self.conv1d2.forward(relu1_out1);
        let relu2_out1 = burn::tensor::activation::relu(conv1d2_out1);
        let conv1d3_out1 = self.conv1d3.forward(relu2_out1);
        let relu3_out1 = burn::tensor::activation::relu(conv1d3_out1);
        let flatten1_out1 = relu3_out1.flatten(1, 2);
        let linear1_out1 = self.linear1.forward(flatten1_out1);
        let relu4_out1 = burn::tensor::activation::relu(linear1_out1);
        let linear2_out1 = self.linear2.forward(relu4_out1);
        let softmax1_out1 = burn::tensor::activation::softmax(linear2_out1, 1);
        softmax1_out1
    }
}

But I will investigate a bit myself first and make a new issue if I cannot figure it out.


PyPI archive: bash completion line end using '\r\n'

As I recently found out, the bash completion script from the legit PyPI archive has its line end using CRLF:
$ xxd legit-1.2.0/extra/bash-completion/legit
00000000: 5f6c 6567 6974 2829 0d0a 7b0d 0a20 2020  _legit()..{..
00000010: 206c 6f63 616c 2063 7572 2070 7265 7620   local cur prev
00000020: 6f70 7473 0d0a 2020 2020 434f 4d50 5245  opts..    COMPRE

However, the release archive from GitHub, has its line end using LF:
$ xxd legit-1.2.0/extra/bash-completion/legit
00000000: 5f6c 6567 6974 2829 0a7b 0a20 2020 206c  _legit().{.    l
00000010: 6f63 616c 2063 7572 2070 7265 7620 6f70  ocal cur prev op
00000020: 7473 0a20 2020 2043 4f4d 5052 4550 4c59  ts.    COMPREPLY

Such line end using CRLF in the bash completion script will cause error message when opening a new Terminal on macOS:
-bash: /usr/local/etc/bash_completion.d/legit: line 1: syntax error near unexpected token `$'\r''
'bash: /usr/local/etc/bash_completion.d/legit: line 1: `_legit()

I suppose there're some issues with your distribution workflow...
NB: this issue was original raised from Homebrew/homebrew-core#51719

Oh, I will cut a new patch release to fix this issue


telemetry: fix crash with bad provider config

apiVersion: telemetry.istio.io/v1alpha1
kind: Telemetry
metadata:
  name: default
spec:
  tracing:
  - providers:
    - name: envoy

causes a crash.
This fixes the crash. In addition, it adds a per-proxy debug interface to telemetryz for better debugging, and tries to improve the exhaustiveness checks to prevent future bugs (Go can't handle this well :shrug: )

@howardjohn: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

release-notes_istio
103bbdead8fde616a011779d75c67175b8721495
link
true
/test release-notes

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

release-notes_istio
103bbdead8fde616a011779d75c67175b8721495
link
true
/test release-notes

unit-tests-arm64_istio
103bbdead8fde616a011779d75c67175b8721495
link
true
/test unit-tests-arm64

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

release-notes_istio
103bbdead8fde616a011779d75c67175b8721495
link
true
/test release-notes

unit-tests-arm64_istio
103bbdead8fde616a011779d75c67175b8721495
link
true
/test unit-tests-arm64

lint_istio
103bbdead8fde616a011779d75c67175b8721495
link
true
/test lint

integ-ambient_istio
103bbdead8fde616a011779d75c67175b8721495
link
true
/test integ-ambient

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@howardjohn: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

integ-ambient_istio
e918dc9ff6965fb16e856e87464e3bf24fd8dd8a
link
true
/test integ-ambient

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

In response to a cherrypick label: #46528 failed to apply on top of branch "release-1.17":
Applying: telemetry: fix crash with bad provider config
Using index info to reconstruct a base tree...
M	pilot/pkg/model/push_context.go
M	pilot/pkg/model/push_context_test.go
M	pilot/pkg/model/telemetry.go
M	pilot/pkg/model/telemetry_logging.go
M	pilot/pkg/model/telemetry_logging_test.go
M	pilot/pkg/model/telemetry_metric_test.go
M	pilot/pkg/networking/core/v1alpha3/tracing.go
M	pilot/pkg/networking/core/v1alpha3/tracing_test.go
M	pilot/pkg/xds/debug.go
Falling back to patching base and 3-way merge...
Auto-merging pilot/pkg/xds/debug.go
Auto-merging pilot/pkg/networking/core/v1alpha3/tracing_test.go
CONFLICT (content): Merge conflict in pilot/pkg/networking/core/v1alpha3/tracing_test.go
Auto-merging pilot/pkg/networking/core/v1alpha3/tracing.go
CONFLICT (content): Merge conflict in pilot/pkg/networking/core/v1alpha3/tracing.go
Auto-merging pilot/pkg/model/telemetry_metric_test.go
Auto-merging pilot/pkg/model/telemetry_logging_test.go
Auto-merging pilot/pkg/model/telemetry_logging.go
Auto-merging pilot/pkg/model/telemetry.go
CONFLICT (content): Merge conflict in pilot/pkg/model/telemetry.go
Auto-merging pilot/pkg/model/push_context_test.go
Auto-merging pilot/pkg/model/push_context.go
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
Patch failed at 0001 telemetry: fix crash with bad provider config
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".

In response to a cherrypick label: new issue created for failed cherrypick: #46547

In response to a cherrypick label: new pull request created: #46548

In response to a cherrypick label: new pull request created: #46549