CC Security Solution Explained #1：Pre-OS Measured/Trusted Boot

Motivation
The motivation of writing this article is inspired by this discussion. This discussion inspired me to think about the specific solutions for the TEEs of SEV(-ES) and TDX used by Kata CC, so this article is full of detail and analysis.  This is how people think, from understanding technical details to solution selection to high-level abstraction and modeling.
In current CCv0, the most of our attentions about security is paid to image pulling. Right now it is time to consider the security solution out of it. This article mainly discusses the sub-stage Pre-OS in the pre-container stage[1], but inevitably it also mentions other components. In addition, the question "what image encryption/signing means for CC?" is clearly answered in this article.
Note that this article is not a formal document about the overall security design, but an attempt to let people understand how the security of CC is established. I would be happy to provide some references for component designers and the drafts of trust model and threat model.
This is the first article in series. I will decide the topic of next article based on your feedback.
Deploy Protected Boot Image or Unprotected Boot Image
Boot Image refers to the Guest OS image used by Pod. It is composed of pre-container[1] components and can be deemed as a system volume. The decision to use which Boot Image solution is so important so I discuss it first.
Protected Boot Image refers to the use of the Boot Image with confidentiality and integrity protections provided by workload provider[2] in a specific solution. Protected Boot Image is usually pre-provisioned with partial or all of the artifacts[3]. The artifacts may include but are not limited to:

kata-agent configuration file
policy.json
sigstore
pubkey (for signature verification)
KEK or PLBCO (with LEK)
Registry login

These artifacts are sensitive information, namely secrets, because they directly affect the behavior in Pod,  This is why needing to provide confidentiality and integrity protections for the Boot Image in order to store them. The Pod-level system security solution based on Protected Boot Image can simplify the security design for Pod, because the TCB state of Pod is trusted as long as the deployment based on Protected Boot Image is correctly implemented. Someone may consider to use KBS provisioning to retrieve all the artifacts for persistent and secure storage, because Protected Boot Image itself can be deemed as a secure data volume for, e.g, container snapshot storage. The decryption procedure of Protected Boot Image usually occurs when the bootloader (such as grub) mounts the LUKS volume[4,  Configure GRUB], and the master key[5, 1. Overview] in LUKS volume needs to be obtained by the guest firmware through attestation in advance.
The opposite of Protected Boot Image is Unprotected Boot Image. It is provided by CSP instead of the workload provider. As its name suggests, "Unprotected" means the confidentiality and integrity protections are irrelevant because it is not trusted by tenants/workload provider. So all above artifacts must be provisioned through KBS provisioning. This solution is based on a business requirement that the workload provider/tenant only needs to provide container images. It is reasonable because traditionally Boot Image is supplied by the CSP.
Theoretically, the TCB state of Pod is trusted as long as the Protected Boot Image solution is implemented correctly. The TCB state of Pod is unverified and may even be malicious if using the Unprotected Boot Image solution. Therefore, the Unprotected Boot Image solution faces more security challenges and design difficulty than the Protected Boot Image solution due to its characteristics.

Malicious K8s admin can replace any components in Unprotected Boot Image at any time.
The TCB state of Pod is unverified before launch of attestation, so you cannot assume it can authenticate a KBS server, authenticate pulling an image or verify an image signature as expected.
Even the TCB state of Pod was verified, it only proves the TCB state of Pod reflected in attestation evidence at that moment is trust. In other words, a full attestation verification must be taken again in the next attestation procedure.

These Boot Image solutions between Protected Boot Image and Unprotected Boot Image have a huge impact on the security design of Pod, so the following discussions will be made around them.
What image encryption means for CC？
To answer this question, let's take a look at how the full-disk encryption[6] works in the scenario of trusted computing.
In setup phase, the security management program seals the encryption key to the TPM. Sealing permits the key to be protected not only by a TPM but by a policy which locks the key to PCR values (the TCB state) current at the time of sealing. The setup phase is completely executed in a secure environment, so the PCR values reflect the uncompromised state of the pre-installed components, including hardware, firmware, and software environments (It is noted that how to ensure the sources of the various components pre-installed in the setup phase are trusted is addressed by the security of the software supply chain. I will write another article to discuss this topic, which involves the implementation of reproducible build and verifiable build in DevSecOps and other technologies, with the considerations for CC).
In fact, the use of image encryption in CC is highly similar to the above scenarios. Relying Party/KBS can use the policy to prevent from provisioning any resources to the Pod when the TCB state does not meet expectations, including the KEK/PLBCO used for decrypting container images, or the secret used for decrypting Protected Boot Image. The policy can specify any combinations of, such as the measurements in Attestation Evidence, the hash value of a pre-container component, and so on. For example, you can use OPA policy to do this:
guest_fw = ["hash_A", "hash_B"]
attestation_agent = ["hash_C", "hash_D", "hash_E"]

default allow = false

# Only true iff the measurements of guest fw and attestation-agent equal to
# the known reference values.
allow = true {
        input.event_log.guest_fw == guest_fw[_]
        input.event_log.attestation_agent == attestation_agent[_]
}

The above policy will decide to reject unsealing secret if the measurement of either guest firmware or attestation-agent is not equal to the reference value. In other words, Relying Party will not respond to the current provisioning request from Pod, unless the policy approves it. We can call this process as Attestation Evidence for Authorization.
It is clear that image encryption can enforce to inject an explicit authorization during CC bootstrap through Attestation Evidence for Authorization, causing a Pod has to request Relying Party to verify its TCB state. Because of this, we can detect and block any malicious Pods. This is the inevitable result of image encrypting as full-disk encryption. This use of image encryption is particularly useful for the Unprotected Boot Image solution, because the TCB state, before the first Attestation Evidence for Authorization, is unverified. Why image signature cannot take this effect? Because the signature verification is a process in which a verified Pod or trusted Pod performs the identity authentication. Instead, the signature verification is irrelevant or even cheated by a malicious Pod.
The most useful aspect of the Attestation Evidence for Authorization is that it can bind the unsealing of decryption key to a TCB state of Pod. Obviously, the TCB state of a malicious Pod is inconsistent with the expectation, so the decryption key cannot be obtained. Image encryption can effectively prevent an attacker from pulling and running the tenant's application container image in a malicious Pod controlled by the attacker, leaking sensitive data, or pretending to be a legitimate workload to further steal more external data. Both the Protected Boot Image or the Encrypted Container Image can benefit from the encryption.
As mentioned by James, container images are not always encrypted in the scenario of CC. So we need to clarify that one of the premise of using container images encryption is using the application container images which contain sensitive data. Some people may ask: Is it necessary to use container images encryption if my application container images does not contain any sensitive data? The answer is uncertain. It is related to the solution of Boot Image actually used, and whether the signature verification of container image is used. Let's answer this question in next section.
What container image signing means for CC？
The signature verification of container image currently used by CC is not a mechanism to provide integrity protection for the content of container image at all, because OCI container image format naturally builds in a hash-based addressing mechanism, meaning that the integrity of the container image can be verified when it is parsed.
The image signature provided by a specific entity is an authenticated claim and endorsement for a specific image identity[7].
The verification of container images signature depends on three artifacts:

sigstore: store the signature claims that container images have been approved.

pubkey: the public key used to verify the signature claim.

policy.json：provides the black and white list to check the validity of registry and docker reference, and authorizes or rejects to pull a specified or a group of container images from the specified registry.

If CC lacks the signature verification of container image, a malicious k8s admin and IaaS SRE can inject a malicious container into the Pod (image encryption is irrelevant) to try to attack the other application containers deployed by tenants in the Pod, or steal the sensitive data in the protected boot image (if any) and/or data volume.
Whether the signature verification can work as expected or not is heavily depends on the TCB state of Pod. Only when the TCB state of Pod is trusted or verified, image signing can work as expected. Otherwise, the premise is lost. This has been mentioned in the previous section. The following discussions are made around the solution of Boot Image.

Protected Boot Image
The TCB state of Pod is trustworthy as long as the Protected Boot Image solution is implemented correctly. The above three artifacts may be pre-provionned, or provionned through KBS provisioning. For the latter case, it is not a big deal to secure the provisioning process (The authentication to KBS will be discussed in subsequent article). It is noted that the KBS provisioning process for above three artifacts requires Attestation Evidence for Authorization, as image encryption, although it occurs after signature verification.
In addition, we need to prevent attackers from injecting malicious containers into Pod. Only the container images approved by the signature claims can be pulled as long as policy.json is configured correctly. This means that the Protected Boot Image solution can reliably support encrypted and signed images, and unencrypted and signed images. If you use the whitelist function to mark a specific registry as trusted, you can also successfully pull and run encrypted but unsigned image, or even unencrypted and unsigned images (with a secure channel to download, e.g, HTTPS). It is noted that CSP may host a registry service. A strict signature verification to the images from this sort of registry must be taken.

Unprotected Boot Image
In this solution, the three artifacts need to be provisioned into Pod through KBS provisioning, which is similar with the Protected Boot Image solution, except the authentication to KBS. This topic will be discussed in the subsequent article.
Additionally, we need to prevent attackers from successfully pulling and running the tenant's application container images in a malicious Pod, leaking sensitive data, or pretending to be a legitimate workload to further steal more external data. The value of the Attestation Evidence for Authorization mechanism has been explained in above section. Therefore, if the application container image doesn't contain sensitive data, the encryption of application container image is optional for the Protected Boot Image solution, but it is necessary for the Unprotected Boot Image solution. Once Attestation Evidence for Authorization returns positive result, the TCB state of Pod is verified. If other security principles are implemented correctly (will be discussed in subsequent articles), the other threat vectors are same as Protected Boot Image solution mentioned above: further prevent attackers from injecting additional malicious containers into the Pod. To sum up: the Unprotected Boot Image solution requires the application container images are always encrypted, and the images are signed, or whitelist their provenances of registry. The last thing to note here is that the signature verification and white list check occur prior to image decryption, which means that there's no guarantee the security state of Pod if it doesn't retrieve those three artifacts through a genuine KBS provisioning with Attestation Evidence for Authorization mechanism. Therefore, it is necessary to confirm that the event of provisioning three artifacts have indeed occurred through checking the event log[8] in the procedure of Attestation Evidence for Authorization when decrypting the application container image.

At last, it should be noted that if you want to persist those three artifacts, they cannot be stored to insecure storage, such as Unprotected Boot Image itself. Instead, a data volume with confidentiality and integrity protections must be used. This is another topic to be discussed in the future.
Pre-OS Measured/Trusted Boot
The main purpose of a trust model is to continuously measure trustworthiness of a set of entities based on their behaviors, and it must include implicit or explicit validation of an entity's identity or the characteristics necessary for a particular event or transaction to occur[9].
The trust model of confidential container closely focuses on the establishment and extension procedure of chain of trust, which is highly similar to Measured/Trusted Boot[10] used in trusted computing.

TPM
CC

Measurement
PCR
Measurement Registers（e.g, MRTD and RTMRs in TDX）

Evidence
PCR Quote + TPM Event Log
Attestation Report + Event Log (e.g, TDEL[11,  4.4.3 TD-Event Log] in TDX）

Attestation
TPM Attestation
SEV(-ES) Pre-Attestation / TDX/SEV-SNP Attestation+vTPM

Unseal
PCR Authorization and Policy
Attestation Evidence for Authorization and Policy*

Attestation Evidence for Authorization and Policy can be equivalent to PCR Authorization and Policy implemented outside TPM hardware TEE.

According to the design of Measured Boot, CC needs to continuously and dynamically measure the TCB state of Pod in the pre-container stage, and then verify the TCB state through the Attestation Evidence for Authorization. The hardware platforms that support dynamic measurement such as TDX may choose this design.
According to the design of Trusted Boot, CC needs to perform a static measurement for Pod when loading guest firmware (and other necessary data) in the pre-container stage, then verify the initial TCB state of Pod through the Attestation Evidence for Authorization, and eventually use UEFI Secure Boot or similar mechanisms to check and authenticate dynamically loaded entities. This design is the only choice for the hardware platforms without the support for dynamic measurement, such as SEV(-ES).
Next let's investigate the possible designs for different hardware platforms.
SEV(-ES) Pre-OS Trusted Boot Flow with OVMF
The startup process can be divided into 3 stages:

Pre-Attestation stage [12, Enclave launch time][13, 2.4 SEV: Establish Secure Channel]
Bootloader decrypts the encrypted disk stage [14, Plugging the Gaps: Grub]
Pre-OS boot phase

[14] explains the design principles about the startup process. It is worth noting that the deployment method of bootloader in this article is integrated into OVMF as FV:

The final OVMF change (which is in the same patch set) is to pull grub inside a Firmware Volume and execute it directly. This certainly isn’t the only possible solution to the problem (adding secure boot or an encrypted filesystem were other possibilities) but it is the simplest solution that gives a verifiable component that can be invariant across arbitrary encrypted boots (so the same OVMF can be used to execute any encrypted VM securely). This latter is important because traditionally OVMF is supplied by the host owner rather than being part of the VM image supplied by the guest owner. The grub script that runs from the combined volume must still be trusted to either decrypt the root or reboot to avoid leaking the key. Although the host owner still supplies the combined OVMF, the measurement assures the guest owner of its correctness, which is why having a fairly invariant component is a good idea … so the guest owner doesn’t have potentially thousands of different measurements for approved firmware.

Usually, the certificate of UEFI Secure Boot is tenant-specific and the OVMF firmware is traditionally provided by CSP. UEFI Secure Boot is a possible way, but has maintenance issue for the certificates of UEFI Secure Boot provided by the tenants.
Essentially, due to the fact that SEV(-ES) does not support dynamic measurement, the bootloader has to be wrapped into the guest firmware for static measurement together. Otherwise, there is no way to measure the subsequently loaded bootloader after launch of guest. It is not a problem any more on TDX platform that support dynamic measurement.
In fact, it is not the best solution to integrate bootloader as FV into OVMF. OVMF and bootloader have completely different upgrade and maintenance cycles. It is possible to inject the secret with CFV (will be discussed below) containing the UEFI Secure Boot certificate into the guest during Pre-Attestation. However, it is said that SEV(-ES) cannot support UEFI Secure Boot due to SMM.
This solution of Boot Image obviously uses Protected Boot Image, so the pre-container components except the guest firmware and bootloader are all stored in the encrypted system volume. Only the Pod verified by Attestation Evidence for Authorization implemented by GOP during the Pre-Attestation process can obtain the secret used for decrypting the Protected Boot Image. The TCB state of Pod can reflect the measurements of all components loaded with OVMF together). Eventually, the secret is passed to the guest OS through a designated EFI reserved memory area[15].
How the kernel, kernel commandline and initrd are verified? According to the current implementation of OVMF, the hash values of these 3 artifacts are written in the SEV hash table in OVMF in advance.  During boot time, OVMF will measure and verify the integrity of three artifacts using these three hashes as reference values. This is a specific implementation of trusted boot which is different from UEFI Secure Boot. I'm not sure whether this implementation has maintenance issue or not. Traditionally, OVMF is provided by CSP, and tenants provide the Protected Boot Image, not kernel and initrd. So tenants have to send the hash values of the kernel and initrd to the CSP through another control plane, and then the hash values are written into OVMF at the background. Meanwhile, GOP side pre-calculates the hash value of OVMF injected with 3 hash values. Therefore, CSP needs to maintain one OVMF image for each tenant. Furthermore, the update of any of 3 artifacts would bring more maintenance costs.
A better solution is to provision the secret with the hash values of the three artifacts and the bootloader to OVMF through Pre-Attestation.
Essentially, we can use a general methodology to resolve this kind of problem so-called as the software supply chain security. For example, a neutral organization certificated by CCC and several crucial open source communities is jointly established to provide endorsements and signature signing for the components being part of TCB for Confidential Computing ecosystem. Specific to edk2 (the codebase of OVMF and TDVF), this organization can provide pre-built guest firmware with the 3rd security audit report. Tenants download the binaries and the provenances provided by the organization in advance as reference values. Through Attestation Evidence for Authorization, whether the CSP actually deploys the binaries provided by the organization can be detected. This model can be extended to bootloader, linux kernel, initrd, kata-agent and attestation-agent. If a project can support reproducible build, the auditability of the source code will become more transparent. Of course, mobilizing so many resources to participate in one thing is not that simple. Alternately, the workload providers can also deploy their own private software supply chain to meet specific requirements which cannot be addressed by the general software supply chain mentioned above. In addition, it is a feasible way to allow the software supply chain to be delegated to a trusted third party. This topic will be discussed in subsequent article.
TDX Pre-OS Trusted Boot Flow with TDVF
This startup process can be divided into three stages[16, 2.3 boot flow]:

TDVF (guest firmware) setup: VMM loads TDVF into the private memory in guest, then the hardware automatically measures TDVF and saves the measurement in the static measurement register MRTD.

TDVF boots and enables UEFI secure boot: TDVF in the guest executes the first instruction from the reset vector, then dynamically measures the CFV [16, 3.2 configuration firmware volume (CFV)] containing the certificates of UEFI Secure Boot, and saves the measurement in the dynamic measurement register RTMR[0]. Next, enable UEFI Secure Boot with CFV.

TDVF prepares TD event log and launches the OS loader: TDVF creates ACPI table TDEL[11, 4.4.3 TD event log] to describe TD event logs generated by all dynamic measurement operations, and then passes it to the guest kernel[17]. Next, verify and start the bootloader.

TDVF design a relatively complete Pre-OS Trusted Boot Flow. Comparing to SEV(-ES), TDX does not support a mechanism similar to Pre-Attestation in hardware. In addition, TDVF currently doesn't explicitly mention that it supports attestation, so this implies that TDVF will adopt the Unprotected Boot Image solution, that is, the TCB state of Pod is not verified until the attestation-agent launches a TDX remote attestation.
However, [11, 4.4 storage volume key data] mentions that TDVF needs to be capable of attestation to provision the secret to the guest kernel through the ACPI table SVKL. Essentially, if TDVF supports attestation, it can support the Protected Boot Image solution. In addition, even through TDVF doesn't support attestation, it can also support data volume encryption, as long as the operation of mounting the encrypted data volume occurs after the attestation-agent runs.
TDX Pre-OS Measured Boot Flow with td-shim
td-shim is a lightweight guest firmware designed for CC. At present, it only supports TDX.
The boot process of td-shim is similar to TDVF, except that it is heavily simplified, and supports direct kernel boot without bootloader.
td-shim can support UEFI Secure Boot[18, secure boot support], but the typical use cases of td-shim in CC uses Measured Boot for fast boot and simplification.
Like TDVF, td-shim does not explicitly mentions that it doesn't support attestation. In addition, td-shim doesn't support device management, which means Boot Image solution is not a meaningful question to td-shim.
Summary
The following is a summary about two Boot Image (and General Intird Image) solutions:

Protected Boot Image solution
Unprotected Boot Image/General Intird Image solution

Provider
Workload Provider
CSP

Need to protect the content of boot image?
Required
Not Required

Initial TCB state of Pod
Trusted
Unverified

Need to encrypt the application container image?
Optional
Not Required

Need to sign / whitelist the application container image?
Required
Required

Threat vector
A malicious k8s admin can inject a malicious container into the Pod to try to attack the other application containers deployed by tenants in the Pod, or steal the sensitive data in the protected boot image and/or data volume.
The attacker-controlled Pod successfully pulls and runs the tenant's application container images, leaks sensitive data, or pretends to be a legitimate workload to further steal more external data.*

* After accomplishing the initial Attestation Evidence for Authorization, the threat vector of Protected Boot Image solution should also be considered too.
When choosing the Boot Image solution, we need to make a balance between technical implementation and business requirements, and we must remember that the security of Boot Image will affect the initial TCB state of Pod. The security of subsequent stages, especially the image pulling and workload operation stages, will be affected by it for sure.
References

[1] https://github.com/magowan/documentation/blob/TrustModel/TrustModel.md#lifecycle-stages
[2] https://github.com/magowan/documentation/blob/TrustModel/TrustModel.md#workload-provider
[3] https://github.com/magowan/documentation/blob/ThreatModel/ThreatModel.md
[4] https://www.pavelkogan.com/2014/05/23/luks-full-disk-encryption/
[5] https://gitlab.com/cryptsetup/cryptsetup/-/wikis/LUKS-standard/on-disk-format.pdf
[6] http://chaosophia.net/papers/luks_tpm_and_full_disk_encryption_without_password.html
[7] https://github.com/containers/image/blob/main/docs/containers-signature.5.md
[8] https://twobit.org/2016/05/31/tpm2-uefi-measurements-and-event-log/
[9] https://github.com/magowan/documentation/blob/ThreatModel/ThreatModel.md#artifacts
[10] https://opensource.com/article/20/10/measured-trusted-boot
[11] https://www.intel.com/content/dam/develop/external/us/en/documents/intel-tdx-guest-hypervisor-communication-interface-1.0-344426-002.pdf
[12] https://trmm.net/SEV/
[13] https://arxiv.org/pdf/1908.11680.pdf
[14] https://blog.hansenpartnership.com/deploying-encrypted-images-for-confidential-computing
[15]<EMAIL_ADDRESS>[16] https://www.intel.com/content/dam/develop/external/us/en/documents/tdx-virtual-firmware-design-guide-rev-1.01.pdf
[17] https://github.com/intel/tdx/commit/3935906646031fca9a8d20046100e6fb6d0126bf
[18] https://github.com/confidential-containers/td-shim/blob/staging/doc/tdshim_spec.md

Great post. I have a few clarifications about the SEV sections and a few general thoughts.

It is not a problem any more on TDX and SEV-SNP platforms that support dynamic measurement.
By default SNP does not support dynamic measurement. The SNP attestation report contains the launch measurement, which is basically the same as the launch measurement calculated for SEV(-ES). Someday there might be support for vTPM linked to the hardware root of trust that would enable dynamic measurement, but that is not currently supported.

How the kernel, kernel commandline and initrd are verified?

The Grub-based encrypted disk method and the qemu-driven measured direct boot aren't used together. Maybe you know this but it isn't clear from the post. The kernel, initrd, and command line are included in the encrypted disk image for the first approach and don't need to be verified separately. The hashes are only injected on a direct boot which skips over Grub, so you can't use these approaches together even if you wanted to.
We have also been experimenting with using the measured direct boot method to launch a VM with an encrypted disk. As you point out there are some downsides to the Grub-based approach. We are considering some other things as well and it's not yet clear what will be the most widely supported method.

A better solution is to provision the secret with the hash values of the three artifacts and the bootloader to OVMF through Pre-Attestation.

We discussed this approach at length. In some ways it is simpler than what we currently do with injecting the hashes, but there are important considerations. The current approach does not use a secret to provide the direct boot hashes, instead it extends the launch measurement to include the direct boot hashes. This has two implications. First, it means that we are free to use the secret for something else. You can only inject a secret once with SEV, so if we inject the hashes via the secret, we can't inject the material needed for the attestation agent. Of course we could combine the payloads and inject everything together, but that increases complexity. Perhaps the more significant consideration is that SEV-SNP does not have a secret injection mechanism. To get secrets inside the guest you request an attestation report and use it to setup a secure channel with a trusted 3rd party. If possible we'd like to avoid doing this before the OS boots, given that it is a somewhat complex process. Injecting the hashes works great with SNP because it just extends the launch measurement, which is part of the attestation report that the guest can request when appropriate.
I think your distinction between protected and unprotected boot images is interesting. Note that we already have a platform, Z Secure Execution, that provides a protected boot image. The two SEV approaches, encrypted disk vs measured direct boot, illustrate the differences. I like to think about encrypted disk approaches being best for workloads that are mutable, whereas the measured approach is better suited to generic non-confidential workloads. We've been thinking of Kata as a good fit for the second case, given that the Kata Agent is generic and non-confidential. I think part of the work for CCv1 may be making sure that this is really true. We want the agent, and really the entire measured surface of the guest, to be generic because it makes validating the measurement easier. Currently we have some parameters like AA_KBC_PARAMS in the agent configuration file that might be too dynamic. Parameters like this also mean that the CSP can't really provide the boot image.
If we come at things from the encrypted disk perspective, we don't have to worry about the measurement changing, but we would probably have to create a new disk each time we want to change one of the parameters that is set from within the disk i.e. the config file. The logical extreme of this is where we have a 1-1 mapping of VM image to pod. In this case we wouldn't even need the Kata agent. We would just put the containers in an encrypted boot image. Obviously there are some big downsides to this. Like I said, I've mainly thought of our project as leveraging a generic and non-confidential boot image, but it's good that we're thinking about both.
In the final table, you say that the CSP provides the boot image in the non-confidential boot case. It might be important to note that this fits with our trust model since the provided components are all measured. I agree with what you were suggesting about providing auditable boot images. I think that we need to do some more thinking about reproducible builds as well.
One final note is that the protected boot image solution is not a black box. Booting with an encrypted disk will probably involve some of the same processes that we currently use with the attestation agent. With the SEV + Grub approach, for instance, you provide a launch measurement to a KBS to get the disk encryption key. In some ways the security properties are the same as when we get keys for decrypting containers from a KBS via the Attestation Agent.

By default SNP does not support dynamic measurement. The SNP attestation report contains the launch measurement, which is basically the same as the launch measurement calculated for SEV(-ES).

Thanks your clarification! I will modify the article to reflect your comments.

Someday there might be support for vTPM linked to the hardware root of trust that would enable dynamic measurement, but that is not currently supported.

Wonderful. Current vTPM doesn't link to hardware root of trust and even doesn't provide any protections for the sealed secret in the backend because its trust model still follows up the typical cloud scenario based on virtualization.

We have also been experimenting with using the measured direct boot method to launch a VM with an encrypted disk. As you point out there are some downsides to the Grub-based approach. We are considering some other things as well and it's not yet clear what will be the most widely supported method.

Interesting! Do you also want to use the single method qemu-driven direct boot to support both encrypted and unencrypted boot image? As long as you just use one key to protect both container image and boot image,  the format of secret injection is not touched.

The kernel, initrd, and command line are included in the encrypted disk image for the first approach and don't need to be verified separately.

Please let me confirm whether the following declarations are correct around the encrypted boot image approach:

Package grub and OVMF together.
The injected secret is used to decrypt the protected boot image by bootloader.
Kernel, initrd and cmdline stored in the protected boot image are automatically protected.
Without the intentions of extending the format of injected secret, this approach cannot be used by CC.

Note that we already have a platform, Z Secure Execution, that provides a protected boot image.

Thanks for your info! I'm curious that the protected boot image is provided by tenant rather than CSP/workload provider/3rd trusted vendor right?

We've been thinking of Kata as a good fit for the second case, given that the Kata Agent is generic and non-confidential. I think part of the work for CCv1 may be making sure that this is really true. We want the agent, and really the entire measured surface of the guest, to be generic because it makes validating the measurement easier. Currently we have some parameters like AA_KBC_PARAMS in the agent configuration file that might be too dynamic. Parameters like this also mean that the CSP can't really provide the boot image.

Agreed. This work is challenging, but it's worthy.

We discussed this approach at length. In some ways it is simpler than what we currently do with injecting the hashes, but there are important considerations. The current approach does not use a secret to provide the direct boot hashes, instead it extends the launch measurement to include the direct boot hashes. This has two implications. First, it means that we are free to use the secret for something else. You can only inject a secret once with SEV, so if we inject the hashes via the secret, we can't inject the material needed for the attestation agent. Of course we could combine the payloads and inject everything together, but that increases complexity. Perhaps the more significant consideration is that SEV-SNP does not have a secret injection mechanism. To get secrets inside the guest you request an attestation report and use it to setup a secure channel with a trusted 3rd party. If possible we'd like to avoid doing this before the OS boots, given that it is a somewhat complex process. Injecting the hashes works great with SNP because it just extends the launch measurement, which is part of the attestation report that the guest can request when appropriate.

Understood. So the root cause of not extending secret format is due to the fact that you plan to use injecting the hashes as a common implementation to support both SEV(-ES) and SEV-SNP, right?

If we come at things from the encrypted disk perspective, we don't have to worry about the measurement changing, but we would probably have to create a new disk each time we want to change one of the parameters that is set from within the disk i.e. the config file. The logical extreme of this is where we have a 1-1 mapping of VM image to pod. In this case we wouldn't even need the Kata agent. We would just put the containers in an encrypted boot image. Obviously there are some big downsides to this. Like I said, I've mainly thought of our project as leveraging a generic and non-confidential boot image, but it's good that we're thinking about both.

This is important to me and @magowan. Because protected vs unprotected boot image will use different trust/threat model according to your extreme example. I can add another extreme example: all artifacts with tenant specific configurations or data can be placed in such a protected boot image, including decryption keys for encrypted container images, and thus attestation-agent, remote attestation and KBS become unnecessary. I mean if we mark protected boot image as non-goal, it will be much simpler (and show what we are doing and experimenting is challenging and meaningful).

One final note is that the protected boot image solution is not a black box. Booting with an encrypted disk will probably involve some of the same processes that we currently use with the attestation agent. With the SEV + Grub approach, for instance, you provide a launch measurement to a KBS to get the disk encryption key. In some ways the security properties are the same as when we get keys for decrypting containers from a KBS via the Attestation Agent.

Yes. According to the current practice, getting the disk encryption key only happens to SEV(-ES) pre-attestation. There is no sign that TDX and SNP plan to support attestation in guest fw. Of course it is possible to get the encryption key through attestation-agent in the non-confidential use cases. This is another reason I suggest why we need to consider whether we exclude protected boot image, because the protected boot image solution may allow any of all required artifacts defined by CC is provisioned into boot image during image creation, or remotely provisioned through attestation-agent. There will be a lot of combinations, but not any combination fits for trusted/threat model. Explicitly listing all valid combinations is difficult.

Interesting! Do you also want to use the single method qemu-driven direct boot to support both encrypted and unencrypted boot image? As long as you just use one key to protect both container image and boot image, the format of secret injection is not touched.

We've been looking into this although there are some issues. For instance if you update the kernel from inside the guest the measurement will be different the next time you boot and it's a bit difficult to keep track of the changes in the image and kernel/initrd.

Without the intentions of extending the format of injected secret, this approach cannot be used by CC.

The disk encryption key is injected as part of a secret table. Our efi_secret kernel module expects a secret table with the same format. We use that module to expose the secrets to userspace so that they can be read by the offline_sev_kbc. So we potentially could inject both the disk encryption key and the secrets that are needed by the offline KBC simultaneously. Hypothetically we could use encrypted disk with Kata. We haven't really thought about this and I'm not sure how well it would work. Encrypted disk is not compatible with direct boot so we might have to make some adjustments. The kernel command line would need to be part of the image ahead of time (although you need to know the command line ahead of time with direct boot as well so you can validate the launch measurement). It might be something to look into more if we decide that protected boot images have significant advantages.
Otherwise, your description of encrypted disk is correct.

Thanks for your info! I'm curious that the protected boot image is provided by tenant rather than CSP/workload provider/3rd trusted vendor right?

The boot image is provisioned via the ultravisor on behalf of a tenant. I don't really understand how this works or precisely what the trust model is. @Jakob-Naucke might know more.

Understood. So the root cause of not extending secret format is due to the fact that you plan to use injecting the hashes as a common implementation to support both SEV(-ES) and SEV-SNP, right?

Yeah. To be honest both approaches could work fine. We went with hashes because it seemed like it would be a bit easier to use with SNP.

I can add another extreme example: all artifacts with tenant specific configurations or data can be placed in such a protected boot image, including decryption keys for encrypted container images, and thus attestation-agent, remote attestation and KBS become unnecessary.

This would create an explosion of boot images because every unique workload would require a different boot image. In practice this would probably require some tool that automatically created the boot images from pods. This is pretty similar to the approach that some other projects take when they wrap each workload prior to runtime. If you claim to support standard containers but you need to build a special image for them, either the client has to do a lot more work, or the CSP has to wrap the containers for them. The first option is inconvenient and the second one breaks the trust model.
In short I prefer having a generic image to having a 1-1 mapping between workload and image. Of course there are some options that fall between those two extremes. It's also worth noting that when you have a generic image you need something inside the enclave that talks to the untrusted world (i.e. the kata agent) and this increases the attack surface.

Yes. According to the current practice, getting the disk encryption key only happens to SEV(-ES) pre-attestation. There is no sign that TDX and SNP plan to support attestation in guest fw.

This is actually another advantage of supporting encrypted disk via measured direct boot as discussed above. In SNP it will be more annoying to get a secret before the kernel has booted. If we use direct boot the kernel will be measured so we can decrypt the root disk later (in the initrd phase) and rely on the kernel support for getting the attestation report and setting up a secure channel to get the disk encryption key.

This is another reason I suggest why we need to consider whether we exclude protected boot image, because the protected boot image solution may allow any of all required artifacts defined by CC is provisioned into boot image during image creation, or remotely provisioned through attestation-agent. There will be a lot of combinations, but not any combination fits for trusted/threat model. Explicitly listing all valid combinations is difficult.

Like I said, I don't really like the protected boot image, but I think that what we've done with Secure Execution so far is fine. With the offline_fs_kbs for instance, we have a relatively elegant way to make the AA work with a protected boot image. The security properties there are relatively easy to understand. The new proposal for signatures in image-rs also supports either local configuration files or remote ones. Since there is nothing confidential in these config files it's fairly easy to think about the security properties.

For instance if you update the kernel from inside the guest the measurement will be different the next time you boot and it's a bit difficult to keep track of the changes in the image and kernel/initrd.

Indeed. Sounds like you have to link the following items together when using direct boot:

update kernel/initrd/cmdline
update hash table in guest fw
update allowed_digests in keyset.json in GOP server side

Encrypted disk is not compatible with direct boot so we might have to make some adjustments. The kernel command line would need to be part of the image ahead of time (although you need to know the command line ahead of time with direct boot as well so you can validate the launch measurement). It might be something to look into more if we decide that protected boot images have significant advantages.

and

If we use direct boot the kernel will be measured so we can decrypt the root disk later (in the initrd phase) and rely on the kernel support for getting the attestation report and setting up a secure channel to get the disk encryption key.

and

It's also worth noting that when you have a generic image you need something inside the enclave that talks to the untrusted world (i.e. the kata agent) and this increases the attack surface.

Sure. I think encrypted disk is available for SEV(-ES) only because:

No pre-attestation to provide decryption key for encrypted disk
No attestation support in OVMF at the early stage (and the guest fw TDVF for TDX also doesn't support attestation)

As you mentioned, using encrypted disk solution for SEV(-ES) platform doesn't need to verify kernel, initrd and command line so hash table injection looks useless. This breaks your goal of unifying SEV(-ES) and SEV-SNP. Looks like it is hard to find a way of using encrypted disk solution to the best of both worlds. Therefore, if you seek an unified approach to support both SEV(-ES) and SEV-SNP, it seems that unprotected boot image solution is the only choice.
Considering your info and the attack surface you mentioned above, how about this work flow for all platforms using general image solution?

SEV-SNP: launch OVMF -> verify kernel, cmdline and initrd -> launch AA from initrd*1 -> remote attestation + metadata provisioning -> launch kata-agent from initrd with provisioned config -> image pulling -> unpack the image to protected data volume *2
TDX: launch guest fw -> load and dynamically measure kernel, cmdline and initrd -> launch AA from initrd -> remote attestation + metadata provisioning -> launch kata-agent from initrd with provisioned config -> image pulling -> unpack the image to protected data volume
SEV(-ES): pre-attestation + metadata provisioning -> launch OVMF -> verify kernel, cmdline and initrd -> launch AA from initrd -> remote attestation + metadata provisioning -> launch kata-agent from initrd with provisioned config -> image pulling -> unpack the image to protected data volume

Strictly speaking, there is no general image. Instead, just use a general initrd. The attestations for all platforms must perform the check the measurement of the general initrd and ensures it matches up the known good value. Due to the fact that agent config supplied by tenant is not provisioned yet in *1, launch AA prior to kata-agent. Next, more metadata supplied by tenant through an attested channel are provisioned, such as agent config, image decryption key, policy.json for signature verification and so on. For *2, I would call the encrypted root disk as the protected data volume for snapshot storage, which is under the control of tenant for persistent storage. Of course, providing an encrypted boot disk is also allowed. Just add an extra step to decrypt and mount the encrypted boot disk after metadata provisioning.

Note that we already have a platform, Z Secure Execution, that provides a protected boot image.

Thanks for your info! I'm curious that the protected boot image is provided by tenant rather than CSP/workload provider/3rd trusted vendor right?

The boot image is provisioned via the ultravisor on behalf of a tenant. I don't really understand how this works or precisely what the trust model is. @Jakob-Naucke might know more.

Secure Execution is the confidential computing architecture on s390x (IBM Z & LinuxONE). In Secure Execution, the workload owner has to put the kernel, command line, and initrd into an encrypted blob , which is specified as kernel for direct boot. The (symmetric) key used for this encryption is wrapped by one or more asymmetric keys specific to the machine(s) the image is allowed to run on (somewhat akin to how LUKS works). The public keys are publicly available (by serial number of the machine), the private keys are in the firmware of every machine since z15 & LinuxONE III, unreadable even to administrators. This means that you can put whatever secret data you want in the initrd and it will still be secure, e.g. SSH private keys, TLS certificates, LUKS keys, or keys for Attestation Agent's offline file system KBC. @jiazhang0 this is probably most in line with what you call "Protected Boot".
This is achieved through a so-called ultravisor, which is implemented in hardware and firmware. When a KVM guest is to be launched with a Secure Execution image, the kernel automatically calls into the ultravisor to decrypt and launch the image in a memory-protected context (i.e. when the guest memory is dumped, it will be encrypted). IBM's docs provide some futher information.

@jiazhang0 I generally agree.

Strictly speaking, there is no general image. Instead, just use a general initrd. The attestations for all platforms must perform the check the measurement of the general initrd and ensures it matches up the known good value. Due to the fact that agent config supplied by tenant is not provisioned yet in *1, launch AA prior to kata-agent. Next, more metadata supplied by tenant through an attested channel are provisioned, such as agent config, image decryption key, policy.json for signature verification and so on.

When we talk about configuration data, I think there is some flexibility between putting it in the initrd or getting it via the AA. In CCv0, for instance, we put the agent config file in the initrd. This is fine from a security perspective because the config file is not confidential. The downside is that the measurement of the initrd changes when we change the configuration. In CCv0 the URI of the KBS is in that config file. This is a problem because that is a dynamic parameter that can change somewhat frequently. In general, though, it might be fine to have some more static configuration parameters in the initrd. Of course this could be an issue if the CSP is supposed to provide a totally generic initrd that can be used by every tenant. Even in that case, however, some configuration might be common. For instance all tenants might be happy to share the same API endpoint configuration. I think part of our work for CCv1 is figuring out what configuration parameters can go in the initrd and which need to be dynamically provisioned.
Another place this comes up is when we try to use one initrd for multiple platforms. Is this something we want to support? To do so we may need to build the AA with multiple KBCs.

For *2, I would call the encrypted root disk as the protected data volume for snapshot storage, which is under the control of tenant for persistent storage. Of course, providing an encrypted boot disk is also allowed. Just add an extra step to decrypt and mount the encrypted boot disk after metadata provisioning.

We should probably have a separate issue somewhere to figure out how we are going to do 1) ephemeral block device for holding container images 2) secure volume support.

Secure Execution is the confidential computing architecture on s390x (IBM Z & LinuxONE). In Secure Execution, the workload owner has to put the kernel, command line, and initrd into an encrypted blob , which is specified as kernel for direct boot. The (symmetric) key used for this encryption is wrapped by one or more asymmetric keys specific to the machine(s) the image is allowed to run on (somewhat akin to how LUKS works). The public keys are publicly available (by serial number of the machine), the private keys are in the firmware of every machine since z15 & LinuxONE III, unreadable even to administrators. This means that you can put whatever secret data you want in the initrd and it will still be secure, e.g. SSH private keys, TLS certificates, LUKS keys, or keys for Attestation Agent's offline file system KBC. @jiazhang0 this is probably most in line with what you call "Protected Boot".
This is achieved through a so-called ultravisor, which is implemented in hardware and firmware. When a KVM guest is to be launched with a Secure Execution image, the kernel automatically calls into the ultravisor to decrypt and launch the image in a memory-protected context (i.e. when the guest memory is dumped, it will be encrypted). IBM's docs provide some futher information.

Thanks for your helpful info! I'll learn it from the docs.

@fitzthum Let me sum up about the use of initrd in Measured Boot/Unprotected Boot.

I think this is probably another issue to be discussed further about which configurations of kata-agent are static or dynamic.

Another place this comes up is when we try to use one initrd for multiple platforms. Is this something we want to support? To do so we may need to build the AA with multiple KBCs.

I think so. Choosing which KBCs are built (and anything affecting the contents of initrd) with AA is the top decision made by its provider. For unprotected/measured boot solution, CSP will decide it.

@jiazhang0 I agree with the overall setup of the schema you've posted.

About the capability of reloading kata-agent, I'm not sure whether is really needed.

I'm not a 100% sure either. Way earlier, we had discussed about e.g. metrics and that not every workload owner might have the same confidentiality requirements (easier access to metrics vs. side channel potential), therefore, one might want to hot reload the endpoints. I don't think the "(Optional) kata config" block really makes sense though, it's not an entity like the AA, the Kata agent or the Kata config.
Another minor nit: I know it's not an exhaustive list, but I think "Confidential/Dynamic Artifacts" should really mention image decryption keys, they're pretty much the most important part.

@Jakob-Naucke

Another minor nit: I know it's not an exhaustive list, but I think "Confidential/Dynamic Artifacts" should really mention image decryption keys, they're pretty much the most important part.

Thank you for pointing it out. I upload a new one.

About reloading the remotely provisioned kata-agent config, if a workload provider is not satisfied with the one supplied in the general initrd image provided by CSP, maybe it is useful to allow to reload kata-agent config.

Way earlier, we had discussed about e.g. metrics and that not every workload owner might have the same confidentiality requirements (easier access to metrics vs. side channel potential), therefore, one might want to hot reload the endpoints.

Even a workload owner doesn't need the confidentiality of application container image, the image has to be encrypted as long as the boot image/initrd is not provided by workload owner. This is a conclusion in Summary section.

I think this looks good. I think there are some things, like being able to reload the agent, that we could probably live without. The current approach where the Kata Agent starts the Attestation Agent is very simple and gets us most of what we want.

Therefore, the Unprotected Boot Image solution faces more security challenges and design difficulty than the Protected Boot Image solution due to its characteristics.

Malicious K8s admin can replace any components in Unprotected Boot Image at any time.
The TCB state of Pod is unverified before launch of attestation, so you cannot assume it can authenticate a KBS server, authenticate pulling an image or verify an image signature as expected.
Even the TCB state of Pod was verified, it only proves the TCB state of Pod reflected in attestation evidence at that moment is trust. In other words, a full attestation verification must be taken again in the next attestation procedure.

@jiazhang0, are the challenges associated with the Measured Boot/Unprotected Boot all because Dynamic Configurations are not captured in attestation?
I see two ways to cover Dynamic Configurations in attestation.

CSP dynamically augments the "General initrd" based on Pod Annotation to include the Dynamic Configurations in initrd, then launch the guest VM. The Dynamic Configurations will be covered by measured boot and attestation. This approach does alter the measurement value in the attestation. The Relying Party would have to manage a mapping between the Dynamic Configuration and the expected measurement.
For TEEs that support "Init Time Configuration", such as AMD(r) SEV-SNP's HostData and Intel(R) SGX/TDX's CONFIGID, the hash of the Dynamic Configurations can be included in the attestation (I have submitted an RFC in image-rs repo). With this approach, the "General initrd" measurement does not change.

With these two approaches, it seems that we can achieve security property similar to "Protected Boot Image", even if the CSPs provide the Boot Image, as long as the workload provider audits (directly, or relying on a trusted third-party auditor) and accepts the Boot Image from the CSP and its expected measurement.

@jiazhang0 is this issue still relevant or can be closed?
If it's still relevant to what release do you think we should map it to (mid-November, end-December, mid-February etc...)?


Instructions for setting up environment without colab are slightly outdated

(Porting discussion from TA training)
https://github.com/NeuromatchAcademy/precourse/blob/master/getting-ready-python/02-getting-ready-python-without-colab.md#how-to-get-started-on-python-tutorials-without-google-co-lab
The course-content repo defines an environment.yml (and requirements.txt) file, so we can cut the instructions for writing that (the dependencies are also slightly out of date).

I recently set up my environment like this :
conda env create --file=environment.yml


[homework]  Reading & experience report

Asking

[x] Ask ChatGPT for help completing this homework

Feeling, Writing, Thinking

[ ] Post an experience report (a paragraph or several) as a comment on this issue. This experience report should convey what motivates you to understand and practice data thinking. An experience report may describe a collection of experiences (which may or may not relate to data), and some questions to consider may be helpful: What were you or those involved feeling? Doing? Thinking? Asking? Where were they? What systems were involved, if any? Who would you want to better understand the incentives at play?

if you are stuck, ask for help on Zulip, come to office hours, make a plan of tasks to do and ask for feedback on the plan
try using dictation on iPhone, Android, computer, and ask ChatGPT to summarize it
ask Lex for ideas or ChatGPT for ideas
edit using https://hemingwayapp.com/, Grammarly, or any other tool to help
read the below articles for ideas

Doing

[x] Review the first class: https://panopto.ut.ee/Panopto/Pages/Sessions/List.aspx?folderID=43bb180c-79a6-4324-b055-afa400ecd1a0 - ask on Zulip if you can't access this
[x] Review the Figma whiteboard from the first class: https://www.figma.com/file/SsoNLhaZwnSGkADd4eS0D8/230220-introduction?node-id=0%3A1&t=jR2hISrDchAPapYR-1
[x] Create a LinkedIn profile. (If you are not comfortable using your real name, feel free to use a fake name + email)
[x] Post your LinkedIn profile on Zulip -- Zulip is open source and enables us to analyze our chat messages with natural language processing tools such as ChatGPT, which we will do later in the course!
[x] Read https://matt.might.net/articles/how-to-email/ to understand principles of communication over email or Zulip
[x] Read parts of the Mailchimp Style Guide (https://styleguide.mailchimp.com/voice-and-tone/ or https://styleguide.mailchimp.com/writing-principles/) to understand how written communication can be guided by principles
[x] Read https://georgesaunders.substack.com/p/on-worry to understand a professional writer's perspective on writing.
[x] Read https://every.to/chain-of-thought/writing-essays-with-ai-a-guide to understand how to use voice recording, dictation on your smartphone or computer, and ChatGPT, and Lex to help you write
[x] Read https://simonwillison.net/2022/Dec/5/rust-chatgpt-copilot/ to understand how ChatGPT and CoPilot can help you learn data thinking, programming, and statistics skills
[x] Read http://web.archive.org/web/20170208134519/http://storyshots.tumblr.com/post/25032057278/22-storybasics-ive-picked-up-in-my-time-at-pixar -- especially the "Once upon a time" story telling framework Pixar used for its movies.
[x] Read https://www.newyorker.com/magazine/2023/02/13/the-dubious-rise-of-impostor-syndrome to help understand concepts that can increase our psychological safety and trust in class and how asking ChatGPT can help reduce our fear of asking for help when stuck.
[x] Read https://jaan.io/my-friend-radicalized-this-made-me-rethink-how-i-build-AI/ to learn about my journey in understanding the incentives that can affect our thoughts, feelings, and actions in society and the incentives that affect how AI such as ChatGPT is built.

Programming & Infrastructure

[x] Create a GitHub Codespace using the data thinking template here (from https://github.com/onefact/datathinking.org-codespace)
[x] Send a screenshot to Zulip of a plot you made in the GitHub codespace (of any dataset or data you generated as in the live coding demo from the first lecture; ask for help if this is unclear)
[x] Sign up for Lex - message Jaan on Zulip for an invite code
[x] Sign up for Figma as a student (https://www.figma.com/education/students/)
[x] Install copyq on Windows Powershell or Linux using Chocolatey, or install Maccy on Mac using Homebrew. This will save you a lot of time throughout the class as you copy and paste things from ChatGPT!

Writing tests with Copilot (https://til.simonwillison.net/gpt3/writing-test-with-copilot) was a very excellent article! It is under the "AI assisted learning: Learning Rust with ChatGPT, Copilot and Advent of Code" article (https://simonwillison.net/2022/Dec/5/rust-chatgpt-copilot/). It helps to save time and avoid routine.
I had ChatGPT search for information about Moonsund regattas from 2000 to 2022 (http://muhuvain.ee/tulemused-1/). I had it create a table showing the number of participating sailboats and the year of participation, and I also had it add a column for the number of sailors. Searching for these numbers one by one on the website would be quite a time-consuming task. I instructed ChatGPT to write Python code that generates a comprehensive graph, and it did so successfully. However, one major drawback is that the data is only sometimes accurate. Fortunately, ChatGPT can be trained by pointing out errors, and it will correct them.
Nevertheless, one should only sometimes believe everything ChatGPT responds with. For example, I asked some verification questions, such as who won in the ORC I class in 2019. It answered incorrectly. I am knowledgeable about this area, so I corrected it. I asked about the second place, and it responded poorly again - so mistakenly that the sailboat offered was not even in the ORC I class, and the skipper was not even from that sailboat! Fortunately, the trends depicted on the graph are correct. The number of participants was significantly lower during the economic downturn in 2008 and the year of COVID in 2020.


Soft Delete Retention days

A customer has contacted and is seeking clarification on the following statement from this documentation. The customer says it reads like you should be able to adjust the retention period by means other than the portal (CLI, PS, API, etc).
"The default retention period is 90 days but, during key vault creation, it is possible to set the retention policy interval to a value from 7 to 90 days through the Azure portal. The purge protection retention policy uses the same interval. Once set, the retention policy interval cannot be changed."
My suggestion is to either eliminate "through the Azure portal" or to change it to say, "Once set, the retention policy interval cannot be changed [by any means]." Making this change will make it absolutely clear to customers that this parameter cannot be changed once set.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 5fb5d787-c259-1083-c021-97fb0790122e
Version Independent ID: 352d7d3f-da52-bcd0-3efb-587e99066967
Content: Azure Key Vault soft-delete
Content Source: articles/key-vault/general/soft-delete-overview.md
Service: key-vault
Sub-service: general
GitHub Login: @msmbaldwin
Microsoft Alias: mbaldwin

@jesshaines
Thanks for your feedback! We will investigate and update as appropriate.

Thank you @jesshaines . We have updated the document. #please-close


Pocket world Shadows - <IP_ADDRESS> & earlier

Hi there,
I have had this problem since I started trying to use MCEdit, which has obviously made it unusable because it is a world destroyer.
Once a map has been edited, the chunk will become filled with these extremely dark shadows as soon as you interact with it. Either adding or removing blocks will do it. The more you do to the chunk, the more the shadow spreads.
Here I opened a basic world, added a few structures in MCEdit, and then sent it back to the iPhone for testing.

Cheers!

Does adding a light source to the affected chunk(s) reloading help? Either via pocket or MCEdit.

Hi Khroki,
Adding a light source lights up the surrounding areas in the typical light source fashion. It acts like a light inside a dark cave, rather than recalculating the lighting for the area.

That's strange, it almost looks like there is a block of some sort at the top of the world blocking the sun.

Here is some video in case the behaviour helps you to identify the problem.
https://www.youtube.com/watch?v=Lq-8N_ZX25U

Confirmed, looks like an issue with how sun lighting is handled. We'll get to it when we can.

I've used other world editing tools on MCPE worlds and get the same result - adding or removing large numbers of blocks causes this shadow issue. These are Android apps I have used, and the issue only started with the 0.14 update.

@Robbeedee, @builddroid: I'm closing this issue now, because it has not been updated since a year.
We alos have worked on a new PE support, and Mojang changed the PE world format. (Lights were stored in the world, but in PE 1.1, they're calculated on-the-fly).
Note that, this new PE support is availble only if you run MCEdit from source for now; and using the 'new feature' stuff described here.
The new support will be the default one in the next release :smile:


editor: problem matcher for oxlint errors

What version of Oxlint are you using?
0.9.9
What command did you run?
No response
What does your .oxlint.json config file look like?
No response
What happened?
It would be great if there would be a problem matcher to transfer the found issues to the problems tab

I think we have this feature. What are your specific requirements? Could you describe it?

If I want a task like a watcher task for the complete codebase for example.

Does the language server support something like this? A way to request all issues for the entire workspace?
Is the Problems tab in VS Code the right place for this? From what I can tell, only problems for open files show there.  Are there any other plugins that show problems there for files that are not opened?

We'd have to register a problemMatcher capability in our package.json

Does the language server support something like this? A way to request all issues for the entire workspace?

Any idea on this @DonIsaac ?
I can probably come up with some workarounds like identifying all files and requesting diagnostics for each, but I’m not sure if there is a better alternative.

Does the language server support something like this? A way to request all issues for the entire workspace?

Any idea on this @DonIsaac ?
I can probably come up with some workarounds like identifying all files and requesting diagnostics for each, but I’m not sure if there is a better alternative.

Looking at the problem matcher, this probably isn’t relevant for VS Code, but I do think it is for IntelliJ. I’ll review it some more and make an IntelliJ specific issue when I know some more about it.


Wallet with Tokens disappeared after Migration and Upgrade to 1.2.0

Issue description
I have a few different Trinity wallets that needed to be migrated to Firefly and IOTA 1.5. I migrated 1 successfully yesterday, and then noticed that a new version of Firefly was available. I upgraded to 1.2.0, and soon after, migrated a second Trinity wallet to Firefly successfully. However, when I accessed my first Firefly account today, the interface stated that I had "no wallets" and showed 0 Miota (which is not correct). I tried to "Create a Wallet," which resulted in the error report copied below.
The second account appears to be fine and the wallet and all tokens are accounted for. I have no idea why the wallet / tokens with the first account disappeared. I'm not sure if it's related to the upgrade or not.
Wallet version
1.2.0 now (1.1.0 recently)
Error Report
App Version: 1.2.0
platform: macOS
platformVersion: 10.15
platformArchitecture: x64
Error Type: Render Context Error
TypeError: Cannot read property 'signerType' of undefined
at a (file:///Applications/Firefly.app/Contents/Resources/app.asar/public/build/index.js:2:2911806)
at file:///Applications/Firefly.app/Contents/Resources/app.asar/public/build/index.js:2:2912401
at n (file:///Applications/Firefly.app/Contents/Resources/app.asar/public/build/index.js:2:2843814)
at m (file:///Applications/Firefly.app/Contents/Resources/app.asar/public/build/index.js:2:2843984)
at HTMLInputElement. (file:///Applications/Firefly.app/Contents/Resources/app.asar/public/build/index.js:2:2201023)

Just FYI - I was easily able to create another profile, enter my 24 word recovery phrase, and the wallet / token amount from the 1st migration (discussed above) appeared again.

This issue can be closed as the tokens appeared after creating a new profile


ImportError: cannot import name 'DtypeArg' from 'pandas._typing'

which version of python is used？
Traceback (most recent call last):
File "main.py", line 7, in
from train import train
File "/home/robin/cuiyajie/FastFlow-master/train.py", line 12, in
import neptune.new as neptune
File "/home/robin/anaconda3/envs/patchcore/lib/python3.7/site-packages/neptune/init.py", line 27, in
from neptune.internal.api_clients import backend_factory
File "/home/robin/anaconda3/envs/patchcore/lib/python3.7/site-packages/neptune/internal/api_clients/init.py", line 17, in
from neptune.internal.api_clients.hosted_api_clients.hosted_backend_api_client import (
File "/home/robin/anaconda3/envs/patchcore/lib/python3.7/site-packages/neptune/internal/api_clients/hosted_api_clients/hosted_backend_api_client.py", line 40, in
from neptune.internal.api_clients.hosted_api_clients.hosted_alpha_leaderboard_api_client import (
File "/home/robin/anaconda3/envs/patchcore/lib/python3.7/site-packages/neptune/internal/api_clients/hosted_api_clients/hosted_alpha_leaderboard_api_client.py", line 37, in
from neptune.internal.api_clients.hosted_api_clients.mixins import HostedNeptuneMixin
File "/home/robin/anaconda3/envs/patchcore/lib/python3.7/site-packages/neptune/internal/api_clients/hosted_api_clients/mixins.py", line 35, in
from neptune.utils import with_api_exceptions_handler
File "/home/robin/anaconda3/envs/patchcore/lib/python3.7/site-packages/neptune/utils.py", line 30, in
import pandas as pd
File "/home/robin/anaconda3/envs/patchcore/lib/python3.7/site-packages/pandas/init.py", line 142, in
from pandas.io.api import (
File "/home/robin/anaconda3/envs/patchcore/lib/python3.7/site-packages/pandas/io/api.py", line 8, in
from pandas.io.excel import ExcelFile, ExcelWriter, read_excel
File "/home/robin/anaconda3/envs/patchcore/lib/python3.7/site-packages/pandas/io/excel/init.py", line 1, in
from pandas.io.excel._base import ExcelFile, ExcelWriter, read_excel
File "/home/robin/anaconda3/envs/patchcore/lib/python3.7/site-packages/pandas/io/excel/_base.py", line 32, in
from pandas.io.parsers import TextParser
File "/home/robin/anaconda3/envs/patchcore/lib/python3.7/site-packages/pandas/io/parsers/init.py", line 1, in
from pandas.io.parsers.readers import (
File "/home/robin/anaconda3/envs/patchcore/lib/python3.7/site-packages/pandas/io/parsers/readers.py", line 17, in
from pandas._typing import (
ImportError: cannot import name 'DtypeArg' from 'pandas._typing' (/home/robin/anaconda3/envs/patchcore/lib/python3.7/site-packages/pandas/_typing.py)

I used version 3.8 because it seems that it is required for Neptune to work


Inherited DataTemplate not rendering in WinUI for Desktop

The following markup renders correctly
<ContentControl>
    <ContentControl.ContentTemplate>
        <DataTemplate>
            <TextBlock Text="A"/>
        </DataTemplate>
    </ContentControl.ContentTemplate>
</ContentControl>

But if DataTemplate is inherited, nothing is rendered
public class MyDataTemplate : DataTemplate
{
}

<ContentControl>
    <ContentControl.ContentTemplate>
        <local:MyDataTemplate>
            <TextBlock Text="A"/>
        </local:MyDataTemplate>
    </ContentControl.ContentTemplate>
</ContentControl>

Here it is the repo project
App9.zip

More info. Calling LoadContent of the inherited DataTemplate doesn't work either. It returns null.

@arivoir do you know if this works in a UWP?
@MikeHillberg FYI

No. I know it works in WPF

@arivoir For your repro the derived class from DataTemplate doesn't actually add anything. I'm assuming that's just to produce a min repro. But can you capture what you intended to do with a derivation of DataTemplate? AKA, what was the scenario?

We're using DataTemplate to allow customers specifying icon templates, which are created from a class inherited from DataTemplate (they can be instantiated multiple times for the same control). It's a way to say customers that template is to specific  for icons.


Systems: SGCI Resource Specification - Support API in Tapis V3

[ ] Design endpoint to import from SGCI catalog

The SGCI Resource Specification effort (aka Tech Summit) is creating a reference database (via github) and reference API implementation to be used for sharing HPC and Cloud resources. The goal is for Tapis V3 to support the reference API allowing for a Tapis site to pull in resource definitions, store them locally and make them available to site users. Except from 4/24/20 e-mail written by Joe S:

Gateways and frameworks can and should utilize a local, database-backed catalog of the resources that provides an API for searching for and reading the descriptions. That system should periodically sync/receive updates from github for changes. We will provide a reference implementation of this API system, and the major frameworks (Airavata, HubZero and Tapis) will commit to hosting said API system and incorporating it into their services.
Suresh and Dimuthu made some serious progress on the github catalog and reference implementation of the API, and they gave a walkthrough and demo of initial versions of each:
[1] https://github.com/SGCI/computational-resource-catalog
[2] https://github.com/SGCI/computational-resource-reference-implimentation

An instance of the reference API is deployed here, ready to be tried out:
https://resources.sgci.scigap.org

and there is some very nice documentation, including an overall workflow diagram here:
https://docs.google.com/document/d/1gr5dl-5qqlsajKYeUupc5seeDWAE2BaRGMmaJoKUaPY/edit

Big thanks to Suresh and Dimuthu for all of the above!

Our discussion also included the following points and action items:

1. The group thought it would be best to prioritize creating designs for how the resource API would be incorporated into the existing frameworks: Airavata, HubZero and Tapis. We expect the exercise of working out the designs to help improve and finalize the resource spec itself. Steve and Joe will work on Tapis and think about HPC resources, Suresh and Dimuthu will work on Airavata and think about HPC and cloud resource, Daniel will work on HubZero.

Add Tapis v3 Systems endpoints to create and update system definitions based on an SGCI Resource with the latest information from the SGCI Resource catalog.


how to highlight the cpp <stdio.h>

		<title>Highlight</title>
		<link rel="stylesheet" href="../highlight/styles/default.css">

#include <stdio.h>

int main(int argc, char **argv)
{
	printf("%s\n", "hello world!");

	return 0;
}

i use this code to test, but in the web page, <stdio.h> is gone,how can i do with this problem.
thanks for help.

You need to escape < and > since it ends up being turned into an html tag, even if it isn't a valid tag.
So change <stdio.h> into <stdio.h>


Fiddle: update to webpack-dev-server v4

Description

Checklist:

[ ] I have updated the CHANGELOG with a summary of my changes

https://github.com/webpack/webpack-dev-server/blob/master/migration-v4.md

HMR doesnt seem to work right now :(


Option to remove extra hunger

when sleeping with mods that do more ticks while sleeping i die because my hunger drains too fast

Closing because not going to happen


Fix a false negative for Rails/HasManyOrHasOneDependent

Follow up of https://github.com/rubocop-hq/rubocop/issues/6266#issuecomment-422651337.
This PR fixes a false negative for Rails/HasManyOrHasOneDependent when using the following mix-in case.
module Foo
  extend ActiveSupport::Concern

  included do
    has_many :bazs # 0.59.1 emits offense for this line
  end
end

class Bar < ApplicationRecord
  include Foo
end

In #6298, only subclasses of AactiveRecord::Base or ApplicationRecord were scoped. But in that approach false negatives occur in the above case.
This PR changes the approach to solve the issue #6266 by skipping it in case of subclass of ActiveResource::Base.
Also, this PR integrates a series of change log entry which has not yet been released for the fixing of #6266.
Cc @tejasbubane @yskkin

Before submitting the PR make sure the following are checked:

[x] Wrote good commit messages.
[ ] Commit message starts with [Fix #issue-number] (if the related issue exists).
[x] Feature branch is up-to-date with master (if not - rebase it).
[x] Squashed related commits together.
[x] Added tests.
[x] Added an entry to the Changelog if the new code introduces user-observable changes. See changelog entry format.
[x] The PR relates to only one subject with a clear title
and description in grammatically correct, complete sentences.
[x] Run rake default. It executes all tests and RuboCop for itself, and generates the documentation.

Thank you for looking into this!
It would be great if rejected list of parent class is configurable via rubocop.yml.
For example, active_model_serializer also has has_many and rubocop cannot fully enumerate such classes.

Good catch! I think that makes sense.
This PR aims to fix known false negative. I'd like to consider it as another PR.


td-agent fails to install

hi
I attempted to install using curl -L https://toolbelt.treasuredata.com/sh/install-ubuntu-xenial-td-agent2.sh | sh
on ubuntu Xenial  I get the follwoing error:
Setting up td-agent (2.3.3-0) ...
Conffile /etc/td-agent/td-agent.conf has been modified. Remain untouched.
Job for td-agent.service failed because the control process exited with error code. See "systemctl status td-agent.service" and "journalctl -xe" for details.
invoke-rc.d: initscript td-agent, action "start" failed.
dpkg: error processing package td-agent (--configure):
subprocess installed post-installation script returned error exit status 1
Errors were encountered while processing:
td-agent
W: --force-yes is deprecated, use one of the options starting with --allow instead.
E: Sub-process /usr/bin/dpkg returned an error code (1)
Output of 'systemctl status td-agent.service':
● td-agent.service - LSB: data collector for Treasure Data
Loaded: loaded (/etc/init.d/td-agent; bad; vendor preset: enabled)
Active: failed (Result: exit-code) since Sun 2016-12-04 17:11:43 AEDT; 2min 3
Docs: man:systemd-sysv-generator(8)
Process: 5574 ExecStart=/etc/init.d/td-agent start (code=exited, status=1/FAIL
Dec 04 17:11:43 db.ufsrv systemd[1]: Starting LSB: data collector for Treasure D
Dec 04 17:11:43 db.ufsrv td-agent[5574]: Starting td-agent: start-stop-daemon: u
Dec 04 17:11:43 db.ufsrv td-agent[5574]:  * td-agent
Dec 04 17:11:43 db.ufsrv systemd[1]: td-agent.service: Control process exited, c
Dec 04 17:11:43 db.ufsrv systemd[1]: Failed to start LSB: data collector for Tre
Dec 04 17:11:43 db.ufsrv systemd[1]: td-agent.service: Unit entered failed state
Dec 04 17:11:43 db.ufsrv systemd[1]: td-agent.service: Failed with result 'exit-

This is fluentd repository, not td-agent.
Please re-open on omnibus-td-agent repository.


Add numberOfLines as a prop for the dropdown text

If dropdown options are too long, they are shown with an ellipses.  This option allows you to specify that you want multiple lines of text to show.

Hi @teamneem if dropdown option is too long they are getting cut from the start of the option (few letters getting cut on android), is there any fix ?


Review OAuth2 status

This is in principle implemented, but we never really test it.
First goal: let's get a basic example working (with either github or google OAuth2 system), and document the way it works.
If fractal-server causes any issue, we can even test a vanilla fastapi+fastapi_users example.

Log for @ychiucco

Fix variable names in tests/data/example_server_startup/startup.sh
Register application on https://github.com/settings/applications/new, with callback something like http://<IP_ADDRESS>:8000/auth/callback
Add your own OAUTH_GITHUB_CLIENT_ID and OAUTH_GITHUB_CLIENT_SECRET to env variables.
Startup example fractal-server instance
Update admin email to your own github one.
Issue a command like

curl -X 'GET' \
  'http://<IP_ADDRESS>:8000/auth/github/authorize?scopes=string' \
  -H 'accept: application/json'

Open the URL that you get in the reponse
Authorize (the first time)
500 internal server error, due to email=None ???

Great, thanks!
After re-discussing with @ychiucco, the next steps would be a similar minimal-working example (or even a Fractal example) where the authorize/callback endpoints are exposed via fastapi-users, and not custom. This would require interacting with the browser first, obtaining a fractal_server token (rather than a github token), putting it in the right file (~/.cache/fractal/session), and trying to run fractal whoami from the command line.
NOTE: the user must already exist in the DB with the same email (at least for now). Note that this also requires to have a public email set on Github, which is OK for testing.


Remove related flows together with PFCP sessions

This is needed to avoid losing IPFIX reports for the flows that still
exist when a PFCP session is deleted.
As part of the change, flowtable hooks are replaced with flowtable
events which are more flexible (multiple handlers per event).
Possible inconsistencies related to flows that don't correspond to any
active session are now being avoided.

A bit of explanation:

there was a “hook” mechanism that enabled to lessen circular
dependencies between the flowtable code and the rest of UPG; the
problem with it was that it only allowed one handler per hook, which
became inconvenient with the new changes.  It has been replaced with
'events', with some added comments:
https://github.com/travelping/upg-vpp/blob/56b2959241ebd859d5cf07c77f4c5f8180f2e539/upf/flowtable.h#L237-L267
Sessions' flows are organized in a singly-linked list. Each flow
points to the next one belonging to the same session:
https://github.com/travelping/upg-vpp/blob/56b2959241ebd859d5cf07c77f4c5f8180f2e539/upf/flowtable.h#L168
and the session contains the index of the first flow:
https://github.com/travelping/upg-vpp/blob/56b2959241ebd859d5cf07c77f4c5f8180f2e539/upf/upf.h#L800
when a flow is allocated, it's linked into the chain by making it
point to the previously-first flow for the session, and then making
session's first flow index point to it:
https://github.com/travelping/upg-vpp/blob/56b2959241ebd859d5cf07c77f4c5f8180f2e539/upf/upf_flow_node.c#L225-L237
when a flow is deleted, first FLOW_EVENT_REMOVE occurs which is
handled by the IPFIX code to send the final report, if needed
https://github.com/travelping/upg-vpp/blob/56b2959241ebd859d5cf07c77f4c5f8180f2e539/upf/upf_ipfix.c#L492
and also by the proxy
https://github.com/travelping/upg-vpp/blob/56b2959241ebd859d5cf07c77f4c5f8180f2e539/upf/upf_proxy.c#L1403
so as to remove any connections related to this flow. There's some
complication with half-open active-side connections, when such one
is present for the flow, the proxy session is unlinked from the flow
and kept till the callback is invoked, which will then proceed with
further cleanup.
after all FLOW_EVENT_REMOVE handlers are called, FLOW_EVENT_UNLINK
is invoked which is handled here
https://github.com/travelping/upg-vpp/blob/56b2959241ebd859d5cf07c77f4c5f8180f2e539/upf/upf_pfcp.c#L1169-L1208
Basically, it removes the flow from the linked list of the flows
belonging to the particular session. There's a bit of compromise in
this part: removing the flow costs O(n), n being the number of
flows per session. We could use a doubly-linked list instead, so as to
have O(1) during flow deletion, but that would further enlarge the
flowtable entries, decreasing performance.  (Alternative, maybe
implement later: keep flow list linkage as a separate vector) The
in pfcp_free_session, all of the flows belonging to it are
forcibly removed:
https://github.com/travelping/upg-vpp/blob/56b2959241ebd859d5cf07c77f4c5f8180f2e539/upf/upf_pfcp.c#L1140-L1151
This will invoke FLOW_EVENT_REMOVE and FLOW_EVENT_UNLINK events
for each flow, making sure IPFIX reports are sent and proxy
connections are dealt with.


[DOC]

Report incorrect documentation
Location of incorrect documentation

https://github.com/NVIDIA-Merlin/Transformers4Rec/blob/main/examples/README.md?plain=1
Line number 26 and 35
Describe the problems or issues found in the documentation

Links for "Tutorial - End-to-End Session-Based Recommendation on GPU" and "Transformers4Rec paper experiments reproducibility" are missing "examples/" in the path.
Steps taken to verify documentation is incorrect

To click on the links:
Tutorial - End-to-End Session-Based Recommendation on GPU
Transformers4Rec paper experiments reproducibility
Expected: to follow the link
Actual: 404 error
Suggested fix for documentation

To update:
https://github.com/NVIDIA-Merlin/Transformers4Rec/tree/main/tutorial/ to https://github.com/NVIDIA-Merlin/Transformers4Rec/tree/main/examples/tutorial/
and
https://github.com/NVIDIA-Merlin/Transformers4Rec/tree/main/t4rec_paper_experiments/ to https://github.com/NVIDIA-Merlin/Transformers4Rec/tree/main/examples/t4rec_paper_experiments/

Report needed documentation
Report needed documentation

Describe the documentation you'd like

Steps taken to search for needed documentation**

@rnyak great, I'm closing the issue


chore: release v1.7.1

:tada: This PR is included in version 1.7.1 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


components: refactor handleUpdateComponent

There is an implicit_this coverity issue #1009 with the call to handleUpdateComponent from the ComponentDetailsView. The function call has been refactored to follow the same pattern as the call to handleAddComponent which does not have a coverity issue.

Thanks @jkozol!
osbuild-* landed on fedora 31 finally, NICE!!
Can we enable coverity based on pull request? Thanks!

@henrywang Unfortunately the free version of coverity only allows for a limited number of builds. This is why we have a daily trigger in our github actions.

The number of weekly builds per project are as follows:

Up to 28 builds per week, with a maximum of 4 builds per day, for projects with fewer than 100K lines of code
Up to 21 builds per week, with a maximum of 3 builds per day, for projects with 100K to 500K lines of code
Up to 14 builds per week, with a maximum of 2 build per day, for projects with 500K to 1 million lines of code
Up to 7 builds per week, with a maximum of 1 build per day, for projects with more than 1 million lines of code

Make sense to me. Thanks @jkozol!


Provide low-level API for db-sync

The integration work of the first version of UTxO-HD with db-sync was intended to avoid using tables and just carrying an EmptyMK ledger state all over the execution, with the full UTxO set inside, just as before UTxO-HD.
However, it turns out that our API is too restrictive as it requires ValuesMK for ticking and applying blocks, which will then be stowed. It is perfectly fine for Consensus' concerns but not for this db-sync integration as stowing/unstowing the full UTxO set duplicates the map on each step and therefore memory consumption explodes.
We should provide lower level alternatives to the tickThenReapply function that don't expect to have the tables unstowed, i.e.
tickThenReapply' :: ... -> LedgerState blk EmptyMK -> LedgerState blk EmptyMK

Possibly this just means factoring out some of the things we do (the stowing and unstowing of values) into a separate function, something like tickThenReapplyWithTables or some such, to get:
tickThenReapplyWithTables :: ... -> LedgerState blk ValuesMK -> LedgerState blk ValuesMK
tickThenReapplyWithTables = unstow . tickThenReapply' . stow

(Probably with some applyDiffs in between).
db-sync has a half-working integration which uses ouroboros-network at ea8f657533b64ad3494d3695c945c8fb24c6a0f1.
I think we should just start a branch there and do whatever we need to do to provide this API, but even maybe only in the packages needed by db-sync, to not get into trouble fighting the tests, as this is exploratory integration work. In particular db-sync uses:

ouroboros-consensus
ouroboros-consensus-diffusion
ouroboros-consensus-protocol
ouroboros-consensus-byron
ouroboros-consensus-shelley
ouroboros-consensus-cardano

Thanks! Fyi this was my attempt to use the existing api https://github.com/input-output-hk/cardano-db-sync/blob/e841d95123d25f054ef355a5734bc123bb6dcae1/cardano-db-sync/src/Cardano/DbSync/LedgerState/ConsensusApi.hs#L30. It had performance issues, because of the need to unstow and stow back the ledger state.


More Test for Mewbase Options

The tests for mewbase options need to cover the default and modified values for EventSource
and EventSink.
Easy way into the code base for the Event side of the Library

Closed by #149


DOC: Fix path to built file

Apart from this minor fix in the doc file, I have a question about building KaTeX with a custom path to fonts.
Turns out when you modify @font-folder in submodules/katex-fonts/fonts.less, webpack will no longer find them in submodules/katex-fonts/fonts/ and the dist/fonts/ folder will be missing in the build. Two-step process is needed in this case:

Build with default fonts.less file, and copy dist/fonts/ folder where you need.
Modify fonts.less and build again to get correct css files.

I wonder whether this could be improved, or perhaps at least stated a bit more explicitly in this doc?

@danielhollas what kind of change were you making to @font-folder?

@kevinbarabash I was rebuilding KaTeX for Discourse Math plugin, which requires:
@font-folder: "/plugins/discourse-math/katex/fonts";

Thanks for the fix. Perhaps worth moving the rest to another issue? I imagine this is standard NPM/webpack stuff that has a solution, but not being a webpack expert, I don't know what it is.

[clabot:check]


[Collect approvers] [$500] Offline Pay of an expense will apply even if the submitter edits an expense amount

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Version Number: 1.4.41-1
Reproducible in staging?: y
Reproducible in production?: y
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Expensify/Expensify Issue URL:
Issue reported by: @garrettmknight
Slack conversation: https://expensify.slack.com/archives/C049HHMV9SM/p1707832645778869
Action Performed:

User A invites User B to Free workspace
User B submits an expense
User A goes offline
User A taps ‘Pay’ on the original expense
User B edits the expense amount OR adds an additional expense while User A is offline
User A comes back online

Expected Result:
Payment/Approval is stopped and the admin/approver is notified of the updated amount
Actual Result:
The updated expense amount is paid.
Workaround:
unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[ ] Android: Native
[ ] Android: mWeb Chrome
[x] iOS: Native
[ ] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[ ] MacOS: Desktop

Screenshots/Videos
Add any screenshot/video evidence

View all open jobs on GitHub
Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~01678bf0efdcb7e4a2
Upwork Job ID:<PHONE_NUMBER>892195328
Last Price Increase: 2024-02-13

Asked for clarification here: https://expensify.slack.com/archives/C049HHMV9SM/p1707941200321529?thread_ts=1707832645.778869&cid=C049HHMV9SM

@Gonals do you know the answer to that ^?

#36983

Yep. this should fix both.

What's the next steps here?

I've done the C+ review and testing, looking for the internal engineer feedback/merge.

Cool, @Gonals looks like we're ready to rock here!

@mananjadhav friendly reminder about the checklist so that we can pay tomorrow.

@mananjadhav bump!

I'll work on the checklist today.

I tried finding the offending PR but couldn't find one. We added another property in the optimistic data, and I am not sure at what point we should've included this along with Math.abs.
But I do think we should add a regression test for this one.
Regression test for this one:

User A invites User B to Free workspace
User B submits an expense
User A goes offline
User A taps ‘Pay’ on the original expense
User B edits the expense amount OR adds an additional expense while User A is offline
User A comes back online
Confirm the money request is not paid and User A sees an error message: The requested amount has changed.

This will serve the linked duplicate issue as well.
@puneetlath Can you confirm the payout summary posted by the bot?

Issue for regression test: https://github.com/Expensify/Expensify/issues/379640
I am confirming that the payment summary here is accurate. @mananjadhav please request on NewDot.

$500 approved for @mananjadhav based on summary.


Error occurs when navigating to Clouds -> Instances

INFO -- : Completed 200 OK in 1152ms (Views: 1005.2ms | ActiveRecord: 34.4ms)
INFO -- : Started GET "/vm_cloud/explorer" for <IP_ADDRESS> at 2015-08-14 13:27:15 +0200
INFO -- : Processing by VmCloudController#explorer as HTML
FATAL -- : Error caught: [ActiveRecord::SubclassNotFound] The single-table inheritance mechanism failed to locate the subclass: 'AvailabilityZoneOpenstackNull'. This error is raised because the column 'type' is reserved for storing the class in case of inheritance. Please rename this column if you didn't intend it to be used for storing the inheritance class or overwrite AvailabilityZone.inheritance_column to use another column for that information.
lib/ruby/gems/2.2.0/gems/activerecord-4.2.3/lib/active_record/inheritance.rb:186:in `rescue in find_sti_class'
lib/ruby/gems/2.2.0/gems/activerecord-4.2.3/lib/active_record/inheritance.rb:180:in `find_sti_class'
lib/ruby/gems/2.2.0/gems/activerecord-4.2.3/lib/active_record/inheritance.rb:169:in `discriminate_class_for_record'
lib/ruby/gems/2.2.0/gems/activerecord-4.2.3/lib/active_record/persistence.rb:67:in `instantiate'
lib/ruby/gems/2.2.0/gems/activerecord-4.2.3/lib/active_record/querying.rb:50:in `block (2 levels) in find_by_sql'
lib/ruby/gems/2.2.0/gems/activerecord-4.2.3/lib/active_record/result.rb:51:in `block in each'
lib/ruby/gems/2.2.0/gems/activerecord-4.2.3/lib/active_record/result.rb:51:in `each'
lib/ruby/gems/2.2.0/gems/activerecord-4.2.3/lib/active_record/result.rb:51:in `each'
lib/ruby/gems/2.2.0/gems/activerecord-4.2.3/lib/active_record/querying.rb:50:in `map'
lib/ruby/gems/2.2.0/gems/activerecord-4.2.3/lib/active_record/querying.rb:50:in `block in find_by_sql'
lib/ruby/gems/2.2.0/gems/activesupport-4.2.3/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
lib/ruby/gems/2.2.0/gems/activerecord-4.2.3/lib/active_record/querying.rb:49:in `find_by_sql'
lib/ruby/gems/2.2.0/gems/activerecord-4.2.3/lib/active_record/relation.rb:638:in `exec_queries'
lib/ruby/gems/2.2.0/gems/activerecord-4.2.3/lib/active_record/association_relation.rb:32:in `exec_queries'
lib/ruby/gems/2.2.0/gems/activerecord-4.2.3/lib/active_record/relation.rb:514:in `load'
lib/ruby/gems/2.2.0/gems/activerecord-4.2.3/lib/active_record/relation.rb:243:in `to_a'
lib/ruby/gems/2.2.0/gems/activerecord-4.2.3/lib/active_record/relation/finder_methods.rb:500:in `find_nth_with_limit'
lib/ruby/gems/2.2.0/gems/activerecord-4.2.3/lib/active_record/relation/finder_methods.rb:484:in `find_nth'
lib/ruby/gems/2.2.0/gems/activerecord-4.2.3/lib/active_record/relation/finder_methods.rb:127:in `first'
lib/ruby/gems/2.2.0/gems/activerecord-deprecated_finders-1.0.4/lib/active_record/deprecated_finders/relation.rb:129:in `first'
app/models/rbac.rb:391:in `search'
app/models/rbac.rb:288:in `filtered'
app/presenters/tree_builder.rb:360:in `rbac_filtered_objects'
app/presenters/tree_builder.rb:342:in `rbac_filtered_objects'
app/presenters/tree_builder_instances.rb:28:in `x_get_tree_ems_kids'
app/presenters/tree_builder.rb:252:in `x_get_tree_objects'
app/presenters/tree_builder.rb:306:in `x_build_node'
app/presenters/tree_builder.rb:319:in `x_build_node_dynatree'
app/presenters/tree_builder.rb:219:in `block in x_build_dynatree'
app/presenters/tree_builder.rb:214:in `each'
app/presenters/tree_builder.rb:214:in `each_with_object'
app/presenters/tree_builder.rb:214:in `x_build_dynatree'
app/presenters/tree_builder.rb:152:in `build_tree'
app/presenters/tree_builder.rb:91:in `initialize'
app/controllers/application_controller/feature.rb:28:in `new'
app/controllers/application_controller/feature.rb:28:in `build_tree'
app/controllers/mixins/vm_show_mixin.rb:30:in `block in explorer'
app/controllers/mixins/vm_show_mixin.rb:30:in `collect'
app/controllers/mixins/vm_show_mixin.rb:30:in `explorer'

Not able to reproduce in the latest master.