Bazel: fatal error: 'runtime/cpp/emboss_cpp_util.h' file not found

Clang seems unable to find the cpp_utils headers when the emboss_cc_library rule is used.
bazel-out/k8-fastbuild/bin/external/pigweed/pw_bluetooth/public/pw_bluetooth/hci_common.emb.h:12:10: fatal error: 'runtime/cpp/emboss_cpp_util.h' file not found
   12 | #include "runtime/cpp/emboss_cpp_util.h"
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

It seems like private_hdrs passed to _cc_emboss_aspect_impl are not working for a repo outside of emboss. I wonder why this is working in other projects. Maybe a different version of Bazel?

cc_common.compile() appears to no longer even be supported in Bazel 7: https://bazel.build/rules/lib/toplevel/cc_common
I am using 6.4.0 though.

Your team may be the only active Emboss + Bazel users.
I am tempted to just revert 378cd5dcdb0e8baa74c3e8ac5815727d53de4328 and handle the import-of-imports-across-projects problem a different way, because it really feels like the aspect stuff isn't quite stable enough, and it doesn't seem to be working quite the way I thought it did.
There are some non-public issues, as well.

This is the depset that is passed as private_hdrs to cc_common.compile:
depset([<source file runtime/cpp/emboss_arithmetic.h>, <source file runtime/cpp/emboss_arithmetic_all_known_generated.h>, <source file runtime/cpp/emboss_arithmetic_maximum_operation_generated.h>, <source file runtime/cpp/emboss_array_view.h>, <source file runtime/cpp/emboss_bit_util.h>, <source file runtime/cpp/emboss_constant_view.h>, <source file runtime/cpp/emboss_cpp_types.h>, <source file runtime/cpp/emboss_cpp_util.h>, <source file runtime/cpp/emboss_defines.h>, <source file runtime/cpp/emboss_enum_view.h>, <source file runtime/cpp/emboss_maybe.h>, <source file runtime/cpp/emboss_memory_util.h>, <source file runtime/cpp/emboss_prelude.h>, <source file runtime/cpp/emboss_text_util.h>, <source file runtime/cpp/emboss_view_parameters.h>, <generated file external/pigweed/pw_bluetooth/public/pw_bluetooth/hci_common.emb.h>, <generated file external/pigweed/pw_bluetooth/public/pw_bluetooth/hci_vendor.emb.h>])

I found a terrible workaround. If I prefix all internal emboss C++ header includes with "external/com_googletest_emboss", then clang can find the headers:
#include "external/com_google_emboss/runtime/cpp/emboss_cpp_util.h"

I found what seems to be a fix:
    runtime_cc_info = ctx.attr._emboss_cc_runtime[CcInfo]
    transitive_headers = depset(
        direct = headers,
        transitive = [
                         dep[EmbossCcHeaderInfo].transitive_headers
                         for dep in ctx.rule.attr.deps
                     ],
    )
    (cc_compilation_context, cc_compilation_outputs) = cc_common.compile(
        name = ctx.label.name,
        actions = ctx.actions,
        feature_configuration = feature_configuration,
        cc_toolchain = cc_toolchain,
        public_hdrs = headers,
        private_hdrs = transitive_headers.to_list(),
        compilation_contexts = [runtime_cc_info.compilation_context]
    )

Instead of adding the cpp_utils headers to private_hdrs, add the cpp_utils compilation context to cc_common.compile.

I'm happy to go with the compilation context fix, if that actually works.  Hopefully that is the end of the problems, at least for now.


BAEL-3341 A Guide to Hibernate Types Project

the build seems to be failing, let's try to rebase with master and push again, maybe we have a broken version of the repo

Fixing. I think the pom needs updating

@rpvilao All issues resolved and build is passing


Implement helm.sh/resource-deletion-policy annotation to override deletion cascade per resource

What this PR does / why we need it:
Helm uninstall command allows to provider an optional --cascade flag to specify a deletion policy to use for all resources. In some cases this policy has to be foreground, to ensure the ownership resources are deleted in order, and the deletion is finished until the next hook phase is executed.
To achieve this, a new annotation is provided, helm.sh/resource-deletion-policy. When it is set on the resource, the policy from the value is applied on delete request, regardless of the --cascade flag value.
Special notes for your reviewer:
If applicable:

[ ] this PR contains documentation
[x] this PR contains unit tests
[x] this PR has been tested for backwards compatibility
(backwards compatibility is ensured as it is just adding an annotation. Previous releases would ignore it)

Not to be difficult but shouldn't this have some testing?

Hi @robertsirc yeah, tests sound good. Just added some, can this be ok-to-test now? It is passing for me locally.

Hey @Danil-Grigorev this is a great question if you could join us for the weekly developer meeting on Thursday and ask one of their maintainers. I was just helping read through these and as i looked at all the code you added I wanted to question if it made sense for us to write up some test as we could ensure we are getting the functionality you added.


SageMaker Studio long start-up time

Describe the bug
As a user, when I open SageMaker Studio from the data.all portal I have to wait several minutes before I can start using Studio.
How to Reproduce
*P.S. Please do not attach files as it's considered a security risk. Add code snippets directly in the message body as much as possible.*

New user:

Log in to data.all with a user who has access to create SageMaker Studio profiles
Go to ML Studio and create new ML Studio profile
Click on launch JupyterLab
Amazon SageMaker Studio creates the app
Studio starts loading and displays "Loading... the loading screen is taking a long time. Would you like to clear the workspace or keep waiting?"
The user has to click Keep Waiting or clear workspace for 2-3 times until Studio opens probably 10 or more minutes later

Returning user:

Log in to data.all with a user who has access to SageMaker Studio profiles
Go to ML Studio and click on launch JupyterLab
Studio starts loading and displays "Loading... the loading screen is taking a long time. Would you like to clear the workspace or keep waiting?"
The user has to click Keep Waiting or clear workspace for 2-3 times until Studio opens probably 10 or more minutes later

Expected behavior
For existing users, Studio load within seconds after the use clicks to launch Jupyter Lab icon.
For new users,  Studio should launch as soon as the app is created.
Your project
No response
Screenshots
No response
OS
Mac
Python version
3.8
AWS data.all version
1.4.1
Additional context
No response

Hi @atzev, I can confirm this behavior. I will pass this information internally to see if we can pick up this task and improve the user experience when using ML Studio from data.all

Hello Team, do we have any fix or workaround for this issue?

A colleague has done some investigation and has a explanation for the issue. Basically in the CDK code, the SageMaker Domain is defined with “VPCOnly” together with the default VPC (which has only public subnets):
https://github.com/awslabs/aws-dataall/blob/main/backend/dataall/cdkproxy/stacks/environment.py#L581
SageMaker Domain doesn’t support this (VPCOnly + public subnet):
Doc link stating the requirements (no 1.) that when using VPCOnly mode the Subnets must be private
All the subnets available in the chosen VPC are used to create the Domain (might include a mix of public, private and isolated - probably we just want private and isolated and fail the deployment otherwise if the list is empty)
https://github.com/awslabs/aws-dataall/blob/main/backend/dataall/cdkproxy/stacks/environment.py#L555
In other words, it assumes that the environment account has a default VPC with Private Subnets. From my tests (I created 2 environment accounts from scratch in Isengard) the default VPC does not have private subnets by default, so you have to manually modify the VPC to create them.
A snippet of code that would work is to create the right VPC for SageMaker Studio with VPCOnly mode:
# LINE 553
            # try: # COMMENTING THIS PIECE WHERE WE DO NOT WANT TO USE A DEFAULT VPC
            #     default_vpc = ec2.Vpc.from_lookup(self, 'VPCStudio', is_default=True)
            #     vpc_id = default_vpc.vpc_id
            #     subnet_ids = [private_subnet.subnet_id for private_subnet in default_vpc.private_subnets]
            #     subnet_ids += [public_subnet.subnet_id for public_subnet in default_vpc.public_subnets]
            #     subnet_ids += [isolated_subnet.subnet_id for isolated_subnet in default_vpc.isolated_subnets]
            # except Exception as e:
            #     logger.error(f"Default VPC not found, Exception: {e}. If you don't own a default VPC, modify the networking configuration, or disable ML Studio upon environment creation.")

            # Create VPC with 3 Public Subnets and 3 Private subnets wit NAT Gateways
            log_group = logs.LogGroup(
                self,
                f'SageMakerStudio{self._environment.name}',
                log_group_name=f'/{self._environment.resourcePrefix}/{self._environment.name}/vpc/sagemakerstudio',
                retention=logs.RetentionDays.ONE_MONTH,
                removal_policy=RemovalPolicy.DESTROY,
            )
            vpc_flow_role = iam.Role(
                self, 'FlowLog',
                assumed_by=iam.ServicePrincipal('vpc-flow-logs.amazonaws.com')
            )
            vpc = ec2.Vpc(
                self,
                "VPC",
                max_azs=3,
                cidr="<IP_ADDRESS>/16",
                subnet_configuration=[
                    ec2.SubnetConfiguration(
                        subnet_type=ec2.SubnetType.PUBLIC,
                        name="Public",
                        cidr_mask=24
                    ),
                    ec2.SubnetConfiguration(
                        subnet_type=ec2.SubnetType.PRIVATE_WITH_NAT,
                        name="Private",
                        cidr_mask=24
                    ),
                ],
            )
            ec2.FlowLog(
                self, "StudioVPCFlowLog",
                resource_type=ec2.FlowLogResourceType.from_vpc(vpc),
                destination=ec2.FlowLogDestination.to_cloud_watch_logs(log_group, vpc_flow_role)
            )

            vpc_id = vpc.vpc_id
            subnet_ids = [private_subnet.subnet_id for private_subnet in vpc.private_subnets]
            sg = vpc.vpc_default_security_group
            sagemaker_domain = sagemaker.CfnDomain( # this is the same as before from here to the end

Fixed in #409


GPT4 Diailog message is reduced to zero

Hey, thanks for the update. When I try to select the chat GPT-4 and write any message, even "hello", my telegram bot replies me:
Something went wrong during completion. Reason: Dialog messages is reduced to zero, but still has too many tokens to make completion. How do i fix that? Just wanna know - should i have chat GPT-4 API key to access for that settings mode?

Yes you should get a GPT4 api from the address below.
https://openai.com/waitlist/gpt-4-api


CNCF TAG-Runtime Discussion/Presentation

Hello Kubean maintainers,
As part of CNCF TAG Runtime, I am reaching out to you to present/discuss Kubean at one of our meetings, since you have also applied for Sandbox (cncf/sandbox#49).
It would be great to learn about the roadmap of the project, tech, community and the general overview.
If you are interested, there is a slot on 31st Aug on our agenda. Please feel free to add it to the agenda or reach out to us in the #tag-runtime channel on the CNCF slack.
Thanks!
cc: @raravena80 @helayoty @nikhita

@ErikJiang awesome! I have updated the agenda.

@ErikJiang gentle reminder that this is scheduled for tomorrow! https://cloud-native.slack.com/archives/CPBE97SMU/p1693395403253659

Links below:
The Slides for the intro in tag meeting:
https://docs.google.com/presentation/d/1Gh-Y8t4QGrq4s2nYI5YR0rycVHjQcmEOtxB1Xqotqr8/edit?usp=sharing
Recording for the meeting:
https://www.youtube.com/watch?v=CrPr7TqEIrc&list=PL6wYrb-bYwC_iStEfVmBpLKDBqOy23nCA&index=44
More discussion for sandbox:
https://github.com/cncf/sandbox/issues/49


Missing assets 2

Good Morning,
We're doing a second round of updates on our metadata, and part of that is checking if files exist.
We've found a few scenes that there are missing assets from your bucket, as per the below list:
'sentinel-s2-l2a-cogs/37/L/DD/2018/11/S2A_37LDD_20181104_0_L2A/L2A_PVI.tif',
'sentinel-s2-l2a-cogs/33/P/WM/2018/1/S2A_33PWM_20180124_0_L2A/L2A_PVI.tif',
'sentinel-s2-l2a-cogs/34/N/HM/2018/9/S2A_34NHM_20180919_0_L2A/L2A_PVI.tif',
'sentinel-s2-l2a-cogs/34/S/CA/2018/11/S2B_34SCA_20181115_0_L2A/L2A_PVI.tif',
'sentinel-s2-l2a-cogs/32/L/RP/2018/11/S2A_32LRP_20181127_0_L2A/L2A_PVI.tif',
'sentinel-s2-l2a-cogs/39/P/UK/2018/12/S2B_39PUK_20181223_0_L2A/L2A_PVI.tif',
'sentinel-s2-l2a-cogs/34/S/GA/2018/6/S2A_34SGA_20180620_0_L2A/L2A_PVI.tif',
'sentinel-s2-l2a-cogs/28/Q/BM/2018/2/S2A_28QBM_20180205_0_L2A/L2A_PVI.tif',
'sentinel-s2-l2a-cogs/36/N/TH/2018/8/S2A_36NTH_20180801_0_L2A/L2A_PVI.tif',
'sentinel-s2-l2a-cogs/30/S/WD/2018/10/S2B_30SWD_20181013_0_L2A/L2A_PVI.tif',
'sentinel-s2-l2a-cogs/35/M/NM/2018/9/S2B_35MNM_20180905_0_L2A/L2A_PVI.tif',
'sentinel-s2-l2a-cogs/36/P/ZB/2018/11/S2A_36PZB_20181126_0_L2A/L2A_PVI.tif',
'sentinel-s2-l2a-cogs/35/R/KL/2018/11/S2B_35RKL_20181116_0_L2A/L2A_PVI.tif',
'sentinel-s2-l2a-cogs/35/L/KC/2018/12/S2B_35LKC_20181227_1_L2A/L2A_PVI.tif',
'sentinel-s2-l2a-cogs/39/P/VL/2018/12/S2B_39PVL_20181223_0_L2A/L2A_PVI.tif',
'sentinel-s2-l2a-cogs/33/P/TQ/2018/9/S2B_33PTQ_20180909_0_L2A/B01.tif',
'sentinel-s2-l2a-cogs/35/R/LH/2018/7/S2A_35RLH_20180724_0_L2A/L2A_PVI.tif',
'sentinel-s2-l2a-cogs/39/P/TN/2018/10/S2A_39PTN_20181012_0_L2A/L2A_PVI.tif',
'sentinel-s2-l2a-cogs/29/S/PR/2018/11/S2B_29SPR_20181118_0_L2A/L2A_PVI.tif',
'sentinel-s2-l2a-cogs/35/M/KS/2018/10/S2A_35MKS_20181016_0_L2A/L2A_PVI.tif',
'sentinel-s2-l2a-cogs/34/P/DQ/2018/7/S2B_34PDQ_20180722_0_L2A/L2A_PVI.tif',
'sentinel-s2-l2a-cogs/35/K/KT/2018/5/S2A_35KKT_20180523_0_L2A/L2A_PVI.tif',
'sentinel-s2-l2a-cogs/31/P/BS/2018/1/S2A_31PBS_20180112_0_L2A/L2A_PVI.tif',
'sentinel-s2-l2a-cogs/39/N/UE/2018/7/S2B_39NUE_20180726_0_L2A/L2A_PVI.tif'

This time we have done a big check over 2174227 scenes or the whole African continent. As promised last time, I am listing the missing ones.
Thank you for your attention to this matter.

@alexgleith It's not necessarily expected, but appears as though the PVI files don't exist for these scenes.  I'll see if I can find out why

That one that's missing B01 is a bit of an issue, though! Can you re-process that?

Hey, @matthewhanson is there any update about the missing B01?

@carioca-au @alexgleith sorry, been a bit busy this week, I'll have some time Friday afternoon or early next week and will try to reprocess that scene, and will also take a closer look at the othes.

Can you confirm if this is still an issue for you in v1? In v1, sentinel-s2-l2a-cogs has been renamed to sentinel-2-l2a.

Had a quick look at the scene that's missing the band one file and it's missing a whole lot of stuff.
aws s3 ls s3://sentinel-cogs/sentinel-s2-l2a-cogs/33/P/TQ/2018/9/S2B_33PTQ_20180909_0_L2A/

2020-10-02 17:55:16   62278420 B11.tif
2020-10-02 17:55:36   62480861 B12.tif
2020-10-02 17:54:49   62197818 B8A.tif
2020-10-02 17:47:25     425440 L2A_PVI.tif
2022-11-08 15:48:34      16239 S2B_33PTQ_20180909_0_L2A.json
2020-10-02 17:57:37    5269371 SCL.tif
2020-10-02 17:48:42  332933101 TCI.tif
2020-10-02 17:57:22   89962411 WVP.tif
2022-11-08 15:48:33     553096 granule_metadata.xml
2022-11-08 15:48:33     144943 thumbnail.jpg
2022-11-08 15:48:33       1488 tileinfo_metadata.json

Also, the item has links to jp2 files, which doesn't seem right: https://earth-search.aws.element84.com/v1/collections/sentinel-2-l2a/items/S2B_33PTQ_20180909_0_L2A

It looks like the jp2 asset is used for the default named band assets (e.g., red, blue) if the COG doesn't exist. I don't think that's a great behavior, because then you have the same asset names having different data formats among different Items, and that's not user-friendly when you're expecting a cog. It's also contrary to the item-assets definitions advertised in the Collection. We'll likely change that soon.
I'll also look into taking into account when only some of the cogs exist, and make sure they're created before creating the item.  Thanks!

This work has been moved into JIRA and for good housekeeping we'll close this issue.

You got it, that is the plan! We have linked to these original GH issues within JIRA. It is more streamlined housekeeping than tracking open issues in two places.


Add local formatting changes

Addresses repo wide fmt changes emerging from comments on api fields. This PR simply adds the formatting changes that are being generated by go fmt on master currently.
https://github.com/konveyor/mig-controller/pull/954#discussion_r577190761

Thanks @ibolton336!


Adjusted cache paths in module install scripts

Build runs were showing errors on the local paths:
bash -c "$(curl http://example.com/munkireport/index.php?/install)" bash -i ~/Desktop
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 33382    0 33382    0     0  3298k      0 --:--:-- --:--:-- --:--:-- 3622k
Preparing /var/folders/cn/92l5b1dn49z2zk_xt43n84kh0000gg/T/mrpkg.v1uu5Kqo/install_root/usr/local/munki/ and $3/Library/Preferences/MunkiReport
BaseURL is http://example/munkireport/
Retrieving munkireport scripts
Configuring munkireport
+ Installing appusage
touch: /usr/local/munki/preflight.d/cache/appusage.csv: Permission denied
+ Installing ard
+ Installing security
override rw-r--r--  root/wheel for /usr/local/munki/preflight.d/cache/security.txt? y
rm: /usr/local/munki/preflight.d/cache/security.txt: Permission denied
Building MunkiReport v2.13.0 package.
pkgbuild: Inferring bundle components from contents of /var/folders/cn/92l5b1dn49z2zk_xt43n84kh0000gg/T/mrpkg.v1uu5Kqo/install_root
pkgbuild: Adding top-level preinstall script
pkgbuild: Adding top-level postinstall script
pkgbuild: Wrote package to /Users/techhelp/Desktop/munkireport-2.13.0.pkg
Cleaning up temporary directory /var/folders/cn/92l5b1dn49z2zk_xt43n84kh0000gg/T/mrpkg.v1uu5Kqo

The proposed changes allows the module to affect the correct cache paths when building a package or installing locally.

Confirmed this PR fixed for my autopkg issue


Add a base Writer class

Based on the existing Writers, a base class would have:

a write method
a to_string method
a _write_content method

Using a base class may clean up the other writer classes and allow for extension by users using inheritance.

Hmm, I think there was some reason we didn't do this. I wonder if it is in any of the commit messages...
I think the main argument is that while the would all have a write and a to_string method. It does not mean that all the writers would have the same arguments to those methods. This is also a requirement for having a shared base class. I think assuming that they would all have the same arguments is a pretty strong assumption, and I do not generally see the advantage of making it. For example, if you want to write periodic systems, you can imagine an additional argument to do with the periodic cell, which makes no sense to include in the write() method of non-periodic writers. In fact, I think https://stk.readthedocs.io/en/stable/stk.molecular.writers.mdl_mol.html and https://stk.readthedocs.io/en/stable/stk.molecular.writers.pdb.html are actually example of this .
A design point of on the use of a base class. The use of a shared base class, means that users would be able to provide either a MolWriteror a PdbWriter interchangeable, without caring which one is which. But I find it hard to think of an example with treating writers as abstract, interchangeable entities is beneificial. I think normally when you write code, you want to be explicit about which actual writer you are using, therefore referencing them through a shared, abstract interface I think in practice add little value -> though counter-examples are highly appreciated!

This is perfect and very clear. A description of why we made that choice was needed because I know we did it on purpose.


Update build and build-raw commands to accept simple reference minting scripts

Closes: #4086

bors r+


linux: dynamically load libX11

Removing linking of libX11 and replacing it with dlopen to call the
functions dynamically.
For the future wayland support, the library will not require the libX11
library to be installed on the running system.
For compiling it we need it, because of the huge and complicated type
definitions, but the compiled binary will run also on a system without
libX11.
Maybe in the future we can eliminate also the X11/Xlib.h and X11/Xatom.h.
The only thing that really needs to be declared is the XEvent and it's attributes,
other things can be a void pointer or some c primitives.

So I changed it to load libX11 only once when calling a function. Tests are ok.

Thanks, I'll merge it first and then dig more about this. After a quick inspection on the Go's plugin package, it turns out they are the exact same appraoch.

Thanks! It's always a great feeling when some of my work is merged.
Now I'll try to find out how to wayland :)


[docs] Update URL for @material-ui/system

[x] I have followed (at least) the PR section of the contributing guide.

@NMinhNguyen Yeap, it's definitely better like this!


client keys stuff

[Enter feedback here]

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 4fd20afa-df7a-17aa-3c44-8a41e3f7642e
Version Independent ID: 91e44e8f-aedf-8193-73d5-ea3392911863
Content: Microsoft Server Activation
Content Source: WindowsServerDocs/get-started/Server-2016-activation.md
Product: windows-server
Technology: getting-started
GitHub Login: @jaimeo
Microsoft Alias: jaimeo

Note
Updates might be required on the KMS server to support activation of any of these newer clients. If you receive activation errors, check that you have the appropriate updates listed below this table.

@eross-msft . Also, could you update the article with the correct author (which isn't me)?
@gardmkdg1 did you know you can make edits like this directly in the article, rather than going the issue route? Just select the Edit button in the upper right area of the live article.

@gardmkdg1 : Feel free to discuss editing the article with us, to make the most of your suggestions for improvement.

@jaimeo  I updated the author info, but it won't go live until the end of the week.


Added throwing an error on invalid database name

Right now there is no database name validation which leads to hard to understand exceptions during code generation when developer will use character in database name which not allowed in Java class name. You can check it by changing database name from "App" to "App.db" in sample application. I've added validation of database name and throwing an Error if validation doesn't pass which should be much easier to understand what exactly is wrong .

@kamilzych Nice PR! one thing, can you add checks for if the database name ends in .db to throw a different error to say for devs to leave that portion out?
so if they define the name as "somedatabase.db" then it tries to generate:
somedatabase.db$Database.class

@kamilzych actually, why should it matter if the databaseName is a java keyword? We don't generate any code that may produce a java keyword. We simply append $Database to the name by default so if it was void, it would be void$Database as the generated class name.

@agrosner Ok, a made some changes.

renamed method from 'isValidJavaClassName' to 'isValidDatabaseName' since it no longer checks for java class name validation.
moved this method from WriterUtils to DatabaseWriter because it's no longer general java class name validation but specific database name validation.
-added javadoc and more clear exception message.


Fixed BLACK & WHITE pixels for Waveshare Weather Icon

Fixes issue with Waveshare Weather Icon not displaying
Connects-to: #48
Change-type: patch

@louispires thanks for the contribution!

No problem @chrisys - Kindly also look at PR #50 #51 #52

Sure, will do. We're looking into why that build check is failing as well; it shouldn't be. After that this will merge automatically 👍

@balena-ci rebase

@balena-ci retest


(nuget.commandline)  Update using tools.json feed

Description
Update nuget.commandline package using tools.json feed instead from nuget.commandline NuGet package.
Motivation and Context
In the past few versions several time the nuget.commandline package was not updated after the NuGet package reached recommodation/blessing state. This PR changes update to use the JSON file describing all versions. It also updates as soon as a new released version is available even if it is not the recommended/blessed version yet (like currently 4.8.1)
Fixes #1076
How Has this Been Tested?
Tested update locally
Screenshot (if appropriate, usually isn't needed):
Types of changes

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Migrated package (a package has been migrated from another repository)

Checklist:

[x] My code follows the code style of this repository.
[ ] My change requires a change to documentation (this usually means the notes in the description of a package).
[ ] I have updated the documentation accordingly (this usually means the notes in the description of a package).
[x] All files are up to date with the latest Contributing Guidelines
[ ] The added/modified package passed install/uninstall in the chocolatey test environment.
[x] The changes only affect a single package (not including meta package).

:white_check_mark: Package verification completed without issues. PR is now pending human review

@majkinetor Or should we better implement this with streams? One package for released (including preview versions) and the other for recommendet version?

I don't think streams are needed here but prereleases. So I think we should just add -preview to current package version if its preview and call it a day. Streams make sense if both versions are independently updated and I don't think its the case here - it seems linear: preview, preview, preview, release, preview, preview ... If that is not the case, streams are the only option and IMO better then 2 packages as its easier to maintain.

@majkinetor They have three steps: Preview -> Release -> Blessed (what they mention as recommended on the page and should be reflected in the nuget.commandline package). See https://github.com/NuGet/Home/issues/4851
If we just provide one package and people update they're on the Release track. If they only want Blessed releases they need to check the nuget.org website. I'm OK with doing it like this and avoid two separate packages with all the issues involved (like dependency issues, etc).
But they release updates to the Release or Blessed version, even if newer preview versions already exists. E.g. here (you can check release dates in the https://dist.nuget.org/tools.json file):
4.7.0: 2018-05-07T23:00:00.0000000Z
4.8.0-preview3: 2018-07-06T23:00:00.0000000Z
4.7.1: 2018-08-10T23:00:00.0000000Z

So I assume three streams for preview, release and blessed makes sense?

:white_check_mark: Package verification completed without issues. PR is now pending human review

I've updated the PR to have two streams:

One for release and preview
One for blessed releases

But they release updates to the Release or Blessed version, even if newer preview versions already exists

It doesn't matter since they will alway go as -preview and can be recognized upstream as such. The probleem is release/blessed. I propose simple solution - add 4th number in the build if blessed (for example 100, high enough to allow for some fixes to be used on it - 666 or 999 sound interesting as another example). So you will know without looking at official site what is blessed  and you can even act on it in a script via choco info

For example this would look like this on gallery:
4.8.0-preview3
4.7.1
<IP_ADDRESS>6
4.7.0
Another idea, blessed package could be virtual package pointing to blessed release, but not DRY ....

I'm not really sold on the idea of implementing a custom versioning system for handling this case. As it might confuse people more than that they will get it (and still requires custom scripting).
How about going with the current approach in this PR and pushing everything to Chocolatey, with an update to the description that package also contains non-blessed versions.
If someone asks for a package containing only blessed releases there still can be a virtual package containing only blessed release.
Btw. Another side effect of this PR is that we went (intentionally) out of sync with the nuget.commandline package. I originally implemented this package like this, with checking the json, but I think @ferventcoder prefered to have it mirroring the NuGet package.

What I meant here is not fixed words such as this package contains release and blessed versions but to actually change README.md via au_SearchReplace and insert somewhere BLESSED RELEASE as you see one.

I don't think this makes sense as this is a mutable value, while the Chocolatey package description is immutable. Like 4.8.1 is now on state Released but probably would become ReleasedAndBlessed at some point.

:white_check_mark: Package verification completed without issues. PR is now pending human review

I don't think this makes sense as this is a mutable value, while the Chocolatey package description is immutable. Like 4.8.1 is now on state Released but probably would become ReleasedAndBlessed at some point.

If that is so, the only way to keep single package and have everything without information loss is adding 4th version number. I don't believe users would be confused if appropriate note is given.
However, IMO all that is overkill for this package. Lets go with simplest thing like you proposed, and if ever needed, we can eventually add virtual package for blessings.

Like 4.8.1 is now on state Released but probably would become ReleasedAndBlessed at some point.

What is the point of streams then ? Blessed stream will always have push conflict, right?

I believe adding just 1 new line in blessed section: PackageName = 'nuget.commandline.blessed' (or something shorter) would make it work with 2 packages being updated.

:white_check_mark: Package verification completed without issues. PR is now pending human review

@majkinetor I've now implemented it simpler, as it just checks for the newest entry in the list regardless of the release state


Add GFM

Can you please add support for GitHub Flavored Markdown?

Just something: as an option please, cause not everyone wants GitHub flavored Mardown...


hashcash/pow on rendezvous server as DoS mitigation?

As discussed in #107, it might be useful to make the server require some sort of proof-of-work when it appears to be under a DoS attack. Clients would need to handle a challenge message, perform the PoW computation, then return the response. The server could be manually configured for the workload required, or automatically increase it when a large number of "scary" (aka PAKE-failing) connections are observed.

I looked into the specifics of the hashcash proof-of-work spec. On the minus side, it uses SHA1.
Format-wise, one looks like this: "{ver}:{bits}:{date}:{resource}::{rand}:{counter}". Glossing over most of the details (see http://hashcash.org/docs/hashcash.html#stamp_format__version_1_) it needs a "resource" string (e.g. for email this is user@example.com). The rand valid is to prevent double-spending one token, and the counter is the thing you can change to try and get the whole thing to SHA1-hash to something with the right number of leading zero bits.
Perhaps we don't care what API you're calling, you just need hashcash per-connection (and thus the resource is just the "connection id"). The "resource" could then just be the IP address and the client doesn't have to be told an id, but it's not very easy to know what IP address you appear to the outside world (or, specifically, to the wormhole server) as.
Worth considering putting e.g. the "nameplate id" into the resource (if per-API call)?
One way I can imagine this all tying together is this:

upon "welcome", the server has some way to send some extra information. If it's there, it can specify whether hashcash is required (perhaps even which API methods require it?). This would include the server's notion of "connection id" for this client. It would also send the difficulty (i.e. number of bits).
a client can look through that and figure out if it needs to send hashcash or not (and the necessary parameters, namely the resource string and difficulty).
clients can send extra arguments with API requests, possibly including hashcash.
if the server is demanding hashcash, all requests (possibly only of one type) fail unless there's a valid hashcash token from the client.

Questions:

per-API-method, or just per-connection hash-cash? The former seems more flexible (e.g. maybe I only want to PoW-limit clients allocating nameplates).
is using hashcash directly a good idea, given SHA1 is burned into the spec?
is "announce through server 'welcome'" and "present hashcash via extra API args" a good approach?
do we need to count double-spends (i.e. in the database, via the rand= param as the spec intends), or is the "connection id" being in the resource string enough (because we make it unique per-connection ourselves).

Meejah and I were brainstorming about this, and we figured out an initial client change that could smooth the way forward if/when we need to turn something on in the future:

make the client look for a new Welcome message field, maybe named permission-token-url
if present, the client should display a message to the user, basically "please point a web browser at this URL, do what it tells you, then paste back in the token that it shows you"

after a few seconds, maybe have the client can pop open a browser window to the URL
but if the client is running in --tor mode, maybe don't: we don't want to make it easy for the server to force the client to reveal anything identifying, if the user has a better/safer way to get the token

the client should have a box or an input field for the token, which must be ASCII and easy to cut/paste
when the client gets the token from the user, it should send it in a new message to the server, maybe {type="submit-permission", token=TOKEN}, then continue with the protocol

The idea is that we don't know what sorts of PoW or CAPTCHA we might need in the future, but there's a decent chance that it will be implementable by a web browser. So if we make the client capable of delegating this work (through the user) to a browser today, then we can wait until we actually need it before we go and build something.
This doesn't limit us to a CAPTCHA.. anything that serves a rate-limiting or auditing purpose would do. "Log in with github". Even just demonstrating control over an IP address would let the server do some anti-abuse classification (e.g. if we see a lot of nameplate allocation or failed guesses coming from the same IP address, treat it as malicious and block or rate-limit it).
GUI clients that can open up web views (e.g. the Tahoe setup tool named https://github.com/gridsync that @crwood is building, which I think is a QT app) could open the requested URL directly in the app, rather than using an external web browser. If the app-webview API allows for manipulation of the page environment, we could create a return pathway for the token that wouldn't require the user to copy/paste anything (we did something like this in Persona a few years ago, basically the web page calls window.MagicWormhole.deliver_token(TOKEN) if that object exists).
Decisions to make:

name of the Welcome message field
name of the token-return message
ordering of the messages

something that plays well with current clients (backwards-compatible, graceful upgrade)
current servers send WELCOME right away. current clients send BIND and ALLOCATE or CLAIM right away
option 1: servers buffer the BIND/ALLOCATE/CLAIM if they want a token, send demand for token in WELCOME, wait for new SUBMIT-PERMISSION message, then allow buffered messages to be delivered
option 2: servers reject BIND without a token, by sending WELCOME with an error=. Sounds unworkable, would need to behave sort of like HTTP "Authorization" header where the first request is rejected but includes instructions for how to make a successful second request. We need some way to carry state from the challenge to the response.
option 3: don't buffer BIND/ALLOCATE/CLAIM (discard them instead), use WELCOME to indicate need for a token, somehow tell the client that it must re-send BIND (with token=) and ALLOCATE/CLAIM. Maybe do this by resetting the client state machines back to the "not connected" state, then faking a connection event

Discussed in #tahoe-lafs and I think what we concluded is this:

new clients send ABILITIES (abilities, a new message) on connect

new clients wait for WELCOME from the server (if it includes the URL, then it is in mitigation mode and demands a sacrifice/token)

new clients send PERMISSION (submit-permission, a new message) with their token

old clients continue to send BIND right away

new server waits for some message from a client before sending WELCOME; if this message is ABILITIES it's a new client, and if not it's an old-client.

if we're doing mitigation:

if it's an old client, send an error in the WELCOME
if it's a new client, include the URI in the WELCOME

@warner notes this will be an extra "1 round-trip" delay for new client + new server (because the server waits for the new message from the client, and the client waits for the welcome before sending bind).


🦒 colab

Thanks for the project ❤ I made a colab. 🥳 I hope you like it. https://github.com/camenduru/Mistral-colab

Great one @camenduru
Do you know? What would be the ideal embedding sentence transformer for Mistral 7B Instruct?
Will bge-large-en-v1.5 Works with it?


Power the index file with JSON

We're exporting the past 12 hours of data to status.json. Instead of manually populating the energy values within the HTML, do so with jQuery.

Done.

Oh, wait, not quite. The graph is powered by JSON, but the two energy values are still being hard-coded.


Save file unpacking failed with wait result 258

idk why this is an issue but i keep getting this error

If the reasons mentioned the error message don't apply, it's likely caused by a space character in the .sl2 file path. This issue has been partially addressed in release v1.05.
Closing as duplicate of #4


Correctly parse jinja templates

VS code is not able to correctly parse HTML Jinja templates.
When HTML file with  {% %} is opened, I get nice HTML color coding, but lots of parsing errors are reported, since {% %}  correctly identified as non-valid HTML tags
When I change type Jinja manually, then parsing errors are no longer shows, but all color coding are lost
The suggestion is to merge ability to recognize {% %} with HTML parsing

VSCode only support standard HTML.
Jinja support has to come from extension. Please check the marketplace.


Text corruption on Atlas rebuild.

I've run into an issue on NVidia hardware where text will get corrupted when the atlas needs to be rebuilt.  Either the characters will be missing or garbled looking.  I've tracked down and was able to fix the issue by changing
Lines 2463-2468 in nanovg.c
if (!nvg__allocTextAtlas(ctx))
break; // no memory :(
if (nverts != 0) {
nvg__renderText(ctx, verts, nverts);
nverts = 0;
}
To
if (nverts != 0) {
nvg__renderText(ctx, verts, nverts);
nverts = 0;
}
if (!nvg__allocTextAtlas(ctx))
break; // no memory :(
This is merely forcing the render of pending text before reallocating the Atlas.  Waiting till after reallocating the atlas causes the wrong texture data to be drawn for the characters in the old atlas.

Sorry I'd rather not as this change is so simple (literally moving 2 lines of code down 4 lines) and I don't actually have this project checked out as I indirectly use it though qnanopainter.  I figure it would be easier for you to make the change and commit than I to clone, checkout, make change, commit, post pull request, etc.


Share-to-Teams button blocked due to wrong cookie settings

Hi Microsoft Team,
for lack of a better place to provide feedback, I write here feedback for the Share-To-Teams button.
Since this February Chromium has been updated to now allow cookies with SameSite=Lax.
When I try to include the Share-to-Teams button in my web application, Chromium based browsers throw the message:
Indicate whether a cookie is intended to be set in a cross-site context by specifying its SameSite attribute
I know that this is not the correct forum to provide feedback, but as this is a bug from my perspective, the feature request on uservoice.com is not correct as well.
Regards,
Nikolas

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 5964e0a9-bba0-c5a3-8099-f3845037726c
Version Independent ID: 24565c3b-2bb5-2482-a6c7-7b1f62e38abb
Content: Create a Share-to-Teams button - Teams
Content Source: msteams-platform/concepts/build-and-test/share-to-teams.md
Product: msteams
GitHub Login: @laujan
Microsoft Alias: lajanuar

Hi @NikolasEisberg, I am able to repro the issue from my end. Seems like it's a bug, yet need to get confirmation from the internal team. Will be raising the same as bug if it gets confirmed and will be sharing the details with you.

Hi @NikolasEisberg, I have tested Share-To-Teams button from my end with latest Chromium version and it is working as expected. Could you please try in latest Chromium version. If the issue is still persisting  then please share repro steps, code snippets.

Hi @Sridevi-MSFT  Sorry for the late reply.
Yes now it work also for me.
Thank you for your Help :)


Wrong cutting position

Describe the bug
Pixel is 1080X1440 image. When cutting, zoom the picture, drag the clipping box and move the picture. Finally, determine that the clipped image is not the desired position, but after rotating 90 degrees, repeat the above operation to achieve its purpose.

The final cut was not the cat, but the white cylinder next to it.

fail:

success:

I don't understand what you are meaning exactly...


ReactiveImagePicker Pass value to Firebase

hi
I'm using ReactiveImagePicker with ReactiveFormBuilder
I have difficult to passing value from the ReactiveImagePicker   to the Firebase
refrence
https://stackoverflow.com/questions/51338041/how-to-save-image-file-in-flutter-file-selected-using-image-picker-plugin
code   file = File(_image.toString());     final picker = ImagePicker();     await ref.putFile(file);     imageurl = await ref.getDownloadURL();     print(imageurl);

@alshueiliproject
This doesn't seem to be a correct repo for your issue. I think you need to ask this in https://github.com/artflutter/reactive_forms_widgets/issues
Closing from here.


Updates to services (selector) being ignored

Describe the bug
We're using blue/green deployments and effect the changeover by updating the selector of the two Ambassador services.
We've found that, although it sometimes works, often this change will be ignored. To test this, I added the commit id to a status API endpoint and can see that the swap doesn't happen (or worse, since we're using two Ambassador instances each with their own IDs, only one updates). After waiting about 10 minutes, no change is visible.
To Reproduce
Steps to reproduce the behavior:

Create two discernibly different API deployments, A and B, with different labels (e.g. API=A, API=B).
Create a single Ambassador service to point at A's label.
Access the API and confirm that it's A.
Edit the Ambassador service (kubebtl edit service ~~~~) to point at B's label.
Access the API and find it's still A. Repeat after 10 minutes. Still A.

Expected behavior
I expect Ambassador to update to point at the other deployment.
Versions (please complete the following information):

Ambassador: 0.40.0
GKE v1.9.7-gke.5

Additional context
If I try to change the values in the two files, and then I do nothing more than scale the Ambassador deployments in to 0 replicas, and back out, it will effect the change, so our config files are definitely correct.
I've found a workaround which is to add a dummy header which contains a 5 character alphanumeric string generated by Helm:
add_request_headers:
x-test-data: "{{ randAlphaNum 5 }}"
Adding this header causes Ambassador to update each time (and in under 30 seconds).

Could you paste the yaml of the service you are editing?

With my exact configuration, it happens every time. With this one, which is very similar to my own (this is basically a compiled version of the Helm template I'm using, with the names changed to protect the innocent), it happens maybe 1/10 times where it just gets stuck on one.
I just keep swapping and eventually it gets stuck. I've got two deployments here so that it's obvious which is being hit: one is Apache and the other is Nginx.
apiVersion: v1
kind: ServiceAccount
metadata:
  name: ambassador-release
  namespace: sandbox
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: ambassador-release
  namespace: sandbox
rules:
- apiGroups: [""]
  resources:
  - services
  verbs: ["get", "list", "watch"]
- apiGroups: [""]
  resources:
  - configmaps
  verbs: ["create", "update", "patch", "get", "list", "watch"]
- apiGroups: [""]
  resources:
  - secrets
  verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: ambassador-release
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: ambassador-release
subjects:
- kind: ServiceAccount
  name: ambassador-release
  namespace: sandbox
---
apiVersion: v1
kind: Service
metadata:
  labels:
    env: "sandbox"
    type: ambassador
  name: ambassador-release-loadbalancer
  namespace: sandbox
spec:
  externalTrafficPolicy: Cluster
  ports:
  - port: 80
    protocol: TCP
    name: web-http
    targetPort: 80

  selector:
    service: ambassador-release-service
  sessionAffinity: None
  type: LoadBalancer
---
apiVersion: v1
kind: Service
metadata:
  labels:
    service: "ambassador-release-admin"
  name: ambassador-release-admin
  namespace: sandbox
spec:
  type: NodePort
  ports:
  - name: "ambassador-release-admin"
    port: 8877
    targetPort: 8877
  selector:
    service: "ambassador-release-ambassador"
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: ambassador-release
  namespace: sandbox
spec:
  replicas: 1
  template:
    metadata:
      annotations:
        sidecar.istio.io/inject: "false"
      labels:
        service: ambassador-release-service
    spec:
      serviceAccountName: ambassador-release
      containers:
      - name: ambassador
        image: quay.io/datawire/ambassador:0.40.0
        resources:
          limits:
            cpu: 1
            memory: 400Mi
          requests:
            cpu: 200m
            memory: 100Mi
        env:
        - name: AMBASSADOR_ID
          value: ambassador-release
        - name: AMBASSADOR_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        livenessProbe:
          httpGet:
            path: /ambassador/v0/check_alive
            port: 8877
          initialDelaySeconds: 30
          periodSeconds: 3
        readinessProbe:
          httpGet:
            path: /ambassador/v0/check_ready
            port: 8877
          initialDelaySeconds: 30
          periodSeconds: 3
      restartPolicy: Always
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
              - matchExpressions:
                - key: nodeType
                  operator: DoesNotExist
---
apiVersion: v1
kind: Service
metadata:
  annotations:
    getambassador.io/config: |
      ---
      apiVersion: ambassador/v0
      kind:  Mapping
      name:  test-mapping
      prefix: /
      service: test-service
      host: "www.example.com"
      ambassador_id: ambassador-release
      timeout_ms: 30000
  name: test-service
  namespace: sandbox
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: api
    slot: green
  sessionAffinity: None
  type: ClusterIP
status:
  loadBalancer: {}
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: api-deployment-green
  namespace: sandbox
spec:
  selector:
    matchLabels:
      app: api
      slot: green
  replicas: 1
  template:
    metadata:
      labels:
        app: api
        slot: green
    spec:
      containers:
      - name: httpd
        image: httpd:2.4.35
        ports:
        - containerPort: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: api-deployment-blue
  namespace: sandbox
spec:
  selector:
    matchLabels:
      app: api
      slot: blue
  replicas: 1
  template:
    metadata:
      labels:
        app: api
        slot: blue
    spec:
      containers:
      - name: nginx
        image: nginx:1.8
        ports:
        - containerPort: 80

Okay so you are changing the selector in test-service from slot: green to slot: blue and back and forth again. That clears some things up thank you.
So I was able to reproduce your issue yesterday but have been unable to today after deploying with helm. Yesterday, however, when I was able to reproduce I took a deeper look into what was going on and it appeared to be an issue with Kubernetes failing to point the service to the correct pod.
How I Tested This

run kubectl edit service test-service
Test the edit through ambassador and notice the change didn't take place
Run kubectl exec -it <ambassador-pod-name> -- /bin/sh to open up a shell in the ambassador pod. This allows us to test within the cluster
Run curl http://test-service/ and see that the wrong response is returned. This curl is bypassing ambassador and simply sending a request to test-service which is returning a response from whichever pod it is pointing to.

I am also skeptical of this being an issue with Ambassador. Ambassador simply takes this mapping annotation and sees to point requests to http://<ambassador-ip>/ to service: test-service. test-service then is in charge of forwarding that request to the correct deployment. For this reason, and with the results of my testing, I am inclined to believe there is some Kubernetes error causing this.
I do believe I have a workaround though. While I was able to replicate this issue using kubectl edit I was unable to when I was editing the service using kubectl apply -f. The helm equivalent of this would be to edit the yaml template and helm upgrade <release-name> <chart-directory>.
I would be very interested in hearing your results of testing the traffic inside Kubernetes. I would also love to know if the workaround I suggested works.

Closing for lack of response


Change limit on request to Oracle API to 250 due to message size limit

Required items, please complete
Change(s):

Change the limit from 1000  to 250 on the requests to Oracle Cloud Streaming API.

Reason for Change(s):

The Oracle Cloud Streaming API have a max limit of 1MB per request. The request limit of 1000 is making the requests to fail due to hitting the max size limit per request. This is changed to 250 to mitigate this. It might be a good idea to have this as an configuration option. For reference see more here https://docs.oracle.com/en-us/iaas/Content/Streaming/Concepts/streamingoverview_topic-Limits_on_Streaming_Resources.htm

Version Updated:

Required only for Detections/Analytic Rule templates
See guidance below

Testing Completed:

See guidance below

Checked that the validations are passing and have addressed any issues that are present:

Yes

@microsoft-github-policy-service agree
From: microsoft-github-policy-service[bot] @.>
Sent: Monday, 15 May 2023 13:27
To: Azure/Azure-Sentinel @.>
Cc: Jan Berg @.>; Mention @.>
Subject: Re: [Azure/Azure-Sentinel] Change limit on request to Oracle API to 250 due to message size limit (PR #8038)
You don't often get email from<EMAIL_ADDRESS>Learn why this is importanthttps://aka.ms/LearnAboutSenderIdentification
@janberg-tshttps://github.com/janberg-ts please read the following Contributor License Agreement(CLA). If you agree with the CLA, please reply with the following information.
@microsoft-github-policy-service agree [company="{your company}"]
Options:

(default - no company specified) I have sole ownership of intellectual property rights to my Submissions and I am not making Submissions in the course of work for my employer.

@microsoft-github-policy-service agree

(when company given) I am making Submissions in the course of work for my employer (or my employer has intellectual property rights in my Submissions by contract or applicable law). I have permission from my employer to make Submissions and enter into this Agreement on behalf of my employer. By signing below, the defined term “You” includes me and my employer.

@microsoft-github-policy-service agree company="Microsoft"
Contributor License Agreement
Contribution License Agreement
This Contribution License Agreement (“Agreement”) is agreed to by the party signing below (“You”),
and conveys certain license rights to Microsoft Corporation and its affiliates (“Microsoft”) for Your
contributions to Microsoft open source projects. This Agreement is effective as of the latest signature
date below.

Definitions.
“Code” means the computer software code, whether in human-readable or machine-executable form,
that is delivered by You to Microsoft under this Agreement.
“Project” means any of the projects owned or managed by Microsoft and offered under a license
approved by the Open Source Initiative (www.opensource.orghttp://www.opensource.org/).
“Submit” is the act of uploading, submitting, transmitting, or distributing code or other content to any
Project, including but not limited to communication on electronic mailing lists, source code control
systems, and issue tracking systems that are managed by, or on behalf of, the Project for the purpose of
discussing and improving that Project, but excluding communication that is conspicuously marked or
otherwise designated in writing by You as “Not a Submission.”
“Submission” means the Code and any other copyrightable material Submitted by You, including any
associated comments and documentation.
Your Submission. You must agree to the terms of this Agreement before making a Submission to any
Project. This Agreement covers any and all Submissions that You, now or in the future (except as
described in Section 4 below), Submit to any Project.
Originality of Work. You represent that each of Your Submissions is entirely Your original work.
Should You wish to Submit materials that are not Your original work, You may Submit them separately
to the Project if You (a) retain all copyright and license information that was in the materials as You
received them, (b) in the description accompanying Your Submission, include the phrase “Submission
containing materials of a third party:” followed by the names of the third party and any licenses or other
restrictions of which You are aware, and (c) follow any other instructions in the Project’s written
guidelines concerning Submissions.
Your Employer. References to “employer” in this Agreement include Your employer or anyone else
for whom You are acting in making Your Submission, e.g. as a contractor, vendor, or agent. If Your
Submission is made in the course of Your work for an employer or Your employer has intellectual
property rights in Your Submission by contract or applicable law, You must secure permission from Your
employer to make the Submission before signing this Agreement. In that case, the term “You” in this
Agreement will refer to You and the employer collectively. If You change employers in the future and
desire to Submit additional Submissions for the new employer, then You agree to sign a new Agreement
and secure permission from the new employer before Submitting those Submissions.
Licenses.

Copyright License. You grant Microsoft, and those who receive the Submission directly or
indirectly from Microsoft, a perpetual, worldwide, non-exclusive, royalty-free, irrevocable license in the
Submission to reproduce, prepare derivative works of, publicly display, publicly perform, and distribute
the Submission and such derivative works, and to sublicense any or all of the foregoing rights to third
parties.
Patent License. You grant Microsoft, and those who receive the Submission directly or
indirectly from Microsoft, a perpetual, worldwide, non-exclusive, royalty-free, irrevocable license under
Your patent claims that are necessarily infringed by the Submission or the combination of the
Submission with the Project to which it was Submitted to make, have made, use, offer to sell, sell and
import or otherwise dispose of the Submission alone or with the Project.
Other Rights Reserved. Each party reserves all rights not expressly granted in this Agreement.
No additional licenses or rights whatsoever (including, without limitation, any implied licenses) are
granted by implication, exhaustion, estoppel or otherwise.

Representations and Warranties. You represent that You are legally entitled to grant the above
licenses. You represent that each of Your Submissions is entirely Your original work (except as You may
have disclosed under Section 3). You represent that You have secured permission from Your employer to
make the Submission in cases where Your Submission is made in the course of Your work for Your
employer or Your employer has intellectual property rights in Your Submission by contract or applicable
law. If You are signing this Agreement on behalf of Your employer, You represent and warrant that You
have the necessary authority to bind the listed employer to the obligations contained in this Agreement.
You are not expected to provide support for Your Submission, unless You choose to do so. UNLESS
REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING, AND EXCEPT FOR THE WARRANTIES
EXPRESSLY STATED IN SECTIONS 3, 4, AND 6, THE SUBMISSION PROVIDED UNDER THIS AGREEMENT IS
PROVIDED WITHOUT WARRANTY OF ANY KIND, INCLUDING, BUT NOT LIMITED TO, ANY WARRANTY OF
NONINFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE.
Notice to Microsoft. You agree to notify Microsoft in writing of any facts or circumstances of which
You later become aware that would make Your representations in this Agreement inaccurate in any
respect.
Information about Submissions. You agree that contributions to Projects and information about
contributions may be maintained indefinitely and disclosed publicly, including Your name and other
information that You submit with Your Submission.
Governing Law/Jurisdiction. This Agreement is governed by the laws of the State of Washington, and
the parties consent to exclusive jurisdiction and venue in the federal courts sitting in King County,
Washington, unless no federal subject matter jurisdiction exists, in which case the parties consent to
exclusive jurisdiction and venue in the Superior Court of King County, Washington. The parties waive all
defenses of lack of personal jurisdiction and forum non-conveniens.
Entire Agreement/Assignment. This Agreement is the entire agreement between the parties, and
supersedes any and all prior agreements, understandings or communications, written or oral, between
the parties relating to the subject matter hereof. This Agreement may be assigned by Microsoft.

—
Reply to this email directly, view it on GitHubhttps://github.com/Azure/Azure-Sentinel/pull/8038#issuecomment-1547678149, or unsubscribehttps://github.com/notifications/unsubscribe-auth/A5XL3KGZADWTRZ7P5YOA7FLXGIHHRANCNFSM6AAAAAAYCBOXEI.
You are receiving this because you were mentioned.Message ID<EMAIL_ADDRESS>

Hi, we are looking into this, thanks!

Hi we are working on this, will update shortly, thanks!


A11y: Links - Make all links clearly recognizable and streamline their appearance where useful

PHP Version: 8.2   | Shopware Version: <IP_ADDRESS>  | Affected area / extension: Platform(Default)

Expected behaviour:

Links are recognizable and differ from text, not only by color.
In most cases the common pattern (implemented by browsers, known by users) with an underline is useful.
Hover states are clearly visible.
Tbd. is changing color - with compliant color contrast - enough?
Focus states are visible not only by changing the link color.

Actual behaviour:

"Display shopping cart" in the mini cart only differs from other content by a different color, unlike "shipping costs" link in the footer. Instead it matches with the unlinked headline "Service hotline".

New version:
!image-20241126-072045.png|width=41.66666666666667%,alt="image-20241126-072045.png"!

The product name "Variant product" in mini cart, cart and checkout listing is an visually unnoticeable link. It doesn't differ from normal text by any format or sign.

New version:
!image-20241126-072424.png|width=41.66666666666667%,alt="image-20241126-072424.png"!

Unlike the shipping costs toggle ("Standard") in the mini cart, "Shipping details" in the cart isn't technically a button, what it should be ideally as a content toggle. But furthermore it doesn't look like the common links in shopware (missing underline and an arrow icon without any useful meaning / not used in other equivalent places).

New version:
!image-20241126-073244.png|width=41.66666666666667%,alt="image-20241126-073244.png"!

"sign up" in the account dropdown from the header differs from text, especially the "or", only by color. So it's visually unnoticeable for users with color blindness.

New version:
!image-20241126-073333.png|width=168,height=90,alt="image-20241126-073333.png"!

The phone number inside the footer column for "Service hotline" isn't visually noticeable as a link. It looks only like bold text and doesn't have a the blue color, like shopware text links have in general. Furthermore, the headline of the column looks more like a link, but isn't linked.

New version:
!image-20241126-073413.png|width=213,height=96,alt="image-20241126-073413.png"!

The phone number in the checkout header differs from text, especially the "Questions regarding your order?", only by color. So it's visually unnoticeable for users with color blindness.

New version:
!image-20241126-073616.png|width=184,height=61,alt="image-20241126-073616.png"!

The login link on the checkout page ("You already have an account? Click here to login.") differs from bold text or headlines, e.g. see the different text snippets around on the same page, only by color.

Furthermore the default link text already demonstrates, that isn't easy to identify the link by it's text, until you read it in detail.
As an alternative to a text link, it could be a button link. Perhaps a secondary one in contrast to the primary "Continue" button.

New version:
!image-20241126-073702.png|width=33.333333333333336%,alt="image-20241126-073702.png"!

The forgotten pw link on the login/registration page ("I have forgotten my password.") differs from text, only by color. So it's visually unnoticeable for users with color blindness (until the read it in detail). Furthermore it's inconsistent styled to other text links on the same page.

New version:
!image-20241126-073758.png|width=33.333333333333336%,alt="image-20241126-073758.png"!

How to reproduce:
View the default storefront.

[public] Shopware automated response: This issue is linked to the internal issue: NEXT-39630
[created from NEXT-39630, comment 512893]

[public]

FixVersion Change: <IP_ADDRESS>

This issue will be / was solved in Shopware version <IP_ADDRESS>.
[created from NEXT-39630, comment 519472]


[IA-4432] HACK: Quietly force role=creator param into all listApp queries

Only the Cloud Envs page makes these queries; the one for Apps times out after 30s on prod and is likely the cause of memory leaks. This is a HACK pending fix of app query performance.
Jira Ticket: https://broadworkbench.atlassian.net/browse/IA-4432
Testing strategy

[ ] As user U1...
[ ] in workspace W1, create app A1
[ ] in workspace W2, create runtime R1
[ ] Share workspaces W1 and W2 with user U2
[ ] As user U2...
[ ] Goto Cloud Environments page
[ ] Check and uncheck the checkbox
[ ] See just your environments (when checked)
[ ] See all permitted envs (when unchecked) INCLUDING runtime R1 and NOT INCLUDING app A1

Not doing; see https://github.com/DataBiosphere/terra-ui/pull/4105

Not doing, see https://github.com/DataBiosphere/terra-ui/pull/4105


[Feature] Port ethers-addressbook to alloy

Component
contract, provider, pubsub
Describe the feature you would like
Port ethers-addressbook to alloy
Additional context
No response

@mattsse @prestwich Supportive? Will be implemented as another crate in alloy.

This has a very low surfsce area but a potentially very high burden of maintenance via update requests. Thinking about whether it should be part of alloy or in a separate utils workspace. Do we know the actual number of ethers PRs that touched the JSON?


Accept alternate readthedocs.org domain names

According to your code, the only accepted domain name for documentation is http://readthedocs.org. However, other valid domain names are available (see the readthedocs documentation): http://rtfd.io, http://rtfd.org, http://readthedocs.io.
Shouldn't these be accepted as well?
Regards,
-- Louis

After considering this back and forth I decided I shouldn't recommend any places to host documentation.


Converting checkpoints

Are there any scripts available for converting trained Gemma/Llama/Mistral MaxText checkpoints to  HuggingFace?

@A9isha

Hi @peregilk , there isn't one yet but we will add one very soon! thanks for your patience.

@A9isha Thanks a lot for the answer. Really looking forward to this.

Sorry for bothering you again with this @A9isha. Do you have a rough estimate of when the  HF conversion will be ready?

Hi @peregilk ,
Sorry for the delay. We have a PR in the works: https://github.com/google/maxtext/pull/581
If you are up for a bit of an experiment, do you want to give it a shot and let us know if you hit any issue with this script.

Awesome. Ill give it a shot tomorrow, and report back.

Hi @A9isha,
I have now given it a try. Had some minor issues before I ran into a major issue. Ill report the small ones as well because they are mainly related to documentation. Maybe updating the documentation will save others from problems.
I have a checkpoint saved in:
gs://mybucket/north_mistral_warm_norwegian/checkpoints/150000
This is a continual training of a Mistral-7b-model on a Norwegian dataset. It has by default saved checkpoints every 10k steps, I am targeting the last checkpoint.
Your comments refer to running MaxText/llama_or_mistral_ckpt.py first. I assume this is just when converting from the meta-checkpoints, and not needed in my case.
I am starting by creating and cloning a HF-repo (where I plan to place to finished files) and a tmp-directory called /home/user/checkpoint/test-mistral-warm-nortoken.
I made two minor changes from the documentation here:

mistral -> mistral-7b (doc says to choose between mistral and llama)
Added "/items" to the end of the checkpoint-path

I am not really sure what the purpose of run_name is, but set it to "test".
My final command looks like this:
python MaxText/llama_or_mistral_orbax_to_huggingface.py MaxText/configs/base.yml base_output_directory=/home/user/checkpoint/test-mistral-warm-nortoken load_parameters_path=gs://mybucket/north_mistral_warm_norwegian/checkpoints/150000/items run_name=test model_name=mistral-7b hf_model_path=/home/user/test-mistral-warm-nortoken
This now runs for a couple of minutes. I see a couple of warnings that might indicate errors:
Found 0 checkpoint steps in /home/user/checkpoint/test-mistral-warm-nortoken/test/checkpoints

and:
I0411 10:23:46.518555<PHONE_NUMBER>31872 checkpointer.py:168] Restoring item from gs://maxlog-eu/north_mistral_warm_norwegian/checkpoints/150000/items.
W0411 10:23:51.243256<PHONE_NUMBER>31872 transform_utils.py:229] The transformations API will eventually be replaced by an upgraded design. The current API will not be removed until this point, but it will no longer be actively worked on.
I0411 10:23:51.243912<PHONE_NUMBER>31872 transform_utils.py:286] The following keys are not loaded from the original tree after applying specified transforms: params/params/decoder/decoder_norm/scale, params/params/decoder/layers/mlp/wi_0/kernel, params/params/decoder/layers/mlp/wi_1/kernel, params/params/decoder/layers/mlp/wo/kernel, params/params/decoder/layers/post_self_attention_layer_norm/scale, params/params/decoder/layers/pre_self_attention_layer_norm/scale, params/params/decoder/layers/self_attention/key/kernel, params/params/decoder/layers/self_attention/out/kernel, params/params/decoder/layers/self_attention/query/kernel, params/params/decoder/layers/self_attention/value/kernel, params/params/decoder/logits_dense/kernel, params/params/token_embedder/embedding
I0411 10:23:51.244194<PHONE_NUMBER>31872 checkpointer.py:171] Finished restoring checkpoint from gs://maxlog-eu/north_mistral_warm_norwegian/checkpoints/150000/items.

a while after that the conversion however crashes with this message:
In input checkpoint Number of model params=7.242 billion
Traceback (most recent call last):
  File "/home/user/maxtext/MaxText/llama_or_mistral_orbax_to_huggingface.py", line 215, in <module>
    app.run(main)
  File "/home/user/.t5x/lib/python3.10/site-packages/absl/app.py", line 308, in run
    _run_main(main, args)
  File "/home/user/.t5x/lib/python3.10/site-packages/absl/app.py", line 254, in _run_main
    sys.exit(main(argv))
  File "/home/user/maxtext/MaxText/llama_or_mistral_orbax_to_huggingface.py", line 211, in main
    convert_orbax_hf(hf_model_path, pyconfig.config)
  File "/home/user/maxtext/MaxText/llama_or_mistral_orbax_to_huggingface.py", line 198, in convert_orbax_hf
    new_hf_model_params = convert_state_to_hf(training_state, config.model_name)
  File "/home/user/maxtext/MaxText/llama_or_mistral_orbax_to_huggingface.py", line 119, in convert_state_to_hf
    hf_model_params["model.embed_tokens.weight"] = torch.tensor(
TypeError: can't convert np.ndarray of type numpy.object_. The only supported types are: float64, float32, float16, complex64, complex128, int64, int32, int16, int8, uint8, and bool.

@peregilk
Right I think this is caused by a recent breaking change in the way we are generating MaxText's Orbax checkpoints.
https://github.com/google/maxtext/pull/568
Could you please regenerate your MaxText checkpoint with the latest code (i.e., after including the PR#568), and try out the script llama_or_mistral_orbax_to_huggingface.py?

expansion_factor_real_data  was added in this PR https://github.com/google/maxtext/pull/187
But it has a default value in base.yml https://github.com/google/maxtext/blob/main/MaxText/configs/base.yml#L167
Could you check if your current (re)cloned repo has this PR's updates for e.g., in base.yml?

My bad. I tried replicating the the experiment with my custom .yml. I did not realise that there were updates in base.yml. The model is now training, and at the first checkpoint I'll be able to test the export again. I will report my results here. Thanks @A9isha

Hi @A9isha. I am trying to recreate my experiments here so that I am able to convert my models to HF. My first models were trained 3 weeks ago. If I understand correctly, there are also some updates to the conversion script here, so to restart the models I also need to run MaxText/llama_or_mistral_ckpt.py again.
My main sanity check here is if I am able to do a warm restart of the Mistral-7b model using the same tokenizer and a Norwegian c4-corpus from tfds. I am trying to use the exact same settings as eariler, though I see there are some changes to base.yml.
However, the result really puzzles me:

The graphs should be self explanatory.
I am training on v5e-128 with these parametersl:
per_device_batch_size=4
ici_fsdp_transpose_parallelism=16
remat_policy=minimal
This might not be related to the checkpointing at all. Tell me if you want to open a separate issue on it.

Hi @A9isha, does Maxtext support the other way round now? That's converting HF's Llama or Mistral weights to MaxText checkpoints. Thanks

@peregilk Apologies for the delayed response, I was OOO for sometime.

My main sanity check here is if I am able to do a warm restart of the Mistral-7b model using the same tokenizer and a Norwegian c4-corpus from tfds
Yes, the loss curve seems terrible, but yet your idea is correct that you should be definitely able to use your tokenizer and dataset for finetuning the converted checkpoint.

If I understand correctly, there are also some updates to the conversion script here
The changes were not made to checkpoint conversion script but actually to the way the checkpoints are written out and those changes would not have any effect of loss, it's just cosmetic changes.

Let me know if you were able to investigate more on this

Hi @A9isha, does Maxtext support the other way round now? That's converting HF's Llama or Mistral weights to MaxText checkpoints. Thanks

We have the script llama_or_mistral_ckpt.py to convert the original PyTorch Llama2 checkpoint that Meta provides into MaxText checkpoint.
You can see the usage [here] for Llama2-7b for e.g.: (https://github.com/google/maxtext/blob/main/end_to_end/tpu/llama2/7b/test_llama2_7b.sh#L35)

We have the script llama_or_mistral_ckpt.py to convert the original PyTorch Llama2 checkpoint that Meta provides into MaxText checkpoint.
You can see the usage here for Llama2-7b for e.g.

Thanks for the pointer! I'm still wondering if there's  a direct script for converting HF's LLaMA2-like weight to MaxText weight. Since I might want to use another version of LLaMA2 trained by others hosted on HuggingFace. Thanks!

I see, unfortunately no there isn't the conversion script at the moment. It should be a modification of llama_or_mistral_ckpt. If you are interested, please feel free to send across a PR.

I see, unfortunately no there isn't the conversion script at the moment. It should be a modification of llama_or_mistral_ckpt. If you are interested, please feel free to send across a PR.

Thanks @A9isha, I'm working on it and will try to open a PR for it soon :)

@peregilk Apologies for the delayed response, I was OOO for sometime.

My main sanity check here is if I am able to do a warm restart of the Mistral-7b model using the same tokenizer and a Norwegian c4-corpus from tfds
Yes, the loss curve seems terrible, but yet your idea is correct that you should be definitely able to use your tokenizer and dataset for finetuning the converted checkpoint.

If I understand correctly, there are also some updates to the conversion script here
The changes were not made to checkpoint conversion script but actually to the way the checkpoints are written out and those changes would not have any effect of loss, it's just cosmetic changes.

Let me know if you were able to investigate more on this

@A9isha. Sorry for taking some time to reply to this. I had to give up on this. I am mainly doing experiments on dataset composition, and needed something that worked and could be converted to HuggingFace. I was unable to accomplish that with MaxText.

@peregilk I am very sorry to hear that. Does the new feature of supporting HuggingFace dataset on MaxText help you with the data composition effort?
Please feel free to reopen/create a new issue with more findings.
Good luck! :)

Yes. The added HuggingFace dataset support actually makes it super easy to try to replicate some of the experiments I am currently running on Levanter. It would be great to check this out. Maybe some of the issues I was struggling with is fixed now. MaxText definitively seemed to be faster.

@A9isha I have not converted back my experiments from Levanter to MaxText, and will continue using it. I am super impressed by the speed and stability of MaxText. Though it takes some steps, I am able to convert the results back into HuggingFace, and have verified that the results looks good. Thanks a lot for implementing this!!
I am also using HF Datasets for training here, and it works super smooth (Thanks @aireenmei!).
With fixing checkpoint-conversions and dataset loading, I have solved my main issues with using MaxText for my research (that is more related to effects of dataset composition). It would however also be great to do experiments on the newest versions of the current models. I have opened issue #683 related to that. Do you know the status here, @A9isha?

@A9isha is out of office. I believe Llama3 is being worked on cc @khatwanimohit

@A9isha @aireenmei Attaching to this thread since it is related. I am trying to convert a Llama 70B model to HF format. I am using a v4-8. I am using the llama_or_mistral_orbax_to_huggingface.py referred to above.
However, the script keeps getting killed fairly late in the process (after 5-6 hours), before saving anything. Most likely this is an OOM issue.
Do you know if it is possible to convert this model sizes on a v4-8? Are there any tricks for making it possible? It does not seem like pods are supported by the script.

@jwyang-google for questions regarding usage of llama_mistral_mixtral_orbax_to_hf.py

@aireenmei @jwyang-google Just reporting on my own problem here. It turns out that the script runs fine. However, memory requirements are huge. Converting the Llama3.1 70B model was possible with 1.4TB of memory. I was able to do it on CPU with adding a swap disk.


No work anymore

Describe the bug
Seem to doesn't work anymore. I get this message when I try to watch a replay

VOD / Channel link
https://www.twitch.tv/videos/1846239165
Browser
Chrome
Additional context
I have tried several replays and I always have the same error message.

I don't think this error has anything to do with the extension, but I might be wrong. "Error 2000" usually means that there has been a connection failure between your internet and the server you're connecting to, which usually is Twitch's fault. I get this error all the time, on VODs or lives, and it's not specifically tied to any extensions, as far as I know, and I've been using TwitchNoSub for literally less than a day, while getting this error for years since Amazon bought off Twitch.

I don't think this error has anything to do with the extension, but I might be wrong. "Error 2000" usually means that there has been a connection failure between your internet and the server you're connecting to, which usually is Twitch's fault. I get this error all the time, on VODs or lives, and it's not specifically tied to any extensions, as far as I know, and I've been using TwitchNoSub for literally less than a day, while getting this error for years since Amazon bought off Twitch.

Personally, I had never had this error before.
It seems to only occur on sub-only videos so I think it's related to the extension. I don't get this error anywhere else on twitch.

You issue is not reproductible on my side. Maybe try to clean application file, or private navigation (make sure the extension is enabled is private mode).
But it seems it's linked to your local setup since on my side I have access to the video.

Got the same issue, it was the Better Twitch Adblock extension for me, if it can help


Question: Login with browser and export cookies to...

Login with browser and export cookies to "/home/xdshot/.config/lgogdownloader/cookies.txt"
Which exactly cookies do I need to get from my web browser? And how should I paste them into cookies.txt file?

Currently my cookies.txt contains these cookies
galaxy-login-s
galaxy-login-al
galaxy-login-tsa
gog_us
gog_lc
cart_token
gog-al
gog_verify

For Firefox you can use any cookie export add-on that exports cookies in netscape format
https://addons.mozilla.org/en-US/firefox/search/?q=cookie+export
Other browsers should have similar add-ons

Thanks


Fixes for Virtual Directories #156

Fixes for Virtual Directories #156
Please review the changes!

Oh snap ;)
Thanks, I'll try to get this looked at probably early next week. :+1:

Thanks!

@kipusoep Looks good from here too!  I ended up abandoning the virtual directory route as there were too many issues that kept cropping up.  In the end we setup the backoffice server to run at the root, and the front-end nodes to use virtual directories...seems to be working well!
Anyway, one note - the tilde support in PropertyEditor and PreValueField were added in later versions of Umbraco, so we should probably mark the next release with a minimum Umbraco version.  I believe they were added in 7.1.5 and 7.3.0-beta (according to the tags at least [1] [2]).
Alternatively, we might think about using relative paths instead to maintain backward compatibility, ie ../App_Plugins/Archetype/....  Shannon seems to think this should work fine in this comment.

I just ran into an issue with the config page throwing 404. Trying to track that down.

@tomfulton Thanks man. I think we're up to min version of 7.1.6 so far. Any other gotchas?

I think this will fix it: https://github.com/umbraco/Umbraco-CMS/commit/048a8683dd1dfc623805c83096929f7a73ff164c as @tomfulton suggests. So maybe this will have to wait for core 7.3

Curious why the editor doesn't break for the same reason... hmmm.

@kgiszewski Not that I can think of - as mentioned I abandoned this setup, but this looks like everything I did to get it working initially ;).
Yeah, that's the update I was thinking of and was introduced in 7.3.0-beta.  Instead of relying on that update you can try using relative paths as mentioned above...should work.
One other thing you might want to test, is make sure debug=false in the web.config and everything still works.  I recall some issues with ClientDependency playing nice with the virtual routes.

@kgiszewski because [PropertyEditor] supports the virtual paths as of 7.1.5, but [PreValueField] does not until 7.3.0-beta ;)

@tomfulton Awesome for the pain you've already done on this.
So if I change to relative (../), I think we can forego the need for users to go to v7.3.x right?
It appears to work find with the relative path in place but I haven't actually tried with a virtual directory yet.

Yep, that's correct in theory ;)

@tomfulton super, I'm adding this and I don't want to have to wait on nor depend on 7.3 :)


Do the opaque check earlier in CompInfo::codegen

Right now we generate all of a compound type's fields and methods eagerly, and then throw away the result of that computation for opaque types here:
https://github.com/rust-lang-nursery/rust-bindgen/blob/8aa41d195b278d36b9cde9d17c01f5651d9b79b3/src/codegen/mod.rs#L1657-L1675
Instead, we should only codegen the fields and methods if the type isn't opaque in the first place.

Hi! If you have any questions regarding this issue, feel free to make a comment here, or ask it in the #servo channel in IRC.
If you intend to work on this issue, then add @highfive: assign me to your comment, and I'll assign this to you. :smile:

@highfive: assign me

@MaloJaffre, were you interested in picking this issue up as well? I saw you left a @highfive: assign me comment, but then deleted it. It seems like our highfive bot integrations is half broken right now (probably since the move to rust-lang-nursery)

@fitzgen
Sorry, I've mixed up the two issues.

@fitzgen, I have a fix which is passing tests, but I am not completely sure of its correctness - it doesn't call struct_layout.saw_vtable() when item is opaque.
I pretty much just added some if !is_opaque {...} wrappers around the code paths that add to fields and methods

@jhod0 can you make a WIP pull request? That's the easiest way to have discussion about the code. Thanks!

@fitzgen I'll make one later, and ping you when I do

@fitzgen there is now a PR!

@fitzgen  should this be closed?

Yep, thanks!


Upgrade tins to version 1.8.1

Libraries.io has found that there is a newer version of tins that this project depends on.
More info: https://libraries.io/rubygems/tins/1.8.1

Subdependencies (coveralls and rubocop) block this upgrade.


Enhance Intune Win32 App Lifecycle Automation with Improved programmatic Auth and upload Retry Logic

Summary
Overview:
This pull request introduces several key changes aimed at enhancing the reliability, flexibility, and automation capabilities (e.g., GitHub Actions, etc.) of the Intune Win32 app lifecycle management using modern authentication (MSAL.PS is not recommended). The changes primarily address issues related to token handling, authentication flow, Azure Storage blob upload processes, and error handling. Designed to automate via Azure Service Principal auth.
Key Changes:

Enhanced Token Handling and Authentication Flow:

Addition of ExpiresOn Property: The New-ClientCredentialsAccessToken function now calculates and adds an ExpiresOn property to the token object for easier expiration checks.
Improved Token Expiration Check: The Test-AccessToken function now uses the ExpiresOn property to determine token expiration, enhancing reliability.
Refinements in Connect-MSIntuneGraph: Improved error handling and dynamic installation of the MSAL.PS module if still required for other auth methods.
New-AuthenticationHeader: Updated to support tokens with the ExpiresOn property directly.

Content-Type Header Addition:

Invoke-AzureStorageBlobUploadFinalize: Added a content-type header to the REST request to ensure correct handling of the request body during upload finalization.

Fixing System.DateTime Error:

Addressed a specific error related to invalid System.DateTime usage in Azure blob upload processes.

Retry Logic and SAS URI Renewal:

Retry Mechanism: Implemented a retry mechanism for chunk uploads and finalization steps, with a maximum of 5 attempts.
SAS URI Renewal Adjustments: Initially added but removed SAS URI renewal on the first retry, opting for straightforward retries instead, and maintaining renewal attempts if needed based on a timer.

Module Configuration:

Removed MSAL.PS from the required modules to support environments where it is dynamically loaded.

This PR also introduces retry logic to improve the robustness of the Add-IntuneWin32App function, in hopes to help mitigate some transient errors in the following potential areas: (see Issue #8)

Win32 App Creation
Content Version Creation
File Content Creation

Detailed Changes:

Token Handling Enhancements:

New-ClientCredentialsAccessToken: Added to handle modern OAuth2.0 client credentials flow without using MSAL.PS which is not recommended to use anymore.
Test-AccessToken: Updated to utilize the ExpiresOn property.
New-AuthenticationHeader: Updated to handle the ExpiresOn property directly.
Connect-MSIntuneGraph: Improved integration with new token functions and error handling.

Azure Storage Blob Upload:

Invoke-AzureStorageBlobUpload: Enhanced retry logic for chunk uploads and finalization.
Invoke-AzureStorageBlobUploadChunk: Added exception throwing to support retry logic.
Invoke-AzureStorageBlobUploadFinalize: Added exception throwing to support retry logic during finalization.
Invoke-AzureStorageBlobUploadRenew: Added loop to check the status of the SAS URI renewal process.

Module Configuration:

IntuneWin32App.psd1: Removed MSAL.PS from the required modules to allow dynamic loading.

Error Handling Improvements:

Improved error messages and handling in various functions to provide more informative feedback and enhance robustness.

Add-IntuneWin32App: (see Issue #8) Adds retry logic to the following:

Win32 App Creation
Content Version Creation
File Content Creation

Impact:
These improvements are expected to significantly enhance the automation capabilities of the Intune Win32 app lifecycle management process. They address previous limitations and errors, making the system more robust against issues like authentication, throttling / rate limitations, blob upload issues, and some other minor fixes.
Testing:
I have tested this using a fully automated GitHub Actions Intune app management lifecycle workflow, and locally on up to date PS 7. If others can help test other scenarios I would appreciate the help.
Example of the fix working successfully during a GitHub Action (throttling / network / etc issue):

Please review the changes and provide feedback. Thank you!

I believe the upload retry logic is a badly needed feature of IntuneWin32App since Intune gets unpredictable too often. Especially for automation scenarios. So good job on implementing that Tim.
I'm now testing specifically this part of the PR but all the uploads end up with:
WARNING: An error occurred while creating the Win32 application. Error message: Cannot convert the "9/27/2024 10:30:07 AM +00:00" value of type "System.DateTimeOffset" to type "System.DateTime".

As it turns out there's a problem with a cast in this line
https://github.com/MSEndpointMgr/IntuneWin32App/blob/84bc50a61b206bd3482d4e90bb0d6200f3914345/Private/Invoke-AzureStorageBlobUpload.ps1#L58
Looking at the original it looks like you just lost a call to the UtcDateTime property :)
https://github.com/MSEndpointMgr/IntuneWin32App/blob/21b0c09691d864f30a00d611f568a6fde10b7afb/Private/Invoke-AzureStorageBlobUpload.ps1#L58
And honestly I believe a cast to datetime is redundant here since $Global:AccessToken.ExpiresOn.UtcDateTime property is already a DateTime. We can just rewrite the line like this:
$TokenExpireMinutes = [System.Math]::Round(($Global:AccessToken.ExpiresOn.UtcDateTime - $UTCDateTime).TotalMinutes)

On my end, I got errors when using the original:

https://github.com/MSEndpointMgr/IntuneWin32App/blob/21b0c09691d864f30a00d611f568a6fde10b7afb/Private/Invoke-AzureStorageBlobUpload.ps1#L58

Once I removed .UtcDateTime, it worked without issue locally and in GitHub Actions. I'm strictly using it in an automation scenario there since the PR. I'm curious why that part doesn't work for you but does for me and in automation?
I'll test your below suggestion to see if that works for me as well:

$TokenExpireMinutes = [System.Math]::Round(($Global:AccessToken.ExpiresOn.UtcDateTime - $UTCDateTime).TotalMinutes)`

My end goal is Github Actions as well - I already have an instance of Intune App Factory running there. But I'm testing all the code changes in Windows Sandbox, that's where I got the error above.
Not sure why the statement behaves differently for us, but it could be due to different locales and date/time formats.

My end goal is Github Actions as well - I already have an instance of Intune App Factory running there. But I'm testing all the code changes in Windows Sandbox, that's where I got the error above.
Not sure why the statement behaves differently for us, but it could be due to different locales and date/time formats.

Are you able to test this latest change? It is working well on everything I can test it on. I'd appreciate it if you could try it when you have time.

Next up, I need to add retry logic to almost all calls to MS. It's actually fairly frequent that I get request timeouts, and simply waiting a few seconds fixes it.

@obuolinis Are you able to test this latest change? It is working well on everything I can test it on. I'd appreciate it if you could try it when you have time.

Managed to test it just now. It does work for me as well, both in Github and within Windows Sandbox. However, I'm really puzzled about this "black magic" trick you went for :)
https://github.com/MSEndpointMgr/IntuneWin32App/blob/75901a50ce607e14fb1326d7ead441d49b3141b6/Private/Invoke-AzureStorageBlobUpload.ps1#L57-L62
Why did you have to convert a DateTimeOffset to String and then parse back to DateTimeOffset? I don't think DateTimeOffset is culture-dependent.
In my eyes token expiration calculation here has got to be just a simple subtraction of either two Datetimes or two Datetimeoffsets, whatever the preference.
If you like Datetimeoffset, then ExpiresOn property is already what you need:
PS C:\> $AccessToken.ExpiresOn.GetType()

IsPublic IsSerial Name                                     BaseType
-------- -------- ----                                     --------
True     True     DateTimeOffset                           System.ValueType

If you go for Datetime, like in the original, then there's UtcDateTime property:
PS C:\> $AccessToken.ExpiresOn.UtcDateTime.GetType()

IsPublic IsSerial Name                                     BaseType
-------- -------- ----                                     --------
True     True     DateTime                                 System.ValueType

Did you test any of the above?

@obuolinis Are you able to test this latest change? It is working well on everything I can test it on. I'd appreciate it if you could try it when you have time.

Managed to test it just now. It does work for me as well, both in Github and within Windows Sandbox. However, I'm really puzzled about this "black magic" trick you went for :)
https://github.com/MSEndpointMgr/IntuneWin32App/blob/75901a50ce607e14fb1326d7ead441d49b3141b6/Private/Invoke-AzureStorageBlobUpload.ps1#L57-L62
Why did you have to convert a DateTimeOffset to String and then parse back to DateTimeOffset? I don't think DateTimeOffset is culture-dependent.
In my eyes token expiration calculation here has got to be just a simple subtraction of either two Datetimes or two Datetimeoffsets, whatever the preference.
If you like Datetimeoffset, then ExpiresOn property is already what you need:
PS C:\> $AccessToken.ExpiresOn.GetType()

IsPublic IsSerial Name                                     BaseType
-------- -------- ----                                     --------
True     True     DateTimeOffset                           System.ValueType

If you go for Datetime, like in the original, then there's UtcDateTime property:
PS C:\> $AccessToken.ExpiresOn.UtcDateTime.GetType()

IsPublic IsSerial Name                                     BaseType
-------- -------- ----                                     --------
True     True     DateTime                                 System.ValueType

Did you test any of the above?

Hey, sorry I had no idea you responded to this until now.
It's been a bit, but what I remember is that it was EXTREMELY sensitive and that was the only way I could reliably get it to work across all my tests while also considering the potential culture specific differences. They were both different enough that you couldn't do a simple subtraction between them reliably. I don't remember all the details anymore, but I did try a lot of options. This was the simplest way I could make it work without changing or affecting things upstream or downstream.

@obuolinis Are you able to test this latest change? It is working well on everything I can test it on. I'd appreciate it if you could try it when you have time.

Managed to test it just now. It does work for me as well, both in Github and within Windows Sandbox. However, I'm really puzzled about this "black magic" trick you went for :)
https://github.com/MSEndpointMgr/IntuneWin32App/blob/75901a50ce607e14fb1326d7ead441d49b3141b6/Private/Invoke-AzureStorageBlobUpload.ps1#L57-L62
Why did you have to convert a DateTimeOffset to String and then parse back to DateTimeOffset? I don't think DateTimeOffset is culture-dependent.
In my eyes token expiration calculation here has got to be just a simple subtraction of either two Datetimes or two Datetimeoffsets, whatever the preference.
If you like Datetimeoffset, then ExpiresOn property is already what you need:
PS C:\> $AccessToken.ExpiresOn.GetType()

IsPublic IsSerial Name                                     BaseType
-------- -------- ----                                     --------
True     True     DateTimeOffset                           System.ValueType

If you go for Datetime, like in the original, then there's UtcDateTime property:
PS C:\> $AccessToken.ExpiresOn.UtcDateTime.GetType()

IsPublic IsSerial Name                                     BaseType
-------- -------- ----                                     --------
True     True     DateTime                                 System.ValueType

Did you test any of the above?

Hey, sorry I had no idea you responded to this until now.
It's been a bit, but what I remember is that it was EXTREMELY sensitive and that was the only way I could reliably get it to work across all my tests while also considering the potential culture specific differences. They were both different enough that you couldn't do a simple subtraction between them reliably. I don't remember all the details anymore, but I did try a lot of options. This was the simplest way I could make it work without changing or effecting things upstream or downstream.