HTTP POST fails with 401 Unauthorized

Hi,
I have setup a dev tunnel to a locally running ASP.NET Core app which uses Azure AD for SSO.
The tunnel was setup using these commands:
devtunnel create my-test-dev-tunnel

devtunnel port create my-test-dev-tunnel --port-number 5000 --protocol https
devtunnel port create my-test-dev-tunnel --port-number 5001 --protocol https
devtunnel port create my-test-dev-tunnel --port-number 5173 --protocol https

devtunnel access create my-test-dev-tunnel --port-number 5000 --tenant --expiration 5h
devtunnel access create my-test-dev-tunnel --port-number 5001 --tenant --expiration 5h
devtunnel access create my-test-dev-tunnel --port-number 5173 --anonymous --expiration 5h

When I navigate to a URL that requires authentication using through the tunnel, I get redirected properly to the Azure AD login page. Upon a successful login, Azure AD responds back with a simple HTML page containing a form that is submitted automatically in the browser to my-app-url/signin-oidc and sends some auth parameters back to my app. The form is submitting using a HTTP POST.
However, the HTTP POST request always fails with 401 and never actually hits my app. I thought maybe /signin-oidc is conflicting with something in the tunnel itself so I changed that callback url to /hello-world which did not fix the issue.
I then noticed that HTTP POST requests are all failing.
I tried adding X-Tunnel-Skip-AntiPhishing-Page header and setting its value to True thinking it was because of the anti-phishing measures (shouldn't be since the request is POST) but that did not help either.
I'd appreciate any help and guidance you can offer. This is the last thing for me to solve before we can adopt dev tunnels into our internal processes.
Thank you.

When I navigate to a URL that requires authentication using through the tunnel, I get redirected properly to the Azure AD login page.
Is the URL used here the port that you have anonymous access enabled on, i.e. 5173?

However, the HTTP POST request always fails with 401 and never actually hits my app.
I then noticed that HTTP POST requests are all failing.

Can you share a tunnelId and request/correlation id for the POST requests returning 401?

Hi @derekbekoe, thanks for the reply.
Let me explain the setup a bit more. There are 3 ports that we need open when running the website locally:

5000: Website 1
5001: Website 2
5173: Vite (Front-End tooling) Development Server and Hot Module Reload.

5173 has anonymous access because the front-end of the website (either 5000 or 5001) will need to open a WSS connection and it will fail if it gets redirected to a login page.
The issue I described above happens when I navigate to my-website-host/login. My app redirects with a 302 to https://login.microsoft..... Once the login is successful, a 200 response containing a HTML form with some JS to submit it on page load. The HTTP POST from that form always returns a 401 error.
I observed my app logs at debug level and the request doesn't make it to my app at all. Inspecting the headers and comparing them with a request that made it to my app's server, confirms that the response is not generated by my app.
Here's a POST request sent to my website's home page that is also failing:
* Preparing request to https://0lw58g42.aue.devtunnels.ms:5000/
* Current time is 2024-05-08T23:24:20.209Z
* Enable automatic URL encoding
* Using default HTTP version
* Disable SSL validation
* Found bundle for host 0lw58g42.aue.devtunnels.ms: 0x23f8b439cb0 [can multiplex]
* Re-using existing connection! (#18) with host 0lw58g42.aue.devtunnels.ms
* Connected to 0lw58g42.aue.devtunnels.ms (<IP_ADDRESS>) port 5000 (#18)
* Using Stream ID: 3 (easy handle 0x23f8b84ec70)
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):

> POST / HTTP/2
> Host: 0lw58g42.aue.devtunnels.ms:5000
> user-agent: insomnia/2023.5.8
> accept: */*
> content-length: 0

* TLSv1.2 (IN), TLS header, Supplemental data (23):

< HTTP/2 401
< date: Wed, 08 May 2024 23:24:20 GMT
< content-length: 0
< www-authenticate: tunnel
< x-content-type-options: nosniff
< ratelimit-limit: HttpRequestRatePerPort:1500/m
< ratelimit-remaining: HttpRequestRatePerPort:1499
< ratelimit-reset: HttpRequestRatePerPort:49s
< vssaas-request-id: c7b1954b-ae2f-41c4-a40f-4cae66e63683
< strict-transport-security: max-age=31536000; includeSubDomains
< x-served-by: tunnels-prod-rel-aue-v3-cluster

* Connection #18 to host 0lw58g42.aue.devtunnels.ms left intact

The above is an empty POST request (no json or form body).
My tunnel ID is: carlisle-homes-dev-tunnel. I am not sure where I can find the correlation ID but the above log has vssaas-request-id.
Some more info:
> devtunnel show carlisle-homes-dev-tunnel

Tunnel ID             : carlisle-homes-dev-tunnel.aue
Description           :
Labels                :
Access control        : {}
Host connections      : 1
Client connections    : 0
Current upload rate   : 0 MB/s (limit: 20 MB/s)
Current download rate : 0 MB/s (limit: 20 MB/s)
Upload total          : 1799 KB
Download total        : 26 MB
Ports                 : 3
  5000  https  https://0lw58g42-5000.aue.devtunnels.ms/    (Host:unchanged, Origin:unchanged)
  5001  https  https://0lw58g42-5001.aue.devtunnels.ms/
  5173  https  https://0lw58g42-5173.aue.devtunnels.ms/
Tunnel Expiration     : 30 days

Looking forward to your reply.
Cheers

I am not sure where I can find the correlation ID but the above log has vssaas-request-id.
Yes I meant vssaas-request-id, thanks.

The issue I described above happens when I navigate to my-website-host/login(port 5000 or 5001). My app redirects with a 302 to https://login.microsoft..... Once the login is successful, a 200 response containing a HTML form with some JS to submit it on page load. The HTTP POST from that form always returns a 401 error.
I observed my app logs at debug level and the request doesn't make it to my app at all. Inspecting the headers and comparing them with a request that made it to my app's server, confirms that the response is not generated by my app.

I believe the initial redirection you see to https://login.microsoft..... is not from your app but from the Dev Tunnels service. Once that login is successful, it's Dev Tunnels that redirects to your HTML.

I'll try to explain:
devtunnel access create my-test-dev-tunnel --port-number 5000 --tenant --expiration 5h
devtunnel access create my-test-dev-tunnel --port-number 5001 --tenant --expiration 5h

When you run the commands above, you're telling Dev Tunnels to authenticate requests on your behalf. Specifically in this case, to only allow access for identities in your Entra ID tenant. When the initial GET request to my-website-host/login is made, Dev Tunnels redirects you to log in to your tenant via Microsoft Entra and after login, you see your content. But when the POST request is made, you get the 401 response and that response is coming from the Dev Tunnels service (not your app) and it's because that request is not authenticated.

I have setup a dev tunnel to a locally running ASP.NET Core app which uses Azure AD for SSO.

Given you initially mentioned this, it sounds like your app (running on port 5000 / 5001) is already doing its own auth. Therefore, instead of --tenant, you could specify --anonymous. That way, Dev Tunnels will not auth and it'd be up to your own app to authenticate the requests. This YouTube video covers that topic - https://www.youtube.com/watch?v=yCYLurylgj8.
On the other hand, if your app does not authenticate the POST endpoint or there's another reason you do not want to enable --anonymous, you can specify a Tunnel Access Token as a request header on the POST request - https://learn.microsoft.com/en-us/azure/developer/dev-tunnels/security#tunnel-access.

Thank you @derekbekoe that was super helpful using --anonymous worked perfectly.
I watched the video but I am missing one piece of info here and would appreciate a quick explanation from you if you can:
When I have --tenant enabled and navigate to the tunnel URL, I go past the anti-phishing screen and login to the tenant. At this point, my browser will hold cookies that ensure the next time I navigate to that domain, I won't see the anti-phishing screen nor would have to login again.
If I then navigate to my /login URL, I get taken to the app's own AzureAD login page. When I login successfully, the HTTP post from AzureAD back to my tunnel's domain fails. Are you suggesting that this failure (error 401) is happening because the POST is going to MS servers? Why would it do that if my browser is still carrying the cookies that log me into the tunnel?
Essentially, since that POST request is happening from the same browser, shouldn't it also be authenticated?
Again, thank you for taking the time to answer.
Cheers

Glad to hear --anonymous on all ports works.
Dev Tunnel authentication is per-port, not per-tunnel. For the cookies, you should see one for each port. So, if you go through the login flow for port 5000 and then try to submit a POST request to port 5001, that request will not be authenticated and so the Dev Tunnel service would return a 401.
If all the requests are happening on the same port and you're still getting a 401, we may need some more details on each request of the flow you are describing.

If all the requests are happening on the same port and you're still getting a 401, we may need some more details on each request of the flow you are describing.

Yes, this is what happened. The flow I described was on one port the entire time. What kind of details do you need to investigate that?

Please share recent vssaas-request-ids for each of the requests including the timestamp it occurred. This should include the first GET request and then the POST request as well. Also, please check that in your browser, the dev tunnels auth cookie is sent along with the POST request.

Closing this as stale but please comment to clarify your use case if you continue to see an issue.


fix(index):changed verbage from orginization to agencies

#722
@mannyrivera2010 @lsemesky @blynch11p

:+1:

:+1:


Content Security Policy

The crop fails when I add
meta http-equiv="Content-Security-Policy" content="default-src 'self'"
for security perposes regarding xss how can I be able to fix this???
I think the cause of this is because your code creates script or img tags somewhere but I dont know where, can you help me with that please

This may be related to your issue: https://github.com/jwagner/smartcrop.js/issues/26

Hello @Xaplink smartcrop only creates canvas elements. It doesn't create any images or script tags (as far as I know). Please provide a link with a minimal testcase that reproduces the issue.

Please provide a link with a minimal testcase that reproduces the issue.

I cannot provide that as my website is still underconstruction please put the meta tag on the head of your html page that has the smartcrop you will see that it will not work here is the meta tag


E-mail regex update

Updated the e-mail regex. Now it is using the regex retrieved from the sixarm e-mail validation gem.
If approved, it closes #1014.

This got closed, but never integrated. I'd love to have more robust email validation
What's the story?

You can customize the e-mail regexp in your app. Devise is keeping it simple by choice.

Understood. The comment from @rafaelfranca followed by the close was confusing. :-)
Thanks.

Removed xD


[Task]: Explore features list which will not be supported or needs update because of the airgap environment

We are tracking the feature list in following doc:
https://www.notion.so/appsmith/Airgap-implementation-details-WIP-014cc2553f634083979492a0d1948171?pvs=4#c5817673446f45ef99e8a231840944e6

@berzerkeer please close this once your exploration is done.

Closing this now, @berzerkeer please open it if needed.


Multi-GPU single-machine on larger than memory data without Dask

Hi there, I'm trying to get the Python package to work on larger than memory data using GPU training.
I'm using a data iterator with a QuantileDMatrix as shown here.
When I check GPU saturation using nvidia-smi I only see one GPU being saturated.
Is there a way to pass in multiple GPUs without using a local Dask cluster, or alternatively use external memory and data iterator from a local Dask cluster?
Thanks for any pointers!

When multiple GPUs are used, it's by definition a distributed computing task, hence a distributed framework like dask or pyspark.

or alternatively use external memory and data iterator from a local Dask cluster?

I don't quite understand what do you mean by this, could you please elaborate? In addition, could you please explain what you are trying to achieve?

Thanks for the clarifying questions @trivialfis !

I don't quite understand what do you mean by this, could you please elaborate?
I was trying to get something like this to work with Dask or another distributed framework.

In addition, could you please explain what you are trying to achieve?
I'd like to train on larger than GPU-memory datasets using GPUs. Say, 100GB of data on a GPU with 16GB of memory.

Internally, the dask interface with QuantileDMatrix uses the iterator to go through dask data partitions similar to what the demo does.
When input is on CPU, one can squeeze some extra memory so that XGBoost might require lesser GPU memory than the actual dataset as XGBoost GPU has data compression. But that's no where near your 100/16 ratio.
Gradient boosting is not a stochastic algorithm and requires the dataset to be fully loaded into memory. Unlike deep learning where one can train the model with small batches, we can't boost a new base model using only a small fraction of data without significant lost of accuracy.

You can try external memory with gradient based sampling with a small sampling ratio, which can help. But then, it's not going to be 100/16 ratio

I understand, thank you for the guidance @trivialfis !


Use smaller and compressed varients of buttons and form components

Description
Replace instances of EuiButton that don't have an explicit sizing attribute to EuiSmallButton*.
Replace instances of Eui<form elements> that don't have density attributes to EuiCompressed<form elements>.
Check List

[ ] New functionality includes testing.

[ ] All tests pass, including unit test, integration test and doctest

[ ] New functionality has been documented.

[ ] New functionality has javadoc added
[ ] New functionality has user manual doc added

[X] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Snapshots should be dupated

Why does this repository have workflows that are known to fail?

2 workflows fail because the plugin cannot be installed
the link checker is not configured properly
the code is riddled with lint errors

Maybe disable these workflows for good open source citizenry.


Linux suppport

as linux gaming - including VR gaming - becomes more popular with steam play and dxvk, it would be nice to have such tools as openVR advanced settings, playspace mover and chaperone tweak on linux.

Well, ORV Advanced Settings is already on linux (which includes the playspace mover). But I totally agree that ChaperoneTweak would be awesome to have as well :)


Fieldset label margins

.hw-fieldset__content label has some margin declaration that is causing some issues.
First of all - we should prefere bottom margin og these type of elements.
Second - we might not need it.


[$1000] WebP image loads every time while opening in preview

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Action Performed:

Go to any chat
send any webp image
open image in full window 2-3 times

Expected Result:
should show loading for first rime same as other imaged\s
Actual Result:
image loads every time
Workaround:
Can the user still use Expensify without this being fixed? Have you informed them of the workaround?
Platforms:

Which of our officially supported platforms is this issue occurring on?

[ ] Android / native
[ ] Android / Chrome
[ ] iOS / native
[ ] iOS / Safari
[x] MacOS / Chrome / Safari
[ ] MacOS / Desktop

Version Number: 1.3.16-6
Reproducible in staging?: y
Reproducible in production?: y
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation
https://github.com/Expensify/App/assets/43996225/89620dfa-2718-4df1-b0ee-029f829d169d
https://github.com/Expensify/App/assets/43996225/97fa74f9-b8d5-4326-87e1-f8b5cb7e91c8
Expensify/Expensify Issue URL:
Issue reported by: @gadhiyamanan
Slack conversation: https://expensify.slack.com/archives/C049HHMV9SM/p1684400068777829
View all open jobs on GitHub
Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~019f3d2f9c9afc9a99
Upwork Job ID:<PHONE_NUMBER>274568192
Last Price Increase: 2023-05-29

From Slack: https://expensify.slack.com/archives/C01SKUP7QR0/p1684768175300849?thread_ts=1684767732.056779&cid=C01SKUP7QR0
it is doing 4 requests (2 fail?!?!?) when you open the image:

Proposal
Please re-state the problem that we are trying to solve in this issue.
Webp images aren't stored in the browser cache and are reloaded every time.
What is the root cause of that problem?
The root cause of this problem is that the response header for webp images always contains 'Cache-Control: no-cache, no-store', preventing them from being stored in the cache.

Bad webp response header (https://new.expensify.com)

What changes do you think we should make in order to solve the problem?

Modify the response header for webp images to include 'Cache-Control: public, max-age=put_expiry_here'. This will allow the browser to cache the webp images and serve them from the cache for subsequent requests.
Set the Content-Type header to image/webp for webp images. Currently, this header is not set, and explicitly setting it will ensure proper handling of the image format.

Good webp response header (https://staging.new.expensify.com)

What alternative solutions did you explore? (Optional)
Setting Cache-Control manually in the request header like this:
<RNImage
    {...forwardedProps}
    source={{
        uri: source.uri,
        headers: {
            'Cache-Control': 'max-age=3600, immutable'
        }
    }}
/>

The server continued to return no-cache for webp images. Therefore, modifying the response header on the server side is the recommended approach to address the caching issue.

@Naveen3Singh Thanks for the proposal. Your RCA is correct. Looks like this will be handled internally.
cc @srikarparsi

Not overdue. This should be internal. cc @srikarparsi ^

Thanks @s77rt, will take this over.

Issue not reproducible during KI retests. (First week)

Agree, I'm not able to reproduce this issue with this image I downloaded from here. @abekkala, I believe we can close out this issue and reopen it if the issues resurfaces.

@srikarparsi It's reproducible in iOS and Android for jpg image also. Do you need a separate ticket?
Uploading Record_2023-05-31-23-12-46_4f9154176b47c00da84e32064abf1c48.mp4…

@s77rt @Naveen3Singh is this other issue the same or similar root cause? It seems like the same same issue, but only for jpg files on iOS and Android native apps.

@sakluger I don't think they have the same root cause. This issue's root cause is internal (server response). The other seems related to other factors (I guess it's related to react-native-fast-image).

@srikarparsi I was able to reproduce this as well.

I agree @s77rt. I think we should close out this issue in favor of this issue since they're more or less unrelated. I'll close it out for now but if there's anything left to do for this or the issue resurfaces, feel free to reopen it.


Missing most licenses/dependencies starting in webpack 5.55.0

I updated to a recent version of webpack (5.68.0), and some time later noticed the license file generated with this plugin was missing almost all dependencies.  It only showed dependencies for a few webpack related plugins, like css-loader.  It was missing all other dependencies, like react.  I kept downgrading webpack and found that this started happening in webpack 5.55.0.  5.54.0 and before work as expected.
Looking briefly at code of the plugin, it looks like compilation.fileDependencies, used in licenseUtils.js doesn't have the missing dependencies when it's accessed (for example, react is not in the list).

same issue, have you solved this?


Responsive should change layout when a window increases in size

Bug Report
Steps

Checkout homepage starter template

Run template

Resize from desktop width to a mobile width, i.e. show mobile menu and content.

Resize back to a desktop width

Expected Result
Responsive is expected to show the desktop content and menu.
Actual Result
Responsive remains on the mobile content.
Version
0.82.0

Have tested in version 0.75.1 and Responsive is working correctly.

Fixed in master and will be released soon, #2992.


Add tutorial for user to be able to save this app on screen (android)

this was already implemented.


[tests-only] Run phoenix UI tests with OCIS and owncloud storages

Fixes https://jira.owncloud.com/browse/OCIS-706 (some day - see comment below)

The phoenix UI pipelines for "ocis" storage fail because the phoenix test code does not have any special code that knows how to cleanup user files... "under the hood" when a user is deleted and has "ocis" storage. We could add special test code to do that. But PR #755 is in progress - that will make OCIS (on both ocis and owncloud storage) delete the user files correctly when a user is deleted. When that PR is finished, then there will be no special "under the hood" stuff to do when deleting a user.
Note: the core API test code in owncloud/core does do special stuff "under the hood" when deleting users. It has the tricks for what to do when the user is on "ocis" and/or "owncloud" storage. That is why the core API tests can be and are run on both storages.

This is blocked by PR #755 which is blocked by https://github.com/cs3org/cs3apis/pull/95

This is blocked by PR #755 which is blocked by https://github.com/cs3org/cs3apis/pull/95

Need to see if there is code in cs3org/reva yet to implement the delete-home stuff - https://github.com/cs3org/cs3apis/pull/95#issuecomment-778861517

Issue #1987 PR #1990 changes from "owncloud" to "ocis" storage for the webUI tests. We decided not to run all the tests twice - they just run with "ocis" storage now.


Deployment failed both at Vercel and Netlify

All was fine until recently I am unable to deploy anything either on Netlify or Vercel.
I have tried deleting and re-installing Digital Garden in Obsidian, updating the template in the plug-in, deleting all projects on Netlify, Vercel, and Github, and recreating new projects, but it still doesn't work.
I am not a tech person so I don't know how to read the error message and solve the problem.
On Netlify, it says non-zero exit status: 2.
9:31:42 PM: build-image version: 68a2c262fa3c95a08a1716b8dde8fba0e332ec73 (focal)
9:31:42 PM: buildbot version: 68a2c262fa3c95a08a1716b8dde8fba0e332ec73
9:31:42 PM: Fetching cached dependencies
9:31:42 PM: Starting to download cache of 104.0MB
9:31:43 PM: Finished downloading cache in 334ms
9:31:43 PM: Starting to extract cache
9:31:43 PM: Finished extracting cache in 775ms
9:31:43 PM: Finished fetching cache in 1.169s
9:31:43 PM: Starting to prepare the repo for build
9:31:44 PM: Preparing Git Reference refs/heads/main
9:31:45 PM: Parsing package.json dependencies
9:31:46 PM: Starting to install dependencies
9:31:46 PM: Python version set to 3.8
9:31:46 PM: Attempting Ruby version 2.7.2, read from environment
9:31:47 PM: Using Ruby version 2.7.2
9:31:47 PM: Started restoring cached go cache
9:31:47 PM: Finished restoring cached go cache
9:31:47 PM: Installing Go version 1.19.5 (requested 1.19.5)
9:31:53 PM: go version go1.19.5 linux/amd64
9:31:53 PM: Using PHP version 8.0
9:31:54 PM: Started restoring cached Node.js version
9:31:54 PM: Finished restoring cached Node.js version
9:31:55 PM: v16.20.0 is already installed.
9:31:55 PM: Now using node v16.20.0 (npm v8.19.4)
9:31:55 PM: Enabling Node.js Corepack
9:31:55 PM: Started restoring cached build plugins
9:31:55 PM: Finished restoring cached build plugins
9:31:55 PM: Started restoring cached corepack dependencies
9:31:55 PM: Finished restoring cached corepack dependencies
9:31:55 PM: No npm workspaces detected
9:31:55 PM: Started restoring cached node modules
9:31:55 PM: Finished restoring cached node modules
9:31:55 PM: Installing npm packages using npm version 8.19.4
9:31:58 PM: up to date, audited 555 packages in 2s
9:31:58 PM: 98 packages are looking for funding
9:31:58 PM:   run npm fund for details
9:31:58 PM: 14 vulnerabilities (2 moderate, 9 high, 3 critical)
9:31:58 PM: To address issues that do not require attention, run:
9:31:58 PM:   npm audit fix
9:31:58 PM: Some issues need review, and may require choosing
9:31:58 PM: a different dependency.
9:31:58 PM: Run npm audit for details.
9:31:58 PM: npm packages installed
9:31:58 PM: Install dependencies script success
9:31:58 PM: Starting build script
9:31:59 PM: Detected 1 framework(s)
9:31:59 PM: "eleventy" at version "2.0.1"
9:31:59 PM: Section completed: initializing
9:32:00 PM:
9:32:00 PM:   Netlify Build
9:32:00 PM: ────────────────────────────────────────────────────────────────
9:32:00 PM:
9:32:00 PM: ❯ Version
9:32:00 PM:   @netlify/build 29.9.2
9:32:00 PM:
9:32:00 PM: ❯ Flags
9:32:00 PM:   baseRelDir: true
9:32:00 PM:   buildId: 64473b8a0aa74600080d0693
9:32:00 PM:   deployId: 64473b8a0aa74600080d0695
9:32:00 PM:
9:32:00 PM: ❯ Current directory
9:32:00 PM:   /opt/build/repo
9:32:00 PM:
9:32:00 PM: ❯ Config file
9:32:00 PM:   /opt/build/repo/netlify.toml
9:32:00 PM:
9:32:00 PM: ❯ Context
9:32:00 PM:   production
9:32:00 PM:
9:32:00 PM:   1. build.command from netlify.toml
9:32:00 PM: ────────────────────────────────────────────────────────────────
9:32:00 PM:
9:32:00 PM: $ npm install && npm run build
9:32:01 PM: up to date, audited 555 packages in 793ms
9:32:01 PM: 98 packages are looking for funding
9:32:01 PM:   run npm fund for details
9:32:01 PM: 14 vulnerabilities (2 moderate, 9 high, 3 critical)
9:32:01 PM: To address issues that do not require attention, run:
9:32:01 PM:   npm audit fix
9:32:01 PM: Some issues need review, and may require choosing
9:32:01 PM: a different dependency.
9:32:01 PM: Run npm audit for details.
9:32:02 PM: ><EMAIL_ADDRESS>build
9:32:02 PM: > npm-run-all get-theme build:*
9:32:02 PM: ><EMAIL_ADDRESS>get-theme
9:32:02 PM: > node src/site/get-theme.js
9:32:02 PM: ><EMAIL_ADDRESS>build:eleventy
9:32:02 PM: > cross-env ELEVENTY_ENV=prod NODE_OPTIONS=--max-old-space-size=4096 eleventy
9:32:03 PM: [11ty] Problem writing Eleventy templates: (more in DEBUG output)
9:32:03 PM: [11ty] 1. Having trouble rendering njk template ./src/site/notes/xxx/xxx.md (via TemplateContentRenderError)
9:32:03 PM: [11ty] 2. Cannot read properties of undefined (reading 'docId') (via TypeError)
9:32:03 PM: [11ty]
9:32:03 PM: [11ty] Original error stack trace: TypeError: Cannot read properties of undefined (reading 'docId')
9:32:03 PM: [11ty]     at Object.render_footnote_anchor_name [as footnote_anchor_name] (/opt/build/repo/node_modules/markdown-it-footnote/index.js:12:18)
9:32:03 PM: [11ty]     at Object.render_footnote_ref [as footnote_ref] (/opt/build/repo/node_modules/markdown-it-footnote/index.js:30:27)
9:32:03 PM: [11ty]     at Renderer.render (/opt/build/repo/node_modules/markdown-it/lib/renderer.js:332:38)
9:32:03 PM: [11ty]     at /opt/build/repo/src/helpers/utils.js:17:36
9:32:03 PM: [11ty]     at Array.forEach ()
9:32:03 PM: [11ty]     at namedHeadings (/opt/build/repo/src/helpers/utils.js:15:18)
9:32:03 PM: [11ty]     at Core.process (/opt/build/repo/node_modules/markdown-it/lib/parser_core.js:54:13)
9:32:03 PM: [11ty]     at MarkdownIt.parse (/opt/build/repo/node_modules/markdown-it/lib/index.js:524:13)
9:32:03 PM: [11ty]     at MarkdownIt.render (/opt/build/repo/node_modules/markdown-it/lib/index.js:544:36)
9:32:03 PM: [11ty]     at /opt/build/repo/node_modules/@11ty/eleventy/src/Engines/Markdown.js:79:38
9:32:03 PM: [11ty] Copied 6 files / Wrote 0 files in 0.15 seconds (v2.0.1)
9:32:03 PM: ERROR: "build:eleventy" exited with 1.
9:32:03 PM:
9:32:03 PM:   "build.command" failed
9:32:03 PM: ────────────────────────────────────────────────────────────────
9:32:03 PM:
9:32:04 PM: Failed during stage 'building site': Build script returned non-zero exit code: 2 (https://ntl.fyi/exit-code-2)
9:32:03 PM:   Error message
9:32:03 PM:   Command failed with exit code 1: npm install && npm run build (https://ntl.fyi/exit-code-1)
9:32:03 PM:
9:32:03 PM:   Error location
9:32:03 PM:   In build.command from netlify.toml:
9:32:03 PM:   npm install && npm run build
9:32:03 PM:
9:32:03 PM:   Resolved config
9:32:03 PM:   build:
9:32:03 PM:     command: npm install && npm run build
9:32:03 PM:     commandOrigin: config
9:32:03 PM:     publish: /opt/build/repo/dist
9:32:03 PM:     publishOrigin: config
9:32:03 PM:   functionsDirectory: /opt/build/repo/netlify/functions
9:32:03 PM:   redirects:
9:32:04 PM:     - from: /api/*      status: 200      to: /.netlify/functions/:splat    - from: /*      status: 404      to: /404  redirectsOrigin: configCaching artifacts
9:32:04 PM: Started saving node modules
9:32:04 PM: Finished saving node modules
9:32:04 PM: Started saving build plugins
9:32:04 PM: Finished saving build plugins
9:32:04 PM: Started saving corepack cache
9:32:04 PM: Finished saving corepack cache
9:32:04 PM: Started saving pip cache
9:32:04 PM: Finished saving pip cache
9:32:04 PM: Started saving emacs cask dependencies
9:32:04 PM: Finished saving emacs cask dependencies
9:32:04 PM: Started saving maven dependencies
9:32:04 PM: Finished saving maven dependencies
9:32:04 PM: Started saving boot dependencies
9:32:04 PM: Finished saving boot dependencies
9:32:04 PM: Started saving rust rustup cache
9:32:04 PM: Finished saving rust rustup cache
9:32:04 PM: Started saving go dependencies
9:32:04 PM: Finished saving go dependencies
9:32:04 PM: Build failed due to a user error: Build script returned non-zero exit code: 2
9:32:04 PM: Failing build: Failed to build site
9:32:04 PM: Finished processing build request in 22.223s
Here is the error message on Vercel:
[19:17:23.231] Running build in San Francisco, USA (West) – sfo1
[19:17:23.291] Cloning github.com/xxx/xxx (Branch: main, Commit: 8ab3aa4)
[19:17:24.199] Cloning completed: 907.196ms
[19:17:24.493] Restored build cache
[19:17:24.522] Running "vercel build"
[19:17:24.969] Vercel CLI 28.18.5
[19:17:25.940] Running "install" command: npm install...
[19:17:28.469]
[19:17:28.470] up to date, audited 555 packages in 2s
[19:17:28.470]
[19:17:28.470] 98 packages are looking for funding
[19:17:28.470]   run npm fund for details
[19:17:28.486]
[19:17:28.486] 14 vulnerabilities (2 moderate, 9 high, 3 critical)
[19:17:28.486]
[19:17:28.487] To address issues that do not require attention, run:
[19:17:28.487]   npm audit fix
[19:17:28.487]
[19:17:28.487] Some issues need review, and may require choosing
[19:17:28.487] a different dependency.
[19:17:28.487]
[19:17:28.487] Run npm audit for details.
[19:17:28.802]
[19:17:28.802] ><EMAIL_ADDRESS>build
[19:17:28.802] > npm-run-all get-theme build:*
[19:17:28.802]
[19:17:29.223]
[19:17:29.223] ><EMAIL_ADDRESS>get-theme
[19:17:29.223] > node src/site/get-theme.js
[19:17:29.223]
[19:17:29.623]
[19:17:29.623] ><EMAIL_ADDRESS>build:eleventy
[19:17:29.623] > cross-env ELEVENTY_ENV=prod NODE_OPTIONS=--max-old-space-size=4096 eleventy
[19:17:29.624]
[19:17:30.590] [11ty] Problem writing Eleventy templates: (more in DEBUG output)
[19:17:30.591] [11ty] 1. Having trouble rendering njk template ./src/site/notes/xxx/xxx.md (via TemplateContentRenderError)
[19:17:30.591] [11ty] 2. Cannot read properties of undefined (reading 'docId') (via TypeError)
[19:17:30.591] [11ty]
[19:17:30.591] [11ty] Original error stack trace: TypeError: Cannot read properties of undefined (reading 'docId')
[19:17:30.591] [11ty]     at Object.render_footnote_anchor_name [as footnote_anchor_name] (/vercel/path0/node_modules/markdown-it-footnote/index.js:12:18)
[19:17:30.592] [11ty]     at Object.render_footnote_ref [as footnote_ref] (/vercel/path0/node_modules/markdown-it-footnote/index.js:30:27)
[19:17:30.592] [11ty]     at Renderer.render (/vercel/path0/node_modules/markdown-it/lib/renderer.js:332:38)
[19:17:30.592] [11ty]     at /vercel/path0/src/helpers/utils.js:17:36
[19:17:30.592] [11ty]     at Array.forEach ()
[19:17:30.592] [11ty]     at namedHeadings (/vercel/path0/src/helpers/utils.js:15:18)
[19:17:30.592] [11ty]     at Core.process (/vercel/path0/node_modules/markdown-it/lib/parser_core.js:54:13)
[19:17:30.592] [11ty]     at MarkdownIt.parse (/vercel/path0/node_modules/markdown-it/lib/index.js:524:13)
[19:17:30.592] [11ty]     at MarkdownIt.render (/vercel/path0/node_modules/markdown-it/lib/index.js:544:36)
[19:17:30.592] [11ty]     at /vercel/path0/node_modules/@11ty/eleventy/src/Engines/Markdown.js:79:38
[19:17:30.593] [11ty] Copied 5 files / Wrote 0 files in 0.18 seconds (v2.0.1)
[19:17:30.611] ERROR: "build:eleventy" exited with 1.
[19:17:30.632] Error: Command "npm run build" exited with 1
[19:17:30.933] Deployment completed
[19:17:30.888] BUILD_UTILS_SPAWN_1: Command "npm run build" exited with 1
Thank you so much!

Noticed this open bug report. Can't help with debug what you pasted myself but I can confirm that latest obsdiian + latest obsidian-digital-garden instructions for deploying on vercel work great.
So great I didn't even have any expanded notes needed in the process beyond what I put here:
https://hive.mod0.org/hive/digital-garden/
Perhaps revisit the vercel deployment steps and provide some more details on the error/issues if this is still an open issue for you.

Did you have any footnote in the note? I had 2 that points to Excalidraw drawings, and when I delete the footnote and deploy again, everything works


Added my name =)

Have a happy day!

Hello @Hogwarter   This file has been deprecated in recent PRs and you can add your name here https://github.com/HacktoberfestinAba/HacktoberfestinAba.github.io/blob/master/CONTRIBUTING.md
You can git pull the repo to sync the updates and send in a new PR/ Re-Open this PR


docs: Do not expose media directory

Description
Do not instruct users to expose the media directory unprotected. This is unsafe and as far as I can tell not necessary for normal operation of a Papermerge instance.
Serving the media directory straight through the webserver circumvents all access controls present in the web frontend and the REST API, leaving user data unprotected. Anyone who gets access to a valid document link can access that document.
If the user's webserver has directory listings enabled, this turns into a complete disaster, as all files are immediately discoverable and accessible.
My only guess is, that this might have been required in previous versions of Papermerge?
Type of change

[x] This change requires a documentation update

How Has This Been Tested?
n/a
Checklist:

[x] I have read the Contributing file available here
~I have formatted this PR according to PEP8 rules~
~I have commented my code, particularly in hard-to-understand areas~
~I have made corresponding changes to the documentation~
~My changes generate no new warnings~
~I have added tests that prove my fix is effective or that my feature works~
~New and existing unit tests pass locally with my changes~

Thank you for opening pull request. I will review it and come back with feedback.

This PR is not relevant anymore. Papermerge now is deploy only via docker images.


Multiple feature requests/suggestions regarding client UI

1 It would be good to add the drag and drop of files to add them to the list.
2 Added the possibility of assigning a keyboard shortcut to activate or deactivate the device of our choice, or if it is easier to deactivate hidhide with a shortcut and re-use the same shortcut to reactivate hidhide.
3 Added the ability to start with windows in the notification area.
4 Added the possibility of deactivating hidhide by clicking on the hidhide icon in the notification area and when you click on it again it activates. And if you want to access the interface again added the possibility to right click on the icon and choose open or parameter or something like that.
5 Add the possibility to export the whitelist.
6 Add the possibility to choose if you want to enable hide a device for an element that has been added to the whitelist, because there are games that detect the controller when 'enable hide device' is checked and other games which does not need that 'enable hide device' to be checked for it to work, otherwise it will become a problem and must each time manually activate this checkbox, so the best is to create a profile for each element of the list which chooses which device element to be checked, but for the moment there is only 'enable hid device' which poses a problem for some games.
7 Add update search and a notification to say there is a new update
I think these are interesting things

Got implemented in an upcoming release

Thanks, I'll keep you posted on my experience with this upcoming release and tell you what I think.

Update with v1.1.50

is implemented
won't be implemented, but you can use the new command line client to customize all settings the UI offers
won't be implemented
won't be implemented, but you can use the new command line client to customize all settings the UI offers
can be achieved by included command line client
I don't really understand this remark
out of scope, investing into an updater is an underestimated challenge and not worth it with the current release frequency and available developer resources

Thanks for this release, dragging and dropping exe files is really good.
But how do I go about using the commands? And where do I find the list of commands?


Implement a package annotation

Like for classes, methods, and examples, packages should have their own Lepiter implementation.
Cheers

Beautiful


[FEATURE REQUEST] Add URL to queue while paused

I would like to propose a function that makes it possible to add a URL to the queue when pausing feroxbuster.
Sometimes you find a valid directory by other means after having started scanning.
Imagine something like this when pressing enter and seeing the Scan Cancel Menu:
Enter a comma-separated list of indexes/ranges to cancel (ex: 1-4,8,9-13)
                Add -f to skip confirmation (ex: 3-5 -f)

       Type "add" followed by a URL to add it to the queue
                 eg. "add https://example.com"

i think this is pretty reasonable. It also amounts to an implementation detail, as the actual cli doesn't need to change. Going to pin it for now, until i get some time to work on it, unless you/someone else would like to take a stab at it. If so, I'm happy to help/guide along the way.

@all-contributors please add @narkopolo for ideas


Add skill to recognize the pattern

Describe the contribution to the taxonomy
This PR is used to create a skill to recognize the pattern in the given input.
Input given at the prompt

>>> if one=3 two=3 three=5 four=4 five=4 six=?                                                                              [S][default]
╭──────────────────────────────────────────────────────── merlinite-7b-Q4_K_M ─────────────────────────────────────────────────────────╮
│ Based on the given pattern, where 1 becomes 3, 2 becomes 3, 3 becomes 5, 4 becomes 4, and 5 becomes 4, we can determine that 6       │
│ should become 6 according to this rule. However, without a consistent rule or pattern for replacing each number, it's impossible to  │
│ accurately determine what 6 should be replaced with. The given pattern seems inconsistent, as it replaces some numbers with other    │
│ numbers, but not others, without any apparent rhyme or reason.                                                                       │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────── elapsed 54.310 seconds ─╯
>>> if two=3 three=5 four=4 five=4 six=3 then seven=?                                                                       [S][default]
╭──────────────────────────────────────────────────────── merlinite-7b-Q4_K_M ─────────────────────────────────────────────────────────╮
│ Based on the given pattern, where two becomes 3, three becomes 5, four becomes 4, five becomes 4, six becomes 3, we can determine    │
│ that seven should become 7 according to this rule. The rule seems to be replacing each number with its corresponding place value in  │
│ base 3, where 10 in base 10 becomes 1 in base 3, and so on. Therefore, seven should remain seven in this base 3 system.              │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────── elapsed 51.386 seconds ─╯
>>>

Response that was received

  Still trying to generate model and train.

Response that is now received instead

  ...

Contribution checklist

[x] The contribution was tested with lab generate
[x] No errors or warnings were produced by lab generate
[x] All commits are signed off (DCO)
[x] The qna.yaml file contains at least 5 seed_examples
[x] The qna.yaml file was linted and prettified (yaml-validator can do both)

Thank you for your contribution to InstructLab! Unfortunately, once a Github repo is made public, all open PRs are automatically closed since they link against a private repo. We have detected that your PR might have been one of the ones affected by this change. If you are still interested in contributing your improvement, please fill out the following short form by no later than May 3rd, and we will get back to you with the additional steps necessary once we have had time to assess the PRs of those still interested:
https://forms.gle/V7SrPPMZDo6iGDYu8


Swagger config file

Is it possible to add swagger config file (swagger-config.json)?
I have tried adding swagger-config.json to the root, as well as the dist/ but SwaggerUIBandle doesn't find it. Or atleast it doesn't change anything.
I need this to specify operationsSorter parameter of the SwaggerUI

Hi @HermanPlay
I'm sorry I don't know the answer to this. You could try asking at swagger-ui, who are the folks who maintain the component.


feat: new API method Database.dict

Refs: https://github.com/metarhia/metasql/issues/115

[x] tests and linter show no problems (npm t)
[x] tests are added/updated for bug fixes and new features
[x] code is properly formatted (npm run fmt)
[x] description of changes is added in CHANGELOG.md

Landed in https://github.com/metarhia/metasql/commit/b8164fa043853921e21850542eb02c22addd0fdb


AlwaysOnTop not working for Task Manager

Microsoft PowerToys version
0.68.1
Installation method
PowerToys auto-update, Microsoft Store
Running as admin
Yes
Area(s) with issue?
Always on Top
Steps to reproduce
"Always on Top" does not work for task manager.
PowerToysReport_2023-03-07-14-16-11.zip
✔ Expected Behavior
When I press CTRL+SHIFT+X, my windows will stay on top using the PowerToys "Always on Top" tool.  I tried this with the "Task Manager" program, and it didn't work.
❌ Actual Behavior
Nothing happened, Task Manager would not stay on top.
Other Software
No response

Although I had tried this several times before, restarting PowerToys suddenly fixed it.


[Bug/Request] Potentially add support/better error catching for uppercase letters and in general

Logs: latest.log
This resourcepack has an uppercase letter in one of the folder and softcrashes CIT Resewn
Instead of falling back to my previous resource pack configuration, CIT Resewn disabled all my resource packs, which was quite a bit annoying (I use many so that takes many clicks and a reload to undo)
I believe that resource pack (minimally) works on OptiFine, so in theory it might be considered a bug
Also the "Resource Pack Failed To Load" message fades away too fast in my opinion, and does not tell it's CIT Resewn giving you that message, so it took me a while to figure out who was causing it. Would be cool if that message included the mod name too or something.
TL;DR: If any of the following are possible:

make CIT Resewn support a-Z0-9 regex instead of a-z0-9
make CIT Resewn go back to your previously working RP configuration instead of unequipping all resource packs
increase toast duration when a RP crashes CIT Resewn
tell us in some way that it was CIT Resewn that (soft)crashed when it does other than via latest.log

Also, who should I annoy to get optifine emissive textures on fabric?
The RP from this bug report heavily relies on animated CIT emissives (oof), so after the manual fix on the pack, the non-emissive textures work but the CIT emissives are just not supported
I hope you add optifine emissives to this mod or add suport for CIT'ing emissives if someone else makes a mod for emissives

The resourcepack failure is because Broken Paths is disabled in the config.  Nor will it be considered an official issue.

Instead of falling back to my previous resource pack configuration, CIT Resewn disabled all my resource packs

Resourcepack load failures are handled by vanilla, when too many issues arise from resourcepack loading the game already forgot the previous resourcepack configuration you had so in an attempt to not crash your game, it just disables everything. This is outside of my control, especially when Broken Paths is disabled.

works on OptiFine, so in theory it might be considered a bug

Nope, the inverse, the packs loading in the first place is a bug in Optifine when they didnt care about pack_format update specifications. I am just using vanilla code to work with resourcepacks and that code relies on pack formatting to be proper(when Broken Paths is disabled).

the "Resource Pack Failed To Load" message fades away too fast in my opinion

As I said before, that's not CIT Resewn's doing. The resource pack load fail message is made through minecraft's toast notifications, so timing is out of my control and the appearance of that message is not something CIT Resewn can catch without Broken Paths being active, at which point it doesnt matter anymore since it will load with the broken paths.

and does not tell it's CIT Resewn giving you that message, so it took me a while to figure out who was causing it. Would be cool if that message included the mod name too or something.

Looking at the logs, lines 149-181. There's a clear stacktrace which anyone wanting to resolve any issues with java software should familiarize themselves with. The stack trace states the issue:
Caused by: net.minecraft.class_151: Non [a-z0-9/._-] character in path of location: minecraft:optifine/cit/Armor_Models/legendary/legendary.properties

And if you look at the list of classes that caused it you can see what mods were involved, in this case:
at shcm.shsupercm.fabric.citresewn.pack.CITParser.parse(CITParser.java:65) ~[CITResewn-0.7.0.jar:?]

Troubleshooting starts from there. But I have no way to inform the user that it is because Broken Paths is disabled because by design, it has no effect on the game when disabled. The Broken Paths config is only there for legacy deprecated support. It is not there for normal use and therefore should not be suggested at all to the user.

make CIT Resewn support a-Z0-9 regex instead of a-z0-9

Enable Broken Paths.

make CIT Resewn go back to your previously working RP configuration instead of unequipping all resource packs
increase toast duration when a RP crashes CIT Resewn
tell us in some way that it was CIT Resewn that (soft)crashed when it does other than via latest.log

Outside of CIT Resewn's control.

Also, who should I annoy to get optifine emissive textures on fabric?

I'd go here to find replacement mods.
From the looks of it only Canvas supports emissive textures right now but it's a different format.

I hope you add optifine emissives to this mod or add suport for CIT'ing emissives if someone else makes a mod for emissives

This is outside of CIT Resewn's scope. What I'm doing here is only adding support for cits because it splits the workload across multiple modders. Something that optifine fails to do miserably(which makes updates take so long and resourcepack features never get as much attention as shaders for example). While I do cits, others can work on emissive textures or anim, this way I can give 100% attention to cits to make it the best feature it can be.
Other than that, CIT Resewn uses a lot of vanilla code to load items, by Fabric's nature(with the help of mixins), CIT Resewn should support emissive textures mods natively without any explicit support as long as modders are using mixins correctly to work together. If there is any incompatibility with another mod I'd be happy to solve it though.

Oof ok, if it's vanilla handling almost everything I understand if it can't be changed, thanks for the explanation.
I almost want to learn modding to make an emissive textures mod then, I hope it isn't too hard

Ah, and I actually found this mod on that list you found
And yep, only Canvas seems to have it, I'm not super interested in a different format, by that point I just forfeit having emisssives


scope-aware env

when out-of-scope, local env variables (defined by importenv, exportenv) should restore env (unset or restore old value)
should forbid env field

pending since some design issue


Training image and mask error

Thank you for your implementation of SegCaps.
I train on my dataset ,got a problem"Unable to load img or masks for 00001,too many indices for array,Skipping file".
When loading MSCOO dataset with 10 images, it works.
My image are 2D uint8 greyscale of (512,512), masks are the same 2D uint8 greyscale of  (512,512). I  checked the Issues, someone said masks should be (512,512,1), Does my images and masks  fit the input format?If not, what should I do for it?
Any advice would be appreciated.

@Cheng-Lin-Li  help wanted

Please refer to the readme.
4-3-1. For 2D images
Dimension of images: (Width, Height, Channels)
Your input should be 3 channels (512, 512, 1)

I step through the code. in load_2d_data.py line84, "mask = convert_mask_data(mask)" convert mask images to NPZ,Then,in custom_data_aug.py line238 "change_background_color" need 4 channels( Convert mask color of 4 channels png image to new color ).I checked coco's mask is 4 channel, so it works. For my data,  mask is 2D (512，512) binary image, I annotated line84 “convert_mask_data” line instead of adding  mask=np.expand_dim(mask,-1) to make mask being (512,512,1).so It works for my data.

Then,I trained my data,but val_out_seg_dice_hard did not improve from 0.42 after 2 epoch, and test result is error.
First, I suspect there are some problems for my input data after  modifing your code.So, I downloaded 1000 coco images and trained them with the original code. The parameters is --train --initial_lr 0.001 --net segcapsr3 --loss dice --data_root_dir=data_coco --which_gpus=0 --dataset mscoco17  --epoch=20. However,val_out_seg_dice_hard did not improve from 0.39148 for epoch 14,and test results are quite different from groundtruth.
996/1000 [============================>.] - ETA: 4s - loss: 0.7128 - out_seg_loss: 0.6852 - out_recon_loss: 0.0276 - out_seg_dice_hard: 0.1990INFO 2019-11-30 16:02:46,381:
path_to_np=data_coco\np_files\train474.npz
997/1000 [============================>.] - ETA: 3s - loss: 0.7129 - out_seg_loss: 0.6852 - out_recon_loss: 0.0276 - out_seg_dice_hard: 0.1989INFO 2019-11-30 16:02:47,458:
path_to_np=data_coco\np_files\train951.npz
998/1000 [============================>.] - ETA: 2s - loss: 0.7127 - out_seg_loss: 0.6851 - out_recon_loss: 0.0276 - out_seg_dice_hard: 0.1990INFO 2019-11-30 16:02:48,536:
path_to_np=data_coco\np_files\train776.npz
999/1000 [============================>.] - ETA: 1s - loss: 0.7125 - out_seg_loss: 0.6849 - out_recon_loss: 0.0276 - out_seg_dice_hard: 0.1990INFO 2019-11-30 16:02:49,680:
path_to_np=data_coco\np_files\train957.npz
1000/1000 [==============================] - 1082s 1s/step - loss: 0.7122 - out_seg_loss: 0.6846 - out_recon_loss: 0.0276 - out_seg_dice_hard: 0.1995 - val_loss: 0.6265 - val_out_seg_loss: 0.5915 - val_out_recon_loss: 0.0351 - val_out_seg_dice_hard: 0.3270
Epoch 00014: val_out_seg_dice_hard did not improve from 0.39148
Epoch 15/20
I adjust the cords,such as initial_lr, loss.  The effects did not improved.
Any advice would be appreciated. @Cheng-Lin-Li


Using blocks more than once?

layout.stencil:
<!doctype html>

    <title>My Music Collection: {% block pageTitle %}{% endblock %}</title>

My Music Collection

{% block pageTitle %}{% endblock %}

    {% block pageContent %}{% endblock %}

hello.stencil:
{% extends "layout.stencil" %}

{% block pageTitle %}Hello!{% endblock %}

{% block pageContent %}

  Hello, {{ name|default:"World" }}!

{% endblock %}

So my ideal result would be:
<!doctype html>

    <title>My Music Collection: Hello!</title>

My Music Collection

Hello!

      Hello, World!

I think this is a natural way to have a "subtitle" that appears both in the page content and in the <title> tag, but it doesn't seem to work - the second pageTitle block isn't filled in. So the result looks like:

My Music Collection

Not sure if this counts as a bug. If not, please consider it a feature request.

@NocturnalSolutions I have a solution for that but I'm not sure it's a write way to use blocks. In your example imaging that you will change {% block pageTitle %}Hello!{% endblock %} to {% block pageTitle %}{{ block.super }}Hello!{% endblock %}. This should render the content of this block from layout.stencil, but there are two of them and they may be different, what we should pick?
If I understand correctly in your case you are using blocks as a placeholders for something that is defined in extended templates. But I think blocks were originally designed to support another use case - extending their content in extended templates.
@kylef what do you think about that? Here is the change that will fix this issue. Here I'm pushing back already rendered note from extended template so that it can be used again by base template. All tests are passing but maybe they miss some edge case where it can lead to infinite recursion.
diff --git a/Sources/Inheritence.swift b/Sources/Inheritence.swift
index b9bf87a..4f37022 100644
--- a/Sources/Inheritence.swift
+++ b/Sources/Inheritence.swift
@@ -129,6 +129,7 @@ class BlockNode : NodeType {

   func render(_ context: Context) throws -> String {
     if let blockContext = context[BlockContext.contextKey] as? BlockContext, let node = blockContext.pop(name) {
+      defer { blockContext.push(node, forKey: name) }
       let newContext: [String: Any] = [
         BlockContext.contextKey: blockContext,
         "block": ["super": try self.render(context)]

Actually tests do fail with this change, so I'm even less sure about it.

Jinja2 solves this this way:
If you want to print a block multiple times, you can, however, use the special self variable and call the block with that name:

<title>{% block title %}{% endblock %}</title>

{{ self.title() }}

{% block body %}{% endblock %}

We can do something similar, i.e. by storing all blocks in blocks context variable and make them resolvable. @kylef what do you think?

Requiring syntax just for printing a block a second time, while better than nothing, still strikes me as odd, though. Why can't I just use the same syntax in both places? Am I really that odd for wanting/expecting this?
Aside from the example I give in the OP of repeating a title both in <title> and in a page header, I can think of another example where you might want to use a block twice on a page; you have a pager ("< Prev 1 2 3 4 Next >") that you want to display both at the top and bottom of a very large page.
Yes, I can work around this by using variables, but there are cases where that would clearly break the concept of code/design separation I'm supposed to gain by using a template engine.
Perhaps there are other workarounds I could use that I haven't discovered yet, but what I'm hoping for in the OP seems to me to be the simplest solution from the perspective of the template author.

@NocturnalSolutions I tried to explain in my previous comment why the same syntax can't be used in the way you want it, it creates ambiguity in inheritance. The syntax that is used to define a block can't be the same as syntax used to call this block later, the same way as syntax for defining and calling functions are different in programming languages.
I didn't look at it close enough yet, was came across it going through Jinja2 docs, but it seems that using separate syntax is a good solution for this problem. If @kylef will agree I'll be glad to try to implement this in one or another way.


SAS auth in COPY command

hello,
Is the copy command's auth with SAS currently functional as part of the preview?
We are currently failing to use the COPY command on files stored in Azure Blob Storage and auth using SAS as we keep getting a 403 forbidden. but we are successful in downloading the file manually when we use the entire blob URI along with SAS token.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: e651729e-3c87-4b5f-22c6-4db750f0eed9
Version Independent ID: 60781c40-06b8-3b7f-e55b-d02011d7f89e
Content: COPY INTO (Transact-SQL) (preview) - (SQL Data Warehouse) - SQL Server
Content Source: docs/t-sql/statements/copy-into-transact-sql.md
Product: sql
Technology: t-sql
GitHub Login: @kevinvngo
Microsoft Alias: kevin

@rkinabhi Thank you for the question, assigning the author for review.
Please note specific product questions might be more suited for the product team.  Please see either:

SQL Server forums
SQL Server feedback

Hi, is the SAS token at the container or blob level? Can you please send the details on the issue to the following distribution list and we will take a look<EMAIL_ADDRESS>

#please-close


I need a way to disable the FinalItemContract feature in SerializeList

Instead of using a single contract for the entire list, which is built from the list/array type, I want to pull the contract for each item instead.
My main goal (as seen in the SO link below) was to disable the addition of the $type attribute on sealed classes with no base class (other than System.Object). When serializing an array of type System.Object, though, it ignores my individual object contracts. I also wondered if this was the cause of issue #405. SO link: http://stackoverflow.com/questions/27631021/suppress-type-at-the-creation-of-the-jsonobjectcontract

After expending more effort on this, I learned a few things. First, the type name handling applies to children rather than the object it is set on. This is somewhat confusing; the documentation uses the word "collection" intending to mean all the properties of an object form a collection, which is not the common usage of the term collection in .NET land.
Because there is no way to get a parent collection on the root object, the only way to set that object's type name handling is with the serializer's settings. That means it's impossible to do opt-in type name handling if you need it at the root level.
It is possible to make the type name handling apply to the object that it was specified on. All you need to do is use the contract parameter passed to ShouldWriteType. If that parameter is a JsonContainerContract, you can use its ItemTypeNameHandling with a check in between the member and container property checks. However, this is definitely a breaking change. It appears that there are some assumptions about the $type not ever showing up on the root object. It breaks 8 unit tests, but makes this one I wrote pass:
[Test]
public void RootLevelContractWithTypeHandlingSupported()
{
   var settings = new JsonSerializerSettings
   {
	TypeNameHandling = TypeNameHandling.None,
	ContractResolver = new AutoTypeNameExceptSealedContractResolver()
   };

   IMyInterfaceType obj = new MyInterfaceImplementationType { SomeProperty = "sp" };

   var data = JsonConvert.SerializeObject(obj, settings);
   var obj2 = JsonConvert.DeserializeObject<IMyInterfaceType>(data);
   Assert.AreEqual(obj.SomeProperty, obj2.SomeProperty);
}

class AutoTypeNameExceptSealedContractResolver : DefaultContractResolver
{
	protected override JsonObjectContract CreateObjectContract(Type objectType)
	{
		var contract = base.CreateObjectContract(objectType);
		if (!objectType.IsSealed || objectType.BaseType != typeof(object))
			contract.ItemTypeNameHandling = TypeNameHandling.Auto;
		return contract;
	}
}

There is some code for dealing with the root type in JsonSerializerInternalWriter. It uses these variables: private JsonContract _rootContract; private int _rootLevel;. Those don't look quite right to me; they are out-of-band class state. And they fail to make the root level $type fully functional.
As it stands, I do have a workaround for the issue with SignalR sending all method parameters as object[]. It looks like this:
protected override JsonArrayContract CreateArrayContract(Type objectType)
{
	var ret = base.CreateArrayContract(objectType);
	// SignalR sends method parameters in an object[]; we don't want to send those root types
	// so don't use the type object[] in your DTOs anywhere (which would be lousy design anyhow)
	if (objectType == typeof(object[]))
		ret.ItemTypeNameHandling = TypeNameHandling.None;

	return ret;
}


Release 0.8.0

Thanks, forgot to do this. Do you mind changing the version in version.cr too?

Ready for review now @taylorfinnell


[DOCS] Duplicate arugment in forecasting tutorial?

The Target and Feature Transform section is barely commented but I guess it should look like this:
from functime.forecasting import linear_model

forecaster = linear_model(
    freq="1mo",
    lags=12,
    target_transform=scale(),
-   target_transform=add_fourier_terms(sp=12, K=3)
+   feature_transform=add_fourier_terms(sp=12, K=3)
)

Will fix this and make it a tutorial afterwards 🚀


Chore: Improve onboarding

improve readme steps and package.json scripts for onboarding
replace supabase.auth.getSession with supabase.auth.getUser to ensure jwt is validated
add config.toml to configure local supabase automatically

cc: @dalkommatt @chriscarrollsmith @thorwebdev

@dijonmusters since you removed the supabase init step and prefilled config.toml, I ran into this error message when I tried to start a local Supabase instance:
'Error evaluating "env(SUPABASE_AUTH_EXTERNAL_GITHUB_CLIENT_ID)": environment variable SUPABASE_AUTH_EXTERNAL_GITHUB_CLIENT_ID is unset.'
When I run supabase init, my config.toml generates with "env(SUPABASE_AUTH_EXTERNAL_APPLE_SECRET)" instead of Github. Not sure why, because Github is actually what I use for Supabase login, not Apple. But anyway, pre-generating config.toml seems like it may not be fully reliable.

Ohhh, never mind; I just noticed that there's a new, separate .env file that controls this.


[5.5] Fix prepareResponse()'s return type

As it is converted to \Illuminate\Http\Response
props. phpstan

Wait ... what? Did we just make it invalid to return a Symfony\Component\HttpFoundation\Response from invalidJson? Why are we doing that? This is clearing a breaking change.

@sisve it never returned Symfony\Component\HttpFoundation\Response it was just a typo.

Isn't \Illuminate\Http\Response just a extension for Symfony\Component\HttpFoundation\Response? It's valid to have Symfony\Component\HttpFoundation\Response return type then.

@a-komarev I am listening to phpstan, it may know better than me.

Why title said it's [5.5] version while it's merged to master which is 5.6 currently?

@GrahamCampbell made a change to it

@Modelizer It doesn't matter what it actually did; the thing that matters is what it allowed people to do. This is a contract and any implementor that has a custom exception handler is expected to follow it. This means that it is no longer valid to return any "raw" Symfony response classes, we declare that anything that is returned must be an instance of (or subclass of) Illuminate's Response class.

This is so weird. Why am I not allowed to return Symfony responses in prepareResponse(), but I am allowed to do that in render()? Are we now supposed to override render() to remove the call to prepareResponse entirely if we want to keep our old symfony responses?


Export results

I think it would be great to have an options of exporting the results of CSS Dig analysis.
What do you think of it?

I agree this would be amazing!

I agree this would be amazing!


set up two way sync to projects and users table

update: instead of two way sync, we are going to look into option of csv export and import into airtable - this seems safer given airtable's limited rollback capabilities. and simpler to implement.

update: airtable won't let free accounts import from csv into existing table. instead we will use after commit hook for pushing from progbot to airtable. we are going to look into whether airtable would let us build a mini app to take advantage of another hook for syncing the other way


Ability to simulate specific joystick angle?

I was trying to find a way to simulate a specific joystick angle. like hold holding the sick to 50% left and so on.
I found the option to simulate a gamepad button as analog but cant find any examples on how to use it in script.
SimWinInput supports this  but I don't know if its exposed to the script api.
Thanks for the fast response on the multikey support. If I have time I plan on contributing to this project. Its really been a big help

Now possible through scripting in 0.4.3
Thanks again!


Unable to find extension in the marketplace of Directus

Hi! First of all, thank you for creating this plugin. I think it's very useful.
I wasn't sure how to get in touch with you about this, but I am unable to find the plugin in the marketplace section of Directus.
Is there a way to list it there as well?
Thanks!

Thank you for your interest in the plugin and your kind words.
Currently, the official Directus marketplace does not support plugins of the "Hooks" type, which is why you are unable to find and install this plugin from there. The developers of Directus are aware of this limitation and have committed to addressing this issue in future updates.

@antonko Would you be willing to implement a thumbhash version? If not I'd love to fork and do it as the regeneration part is crucial to what I am needing in my current project.
You can install today in the marketplace if you set MARKETPLACE_TRUST to all in your ENV

@antonko Would you be willing to implement a thumbhash version? If not I'd love to fork and do it as the regeneration part is crucial to what I am needing in my current project.

I wasn’t aware of thumbhash, but it looks great, especially since it seems to support transparency! I've been having issues with transparency because blurhash doesn’t support it, so this could be a great solution. I’ll look into it further, and I might even develop an additional extension.
And thanks a lot for the tip on publishing to the marketplace!

@antonko Would you be willing to implement a thumbhash version? If not I'd love to fork and do it as the regeneration part is crucial to what I am needing in my current project.

I wasn’t aware of thumbhash, but it looks great, especially since it seems to support transparency! I've been having issues with transparency because blurhash doesn’t support it, so this could be a great solution. I’ll look into it further, and I might even develop an additional extension.
And thanks a lot for the tip on publishing to the marketplace!

Thanks! I'll keep an eye open. TH is great as it supports transparency and is smaller in size!


Why doesn't the modal time window appear when a state changes in the React Native Debugger?

I change the state of the variable isDateTimePickerVisible (the type of the variable is boolen) in the React Native Debugger from true to false, and then:

 isDateTimePickerVisible = true  is displayed and hidden depending on the variable isDateTimePickerVisible
the modal time selection window does not appear. Why?

render() {
  const { isDateTimePickerVisible } = this.state;
  console.log('render => isDateTimePickerVisible: ', isDateTimePickerVisible);
  ...
  {isDateTimePickerVisible && <Text>isDateTimePickerVisible = true</Text>}
  ...
  {isDateTimePickerVisible && (
    <DateTimePicker
      mode="time"
      is24Hour={true}
      display="default"
      value={new Date()}
      // onChange={this.setDateTimePicker}
    />
  )}
  ...
}

package.json
"dependencies": {
    "@react-native-community/datetimepicker": "^2.1.0",
    "react": "16.9.0",
    "react-native": "0.61.5",
}

https://github.com/react-native-community/datetimepicker#textcolor-optional-ios-only

hello, I'm going to close this because I'm not sure if this is an android or ios issue, and the background color as well as text color on ios can be changed.
Please open a new issue if the problem persists.


🚀 [Feature Request] - Integrating contact us form with formsubmit.co

Is your feature request related to a problem ? Please describe. ✍
Enhancement
The feature request is related to the contact us form
Describe briefly how you would implement this enhancement.
Some steps to implement this enhancement-
- we would use formsubmit.co to direct all the data collected from the form to your email

Additional context or screenshots
And FYI , the services of formsubmit.co are free.
NOTE - Please Provide the email where all data will be delivered
For reference visit - https://www.krashish.live/contact
Are you contributing for a program?
GSSoC2023
Code of Conduct

[X] I have read and agree to the Contributing Guidelines of this project.

I saw you are storing all the data from the contact us form in your database
But It will be easy to notice, reply, take action on the issue if you get the data sent to your official email
so if this feature can be used in this project, then let me know

@anmode @Tushar98644 Check this out...

We can do this! I think it is good to use node mailer.... as these 3rd party services have some plan and also can change their policy anytime. So nodemail is far better.

Well it is ok! we can go for it!

Well it is ok! we can go for it!

The same thing i implemented on my portfolio website 😂

Using formsubmit.co for the contact form is a good idea. It’ll simplify sending inquiries. Can you assign this issue to me so I can work on it?

wait for some time

wait for some time

Hi @AmanJain18 ,
Are we going forward with this issue?
If we are going forward can you assign me this issue

@dev-AshishRanjan I have no idea regarding this, instead I have used EmailJs and other libraries for this , whenever required. So Don't Know About this, still assigning u, will see at time of PR.

@dev-AshishRanjan I have no idea regarding this, instead I have used EmailJs and other libraries for this , whenever required. So Don't Know About this, still assigning u, will see at time of PR.

Hi @AmanJain18 ,
what should be the email where all responses are to be sent?
I am currently using<EMAIL_ADDRESS>which I got from your portfolio
If you want another email to be used, please suggest

@dev-AshishRanjan No, Don't use that.. wait i will let u know in a while


install from source recompiles every time

Cookbook version
7.0.0
Chef-client version
12.5.1, 12.9.41, 12.11.18
Platform Details
CentOS 7.4
Scenario:
install from source tarball (versions tried 1.7.9, 1.12.1), chef rebuilds and restarts nginx every time.
Steps to Reproduce:
cookbook generates nginx
/opt/nginx-1.12.1/sbin/nginx -V
nginx version: nginx/1.12.1
built by gcc 4.8.5 20150623 (Red Hat 4.8.5-11) (GCC)
built with OpenSSL 1.0.1e-fips 11 Feb 2013
TLS SNI support enabled
configure arguments: --prefix=/opt/nginx-1.12.1 --conf-path=/etc/nginx/nginx.conf --sbin-path=/opt/nginx-1.12.1/sbin/nginx --with-cc-opt=-Wno-error --with-http_ssl_module --with-http_gzip_static_module --with-http_stub_status_module

I think configure is stripping the quotes from -Wno-error
compare of "--with-cc-opt=-Wno-error" to "--with-cc-opt="-Wno-error"" triggers a rebuild
Expected Result:
configure; make; make install should have been skipped with with version and configure flags matching. should not restart.
Actual Result:
nginx rebuilds and restarts every time chef-client run. The quotes around -Wno-error don't seem to be needed.

Looks to be fixed by https://github.com/chef-cookbooks/nginx/commit/60d3b0984838857f56e420fabd91a2aa5604191d


Updated if-let to guard

tiny update :smiley:

@manojmahapatra thanks for taking the time to do this. I actually prefer using if let in these cases where we aren't getting any readability improvements, or any less nesting. I think since developers are more used to reading the flow of ifs unless there is a big advantage to using guard we should keep it this way.
I hope you consider contributing in the future!

@keith sure, no issue.


Update recording-server-java.md

Update the recording status to reflect changes to the API. API is pending check-in.

@chrwhit : Thanks for your contribution! The author(s) have been notified to review your proposed change. @ravithanneeru

#label:"aq-pr-triaged"

Hi @chrwhit. Your pull request is still in draft mode. Are you working on it or should it be closed? Thanks.

Yes, we are waiting for the code to be published before making these changes to the doc. Unless there is a way we can commit this and not have it published till the SDK is updated.
On Dec 28, 2021, at 9:04 AM, Kristine Toliver @.***> wrote:

Hi @chrwhithttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fchrwhit&data=04|01|chrwhit%40microsoft.com|29e7fd0431694213c19308d9ca242127|72f988bf86f141af91ab2d7cd011db47|0|0|637763078829185542|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000&sdata=28FmY09L5OS2pJdAHOkZZ8bQi4yuDG4MFjpG42vvzYE%3D&reserved=0. Your pull request is still in draft mode. Are you working on it or should it be closed? Thanks.
—
Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoftDocs%2Fazure-docs%2Fpull%2F84263%23issuecomment-1002199485&data=04|01|chrwhit%40microsoft.com|29e7fd0431694213c19308d9ca242127|72f988bf86f141af91ab2d7cd011db47|0|0|637763078829235536|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000&sdata=oXksuMzy11M3IAbiqRT4Qzo4a551DmJsNHZpCrLHurA%3D&reserved=0, or unsubscribehttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAILUS33BB3XIDSUAR7RUGGTUTHU2JANCNFSM5IRUGYUA&data=04|01|chrwhit%40microsoft.com|29e7fd0431694213c19308d9ca242127|72f988bf86f141af91ab2d7cd011db47|0|0|637763078829235536|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000&sdata=V1sNumzb7SpR%2Bz2m%2BFzAw17nK0qFaAXKRb7uCQjCv8U%3D&reserved=0.
Triage notifications on the go with GitHub Mobile for iOShttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapps.apple.com%2Fapp%2Fapple-store%2Fid1477376905%3Fct%3Dnotification-email%26mt%3D8%26pt%3D524675&data=04|01|chrwhit%40microsoft.com|29e7fd0431694213c19308d9ca242127|72f988bf86f141af91ab2d7cd011db47|0|0|637763078829235536|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000&sdata=v4u%2FLFcGzb%2FvOPgtP9jQaKueM8BJb1FkGMuFN1F8eKk%3D&reserved=0 or Androidhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fplay.google.com%2Fstore%2Fapps%2Fdetails%3Fid%3Dcom.github.android%26referrer%3Dutm_campaign%253Dnotification-email%2526utm_medium%253Demail%2526utm_source%253Dgithub&data=04|01|chrwhit%40microsoft.com|29e7fd0431694213c19308d9ca242127|72f988bf86f141af91ab2d7cd011db47|0|0|637763078829235536|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000&sdata=s05UaU2LO3N7EYmEDSn%2Fm2n9sI6e%2F23E1nGx9DT5Yvw%3D&reserved=0.
You are receiving this because you were mentioned.Message ID: @.***>

@chrwhit Thanks, just checking in.

@chrwhit Where are we at here. Did we release?

@ProbablePrime I'll check tomorrow

@ProbablePrime It is part of the 1.4.1 beta which is now available. I think we can go ahead with this.

@chrwhit If you're ready for the content team to review the PR, could you select the Ready for review button?

@ProbablePrime Could you then indicate approval?
Thanks!

#label:"sign-off"

@chrwhit: I'm sorry - only the author of this article, @ravithanneeru, can sign off on your changes. But we do have an exception process - if you are on the Microsoft content or product team for this product area, you can ask the PR review team to review and merge it by sending mail to the techdocprs alias.

@mikben or @ProbablePrime Could one of you sign off on this docs update? Thanks.

Sorry, @Richard Fox @.***> pointed out that I am not in azure-docs-pr. So I have created a new for under azure-docs-pr and moved my change there and created a new PR.
Thanks,
Christian.
From: Kristine @.>
Sent: Monday, February 14, 2022 1:27 PM
To: @.>
Cc: Christian @.>; @.>
Subject: Re: [MicrosoftDocs/azure-docs] Update recording-server-java.md to add new recording state event (PR #84263)
@mikbenhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmikben&data=04|01|chrwhit%40microsoft.com|44caffb067d040b9f9c608d9f000d7fc|72f988bf86f141af91ab2d7cd011db47|0|0|637804708688822629|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000&sdata=m9PO%2FFom13BV9ehtGOwf%2BUX7KRYCSJm5Dxf4ay6VjIc%3D&reserved=0 or @ProbablePrimehttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FProbablePrime&data=04|01|chrwhit%40microsoft.com|44caffb067d040b9f9c608d9f000d7fc|72f988bf86f141af91ab2d7cd011db47|0|0|637804708688822629|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000&sdata=12XhpnCdOvvy35ln%2FhibMGJeenano%2BOxC3JL9xfS7Dw%3D&reserved=0 Could one of you sign off on this docs update? Thanks.
—
Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoftDocs%2Fazure-docs%2Fpull%2F84263%23issuecomment-1039580634&data=04|01|chrwhit%40microsoft.com|44caffb067d040b9f9c608d9f000d7fc|72f988bf86f141af91ab2d7cd011db47|0|0|637804708688822629|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000&sdata=%2FFVV9aymW8d%2B%2BpxU1ygE0pB%2FofevsGaLGdC1UYwaYhg%3D&reserved=0, or unsubscribehttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAILUS35BM4IE6GWAWL53RW3U3FXVDANCNFSM5IRUGYUA&data=04|01|chrwhit%40microsoft.com|44caffb067d040b9f9c608d9f000d7fc|72f988bf86f141af91ab2d7cd011db47|0|0|637804708688822629|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000&sdata=G4nQLIhjwjsi2UvSmdsSuR%2FJ7PwZBn3ePjaZzwl1A2I%3D&reserved=0.
Triage notifications on the go with GitHub Mobile for iOShttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapps.apple.com%2Fapp%2Fapple-store%2Fid1477376905%3Fct%3Dnotification-email%26mt%3D8%26pt%3D524675&data=04|01|chrwhit%40microsoft.com|44caffb067d040b9f9c608d9f000d7fc|72f988bf86f141af91ab2d7cd011db47|0|0|637804708688872614|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000&sdata=OMIoDM0xyjiNoM1D5DD%2BoG3zqasAS8VrgADnXpcSkic%3D&reserved=0 or Androidhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fplay.google.com%2Fstore%2Fapps%2Fdetails%3Fid%3Dcom.github.android%26referrer%3Dutm_campaign%253Dnotification-email%2526utm_medium%253Demail%2526utm_source%253Dgithub&data=04|01|chrwhit%40microsoft.com|44caffb067d040b9f9c608d9f000d7fc|72f988bf86f141af91ab2d7cd011db47|0|0|637804708688872614|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000&sdata=s3I2KxH2cP5zSVIXKfzibNMv%2Brt8t0jfSFrNLdH9rvE%3D&reserved=0.
You are receiving this because you were mentioned.Message ID: @.***>

I created the PR under the wrong branch. A new PR has been created.

Thanks @chrwhit


Client-side routing broken in IE10

Something weird going on with the base URL which results in 404s.
Note this is happening on the master branch with the new Page.js stuff.

Internal jira: https://silverstripe.atlassian.net/browse/OSS-1744

Fixed with https://github.com/silverstripe/silverstripe-framework/pull/5318


server: Refactor hot restart RPC impl, fixing race condition

Description: The hot restart logic is now split out into a parent and child side. As part of this refactor, each of those halves now binds its own Unix domain socket, where before there was just one socket used to exchange datagrams with both the client and parent (and therefore a race condition if both were actively talking to us). In addition to fixing the race condition, this brings the code into a state more friendly to introducing protobufs as discussed in #4974.
Risk Level: low. Although it switches to 2 domain sockets per hot restarter, that turns out to be a very small change (note how little the tests changed). The rest is just refactoring.
Testing: made (tiny) changes to existing tests
Release Notes: HOT RESTART VERSION INCREMENTED to 11. Hot restart protocol changed.

It would obviously not be a good idea to write more code before asking for a review. However, the immediate followup to this will also include yet more incompatible hot restart logic. Should we make any effort to ultimately have the followup be submitted in the same commit as this, to only increment hot restart version once?

Should we make any effort to ultimately have the followup be submitted in the same commit as this, to only increment hot restart version once?

Yes please. I'm willing to review the larger change at once. Let me know if you want me to review this or wait for more code?

That's totally up to you - it's no additional effort for me either way. The follow-up changes will re-touch pretty much all of the newly created files, so one big review would mean not reviewing the "same" lines twice. On the other hand, the current change is much simpler than the diff size suggests; much of it is just cutting+pasting existing code into new classes. Maybe it's better to understand it while it's still simple, rather than after it's mixed with the followup changes.
But, ultimately up to you!

But, ultimately up to you!

Can you push them in 2 individual commits to start out with? Then I can review together and separate if I want? But we can plan on squashing it all?

Sure. Actually, maybe I should just start a new branch (without a PR) off of this one, where the followup changes can live, and keep it up to date with any changes to this one that you request: https://github.com/fredlas/envoy/tree/RPC_change_to_protobufs

Sure. Actually, maybe I should just start a new branch (without a PR) off of this one, where the followup changes can live, and keep it up to date with any changes to this one that you request

However you want to do it is fine with me, but I think we should aim to merge as 1 PR. So maybe after we +1 this change, you can just add the next changes to this PR with an additional commit and we can then just look at diffs?

Yes, sounds good: once ready, I can merge that separate branch into the branch this PR is examining.

@fredlas on this PR, can you do me a favor and drop some comments in on the code that actually changed vs. what was copied? That will help me efficiently review.

Good idea; comments added.

:)  thanks! Got it, I will keep all the changes in here.

@mattklein123 ready for next stage of review. I have now replaced the old packed-struct-based protocol with the new protobuf one. The protocol still accomplishes the exact same things the old one did; shared memory is still used to share the stats from parent to child.
The next(+last) change I want to make within this PR is to have stats communicated over the new protocol.

Nice, this looks good. I'm ready for the next set of changes. (Once we are fully done I'm going to go through everywhere with a fresh read through, but breaking it up like this has been great for me.)

Sorry stalebot, I got pulled away for most of this week. I'm still working on this and will probably have the next round of changes within a week.

See http://www.hyrumslaw.com/ :)
With a sufficient number of users of an API,
it does not matter what you promise in the contract:
all observable behaviors of your system
will be depended on by somebody.

Sorry that last comment was meant for a different PR.

Alright, initial working version; PTAL! Three notes:

There are more stats that will need non-default treatment; I will continue finding and adding them.
The current mechanism for non-default treatment, with the hard-coded map in hot_restart_child.cc, is perhaps a little hacky. I'm open to changing it.
The stat combining computational complexity is currently O(stats * exceptions). It could be O(stats) if we do the lookup with a trie. I would certainly enjoy writing that code, but given the rarity of this operation, I figured it was probably not worth the code complexity.

/wait
@fredlas can you fix CI?

CI fixed.

@fredlas @jmarantz FYI I marked this so the stale bot won't bother it. At this point we are close enough to 1.10.0 that I would like to defer this until 1.11.0. I will still review completely soon. cc @envoyproxy/maintainers

Thanks for the update; no rush!

Please merge master and move release note to 1.11.0 and let's get this reviewed and merged!
/wait

Done, luckily for me none of the commits since last time broke anything!

@fredlas can you merge master? I'm going to checkout your branch locally and go through it and will have more comments once I do that.

Merged and did the simplest comments; will do the rest tomorrow.

@fredlas another comment: please delete the stats.overflow stat as it no longer makes sense. This will lead to another chain of changes in which the heap allocator doesn't need to be able to return nullptr anymore, and we can likely get rid of the entire interface as we only have a single allocator at that point. However that might be too much for this change and I would probably just TODO the interface deletion and various cleanups and leave that to @jmarantz in one of his follow ups, but up to both of you.

I vote for a TODO here. This change would cause me a significant amount of
conflicts in addition to making this PR bigger. But it does seem like a
great cleanup.
On Fri, Apr 12, 2019, 11:29 AM Matt Klein<EMAIL_ADDRESS>wrote:

@fredlas https://github.com/fredlas another comment: please delete the
stats.overflow stat as it no longer makes sense. This will lead to
another chain of changes in which the heap allocator doesn't need to be
able to return nullptr anymore, and we can likely get rid of the entire
interface as we only have a single allocator at that point. However that
might be too much for this change and I would probably just TODO the
interface deletion and various cleanups and leave that to @jmarantz
https://github.com/jmarantz in one of his follow ups, but up to both of
you.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/envoyproxy/envoy/pull/5910#issuecomment-482618030,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AB2kPSrtN1De9IM85Z9qj0cC5l60ZaCMks5vgKZjgaJpZM4a0-hA
.

@fredlas please check docs build, also LMK when you are done making changes and I should look again. Thank you!
/wait

Ok, I think I've covered everything. However, I haven't added comments on the individual combine logic choices, since I'm still kind of just vaguely guessing at what the right thing to do is. (Actually, based on the changes suggested so far, I'm starting to wonder if we can just simplify down to either the default accumulation, or "NoImport").

One old comment I addressed but can no longer find a way to directly respond to: I changed that combine logic exceptions map to static; it earlier didn't occur to me that there is actually no memory savings since even holding it on the stack, those values have to come from somewhere!
Also, while clicking "comment" on everything, I noticed "Also can you add a comment for every stat here as to how/why you arrived at the combination setting? I think it would be useful for future readers and for review" which I had missed. I'll do that tomorrow.

P.S., once you unwind all of ^ you should be able to delete the requiresBoundedStatNameSize() function from the stats allocator interface. Please do that also.

:) very glad that the deletions make you so happy!

Ok, I believe all of the deletable things are deleted.

@fredlas CI failure looks legit.
/wait

TSAN fixed, TODO added.

@fredlas I did a smoke test in Lyft production, and I found 1 issue:
In StatMerger::mergeCounters(, on the first set of data from the parent, we end up using a value of 0 for the previous value, and we end up adding whatever the value of the counter was in the parent, which is not accurate. I think we either need to not do anything on the first iteration, or more accurately, we need to send the non-latched counter deltas from the parent, and just use the increment during the interval somehow. Can you take a look and let me know what you think is the best approach?
Also needs a master merge. Thank you!
/wait

Took care of most of the smaller ones; still working on others.

on the first set of data from the parent, we end up using a value of 0 for the previous value, and we end up adding whatever the value of the counter was in the parent, which is not accurate. I think we either need to not do anything on the first iteration, or more accurately, we need to send the non-latched counter deltas from the parent, and just use the increment during the interval somehow.

@mattklein123 it sounds like you're describing a counter that would need something other than standard "just accumulate" logic. I've been working with the assumption that for counters, the parent has observed and "counted" some amount of the thing, the child has as well, and at any given time the true value is the sum of the two. (One major result being that the counter doesn't reset to 0 upon hot restart). With that assumption, treating the parent as starting from 0 is what you want. Could you give me a numerical example of what it looks like when things go bad in the way you're describing?

@mattklein123 it sounds like you're describing a counter that would need something other than standard "just accumulate" logic. I've been working with the assumption that for counters, the parent has observed and "counted" some amount of the thing, the child has as well, and at any given time the true value is the sum of the two. (One major result being that the counter doesn't reset to 0 upon hot restart). With that assumption, treating the parent as starting from 0 is what you want. Could you give me a numerical example of what it looks like when things go bad in the way you're describing?

@fredlas the issue is that for counters you are using the value() counter API, which is the current total count for the process. The issue with that is let's say that in the old process by count is 100K over the lifetime of the process, but in every 5s interval I'm only incrementing by 5. With the new hot restart code, instead of adding 5 that I would have gotten during the last flush interval, I'm now going to add 100K in the first update, and then I will start adding the delta, which is not what we want.
Take a look at the counter latch() API and how that is used. Effectively, I think what you want to send over to the new process is the latched value vs. the total value. This would mean that for counters you don't need to keep track of the previous value that was sent. There are multiple ways of dealing with this but IMO this is the cleanest way. It definitely needs some thought though.

Ah, got it, thanks for the explanation. I had definitely entirely forgotten about that aspect of counters.
The approach I implemented was to add a "stealthyAdd()" to the counter API, and use that rather than add(), for just the first time. Feel free to suggest a different name if stealthyAdd is too whimsical!
It would be nice to just send over the deltas and no longer need the map, but I think that would require adding a distinction between "initial value" and "delta" to the protocol, along with requiring the parent keep track of which counter values it has already sent. (And, I think stealthyAdd() would still need to be added). So, a bit more efficient, but more complex code.

@fredlas sorry can you also go through and resolve any comment threads that are fully resolved? That will make it easier to see what is still being discussed. Thank you. cc @jmarantz

OK this looks great. @jmarantz any further comments?  Do you think we are going to have to bump the hot restart protocol version again or could it be done as an in-place optimization/upgrade? To me pagination is not a huge deal but I'm just curious how you are envisioning the implementation.

Regarding pagination, fortunately it will be hot restart compatible. The idea is that the current transfer mechanism is just "hey here is a bunch of stats for you to merge in", and nothing about it requires each transfer message to be a snapshot of the entire state of all stats. So, the hot restart parent just has to have its export process be "ok, here are (say) 10k non-zero-delta counters", and do that repeatedly. The current hot restart child logic is already compatible with that approach.

@fredlas I just did another smoke test, and counters look good, but gauges are broken now. I think the issue is I don't see how you can use a ConstCharStarHashMap for storing parent stats. The c_str() is not stable? Or am I missing something? If so can you add a test that exposes this and fix? Thank you.
/wait

Ah, ok. Maybe this would be a good time to verify what I think I've picked up from context: is it the case that there are few enough gauges that, when it comes to gauges, we can just skip all the memory-saving fancy stat name stuff and still be safe?

@fredlas I will run another smoke test later, but do you have a test coming that shows the error with the previous hash map? It seems like we should have a test that covers that case?

It seems like the issue was that the whole ConstCharStarHashMap type is not, generally speaking, safe to use when you're getting your const char* keys from c_str(). I think the right response there might be additional comments/warnings in the ConstCharStarHashMap header.
As for this particular instance of the problem, I don't think anyone is likely to want to switch it back to ConstCharStarHashMap now that it's working with the even-better StatNameHashMap. (Assuming it does in fact work with your testing...)

I don't think it would be very difficult to have a test that exposed the previous issue? Basically have a few stack allocated proto messages when testing the merging code that are destroyed in between merge invocations? Seems generally useful?

Ok, got it. Enhanced the multiple imports test to do that.

Phew, awesome! Thanks for working through this huge review burden, through all the stages and major changes. There certainly were a lot of improvements you both suggested to get here.

Going by https://github.com/envoyproxy/envoy/commits/master, the coverage test appears to be like 90% flakey recently, due to the same timeout thing as here. I'll try a /retest but maybe it still won't work.

/retest


Revert "Kore Based Rewriter API's Search Functionality Related Code"

Reverts kframework/k#1555.

Why was this reverted?

Looks like the PR may have been the reason behind builds issues on the Jenkins server. I don't understand why it passed in the first place though. In any case, I'll make a new PR, and track down the issue, and then re-merge.

I think the biggest issue is #1634, which doesn't seem to have improved since you reverted (and don't think this code could possbily have triggered that).


Add JSDoc annotations to PlausibleProvider props

Add JSDoc annotations to PlausibleProvider props from the documentation in the README.
This way I wont have to read the readme, ill just look at the Intellisense

Thanks @fredrik-lindseth!

Published as<EMAIL_ADDRESS>