RFC: Teach concat!() to join [u8] and byte str

Rendered

I’d like to see the Motivation section expanded. Why is this useful? Consider describing use cases and adding examples. Currently this section only paraphrases the summary.
The RFC’s title and motivation section mention “byte str” and “byte char”, which I find confusing because there is no such thing in my mental model. Rather, the language syntax contains: string literals of type &'static str, character literals of type char, byte string literals (or bytes literal) of type &'static [u8; N], and byte literals (singular byte) of type u8. In particular, a byte string literal is already an array of integers.
If this is added, I’d prefer it to be a separate macro. concat! today already accepts multiple inputs of different types but always returns &'static str. It would be surprising that adding another input can not only change the return type, but also completely change the handling of integer literals (from ASCII digits serialization to raw byte values).
For that separate macro, the name bconcat! is rather obscure. It is not typical of the standard library’s to do such aggressive abbreviations in public API naming. How about concat_bytes!?

@SimonSapin sounds reasonable!

I’d like to see the Motivation section expanded. Why is this useful? Consider describing use cases and adding examples. Currently this section only paraphrases the summary.

Will expand. I can definitely see the case where someone would want to do concat_bytes!(b"foo", 3, 2).

The RFC’s title and motivation section mention “byte str” and “byte char”, which I find confusing because there is no such thing in my mental model. Rather, the language syntax contains: string literals of type &'static str, character literals of type char, byte string literals (or bytes literal) of type &'static [u8; N], and byte literals (singular byte) of type u8. In particular, a byte string literal is already an array of integers.

The confusion seems to be around "byte char" (b'x'), which seems to be referred to as "byte literal" in the docs, but that seems slightly misleading to me as a single char can have multiple bytes.

If this is added, I’d prefer it to be a separate macro. concat! today already accepts multiple inputs of different types but always returns &'static str. It would be surprising that adding another input can not only change the return type, but also completely change the handling of integer literals (from ASCII digits serialization to raw byte values).

Very reasonable. It would be easy enough to point people in the right direction when extending the current error.

For that separate macro, the name bconcat! is rather obscure. It is not typical of the standard library’s to do such aggressive abbreviations in public API naming. How about concat_bytes!?

Works for me. The discoverability issue could be handled when someone attempts to use concat!(b"").

The discoverability issue could be handled when someone attempts to use concat!(b"").

Ironically, rustc already suggests bconcat! for byte string literals in certain cases.

I agree with @SimonSapin's objection; I think this should use a separate macro concat_bytes!.
Apart from that, though, I'd love to see this.

Whenever any of the arguments to concat!() is a byte literal, its output will be a byte literal, and the other arguments will be evaluated on their byte contents.

In general I agree with the suggestion of making the type with concat_bytes!, but I think it is also necessary - none of the parameters to concat!()/concat_bytes!() need be a string (concat!(1,2,3) => "123", for example), so there wouldn't be any type queue. You could consider trying to infer the type from the context, but that's getting complex.

XX |     concat!(256, b"val");
   |             ^^^ this value is larger than `255`

I think this should have identical behaviour to string concat - ie emit b"256val" in this case. If you want an arbitrary byte then you could do b'\xff'.

Well, b'\xff' is 256_u8. It’s only another literal syntax for the same type, just like 0xFF_u8 is yet another. Technically since macro inputs are token tree concat_bytes! could tell the difference between the two, but since we make it look like a function that takes comma-separated values I think it would feel very weird for this "function" to behave so differently with two arguments that are spelled with different syntax but are the same at the type level and expression-value level.

Maybe, to avoid this confusion, concat_bytes! should not accept single-integer arguments. Only bytes arrays/slices (with either b"foo" byte string literal syntax or [b'f', 0x6F, 111] array literal syntax)

My point is that concat!(123) -> "123": &str, and I think that concat_bytes!() should have identical behaviour, except that its result type is &[u8] - that is, they should accept the same inputs, and produce byte-for-byte identical output, albeit a slightly different type. The only extension is that concat_bytes!() should also accept binary string literals.

@jsgf sounds reasonable for concat and concat_bytes to behave exactly the same with only a difference in the output type. The original PR opened without an RFCS did get concat to change behavior if an b"" was present, but it'd be too "magical" in practice for my taste.

That would be one way to define it and it’s not too terrible but:

It’s almost the same as concat!(…).as_bytes() (except it could return a known-size array rather than a slice)
It would implicitly encode Unicode characters as UTF-8, without a visible indication that this is happening. Compare with byte-string literals, where we’ve chosen to disallow non-ASCII characters entirely to "force" users to consider what character encodings they want. (Non-ASCII bytes can be included with hex backslash escapes like \xFF.)

I would prefer concat_bytes! to not accept "normal" (Unicode) string literals or character literals at all.

It’s almost the same as concat!(…).as_bytes() (except it could return a known-size array rather than a slice)

For some reason I though arbitrary slices couldn't be const, alas they are. TIL.

I would prefer concat_bytes! to not accept "normal" (Unicode) string literals or character literals at all.

Given that the concat_bytes macro would be a builtin, we can perform very deep evaluation of the values being passed in. We could accept ascii only strings and fail otherwise, or (maybe more interestingly), we could warn on these and provide a way of doing it without having to rely on it, for example suggest concat_bytes!("felíz ", "año") -> concat_bytes!("fel\u{ED}z ", "a\u{F1}o") in the same way that b"" could have \xFF suggested. My reasoning being that unicode literals in strings make it clear how wide they are, making the concern less of a foot-gun.

@SimonSapin I'm ok with rejecting unicode strings, as long as we suggest the appropriate hex representation of values outside of the ascii range, as this seems like something easy to include and very useful for end users.

Sure, making a custom error message for this case sounds good.

Any headway on this?

I'd love to see this capability in the stdlib.
My motivation (which I don't think was mentioned yet) is building null-terminated strings at compile-time, for C interop.
macro_rules! zstr {
    ($str:literal) => {
        concat_bytes!($str, b"\0");
    };
}

zstr!(b"abc") == b"abc\0"

You can do that already with normal strings. I use a null_str! macro myself

Very true! That only works with UTF-8 strings though. I'm unfortunately working with Windows-1252.

@nikomatsakis, @Mark-Simulacrum, and I talked about this one today. We generally think this is a good idea. We feel that, while as a builtin macro this is in the lang team's scope, we'd be comfortable with the @rust-lang/libs team evaluating this as if it were not a builtin macro (in which case it would be entirely within the scope of libs).
I do think this should be a separate macro like concat_bytes! rather than overloading concat!.

I'm happy to start an FCP on behalf of the libs team once the RFC text is updated to use concat_bytes!.

I'm interested in moving this forward. Should I open a PR on @estebank's branch to update the RFC text?

@jplatte You should coordinate with @estebank (here or via Zulip). A PR to help update the RFC text seems like a good step, though, after you've coordinated to confirm that'll work.

I was about to reach out to estebank via Zulip because this has been sitting in my inbox for some time, but now I noticed those thumbs up reactions... Seems like I can go ahead, which I'll try to do soon 🙂

First draft is at https://github.com/estebank/rfcs/pull/1.

First draft is at https://github.com/estebank/rfcs/pull/1.

'Reference-level explanation' and 'Prior art' are now still empty. I'd be happy to write them, but am not entirely clear on what to put in either (should the latter just be None?).

'Reference-level explanation' and 'Prior art' are now still empty. I'd be happy to write them, but am not entirely clear on what to put in either (should the latter just be None?).

@rfcbot merge

Libs FCP has started at this point, but I want to reiterate the comment from August last year - https://github.com/rust-lang/rfcs/pull/2509#issuecomment-669329301 - that T-lang feels this is purely in T-libs territory to design the API and such.

This LGTM. Although, I don't think I'd mind seeing string/char literals supported in concat_bytes!. @SimonSapin mentioned above that this would just be similar to concat!("something").as_bytes(), but AIUI, it would enable things like, concat_bytes!("☃", b"\xFF", "foo"). That is, it would let you mix and match. An undoubtedly niche case, but is there any other way to do such a thing?

@rfcbot concern argument types
With concat!() it turned out that it accepting numbers/bools wasn't a great idea: println!(..) expanded to something like print!(concat!(.., "\n")), which caused println!(123) to be accepted, since concat!() converts it silently to a string. This proposed concat_bytes!() macro doesn't do number formatting, but would still accept integers as arguments, which are used as single bytes.
Considering this, I have two small related concerns about the types of arguments to accept:

Inconsistency with concat!(): concat!(123, "\n") produces "123\n", but concat_bytes!(123, b"\n") produces b"\x7b\n".
Making the same mistakes possible as with concat!(): a macro expanding to concat_bytes!($arg, b"\0") such as the zstr!() example above would also accept zstr!(123), which was probably not intended.

@BurntSushi If Unicode arguments are not supported directly you could still mix and match by converting arguments: concat_bytes!("☃".as_bytes(), b"\xFF")

@SimonSapin Ah thanks for that. I'm totally fine with that.

Oh sorry, I got confused with other macros whose expansion includes expression arguments. The point of this one is to expand to a single literal, so it needs to parse the token of its arguments which therefore can’t be arbitrary expressions.

@SimonSapin Ah, maybe that was why I thought the as_bytes() work-around wouldn't work when I wrote my first comment, but it was totally subconscious. Because of that limitation, how do you feel about allowing string literals in concat_bytes!? It's somewhat niche, but there doesn't seem to be a huge drawback to me for allowing it. I suppose we could also just punt on that since it seems like something that could be added backwards compatibly.

I’d prefer for the conversion and choice of encoding to be explicit when converting Unicode to bytes (even if the conversion is only type-level because the memory representation happens to match), but my pre-Rust-1.0 proposal to rename str::as_bytes to str::as_utf8 was not accepted so :shrug:

Actually no, I don't see how that could possibly work. concat_bytes! operates on literals, not constant expressions.

Oh sorry, I got confused with other macros whose expansion includes expression arguments. The point of this one is to expand to a single literal, so it needs to parse the token of its arguments which therefore can’t be arbitrary expressions.

Yeah I share the sentiment. I think my retort would just be that it would otherwise be not possible to mix non-UTF-8 with UTF-8 via the nice syntax of Unicode strings. For me personally, I think that might outweigh the desire for encoding explicitness.
But I do not feel that strongly about this at all.

Is now a good time to also rename the file from 0000-byte-concat.md to 2509-byte-concat.md or is that automated and I don't need to do anything?

We discussed this in today's @rust-lang/libs meeting, and came to a resolution on the question of integer literals:
We'd like to reject bare integer literals, and only allow arrays of integer literals. So, concat_bytes!(b"hello", [1, 2, 3], "world") would be accepted, but concat_bytes!(b"hello", 1, 2, 3, "world") would not be.
It would also be nice to have a diagnostic with a suggestion, that suggests wrapping a series of integer literals with square brackets. So, concat_bytes!(b"hello", 1, 2, 3, "world") should get a rustfix-applicable suggestion to become concat_bytes!(b"hello", [1, 2, 3], "world")

Minor issue that doesn't need to block acceptance of the RFC, but recently there have been some discussions about whether or not new macros ought to be properly namespaced rather than just being effectively added to the prelude outright (I'm not even sure if macro namespacing existed or was mature when this RFC was originally written). Is this new item useful enough that it should live in the prelude, or should it rather be, say, std::array::concat_bytes?

@m-ou-se Is your concern resolved with the latest revision?

Is now a good time to also rename the file from 0000-byte-concat.md to 2509-byte-concat.md or is that automated and I don't need to do anything?

Usually that's done by whoever merges the RFC after the FCP is finished. But feel free to rename it yourself, to save that person one step in the rfc-merging process. ^^

Is your concern resolved with the latest revision?

Yes
@rfcbot resolve argument types

UTF-8

I'd personally prefer for this macro to accept regular string literals for UTF-8, but we can always add that later if that turns out to be useful, as it'd be backwards compatible.

Ping @joshtriplett

:tada:
This is now tracked in https://github.com/rust-lang/rust/issues/87555

This RFC appears to contain a contradiction:

For example, concat_bytes!(42, b"va", b'l', [1, 2]) evaluates to
[42, 118, 97, 108, 1, 2].

And then later

An earlier version of this RFC proposed to support integer literals outside of
arrays, but that was rejected since it would make the output of
byte_concat!(123, b"\n") inconsistent with the equivalent concat!
invocation.


fix:  boots 1.70

The patch fixes issue with boost 1.70: https://github.com/ipkn/crow/issues/340
Patch is based on the patch submitted to other project: https://github.com/moneroexamples/onion-monero-blockchain-explorer/commit/76a0efa8ee3ea5bb466b81d84357d2fd76920cbd

ping @ipkn

get_io_service has been depreciated since at least 1.66[0] so i think it'd be pretty safe to switch this over without the macro
[0] https://www.boost.org/doc/libs/1_66_0/doc/html/boost_asio/reference/basic_stream_socket/get_io_service.html
related: #358

It seems the travis-ci still use libboost1.55-all-dev which fail the tests in the PR.
Ping @ipkn to fix for boost 1.7.

@ipkn If you are busy to maintenance crow, I can do help you.

The latest version doesn't contain this fix, is the a plan to release a new crow_all.h?

@druc0d3s This repository has not been maintained for about 4 years. A few contributors and I are maintaining a fork here.

@druc0d3s This repository has not been maintained for about 4 years. A few contributors and I are maintaining a fork here.

Great, I'm coming over! :-)


only see 2D scene

Describe the bug
I deploy MRTK Example to Holoens 1, but I only see 2D window in scene，and I donot make any change with camera.
To reproduce
Steps to reproduce the behavior:

Build HandInteractionExamples scene （The same is true of other scenes）;
deploy to Hololens 1 via visual studio 2017;
see 2D scene;

Screenshots

Your Setup (please complete the following information)

Unity Version [2019.2.6f1]
MRTK Version [v2.0]

Target Platform (please complete the following information)

HoloLens

Yes, The same bug exists in my project. Does anyone know how to fix it?

Turn off WSA holographic remoting in XR setting before building. That should do the trick.

Turn off WSA holographic remoting in XR setting before building. That should do the trick.

Thank you , it is OK!


rename modules.email

Having a module named email causes an issue when running crawler.py. This is because email conflicts with an internal dependency of the package requests and causes it to import the wrong code.

Renamed email to email_utils
Updated import statement inside app.py

Feel free to suggest a different name - I was feeling uncreative.
See this StackOverflow for an explanation of the error: https://stackoverflow.com/questions/32874326/python-email-module-importerror-no-module-named-utils
Example stacktrace:

Approved! but I'll leave it to you to merge


Particle effects are billboarding incorrectly with an orthographic camera

Forum post: http://forum.playcanvas.com/t/particle-effect-looks-different-in-orthographic/3682
Sample project: https://playcanvas.com/editor/scene/512278
I've attempted a fix in this branch: https://github.com/yaustar/engine/tree/particle-ortho-fix
But I don't like the way I've done it. It feels like it be cleaner and better to pull the rotate and billboard functions out into their own chunks but the generation in particle.js will still be a bit messy
e.g
vshader += chunk.particle_cpu_variablesVS;
vshader += chunk.particle_functionsVS;
vshader += chunk.particle_cpu_mainVS;

Would this be better? Or is the way I've done it already in the branch okay?

I've done the refractor to the way I mentioned in the first post and it does look a lot cleaner: https://github.com/playcanvas/engine/pull/904


feat: fetch target for unknown transactions

This adds a proposed method of fetching the target of the recovery (Delay Modifier) module when TransactionAdded events could not be decoded and thus no Safe address is known.

I'm going to close this as we will aim to use the modules endpoint of the Transaction Service instead.


Badger: v1 to v3 upgrade

Signed-off-by: Ashmita Bohara<EMAIL_ADDRESS>

Codecov Report

Merging #18 (7e18de1) into master (c9d6957) will decrease coverage by 0.04%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master      #18      +/-   ##
==========================================
- Coverage   95.85%   95.81%   -0.05%
==========================================
  Files         235      235
  Lines       10067    10065       -2
==========================================
- Hits         9650     9644       -6
- Misses        346      350       +4
  Partials       71       71

Impacted Files
Coverage Δ

plugin/storage/badger/factory.go
95.69% <ø> (-0.10%)
:arrow_down:

plugin/storage/badger/spanstore/cache.go
100.00% <ø> (ø)

plugin/storage/badger/spanstore/reader.go
95.37% <ø> (ø)

plugin/storage/badger/spanstore/writer.go
97.01% <ø> (ø)

pkg/config/tlscfg/cert_watcher.go
92.20% <0.00%> (-2.60%)
:arrow_down:

cmd/query/app/server.go
95.62% <0.00%> (-1.46%)
:arrow_down:

...lugin/sampling/strategystore/adaptive/processor.go
99.07% <0.00%> (-0.93%)
:arrow_down:

cmd/query/app/static_handler.go
96.77% <0.00%> (+1.61%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c9d6957...7e18de1. Read the comment docs.


Add filtering to Sessions().ListAll()

https://clerk.com/docs/reference/backend-api/tag/Sessions#operation/GetSessionList docs have filtering options, but they are not accessible from the golang SDK

I would love to do this! I can work on it in a bit.

This can be closed


Upgrade or Install 0x87af0001 SQL logic error

Brief description of your issue
I get 0x87af0001 SQL logic error message, when use winget upgrade or winget install.
LOG FILE:
2022-03-04 12:30:03.468 [CORE] WinGet, version [1.2.3411-preview], activity [{5B9FE2A1-4319-41E7-A308-48FC7BDD3A52}]
2022-03-04 12:30:03.468 [CORE] OS: Windows.Desktop v10.0.22000.527
2022-03-04 12:30:03.468 [CORE] Command line Args: "C:\Users\tgsan\AppData\Local\Microsoft\WindowsApps\winget.exe" upgrade
2022-03-04 12:30:03.468 [CORE] Package: Microsoft.DesktopAppInstaller v1.17.3411.0
2022-03-04 12:30:03.468 [CORE] IsCOMCall:0; Caller: winget-cli
2022-03-04 12:30:03.475 [CLI ] WinGet invoked with arguments: 'upgrade'
2022-03-04 12:30:03.475 [CLI ] Found subcommand: upgrade
2022-03-04 12:30:03.475 [CLI ] Leaf command to execute: root:upgrade
2022-03-04 12:30:03.480 [CLI ] Executing command: upgrade
2022-03-04 12:30:03.481 [REPO] GetCurrentSourceRefs: Source named 'microsoft.builtin.desktop.frameworks' from origin Default is hidden and is dropped.
2022-03-04 12:30:03.481 [REPO] Default source requested, multiple sources available, adding all to source references.
2022-03-04 12:30:03.481 [REPO] Adding to source references msstore
2022-03-04 12:30:03.481 [REPO] Adding to source references winget
2022-03-04 12:30:03.481 [REPO] Source past auto update time [5 mins]; it has been at least 5 mins
2022-03-04 12:30:03.482 [REPO] Source past auto update time [5 mins]; it has been at least 27439470 mins
2022-03-04 12:30:04.805 [FAIL] D:\a\_work\1\s\external\pkg\src\AppInstallerCommonCore\MsixInfo.cpp(150)\WindowsPackageManager.dll!00007FF9AD67BDA7: (caller: 00007FF9AD6DCD0E) Exception(1) tid(8578) 80070781 系统无法辨识文件名。

2022-03-04 12:30:04.805 [FAIL] D:\a\_work\1\s\external\pkg\src\AppInstallerRepositoryCore\RepositorySource.cpp(53)\WindowsPackageManager.dll!00007FF9AD7D8A41: (caller: 00007FF9AD6D2946) LogHr(1) tid(8578) 80070781 系统无法辨识文件名。
    Msg:[D:\a\_work\1\s\external\pkg\src\AppInstallerCommonCore\MsixInfo.cpp(150)\WindowsPackageManager.dll!00007FF9AD67BDA7: (caller: 00007FF9AD6DCD0E) Exception(1) tid(8578) 80070781 系统无法辨识文件名。
]

2022-03-04 12:30:04.805 [REPO] Source add/update failed, waiting a bit and retrying: winget
2022-03-04 12:30:07.269 [FAIL] D:\a\_work\1\s\external\pkg\src\AppInstallerCommonCore\MsixInfo.cpp(150)\WindowsPackageManager.dll!00007FF9AD67BDA7: (caller: 00007FF9AD6DCD0E) Exception(2) tid(8578) 80070781 系统无法辨识文件名。

2022-03-04 12:30:07.269 [FAIL] D:\a\_work\1\s\external\pkg\src\AppInstallerRepositoryCore\RepositorySource.cpp(516)\WindowsPackageManager.dll!00007FF9AD7D81B3: (caller: 00007FF9AD5DB9FD) LogHr(2) tid(8578) 80070781 系统无法辨识文件名。
    Msg:[D:\a\_work\1\s\external\pkg\src\AppInstallerCommonCore\MsixInfo.cpp(150)\WindowsPackageManager.dll!00007FF9AD67BDA7: (caller: 00007FF9AD6DCD0E) Exception(2) tid(8578) 80070781 系统无法辨识文件名。
]

2022-03-04 12:30:07.269 [REPO] Failed to update source: winget
2022-03-04 12:30:07.269 [REPO] Multiple sources available, creating aggregated source.
2022-03-04 12:30:07.269 [REPO] Adding to aggregated source: msstore
2022-03-04 12:30:07.269 [REPO] Sending http GET request to: https://storeedgefd.dsx.mp.microsoft.com/v9.0/information
2022-03-04 12:30:08.121 [REPO] Response status: 200
2022-03-04 12:30:08.121 [REPO] Sending http GET request to: https://storeedgefd.dsx.mp.microsoft.com/v9.0/information
2022-03-04 12:30:08.590 [REPO] Response status: 200
2022-03-04 12:30:08.590 [REPO] Adding to aggregated source: winget
2022-03-04 12:30:08.598 [CORE] Examining extension: PFN = Microsoft.Winget.MSStore.Source_8wekyb3d8bbwe, ID = IndexDB
2022-03-04 12:30:08.599 [CORE] Did not find extension: PFN = Microsoft.Winget.Source_8wekyb3d8bbwe, ID = IndexDB
2022-03-04 12:30:08.599 [REPO] Package not found Microsoft.Winget.Source_8wekyb3d8bbwe
2022-03-04 12:30:08.599 [FAIL] D:\a\_work\1\s\external\pkg\src\AppInstallerRepositoryCore\Microsoft\PreIndexedPackageSourceFactory.cpp(205)\WindowsPackageManager.dll!00007FF9AD6DD681: (caller: 00007FF9AD6D2E6E) Exception(3) tid(8578) 8A15000F
2022-03-04 12:30:08.599 [FAIL] D:\a\_work\1\s\external\pkg\src\AppInstallerRepositoryCore\RepositorySource.cpp(540)\WindowsPackageManager.dll!00007FF9AD7D83A7: (caller: 00007FF9AD5DB9FD) LogHr(3) tid(8578) 8A15000F     Msg:[D:\a\_work\1\s\external\pkg\src\AppInstallerRepositoryCore\Microsoft\PreIndexedPackageSourceFactory.cpp(205)\WindowsPackageManager.dll!00007FF9AD6DD681: (caller: 00007FF9AD6D2E6E) Exception(3) tid(8578) 8A15000F ]

2022-03-04 12:30:08.599 [REPO] Failed to open available source: winget
2022-03-04 12:30:08.835 [REPO] Creating PredefinedInstalledSource with filter [None]
2022-03-04 12:30:08.835 [REPO] Creating new SQLite Index [4294967295.4294967295] at ':memory:'
2022-03-04 12:30:08.835 [SQL ] Opening SQLite connection: ':memory:' [6, 0]
2022-03-04 12:30:08.870 [REPO] Examining ARP entries for Machine | X64
2022-03-04 12:30:08.995 [REPO] Examining ARP entries for Machine | X86
2022-03-04 12:30:09.139 [REPO] Examining ARP entries for User | X64
2022-03-04 12:30:09.732 [REPO] Opening SQLite Index for ReadWrite at 'C:\Users\tgsan\AppData\Local\Packages\Microsoft.DesktopAppInstaller_8wekyb3d8bbwe\LocalState\StoreEdgeFD\installed.db'
2022-03-04 12:30:09.732 [SQL ] Opening SQLite connection: 'C:\Users\tgsan\AppData\Local\Packages\Microsoft.DesktopAppInstaller_8wekyb3d8bbwe\LocalState\StoreEdgeFD\installed.db' [2, 0]
2022-03-04 12:30:09.732 [FAIL] D:\a\_work\1\s\external\pkg\src\AppInstallerRepositoryCore\SQLiteWrapper.cpp(173)\WindowsPackageManager.dll!00007FF9AD6F121A: (caller: 00007FF9AD6F21B6) Exception(4) tid(8578) 87AF0001 SQL 错误或缺少数据库
    Msg:[SQL logic error]

2022-03-04 12:30:09.733 [CLI ] Caught wil::ResultException:

Steps to reproduce

Open any terminal (cmd, pwsh, etc.)
Use winget upgrade or winget install packagename

Expected behavior
No error
Actual behavior
Get 0x87af0001 SQL logic error message
Environment
Windows.Desktop v10.0.22000.527
Microsoft.DesktopAppInstaller v1.17.3411.0

I get the same error with  winget list command too.
winget list
An unexpected error occurred while executing the command:
SQL logic error
0x87af0001 : SQL error or missing database

winget information:
Windows Package Manager v1.2.10271
Copyright (c) Microsoft Corporation. All rights reserved.

Windows: Windows.Desktop v10.0.19043.1645
Package: Microsoft.DesktopAppInstaller v1.17.10271.0

What can we do to help more information for debugging?

UPDATE: Since I could not solve it by resetting it or using troubleshooter, I removed winget at all. I used Appx module of PowerShell. Then, I installed winget using msixbundle downloaded from this very repository.

I was having the same problem recently. I deleted the "installed.db" file and solved the problem.
The location of the file "installed.db":
%LOCALAPPDATA%\Packages\Microsoft.DesktopAppInstaller_8wekyb3d8bbwe\LocalState\StoreEdgeFD

I was having the same problem recently. I deleted the "installed.db" file and solved the problem.
[....]
Weeks of headaches, and solution was that simple
Many thanks !

It seems like winget needs a background integrity check for SQLite. Then a background repair or ideally rebuild of database would be cleaner for user experience.

I'm having the same problem. But by deleting the installed.db file, I don't get any changes and the error still occures. Any other solution?

I'm having the same problem. But by deleting the installed.db file, I don't get any changes and the error still occures. Any other solution?

So, as I see, is this a bug that will be patched with the release of the v1.4 o this software (winget-cli), or it's an error that I and we can resolve trough specific commands or other operations? Thanks

If someone could provide the --verbose-logs log file of one of these failures, that would be helpful.  It would also be helpful to have installed.db, although it does contain information about the packages installed from that source (should not be personal information, just package).

I'm open to collaborate to do that. I need to know how I can output a verbose version of the logs and how can I supply the installed.db content. Thanks

So, I managed to output the --verbose-logs for the command winget upgrade --all --verbose-logs. They are pretty long.
WinGet-2022-06-21-19-23-47.195.log. Hope this helps :)

That helps to show that something strange is happening here...  The most SQL basic statement is not compiling.  If you (or anyone else) could attach the installed.db from %LOCALAPPDATA%\Packages\Microsoft.DesktopAppInstaller_8wekyb3d8bbwe\LocalState\Microsoft.Winget.Source_8wekyb3d8bbwe\installed.db I would hopefully be able to reproduce the problem locally.

Ok. I will try to do that

Ok. I don't know how. This morning I started my Laptop, tried to upgrade --all with winget, and it all works clean. Also listing an installing. The problem seems to be gone by itself. If you want I can provide the installed.db file, but I don't think you can now recreate the error

I've also recently been having this issue. I tried deleting my installed.db file but it still returns with a SQL logic error. I've attached a log using winget list --verbose-logs and also my installed.db file (which I've had to zip up to upload it to GitHub). Hopefully these are of some use to trying to debug or reproduce the issue
WinGet-2022-07-18-03-15-59.185.log
installed.zip

No, unfortunately those files did not help.  SQLite uses hidden/system files to store some data that may not have been merged into the main file.  It is possible that you still have something in those files that is affecting things.
I have created a PR (#2352) to improve the logged error message, which might hopefully enable getting to the bottom of the issue.

Found this issue after running into the same error on Windows Insider build 25217, winget version v1.4.2161-preview.
It seems it's no longer possible to uninstall winget from the insider OS, so I couldn't "refresh" my installation to make it work.  The updated error message (from #2352) listed the following:
> winget list
An unexpected error occurred while executing the command:
no such table: metadata
0x871f0001 : SQL error or missing database

I did finally find a solution.  If I delete both of the following files, functionality is restored:
%LOCALAPPDATA%\Packages\Microsoft.DesktopAppInstall_8wekyb3d8bbwe\LocalState\StoreEdgeFD\installed.db
%LOCALAPPDATA%\Packages\Microsoft.DesktopAppInstall_8wekyb3d8bbwe\LocalState\Microsoft.Winget.Source_8wekyb3d8bbwe\installed.db

Another 'Me Too' entry. I have this problem on both Win10 and Win11. Deleting both installed.db files did not solve it for me (but thanks for the idea).

Another 'Me Too' entry. I have this problem on both Win10 and Win11. Deleting both installed.db files did not solve it for me (but thanks for the idea).

I'm also having this issue. Have tried without success:

deleting both installed.db files
repairing from Settings>Apps
resetting from Settings>Apps
installing the pre-release version
winget source update

I had to install the app installer via the windows store and then I could do an update.

@TGSAN - Are you still experiencing this issue with the latest release of WinGet?

@TGSAN - Are you still experiencing this issue with the latest release of WinGet?

I have reinstalled Windows, and it look like working now

@TGSAN - Are you still experiencing this issue with the latest release of WinGet?

I have reinstalled Windows, and it look like working now

This is an unacceptable solution.
What else can be done when resetting winget and updating winget sources both fail?
Does this probablem have to do with certificate issues and a change to winget forcing no SSL decryption (MITM used by proxies)?


Unable to build container on Raspberry 4 - docker: no matching manifest for linux/arm64/v8 in the manifest list entries.

I'm unable to download, build and use the addon, I got the following error: docker: no matching manifest for linux/arm64/v8 in the manifest list entries.
Sorry, maybe this is a lame issue, I'd appreciate every suggestion.
Thanks!

Please try version 1.5.4


Feature/variables

@jayhof sorry that this contains your section changes - have not seen it and I branched away from joblocal-bootstrap4 with git checkout -b. I don't know how this happens, so I need your help later on, thanks in advance and sorry for that.
Besides of that, this PR contains:

all variables (as discussed yesterday)
semantic color formatting as proposed by Bootstrap team
typography, table and button variables

@jayhof something wrong?

@maxstrebel no sorry, that was an accident. I wanted to approve

ah okay :) could also have merged it right away


Please add a rule to fix redundant use of ternaray

Too often I see:
foo ? foo : bar

and it would be better as
foo || bar

They're both pretty terrible in most cases, though

@caitp what would you prefer instead?

in es5 code I agree this is useful for defaults - some people think if is better,  more explicit than ternary or ||, I think thats a code style choice (if vs ||) as I don't agree that one is better than the other.
for this issue, I agree that if you are using a short one line method, || is preferable as it is simpler,  more common and missing the repetition. However I'm not sure if it is a style issue for jscs or a coding improvement for jshint.
My opinion I think is just on the side of jshint and that this is a good suggestion, so we will wait and see what other people think.

They are bad in general because they treat all falsy values as not present, so it can be a poor pattern for falling back on default values. Of course it depends on context, but in many cases it's the wrong approach and needs to be enforced on a case by case basis in code review

I think this is a good suggestion. Another thing JSHint can check are conditions which are always true/false:
undefined ? a : b;
4 ? a : b;


Crashes not uploaded for Android and IOS symbols not loaded for UE5.1 crashes

Hi,
I integrated version 0.2.5 of BugSplat Unreal plugin in UE5.1.
I see crashes and symbols correctly uploaded and parsed for Linux server builds. But I am not seeing the crash at all reported for Android.
For iOS I see the crash but it is not symbolicated.
All the settings are standard. I am currently testing with development builds.
I have tried global as well as Project specific configs to try to make it work.
I am seeing Android symbol upload process start as part of packaged builds but never completes. No error in the logs either.
Is Android symbol upload supported when running UE Editor on Mac?
I see IOS symbols upload successfully.
I do not see any logs for crash being sent from Android client run.

Hi @gamerdoku 👋,
Can you please DM the name of you database to me on Discord, or reach out via the bugsplat.com support tool?
For Android, what device/version are you using? Are you testing on hardware or in an emulator? Android symbol uploads are supported from macOS, but depending on your network connection, they might take a long time.

I have shared the requested database name.
For Android I am testing on One Plus 8 Pro.
Android symbols definitely are failing for me on Mac with the following errors:
BugSplat postprocessing: Start debug symbols upload for iOS
UATHelper: Packaging (Android (ASTC)): BugSplat postprocessing: Unexpected platform Android. Terminating...

On Windows host I am not seeing an error but do not see a success message either.

The iOS issue is mostly fixed... there's still a UX issue https://github.com/BugSplat-Git/BugSplat-iOS/issues/11
Android Symbol Upload issue should be resolved by https://github.com/BugSplat-Git/bugsplat-unreal/issues/61 and a change to increase the timeout on our backend.
Haven't had a chance to dive into the Android crash issue yet but that's next...

Android crash issue was fixed by #77


Handle case sensitive system

There is no need to use a case sensitive executable name, since it will be handled gracefully on windows anyway. This change allow support for Linux system, in particular Debian distribution where:
$ dpkg -L nuget
...
/usr/bin/nuget

Fixes symptoms:
CPackNuGet:Debug: NUGET_EXECUTABLE=`NUGET_EXECUTABLE-NOTFOUND`
CMake Error at /usr/share/cmake-3.13/Modules/Internal/CPack/CPackNuGet.cmake:282 (message):
  NuGet executable not found

Thanks for your interest in contributing to CMake!  The GitHub repository
is a mirror provided for convenience, but CMake does not use GitHub pull
requests for contribution.  Please see
https://gitlab.kitware.com/cmake/cmake/tree/master/CONTRIBUTING.rst
for contribution instructions.  GitHub OAuth may be used to sign in.

Thanks.  Moved to CMake GitLab MR 3427 as per CONTRIBUTING.rst.


Network Doc Updates

This checklist is used to make sure that common guidelines for a pull request are followed.
General Guidelines

[x] The PR has modified HISTORY.rst describing any customer-facing, functional changes. Note that this does not include changes only to help content. (see Modifying change log).

Command Guidelines

[x] Each command and parameter has a meaningful description.
[x] Each new command has a test.

(see Authoring Command Modules)

View a preview at https://prompt.ws/r/Azure/azure-cli/5476
This is an experimental preview for @microsoft users.

@sptramer: it seems to me that each long summary should end with a period, right?
Was this just left off originally?

@twitchax Long-summaries should end in a period, unless their last element is a URL.

Closing in favor of #5660.


[ERROR] Error: Failed to load data, status code:500

Hi,
I'm hitting an unexpected error on Sony TV KD-49X8000C and would appreciate if you could help.

I'm able to get the list of apps by running terminal command so the IP and PSK are correct.
I'm running homebridge as a docker container on a Synology NAS. Other plugins work OK. It's set up through https://github.com/oznu/homebridge-syno-spk with Homebridge Version 0.4.46
I enabled debug log in Homebridge, hopefully this is enough information.

Thanks,
Sam
[2/10/2019, 10:50:33 PM] [BraviaOSPlatform] [ERROR] Sony Android: An error occured by getting inputs, trying again...
[2/10/2019, 10:50:33 PM] [BraviaOSPlatform] [ERROR] Error: Failed to load data, status code:500
[2/10/2019, 10:50:33 PM] [BraviaOSPlatform] [ERROR] Error: Failed to load data, status code:500
    at ClientRequest.<anonymous> (/homebridge/node_modules/homebridge-bravia-tvos/src/platform.js:77:18)
    at Object.onceWrapper (events.js:277:13)
    at ClientRequest.emit (events.js:189:13)
    at HTTPParser.parserOnIncomingClient [as onIncoming] (_http_client.js:556:21)
    at HTTPParser.parserOnHeadersComplete (_http_common.js:109:17)
    at Socket.socketOnData (_http_client.js:442:20)
    at Socket.emit (events.js:189:13)
    at addChunk (_stream_readable.js:284:12)
    at readableAddChunk (_stream_readable.js:265:11)
    at Socket.Readable.push (_stream_readable.js:220:10)
    at TCP.onStreamRead [as onread] (internal/stream_base_commons.js:94:17)

i see this bit of information in the log
Plugin /homebridge/node_modules/homebridge-bravia-tvos requires Node version of ^8.9.1 which does not satisfy the current Node version of v10.15.1. You may need to upgrade your installation of Node. but i saw your comment on https://github.com/SeydX/homebridge-bravia-tvos/issues/5 so i assume it's a non-issue

I tried to install https://www.npmjs.com/package/homebridge-sony-television
It worked right away. I can turn on/off the TV using that plugin, switch inputs, etc.
The interesting part is that toggling that other plugin made your plugin start working.
[2/10/2019, 11:50:53 PM] [BraviaOSPlatform] [INFO] Getting data from Sony Android (<IP_ADDRESS>:80)
[2/10/2019, 11:50:53 PM] [BraviaOSPlatform] [INFO] Sony Android: Found new inputs, checking config for apps...
[2/10/2019, 11:50:53 PM] [BraviaOSPlatform] [INFO] Sony Android: Found apps in config, storing in cache...
[2/10/2019, 11:50:53 PM] [BraviaOSPlatform] [INFO] Sony Android: Checking Sony Android for channels...
[2/10/2019, 11:50:53 PM] [BraviaOSPlatform] [INFO] Sony Android: Found following sources: ["Analog","Dvbt"]
[2/10/2019, 11:50:53 PM] [BraviaOSPlatform] [INFO] Storing Sony TV Data in cache...
[2/10/2019, 11:50:53 PM] [BraviaOSPlatform] [INFO] Connecting to <IP_ADDRESS>
[2/10/2019, 11:50:53 PM] [BraviaOSPlatform] [INFO] Configuring accessory: Sony Android
[2/10/2019, 11:50:53 PM] [BraviaOSPlatform] [INFO] Adding new Input Source: HDMI 1
[2/10/2019, 11:50:53 PM] [BraviaOSPlatform] [INFO] Adding new Input Source: HDMI 2/MHL
[2/10/2019, 11:50:53 PM] [BraviaOSPlatform] [INFO] Adding new Input Source: HDMI 3
[2/10/2019, 11:50:53 PM] [BraviaOSPlatform] [INFO] Adding new Input Source: HDMI 4/ARC
[2/10/2019, 11:50:53 PM] [BraviaOSPlatform] [INFO] Adding new Input Source: ExpressVPN
[2/10/2019, 11:50:53 PM] [BraviaOSPlatform] [INFO] Sony Android: Inputlist finished
[2/10/2019, 11:50:53 PM] [BraviaOSPlatform] [INFO] Sony Android: Welcome to Sony Bravia TV OS 2.0.0 - Below your Identifier list to create rules in 3rd party apps
[2/10/2019, 11:50:53 PM] [BraviaOSPlatform] [INFO] Sony Android: (0) HDMI 1
[2/10/2019, 11:50:53 PM] [BraviaOSPlatform] [INFO] Sony Android: (1) HDMI 2/MHL
[2/10/2019, 11:50:53 PM] [BraviaOSPlatform] [INFO] Sony Android: (2) HDMI 3
[2/10/2019, 11:50:53 PM] [BraviaOSPlatform] [INFO] Sony Android: (3) HDMI 4/ARC
[2/10/2019, 11:50:53 PM] [BraviaOSPlatform] [INFO] Sony Android: (4) ExpressVPN
HAP Warning: Characteristic 000000EA-0000-1000-8000-0026BB765291 not in required or optional characteristics for service 00000113-0000-1000-8000-0026BB765291. Adding anyway.

Status code 500 means internal server error, maybe ip adress incorrect? Or there are too many attempts to connect to your tv, that can be caused if u use many plugins for your tv and everyone tries to poll the state
The tv needs to be turned on for initial request, only the first time or if you change your config

I have the exact same issue with a Sony KD-55A1.
In my case, the Homebridge runs alone on a Raspberry Pi 3
[2019-2-24 23:36:18] [BraviaOSPlatform] [ERROR] Sony TV: An error occured by getting inputs, trying again... [2019-2-24 23:36:18] [BraviaOSPlatform] [ERROR] Error: Failed to load data, status code:500 [2019-2-24 23:36:18] [BraviaOSPlatform] [ERROR] Error: Failed to load data, status code:500     at ClientRequest.<anonymous> (/usr/local/lib/node_modules/homebridge-bravia-tvos/src/platform.js:77:18)     at Object.onceWrapper (events.js:272:13)     at ClientRequest.emit (events.js:180:13)     at HTTPParser.parserOnIncomingClient (_http_client.js:540:21)     at HTTPParser.parserOnHeadersComplete (_http_common.js:117:17)     at Socket.socketOnData (_http_client.js:445:20)     at Socket.emit (events.js:180:13)     at addChunk (_stream_readable.js:274:12)     at readableAddChunk (_stream_readable.js:261:11)     at Socket.Readable.push (_stream_readable.js:218:10)     at TCP.onread (net.js:581:20)

Hello, im currently working on v3 with a rewritten code , i will check this in v3

Hello, im currently working on v3, the beta is now available!
How to download and configure:
https://github.com/SeydX/homebridge-bravia-tvos/issues/32


just  CLA test

[ ] I have registered the PR changes.

Ⅰ. Describe what this PR did
just  CLA test
Ⅱ. Does this pull request fix one issue?

Ⅲ. Why don't you add test cases (unit test/integration test)?
Ⅳ. Describe how to verify it
Ⅴ. Special notes for reviews

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.15 out of 21 committers have signed the CLA.:white_check_mark: tuwenlin:white_check_mark: a364176773:white_check_mark: renliangyu857:white_check_mark: Bughue:white_check_mark: robynron:white_check_mark: l81893521:white_check_mark: doubleDimple:white_check_mark: zw201913:white_check_mark: wangliang181230:white_check_mark: whxxxxx:white_check_mark: slievrly:white_check_mark: xssdpgy:white_check_mark: miaoxueyu:white_check_mark: AlbumenJ:white_check_mark: xingfudeshi:x: neillee95:x: zhangzq7:x: wingchi-leung:x: crazy-sheep:x: zhuyoufeng:x: odidevYou have signed the CLA already but the status is still pending? Let us recheck it.


Can't apply shadow styling to SortRow on iOS

Attempting to apply a shadow to the active row using sortRowStyle prop doesn't seem possible. The shadow simply doesn't appear. If I remove the overflow: 'hidden' from SortRow's base style, the shadow does appear, so I suspect this is an iOS/RN issue (seems described here: https://github.com/facebook/react-native/issues/449).
Any particular reason for the overflow: hidden? Would it be better to leave it off the base style, and let the end user pass it via sortRowStyle if needed? I can submit a PR if you'd like, it's a simple one-liner.

I don't know why overflow: hidden is there, but you can just override it in your sortRowStyle with overflow: visible

Ah, good point, no need to modify the original base style.


bpf: Add complexity-test config for issue #30093

Adds complexity tests for #30093

ty! I switched the PR to draft for now, as we would need to strip down the config a bit to isolate the relevant parts. Let's see if CI is still happy to consume this though...

/ci-verifier


Semantic (Not)BeEquivalent for XDocument/XElement

Also fixed a bug where the location for xml comparisons was incorrect after a selfclosed  element had been compared.
The tests for XDocument/XElement.Should().BeEquivalent() are not very covering. They basically just checks that semantical equivalency is used. The tests for XmlNode are much more extensive, as those are what I used to test the XmlReaderValidator implementation. How do you want the tests? I can think of three options.

Kept as it is
Also do full tests/specs for XDocument/XElement (will be a lot of copied tests)
Move the detailed tests to a separate class that tests XmlReaderValidator directly

Please go for option 2. We've tried to avoid that before by trying to be smart with shared code or base-classes, but it always resulted in a lot of pain.

Updated with tests, copied the namespace related equivalency tests from XmlDocument.

Looks like something is broken in the CI. It says "All checks have passed", but the build contains failed tests: https://ci.appveyor.com/project/dennisdoomen/fluentassertions/build/4.3.2+0 (Build 151)

Yep, noticed that too on master. It's the Silverlight test. Will check out the build script to make sure it will fail on that.

I've fixed the problems on master, so I'm about to merge this PR. However, you could have saved yourself some time by not putting terms like XElement in the names of tests. I follow these principles.

I did that renaming to be able to see directly from the test name if it concerns XDocument or XmlDocument - but if you want I can roll it back and follow the linked principles.

Would you?? :confounded:

Now I think this one is ready to be merged. Do you have anything more you want me to fix?
I've got another one coming with HaveAttribute etc. for XmlElements. I've based it on this commit, so I can't create a PR for it independently. As soon as this one is merged I'll submit the other one.

The additional assertions for XmlElements are now pushed to https://github.com/AndersAbel/FluentAssertions/tree/XmlElementAssertions. I'll make a PR for that when this one is merged.


Arrow position wrong for certain placements

Reproduction demo
https://codesandbox.io/s/blue-leftpad-4of16?file=/src/Tooltip.tsx

I am using example code from the popper docs with styled components.
I want to make sure it is not related to styled components usage. Happy to create an additional example if this repro isn't good enough.
Steps to reproduce the problem
Please see code sandbox. Issue is present on load.
What is the expected behavior?
When I adjust the placement value the arrow should be positioned correctly via styles.arrow
What went wrong?
The arrow is positioned correctly for certain placements like top and bottom but is off for most other placements including variation placements like top-start, bottom-start  etc.

Any other comments?
I might be missing some userland code that I need to add here but my expectation was that the arrow would get positioned automagically via the translate styling being passed via styles.arrow.
Guidance on how to get the correct arrow position for all placement values would be greatly appreciated!
Packages versions

Popper.js: 2.6.0
react-popper: 2.2.4

I can't see anything in the codesandbox, code is missing.
From the screenshot it looks like you may want to add some margin to the arrow element so that it doesn't glitch like in your pictures.

@FezVrasta sorry about VPN glitched when saving. It should be working now 👍
Thanks for the quick response.
I updated my usePopper invocation to set a padding on the arrow and things look good now.
const { styles, attributes } = usePopper(
    referenceElement.current,
    popperElement.current,
    {
      placement,
      modifiers: [
-       { name: "arrow", options: { element: arrowElement } },
+      { name: "arrow", options: { element: arrowElement, padding: 10 } },
        { name: "offset", options: { offset: [0, 8] } }
      ]
    }
  );

I have a couple of questions:

is it possible to configure this padding dynamically based on a placement via this API?
do you think it's worth updating the docs to include this padding in the example code so that all placements show up correctly?

Adding arrow to react-popper is way more complicated than it should be. Amount of time I spent reading the source code, I could probably spend to write popover manually.
@peterjcaulfield where in the docs is example that you mention linked? I only found it through the repo.


Read from local MQSC file

The MQSC file used by the queue_manager module is currently read from the target machine (assuming it has already been copied there during installation, or with a suitable playbook).
The Ansible module runs on the target machine, so it is unlikely that we can read from the host from within the python code.
Instead, a custom MQSC configuration can be moved to any desired target machines using the ansible.builtin.copy module (https://docs.ansible.com/ansible/latest/collections/ansible/builtin/copy_module.html).
We may wish to document this, or integrate it directly with our sample playbooks.

I belive as of now, this issue can be closed out. A local MQSC file on the host is copied to the target machine, which is then read and executed from a specified location on the target. To my knowledge, this has always been the design - but please correct me if I have misunderstood.


7.0.0-alpha.6 has been released as latest instead of alpha

This causes the generated api to be unusable.
see https://www.npmjs.com/package/api?activeTab=versions

Thanks for the flag @spike1292, this has been fixed!


internal and external link in same section break

Steps to reproduce

In empty folder, create a file lib/test.rb with content
# {#both} internal and {External}[http://www.rubydoc.info/] breaks.
def both
  nil
end

# {#inner_only} works.
def inner_only
  nil
end

# {External only}[http://www.rubydoc.info/] also works
def external_only
  nil
end

Run yard at project root.

Actual Output
The section for #both is rendered as
#both} internal and {External

(There is nothing interesting in console output, but I can surely provide it if you are interested in it.)
Expected Output
That section should be
#both
internal and
External

Environment details:

OS: macOS 10.12.5 (16F73)
Ruby version: 2.4.1
YARD version: <IP_ADDRESS>
Relevant software dependency/versions: N/A

[x] I have read the Contributing Guide.

It is mentioned in #210 that the documentation is first translated by RDoc, who does not know the {#method} syntax. Since my documentation heavily rely on Yard functionality, supporting RDoc is impossible anyway (thus unnecessary), so I may as well just use Yard’s link (which works as expected).


Use nox to test different environments

I started integrating nox to solve our problem with the different environments (latest stable and current version). This is not finished yet, but I have a plan for how to do it.
I also added a custom option to pytest named --skip-post-migration to skip the last post-migration file when setting up the database. But this reveals more problems with my pre-/post-migrations that I understand but don’t know how to solve. The main problem is that without the post-migration we are already using procrastinate_job_status_v1 in the current version on multiple occasions (for example, in queries.sql), but the procrastinate_jobs table still uses procrastinate_job_status (without “_v1”) as this conversion is part of the post-migration. But there is no way to do this conversion in the pre-migration as this would break the latest stable version 😞.

Ouch, this was quite nasty 🤪. I now went another route by keeping procrastinate_job_status and procrastinate_job_event_type unversioned. I also kept the aborting status, but indicated that it is unused. We can remove it later on (if necessary). At least I now have a working pre- and post-migration setup with the current version. I am already scared of trying the latest stable version with pre-migration 🫣. nox is not yet run during CI, upcoming.

@ewjoachim The first two nox sessions work fine. I also fixed the migration stuff. Unfortunately, I have a problem with the last remaining session (latest stable release + pre-migration); see the error message below. I tried different things, but I couldn't solve it. You made something similar with pip in your branch. Could you help me understand why this might be happening and why "tests" isn’t being recognized as a module? I’d really appreciate it! You can start the session by running nox -s  stable_version_without_post_migration.
The error message:
Traceback (most recent call last):
  File "/workspaces/procrastinate/.nox/stable_version_without_post_migration/bin/pytest", line 8, in <module>
    sys.exit(console_main())
             ^^^^^^^^^^^^^^
  File "/workspaces/procrastinate/.nox/stable_version_without_post_migration/lib/python3.12/site-packages/_pytest/config/__init__.py", line 201, in console_main
    code = main()
           ^^^^^^
  File "/workspaces/procrastinate/.nox/stable_version_without_post_migration/lib/python3.12/site-packages/_pytest/config/__init__.py", line 156, in main
    config = _prepareconfig(args, plugins)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/procrastinate/.nox/stable_version_without_post_migration/lib/python3.12/site-packages/_pytest/config/__init__.py", line 341, in _prepareconfig
    config = pluginmanager.hook.pytest_cmdline_parse(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/procrastinate/.nox/stable_version_without_post_migration/lib/python3.12/site-packages/pluggy/_hooks.py", line 513, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/procrastinate/.nox/stable_version_without_post_migration/lib/python3.12/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/procrastinate/.nox/stable_version_without_post_migration/lib/python3.12/site-packages/pluggy/_callers.py", line 139, in _multicall
    raise exception.with_traceback(exception.__traceback__)
  File "/workspaces/procrastinate/.nox/stable_version_without_post_migration/lib/python3.12/site-packages/pluggy/_callers.py", line 122, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/procrastinate/.nox/stable_version_without_post_migration/lib/python3.12/site-packages/_pytest/helpconfig.py", line 105, in pytest_cmdline_parse
    config = yield
             ^^^^^
  File "/workspaces/procrastinate/.nox/stable_version_without_post_migration/lib/python3.12/site-packages/pluggy/_callers.py", line 103, in _multicall
    res = hook_impl.function(*args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/procrastinate/.nox/stable_version_without_post_migration/lib/python3.12/site-packages/_pytest/config/__init__.py", line 1140, in pytest_cmdline_parse
    self.parse(args)
  File "/workspaces/procrastinate/.nox/stable_version_without_post_migration/lib/python3.12/site-packages/_pytest/config/__init__.py", line 1494, in parse
    self._preparse(args, addopts=addopts)
  File "/workspaces/procrastinate/.nox/stable_version_without_post_migration/lib/python3.12/site-packages/_pytest/config/__init__.py", line 1398, in _preparse
    self.hook.pytest_load_initial_conftests(
  File "/workspaces/procrastinate/.nox/stable_version_without_post_migration/lib/python3.12/site-packages/pluggy/_hooks.py", line 513, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/procrastinate/.nox/stable_version_without_post_migration/lib/python3.12/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/procrastinate/.nox/stable_version_without_post_migration/lib/python3.12/site-packages/pluggy/_callers.py", line 139, in _multicall
    raise exception.with_traceback(exception.__traceback__)
  File "/workspaces/procrastinate/.nox/stable_version_without_post_migration/lib/python3.12/site-packages/pluggy/_callers.py", line 122, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/procrastinate/.nox/stable_version_without_post_migration/lib/python3.12/site-packages/_pytest/warnings.py", line 151, in pytest_load_initial_conftests
    return (yield)
            ^^^^^
  File "/workspaces/procrastinate/.nox/stable_version_without_post_migration/lib/python3.12/site-packages/pluggy/_callers.py", line 122, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/procrastinate/.nox/stable_version_without_post_migration/lib/python3.12/site-packages/_pytest/capture.py", line 154, in pytest_load_initial_conftests
    yield
  File "/workspaces/procrastinate/.nox/stable_version_without_post_migration/lib/python3.12/site-packages/pluggy/_callers.py", line 103, in _multicall
    res = hook_impl.function(*args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/procrastinate/.nox/stable_version_without_post_migration/lib/python3.12/site-packages/pytest_django/plugin.py", line 356, in pytest_load_initial_conftests
    with _handle_import_error(_django_project_scan_outcome):
  File "/usr/local/python/3.12.1/lib/python3.12/contextlib.py", line 158, in __exit__
    self.gen.throw(value)
  File "/workspaces/procrastinate/.nox/stable_version_without_post_migration/lib/python3.12/site-packages/pytest_django/plugin.py", line 184, in _handle_import_error
    raise ImportError(msg) from None
ImportError: No module named 'tests'

pytest-django could not find a Django project (no manage.py file could be found). You must explicitly add your Django project to the Python path to have it picked up.

@ewjoachim I finally got it working. Some tests of nox -s  stable_version_without_post_migration still fail, but that's expected as I run the current tests with the old stable version. I wonder how we deal with those, or what the idea is here. Wouldn't it make more sense to run the old tests (by cloning the old code) with the migrations (up until "pre")?

Congratulations @medihack !


F4ever review PR

Some aspects of the code need attention

Thanks for the review, we will take your comments into consideration when making a plan for future refactoring

Thanks for the review, we will take your comments into consideration when making a plan for future refactoring


Empty Module Exports Dramatically Affect Bundle Output

Bug report
While integrating TypeScript 3.9 I noticed significant bundle size regressions and confirmed it was triggered by the export * is always retained change by using TS nightlies.
I've created a more minimal repro showing that exporting an empty module with no executable code with contents such the one below has significant impact on bundle output.
//# sourceMappingURL=DockPanel.Props.js.map

What is the current behavior?
Adding exports to modules that contain no executable code cause a significant change in bundle output. These images show the effects of adding an export of an empty module with the contents shown above (shown on the left in the images) on bundle output (shown on the right):
Webpack 4:

Webpack 5:

This also has dramatic effects on split chunk behavior, causing wide swaths of code to move between chunks. I'm hoping this minimal repro is the cause of that behavior, but I'm not sure how to confirm that hypothesis at this point.
If the current behavior is a bug, please provide the steps to reproduce.
Repros

Branch based off webpack-4
Branch based off webpack-5
webpack4-repro patch file
webpack5-repro patch file

All of these contain a new example called side-effects-empty-module which can be run using the standard example steps. The key line is in module-with-export/index.js which shows dramatic changes to bundle output when commented in and out (effects shown above.)
What is the expected behavior?
I've been chatting with @TheLarkInn about this for the past couple of days and agreed we should enter this issue. I don't think empty module exports should affect the bundle output. (Or at least I would like to understand the reason and mitigate the behavior.)
Other relevant information:
webpack version: 4.43.0, 5.0.0-beta.16
Node.js version: 12.16.3
Operating System: Windows 10
Additional tools:
Terms: bundle size regression, ts3.9, typescript 3.9, 3.9.2

@sokra @evilebottnawi Sean thought your input would be really good on this, so I'd like to hear what you think. 😄

Interesting issue. Empty modules seem to cause an problem here because they are no longer considered as ESM, but as CJS module (there is no import or export). Exports are unknown in such a dynamic module and export * is tricky here. But it should not affect direct exports as they have priority over star exports.
Adding export {} to empty modules should force them to ESM.
Why is the module empty? Is this transpiled from a type-only typescript module?

That is correct, this is an example of what type-only TypeScript modules look like.

One thing I wanted to call out here is that my repro has multiple export *'s from the empty module. It seems that all of them are needed to repro this issue and that having just one export will not affect the bundle. I just wanted to point this fact out in case this could be a bug rather than a module specification issue.

Out of curiosity I also did a small sample project with Babel to see how it generates ESM with type-only modules. It seems to take the same approach of generating an export * of an empty module, which would seem to affect webpack in a similar way. This makes me think Babel users would have previously encountered output similar to what I've provided in my repro. (Although again with the requisite multiple layered export *'s of said empty module causing bundling regression.)

@sokra with TS 3.9 people are going to run into this a lot more I fear. How hard is a fix from our side vs. TSC dropping empty modules. (Maybe not even possible given the ts loader scenario?)

Already working on that...
Still we could generate even better code with less interop logic if TS would add an export {} if it removes all imports and exports.

I'm not sure this is TS-specific. The babel repro I provided above also generates the same types of modules. I think the use case here is more about layers of libs exporting empty modules rather than TS's output.

Yeah, it also likely affects anyone using ts-loader with transpileOnly on. If this was optimized away, we might see a nice dip in bundle sizes for similarly structured projects using transpileOnly or Babel.

Here is the status of this problem from webpack side:
Only investigated in the webpack 5 problem so far. I'll check the webpack 4 thing afterwards (Likely the some problems).
There are two problems:

Using module inside of a ESM opts out of used exports optimization (Tree Shaking). In the repro this was caused by this module:

export * from "module-with-export";

module.exports = function(msg) {
	console.log(msg);
};

Note that this module is invalid (module.exports must not be used when using export or import) and would throw an exception at runtime. But there are other cases where it would make sense to allow using module without opt-out. Like const m = module; console.log(m.id);. In CJS using module really need to cause an opt-out, as you can access module.exports with the module object, but in ESM this is not needed.

Side-effects optimization doesn't work when there are unknown exports reexported somewhere in the chain. We only implemented the pure-ESM case, where exports are statically known. An empty module has unknown exports (see earlier comment). We can improve the algorithm to handle unknown exports and I'll do so.

Fixing problem 1 would already fix the large bundle size due to the large export in the repro. Fixing problem 2 will cause modules like ./log to be excluded from bundle, so reduces sized even further.

Case 1 is not really a valid case in our scenario... this is just a side effect of me trying to repro behavior in a minimal scenario. I should have taken module.exports out. Does taking this out fix case 2 as well? Should I work on the repro a bit more?
Just wanted to clarify that in case 2 you aren't changing side effect behavior, only removing unused exports. In our case we don't want other modules with valid side effects to be removed (like log statements) where they previously weren't, either.
Put most simply, I don't think adding an export of a module with no executable code (which would be types modules here) should have any effect on the bundle, regardless of what other non-empty modules are or aren't doing. If the repro isn't showing that clearly enough, I can try and tweak it more.

Let me fix these problems before you put more time into the repro

I also look into detecting these empty modules, but I still think this is worth fixing in typescript and Babel.

@hzoo what do you think about type exports e.g. export type ... making a module a ESM and should leave an export {} resp. "use strict"; + __esModule when removed? In Babel and typescript.

Is it needed because currently TypeScript uses import/export to disambiguate ESM and CJS, and doesn't support ESM without any "module syntax"?

Is it needed because currently webpack uses import/export to disambiguate ESM and CJS, and doesn't support ESM without any "module syntax"?

Yes

If this is the problem, we could inject export {} in the TS Babel plugin when removing an export type/import type doesn't leave any module syntax, and the file's sourceType is "module".

That would be great :)

For webpack 5 I managed to fully optimize the case in your repro. Even with the offending export * from "empty" the result is now: (()=>{"use strict";console.log("a","smallVar")})();
For completeness I add problem 3:

Modules with unknown / dynamic provided exports are not considered for module concatenating. I fixed that by using the used exports as replacement for these information. This works as long the used exports are not dynamic or point to dynamic exports. So reexporting a CommonJs module is fine as long you don't reference (possible) exports from it. Together with the side-effect optimization reexported empty modules can be omitted from the bundle.

btw with
output: {
	module: true
},
experiments: {
	outputModule: true
}

your repro will lead to this output: console.log("a","smallVar");

@sokra From your comments in this issue it sounds like webpack has solved the problem of re-exported empty modules leading to bundle bloat.
Given that, does webpack still need TypeScript to add export {} into ambiguous files?

Yes.
It would allow to concatenate them with other esm.
If there is some code in the file, removing all imports and exports would make it run in non-strict mode.
If the re-export chain leaves the compilation boundary, this would require weird dynamic exporting. Note you can attach exports to a commonjs module from outside, since it's not frozen like a esm namespace object.
Some improvements where only made for webpack 5 and not for webpack 4 to not risk stability.

TypeScript created an emit export issue and merged a PR that I've tested via the 4.0.0-dev.20200522 nightly and the results look very good. It seems to have negated most if not all of the regressions I saw on bundle output.

As pointed out earlier that this doesn't appear to be a TypeScript-only issue, as well. The Babel repro I posted earlier also doesn't emit exports.

webpack is in a position to solve this problem (disambiguation of .js files) without relying on other projects. The "type": "module" declaration in package.json was introduced specifically to solve this problem.  See #10958

Folks, is fix in question available or planned for webpack 4.x? Looks like in 4.x we can't use experiments: { outputModule: true } option.
And being angular app we can't go for typescript 4.x version just yet.
Any course of action you could recommend? Thanks
Also huge thanks to topic starter @JasonGore, it was quite a relief to find this issue after spending many hours on debugging. 🎉

And being angular app we can't go for typescript 4.x version just yet.

What do you mean?

@evilebottnawi, as far as I know angular version is tightly bound with particular TS version, i.e. the most recent angular version 10 is supposed to be used with TS 3.9.7. Hence we can't simply update TS to 4.x.
I.e ngtools/webpack package.json file.

@amalitsky Are you using the latest webpack version?

@sokra, we are on 4.42 right now and bumping it to 4.44 didn't change much.

@amalitsky Do you use mini-css-extract-plugin? Can you try master branch?

@evilebottnawi, we do indeed use mini-css-extract-plugin. We will try to use webpack 5.x but that might take some time.
What I've noticed with index.ts files affected is handling of xxx.types.ts imports has changed. In the past if xxx.types.ts was exporting nothing but TS types index.ts importing that types file would get import * from './xxx.types.ts'; line dropped on some point. Not after upgrade to TS 9.7.
And since xxx.types.ts is empty and not exporting anything (no declaredExports), whole module (index.ts) exports are used and kind of copy pasted in xxx.types.ts.

And since xxx.types.ts is empty and not exporting anything (no declaredExports)

Can you try to add export {}?

@evilebottnawi, I tried exporting random const and it fixed the issue with particular index.ts file. I'll try export {} and update this comment.

@amalitsky friendly ping, problems still exists with latest webpack version?

@evilebottnawi, looks like problem is resolved for us:

We are on Webpack 4.44.1 and TS 4.0.2.
Thanks for the follow up

Great, will be great if you try webpack@5 (most of critical regressions are fixed, so it should be safe), anyway if you faced with the problems again feel free to open new issue(s)


Add a project upgrade mechanism

We often make changes to the default flutter create template. It would be good to have an automated upgrade mechanism that takes various existing native code, yaml, artifacts etc and apply the latest change

[x] Let flutter create leave some sort of version stamp in the produced artifacts indicating the version of the tool when the project was created
[x] Add some mechanism to compare flutter versions. See how it can be harmonized with version.dart.
[ ] Add a flutter project upgrade command that takes in a number of 'steps' recipes in a folder that does modifications gated by version comparison conditions.
[ ] Let it have a --dry-run option to print a diff patch of the changes.

Tasks:

[ ] Let flutter create leave some sort of version stamp in the produced artifacts indicating the version of the tool when the project was created
[ ] Add some mechanism to compare flutter versions. See how it can be harmonized with version.dart.
[ ] Add a flutter project upgrade command that takes in a number of 'steps' recipes in a folder that does modifications gated by version comparison conditions.
[ ] Let it have a --dry-run option to print a diff patch of the changes.

I'm currently targeting explicit Dart awareness of Flutter revision numbers and applying direct file changes.
Maybe one day in the future, it might be worth exploring a automatic codeless way of maintaining and auto applying diffs on various versioned .yaml/.json/.xml/.etc templates but so far, the use cases lean more towards direct targeted changes.

Worth explicitly noting that this will need to support interactive steps with confirmation. In #18102 (migrate iOS devicelab tests to universal binaries), supporting migration came up, but there are some (probably rare) cases where an app may legitimately be 64-bit only, so I think we'd want to confirm before updating.

I great first step would be a version number stored with the generated files and flutter doctor reporting if the projects generated files are up-to-date
(even without providing an automated way to upgrade them).
I created https://github.com/flutter/flutter/wiki/Workarounds-for-common-issues#generated-project-files-outdated and this seemed to help quite a few times already.

Merging into #40460 since there's more details now.


Discuss character normalization scheme

This is an issue opened up to discuss the character normalization scheme in https://github.com/lbryio/lbrycrd/pull/159 . The issue of character normalization is not a topic of expertise that the blockchain team has in particular so this should be opened up for discussion with everyone else. The details of the normalization should not affect the PR outside of the specific normalization function and unit tests.
The current scheme proposed in 159 is to implement NFD normalization (see below link on information)
http://unicode.org/reports/tr15/
http://unicode.org/faq/normalization.html
And than lower case the letters using locale en_US
I suspect that this scheme will have some problems handling other languages. In particular note that international domain names utilizes a custom scheme to deal with various locale needs ( https://en.wikipedia.org/wiki/Internationalized_domain_name - I wonder if there is there some module we can import for this??  ).
Note that any bad scheme we end up implementing here will be extremely troublesome for us to undo since we have to dive into the details of various languages and manually fix the normalization, so it is important that we get it right the first time.

I haven't done a deep dive on this yet, but chiming in to say that we should avoid any US-centric solution.

A few thoughts: lbrycrd is already US-centric in that it's CLI only shows English help, and its RPC commands use English field names. It also returns JSON, which according to that standard, requires US-like periods in floating point numbers. JSON also has invariant date format requirements.
You can see here a chart of the methods that use the locale: http://userguide.icu-project.org/services Unfortunately, it's hard to determine exactly which ICU methods we're using because of the boost wrapper. Generally speaking, though, those formatting methods aren't the features we're looking for.
In Naut's code, we see that he first calls normalize and then to_lower. The former call should put it into a locale-neutral form before the case is lowered. I don't think it's actually using en_US to do the case conversion. I think that this is the best that we can do without taking locale in on claim requests.

From my understanding NFD normalization is not locale specific, but the function "lower" is.

From my understanding NFD normalization is not locale specific, but the function "lower" is.

Unicode is unicode, so picking a normalization form (like NFD) is language agnostic.  It's correct that the lower case algorithm we're using is.  Boost provides the locale aware version as well (via ICU), but I opted not to use that because while it is locale specific, it will not work where that locale is unavailable.  What this means is that the lower case algorithm chosen may not actually lower case the words represented in unicode that cannot be lower cased without the locale awareness, and I think that's a choice that we live with for consistency.

The issue of character normalization is not a topic of expertise that the blockchain team has in particular so this should be opened up for discussion with everyone else.

Agreed, but it's a topic that I have more experience in than many.  Doesn't mean I'm an expert.

Note that any bad scheme we end up implementing here will be extremely troublesome for us to undo since we have to dive into the details of various languages and manually fix the normalization, so it is important that we get it right the first time.

I don't agree since the unicode normalization will not be an issue.  Knowing that there may be international casing issues is something I think we can live with.  In the end, we are still doing a best effort to normalize the code points consistently, which is far better than what we have today (which is just about guaranteed to break or be inconsistent with different languages/locales).

One consequence of our current trie structure is that partial unicode names will no longer make sense; there's actually a data loss on visualization.
There may be other reasons this is a bad idea, in which case I defer to the blockchain team's judgement. But from a UX perspective, I think it's completely unobjectionable to reject unrenderable claim names.

I think this can be closed. Reopen it for further discussion. As it stands, we'll go with the current decisions:

We will use the NFD, en_US locale.

If the incoming UTF8 data is invalid, we will use the data but it will not be normalized or lower-cased, and will compete only against those exact bytes. We assume that current apps and clients will display partial characters (aka, � symbols) and that they will disallow entry of them.

We will eliminate the archaic getclaimtrie RPC method.


Error running npm run start-hot on Windows

C:\Users\x\devel\electron-react-boilerplate> npm run start-hot

<EMAIL_ADDRESS>start-hot C:\Users\x\devel\electron-react-boilerplate
HOT=1 NODE_ENV=development electron .

'HOT' is not recognized as an internal or external command,
operable program or batch file.
npm ERR! Windows_NT 10.0.10240
npm ERR! argv "C:\Program Files\nodejs\node.exe" "C:\Program Files\nodejs\node_modules\npm\bin\npm-cli.js" "run" "start-hot"
npm ERR! node v5.0.0
npm ERR! npm  v3.3.6
npm ERR! code ELIFECYCLE
npm ERR<EMAIL_ADDRESS>start-hot: HOT=1 NODE_ENV=development electron .
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>start-hot script 'HOT=1 NODE_ENV=development electron .'.
npm ERR! This is most likely a problem with the electron-react-boilerplate package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     HOT=1 NODE_ENV=development electron .
npm ERR! You can get their info via:
npm ERR!     npm owner ls electron-react-boilerplate
npm ERR! There is likely additional logging output above.
npm ERR! Please include the following file with any support request:
npm ERR!     C:\Users\x\devel\electron-react-boilerplate\npm-debug.log
PS C:\Users\x\devel\electron-react-boilerplate>

when run on Windows use set HOT=1 && set  NODE_ENV=development && electron .

You must use different env variable syntax as what @guigmaster mentioned or use a unix-like command line tool for Windows .

Is there a way for the script to be made cross platform?

Maybe we can use better-npm-run to solve this problem, it seems to be the easiest solution. @chentsulin you think?

It seems there isn' t a graceful solution here..
Maybe we can just implement a exec script:
http://stackoverflow.com/a/32993213/3012290
https://github.com/benoror/angular-formly/commit/b039b1b990ef137ab0b7242d3c31a2e96077a018

I think better-npm-run is working for this as solution. It uses the spawn method of child_process and there you can set the env option. Should work on Windows I think.
I will do a pull request and then someone need to check on windows (I am using a mac).

Close via #100


Incorrect universe error in typechecking an inductive

The following code:
module Test

noeq
type m (s:Type u#s) : a:Type u#a → Type =
  | Ret: a:Type u#a → x:a → m s a

fails with:
Test.fst(5,22-5,23): (Error 189) Expected expression of type "Type u#(U_name a)"; got expression "a" of type "Type u#(U_name s)"

Fixed in master.


Api 4962 local store client info

Uses the client data-store, for validating /authorization requests. Reference https://github.com/department-of-veterans-affairs/vets-saml-proxy/blob/408455d3629d4118547df3083b7dbea638abd833/oauth-proxy/dynamo_schema.js#L145

To exercise the new functionality make the following changes to you local setup

modify the contents of database with a valid client at https://github.com/department-of-veterans-affairs/vets-saml-proxy/blob/408455d3629d4118547df3083b7dbea638abd833/oauth-proxy/dynamo_schema.js#L240
modify your local dev-config.js by altering one of the routes as follows. In this case the '/veteran-verification/v1' route will reference the local clients table.
     {       "api_category": "/veteran-verification/v1",       "upstream_issuer": "https://deptva-eval.okta.com/oauth2/aus7y0sefudDrg2HI2p7",       "manage_endpoint": "https://staging.va.gov/account",       "client_store": "local"     },

API-4962 Code Review
Confirm Happy Path is working.

Make Authorization request.
client_store = local
client_store has redirect localhost:8080/redirect

Expected results

[x] Redirect to authorization server

[ ] Confirm Client Redirect Miss is working.

Make Authorization request.
client_store = local
client_store does not have redirect localhost:8080/redirect

expected results

[x] expected json

{
    error	"invalid_client",
    error_description	"The redirect URI specified by the application does not match any of the registered redirect URIs. Erroneous redirect URI: http://localhost:8080/redirect"
}

Client miss in client_store

Make Authorization request.
client_store = local
specified client not in client_store

expected results

[ ] expected json

{
    error: "invalid_client",
    error_description: "Invalid client for the authorization path",
}

No Clients in client_store

Make Authorization request.
client_store = local
no clients client_store

expected results

[ ] expected json

{
    error: "invalid_client",
    error_description: "The client specified by the application is not valid."
}

No Okta Redirect URI

Make Authorization request.
client_store = local
redirect uri not registered with okta

expected results

[ ] expected json

{
    error: "invalid_client",
    error_description: "The redirect URI specified by the application does not match any of the registered redirect URIs. Erroneous redirect URI: ${client_redirect}"
}

No Okta Client Id

Make Authorization request.
client_store = local
client id not registered with okta

expected results

[ ] expected json

{
    error: "invalid_client",
        error_description: "The client specified by the application is not valid.",
}

API-4962 Code Review
Confirm Happy Path is working.

Make Authorization request.
client_store = local
client_store has redirect localhost:8080/redirect

Expected results

[x] Redirect to authorization server

Confirm Client Redirect Miss is working.

Make Authorization request.
client_store = local
client_store does not have redirect localhost:8080/redirect

expected results

[x] expected json

{
    "error":	"invalid_client",
    "error_description": "The redirect URI specified by the application does not match any of the registered redirect URIs. Erroneous redirect URI: http://localhost:8080/redirect"
}

Client miss in client_store

Make Authorization request.
client_store = local
specified client not in client_store

expected results

[x] expected json

{
    "error": "invalid_client",
    "error_description": "Invalid client for the authorization path",
}

No Clients in client_store

Make Authorization request.
client_store = local
force dynamo error

expected results

[x] expected json

{
    "error": "invalid_client",
    "error_description": "The client specified by the application is not valid."
}

No Okta Redirect URI

Make Authorization request.
client_store = null
redirect uri not registered with okta

expected results

[x] expected json

{
    "error": "invalid_client",
    "error_description": "The redirect URI specified by the application does not match any of the registered redirect URIs. Erroneous redirect URI: ${client_redirect}"
}

No Okta Client Id

Make Authorization request.
client_store = null
client id not registered with okta

expected results

[x] expected json

{
    "error": "invalid_client",
    "error_description": "The client specified by the application is not valid.",
}

I have confirmed expected functionality. My other comments were only suggestions and thoughts. I would be ok approving either way.


fix: the style for suggestion panel html

Description
Fixes some of the simple styling issues found in https://snyksec.atlassian.net/browse/IDE-227. Verified in IntelliJ.
Checklist

[x] Tests added and all succeed
[x] Linted
[ ] README.md updated, if user-facing
[ ] License file updated, if new 3rd-party dependency is introduced

🚨After having merged, please update the CLI go.mod to pull in latest language server.
Screenshots / GIFs

old panel

new panel

Things to improve:

the size of the font
more padding in the ignore label
the line numbers and - and + need more padding


Awesome Project!!!!

no issue, just wanted to share the love. took two seconds to implement in my dev environment, and saving me a would of pain to see whats going on. thankyou!!

That's great to hear. This is the first comment(or issue) I received on my open source projects. Thanks for the taking the time to appreciate the project. Really means a lot.


403 Error || The request could not be satisfied

I'm making a request for
https://www.deviantart.com/api/v1/oauth2/gallery/folders/copy_deviations
I pass the headers as it is specified in the documentation
Accept-Encoding - gzip, deflate, br
User-Agent - Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/<IP_ADDRESS> Safari/537.36
If I don't pass parameters, I get a 400 Bad request error
with validation errors.
But if I add at least 1 key to Form Data or json
, I immediately start getting an error
in HTML format
CODE: 403
ERROR: The request could not be satisfied

This API does not support multipart form-data or JSON, this is why you get an error.

This API does not support multipart form-data or JSON, this is why you get an error.

With x-www-form-urlencode i has the same mistake

Reading the documentation, this is a GET request, so all parameters should be in the URL. https://www.deviantart.com/developers/console/gallery/gallery_folders_copy_deviations/b635c560c3412cc591a95eec53fb30da?id|default|GALLERY_FOLDER|target_folderid=283EDC74-9CD3-40B5-6A42-484A7D70BD86&id|default|DEVIATION|deviationids[]=35519B6B-A86F-F8EF-9B27-18F0E2310B65&da_version=&mature_content=true&rconly=false&daml=false&is_emulate_loggedout=false&endpoint=%2Fgallery%2Ffolders%2Fcopy_deviations

See the main URL above saying it is a GET request.
I will mention to our team that the parameter section is incorrect.

GET parameters they don't work either

See the main URL above saying it is a GET request.
I will mention to our team that the parameter section is incorrect.

Thanks for that. We're looking into it on our side now.

To help you out in the mean time, the team has shared that it is a POST request.
So a call such as https://www.deviantart.com/api/v1/oauth2/gallery/folders/copy_deviations?target_folderid=283EDC74-9CD3-40B5-6A42-484A7D70BD86&deviationids%5B%5D=35519B6B-A86F-F8EF-9B27-18F0E2310B65 should work.
You can test our API using the console here: https://www.deviantart.com/developers/console/gallery/gallery_folders_copy_deviations/b635c560c3412cc591a95eec53fb30da. Ignore the GET in request in this case.

To help you out in the mean time, the team has shared that it is a POST request.
Ensure you are sending the parameters in the body of the POST request, not as query parameters
You can test our API using the console here: https://www.deviantart.com/developers/console/gallery/gallery_folders_copy_deviations/b635c560c3412cc591a95eec53fb30da. Ignore the GET in request in this case.

My problem remains the original one until I pass the request parameters - I get a validation error.
As soon as I start passing at least some parameter, it doesn't matter from this API request or not
, I can just add test => 1
immediately I start getting an error
403 The request could not be satisfied from Cloud Front

I would be very grateful if you would provide me with an example in a program like postman or its equivalent request so that I can get an example that I can copy. The console does not help - because it does not show the full request that it sends to the API itself, it sends a request to one back, and that in turn already forms the request itself, which I cannot see

Oops, sorry I forgot to change GET to POST, now everything works.
Thank you for your prompt help!


Lay groundwork for front-end testing.

Some parts of the website are stable enough for this to be possible.
Things to be tested:

[ ] Front page.
[ ] Guide creation flow
[ ] Garden management

This will be important:
https://github.com/wrozka/capybara-angular

Actually, can we use protractor? Should be fairly straight forward to set up.


React editor crashes when there is a null byte in the file

echo "\0" > nullbytefile && atom nullbytefile
crashes the textfield and opens the console, with a bunch of TypeError: Cannot read property 'textContent' of null errors, probably because it can't initialize the editor itself and tries it access it anyways. The original error is not shown, as far as I can tell.
TypeError: Cannot read property 'textContent' of null
  at [object Object].module.exports.React.createClass.measureCharactersInLine (/usr/local/share/atom/resources/app/src/lines-component.js:347:43)
  at [object Object].boundMethod [as measureCharactersInLine] (/usr/local/share/atom/resources/app/node_modules/react-atom-fork/lib/ReactCompositeComponent.js:1434:21)
  at [object Object].module.exports.React.createClass.measureCharactersInNewLines (/usr/local/share/atom/resources/app/src/lines-component.js:320:30)
  at [object Object].boundMethod [as measureCharactersInNewLines] (/usr/local/share/atom/resources/app/node_modules/react-atom-fork/lib/ReactCompositeComponent.js:1434:21)
  at [object Object].module.exports.React.createClass.componentDidUpdate (/usr/local/share/atom/resources/app/src/lines-component.js:95:21)
  at ReactMountReady.mixInto.notifyAll (/usr/local/share/atom/resources/app/node_modules/react-atom-fork/lib/ReactMountReady.js:69:18)
  at ReactReconcileTransaction.ON_DOM_READY_QUEUEING.close (/usr/local/share/atom/resources/app/node_modules/react-atom-fork/lib/ReactReconcileTransaction.js:87:26)
  at ReactReconcileTransaction.Mixin.closeAll (/usr/local/share/atom/resources/app/node_modules/react-atom-fork/lib/Transaction.js:242:42)
  at ReactReconcileTransaction.Mixin.perform (/usr/local/share/atom/resources/app/node_modules/react-atom-fork/lib/Transaction.js:175:16)
  at [object Object].ReactComponent.Mixin.performUpdateIfNecessary (/usr/local/share/atom/resources/app/node_modules/react-atom-fork/lib/ReactComponent.js:478:19)
  at [object Object].ReactCompositeComponentMixin.performUpdateIfNecessary (/usr/local/share/atom/resources/app/node_modules/react-atom-fork/lib/ReactCompositeComponent.js:1157:51)
  at Object.enqueueUpdate (/usr/local/share/atom/resources/app/node_modules/react-atom-fork/lib/ReactUpdates.js:107:15)
  at [object Object].ReactCompositeComponentMixin.forceUpdate (/usr/local/share/atom/resources/app/node_modules/react-atom-fork/lib/ReactCompositeComponent.js:1379:18)
  at [object Object].module.exports.React.createClass.requestUpdate (/usr/local/share/atom/resources/app/src/editor-component.js:917:21)
  at [object Object].boundMethod [as requestUpdate] (/usr/local/share/atom/resources/app/node_modules/react-atom-fork/lib/ReactCompositeComponent.js:1434:21)
  at [object Object].module.exports.React.createClass.componentDidMount (/usr/local/share/atom/resources/app/src/editor-component.js:175:19)
  at ReactMountReady.mixInto.notifyAll (/usr/local/share/atom/resources/app/node_modules/react-atom-fork/lib/ReactMountReady.js:69:18)
  at ReactReconcileTransaction.ON_DOM_READY_QUEUEING.close (/usr/local/share/atom/resources/app/node_modules/react-atom-fork/lib/ReactReconcileTransaction.js:87:26)
  at ReactReconcileTransaction.Mixin.closeAll (/usr/local/share/atom/resources/app/node_modules/react-atom-fork/lib/Transaction.js:242:42)
  at ReactReconcileTransaction.Mixin.perform (/usr/local/share/atom/resources/app/node_modules/react-atom-fork/lib/Transaction.js:175:16)
  at [object Object].ReactComponent.Mixin.mountComponentIntoNode (/usr/local/share/atom/resources/app/node_modules/react-atom-fork/lib/ReactComponent.js:537:19)
  at Object._renderNewRootComponent (/usr/local/share/atom/resources/app/node_modules/react-atom-fork/lib/ReactMount.js:309:25)
  at Object.ReactMount.renderComponent (/usr/local/share/atom/resources/app/node_modules/react-atom-fork/lib/ReactMount.js:359:32)
  at ReactEditorView.module.exports.ReactEditorView.afterAttach (/usr/local/share/atom/resources/app/src/react-editor-view.js:81:30)
  at callAttachHook (/usr/local/share/atom/resources/app/node_modules/space-pen/lib/space-pen.js:343:107)
  at [object Object].jQuery.fn.(anonymous function) [as append] (/usr/local/share/atom/resources/app/node_modules/space-pen/lib/space-pen.js:359:9)
  at PaneView.module.exports.PaneView.onActiveItemChanged (/usr/local/share/atom/resources/app/src/pane-view.js:297:24)
  at /usr/local/share/atom/resources/app/src/pane-view.js:3:61
  at /usr/local/share/atom/resources/app/node_modules/emissary/lib/emitter.js:133:30
  at Array.forEach (native)
  at Behavior.module.exports.Emitter.emit (/usr/local/share/atom/resources/app/node_modules/emissary/lib/emitter.js:132:47)
  at Behavior.module.exports.Behavior.emit (/usr/local/share/atom/resources/app/node_modules/emissary/lib/behavior.js:44:38)
  at Behavior.module.exports.Signal.emitValue (/usr/local/share/atom/resources/app/node_modules/emissary/lib/signal.js:82:19)
  at Pane.module.exports.Model.set (/usr/local/share/atom/resources/app/node_modules/theorist/lib/model.js:166:21)
  at Pane.accessor.set (/usr/local/share/atom/resources/app/node_modules/theorist/lib/model.js:57:23)
  at Pane.module.exports.Pane.addItem (/usr/local/share/atom/resources/app/src/pane.js:189:25)
  at Pane.module.exports.Pane.activateItem (/usr/local/share/atom/resources/app/src/pane.js:174:14)
  at /usr/local/share/atom/resources/app/src/workspace.js:214:16
  at _fulfilled (/usr/local/share/atom/resources/app/node_modules/q/q.js:787:54)
  at self.promiseDispatch.done (/usr/local/share/atom/resources/app/node_modules/q/q.js:816:30)
  at Promise.promise.promiseDispatch (/usr/local/share/atom/resources/app/node_modules/q/q.js:749:13)
  at /usr/local/share/atom/resources/app/node_modules/q/q.js:557:44
  at flush (/usr/local/share/atom/resources/app/node_modules/q/q.js:108:17)
  at process._tickCallback (node.js:605:11)
 /usr/local/share/atom/resources/app/src/workspace.js:223
Uncaught TypeError: Cannot read property 'textContent' of null /usr/local/share/atom/resources/app/src/lines-component.js:347
Uncaught TypeError: Cannot read property 'textContent' of null /usr/local/share/atom/resources/app/node_modules/react-atom-fork/lib/ReactMountReady.js:69
Uncaught TypeError: Cannot read property 'textContent' of null /usr/local/share/atom/resources/app/node_modules/react-atom-fork/lib/ReactMountReady.js:69
Uncaught TypeError: Cannot read property 'textContent' of null /usr/local/share/atom/resources/app/node_modules/react-atom-fork/lib/ReactMountReady.js:69

Yeah, that seems like a great approach. It's going to require some reworking of our rendering, so I'd like to do it once we complete the switch to React.

@nathansobo @luto Have you made progress on making the nulltype chars visible? I'm having weird errors where copy and paste introduces null types, which I then need to remove with a bash command:
tr < my_file.py -d '\000' > my_file2.py

Thanks.