x/tools/gopls: integrate test discovery

I would like to migrate the test discovery services of vscode-go into gopls so they can be implemented in Go and utilize the rich type information provided by the go/* packages instead of the current JavaScript implementation which relies on recursive directory walks and analyzing document symbols. Moving test discovery into gopls should improve performance and would bake it far easier to implement improvements such as detecting subtests in cases where that can be done deterministically.
CC @hyangah @suzmue
Related: https://github.com/golang/vscode-go/issues/2445, https://github.com/golang/vscode-go/issues/1602

@gopherbot, please add label FeatureRequest

@findleyr from an information perspective that would be sufficient if the code lens messages include the name of the test. Otherwise we'd still need document symbols or some other mechanism for extracting the test name. However it seems like that might be problematic from a usability perspective. A user may wish to use the test explorer exclusively and disable the test code lens - if the user disables test code lenses does that tell gopls to stop sending those messages?
Off the top of my head I think these are the components we need:

List packages within the workspace/module that contain tests/test files.

This seems like something best implemented as a separate call to gopls.

List files within a package that contain tests/are test files.

It's not hard to do this in VSCode/JS via filesystem calls but presumably gopls already has that information loaded into memory so it would be faster to ask gopls.
Could this be merged with (1)?

List tests within a file.

This requires the test name, the range in the file (of the TestXxx function or of the subtest callback), and whether it's a child of another test. Can we reasonably pack all of that into CodeLens.data?
This would involve interrogating files that are not open. How difficult is it to request code lenses for a file that has not been opened? What is the performance impact of doing so?

With respect to (1) and (2), testing for a _test.go suffix is sufficient (that's the current behavior) but it would be preferable to verify that the file actually contains tests. I sometimes have util_test.go files that contain shared test harnessing/utilities/etc and do not contain any tests. With the current behavior, those appear in the list and then disappear once they are expanded and discovered to contain no tests. If gopls can quickly and efficiently determine if a file contains tests, using gopls to implement (1) and (2) would improve the user experience.
A big motivation for moving (1) into gopls is to address performance issues on large projects (see https://github.com/golang/vscode-go/issues/2504). The current implementation recursively walks the filesystem, which does not perform well on large projects (and likely performs badly if the project is on a remote filesystem). I realize that gopls will also have to walk the filesystem, but it must already be doing that (to supply type information) and I theorize that using the build metadata gopls collects will perform much better than the current JS implementation.

@firelizzard18 thanks for the details.
It sounds like it may be cleanest to have a custom command, as code lenses in general require type information and we may be able to implement this query without fully type-checking. This command may also have a life-cycle that is different from code lenses, typically.
If we do choose to implement this with type information, we can extract test information as part of our package post-processing, and serialize them in our file cache, so that they will be quickly available at startup. This aspect of the change would be tricker for an external contributor.
I have slated this for<EMAIL_ADDRESS>as this keeps cropping up.

@findleyr I think we can separate the major areas of improvement into three groups:

Package discovery

The current implementation (walking the filesystem) is ok but it's not very clean and it performs badly for large projects.

Test discovery

The current implementation (using document symbols) is ok but it's not very clean.
I am pretty sure this can be done without type checking, all it really needs to do is locate func Test{name}(*testing.T) declarations.

Static subtest discovery.

This is not currently supported except in an approximate way for stretchr test suites.
It seems to me this would be much easier to implement using complete type information. There are some cases where it's straight-forward to identify subtests by walking the AST, but type information would be very useful for more complex cases.

I think (1) and (2) are orthogonal and could be implemented separately, if that is more convenient. Whether (3) is dependent on (2) depends on the implementation. (3) could be implemented as an improvement to the normal test discovery; alternatively it could be implemented as a second pass.
Static subtest discovery (3) is the part I find most technically interesting. To be clear I know it is impossible to statically discover all sub tests (the halting problem) so IMO the target should be "statically discover subtests in cases where it is feasible to do so" as a companion to dynamic subtest discovery (that is, analyzing test output). I have functionally zero knowledge of how gopls works internally, so maybe I should wait until the team has implemented some form of static subtest discovery and then I can provide contributions building on that.

Package discovery

The current implementation (walking the filesystem) is ok but it's not very clean and it performs badly for large projects.

Can this improvement be done in away that it considers (or at least leaves the option open for) an option to infer relevant packages from the user's current open workspace state?  As we were discussing on https://github.com/golang/vscode-go/issues/2504, to support a large monorepo environment it becomes not just an issue of indexing speed, but also of relevance.
When many teams use one monorepo, "Show everything" (the current behavior) becomes too broad and shows thousands of irrelevant packages, and switching to nested mode becomes very deep.  On the other hand, "Show only the package of the file I have open" may be too narrow, but is consistent, intuitive, and doesn't require the user to maintain any separate config with relevant paths.  Perhaps the solution could be something like limiting the display to packages within "x" levels of the current directory.

Detecting tests from the syntax alone is possible, but there's a lot of potential for doing better by using typed syntax to infer subtests, at least in the simple cases. It would make most sense architecturally for test inference to be a step executed just after type checking, like references/methodsets/xrefs, to populate an index of tests. This would avoid the potential for requests such as "run all the tests in this module" to reduce to "first type-check all the packages in this module, sequentially". An index-based  approach would make this operation  parallel, fast, and cached, just like a references query.

@adonovan I would be happy to work on subtest discovery once the initial implementation is in place. That seems like a rather interesting problem. I’m also interested in contributing to the initial implementation but it sounds like that might be challenging due to indexing/caching/etc.

Change https://go.dev/cl/548675 mentions this issue: gopls/internal/lsp: test discovery

@adonovan @findleyr How's this for a proof of concept? Issues:

It could be cleaner
gopls/fileTests is the right method to use for the request
We also need "get all tests for package X" and maybe a couple others

But it works (in the test environment) and it's a place to start.

Thanks @firelizzard18 for the CL! I will take a look at it tomorrow.

Note to self

For parity with the existing vscode-go feature set, gopls would need to recognize stretchr tests, otherwise transitioning to gopls-based discovery would lead to a feature regression. That might be ok, as long as its temporary.
Tests returned by gopls need metadata. Special cases like stretchr tests need to be identified, since isolating a specific stretchr test requires special flags.
Return values from gopls test discovery methods should include enough information (i.e. module info) to construct the entire test tree.
vscode-go's test controller will need significant changes to switch to gopls-based test discovery. The controller should support discovery via expanding test explorer nodes, or opening files, or both. If the user does not expand any nodes or has that feature disabled and opens a file with tests, the controller should build the sub-section of the tree required to hold the tests in the file (and opening more files in the same or different packages/modules should work as expected).
Combining the last two points, it should be possible to configure vscode-go to only discover tests in open files, in packages with open files, or modules with open files.
Dynamically discovered sub-tests should be specifically marked, so that they can be cleared separately from statically discovered sub-tests.
Should dynamically discovered sub-tests should be purged when a test is run, instead of any time the file is updated (current behavior, which is can be annoy)?
Dynamic sub-test discovery needs work. Instead of splitting on certain characters (e.g. slashes):

When the test start event for a/b/c is received, search for tests matching /a(\/b?)?/
Attach a/b/c as a child to the test with the longest prefix
If no test is a prefix, add it to the package as an orphan?

@hyangah and I discussed the protocol and agreed that the API should be sufficient for a starting point. The Packages method below would be a new command (see command.Interface in gopls), and its implementation would require type checking (similar to https://go.dev/cl/548675).
// The Packages command returns information about the set of Go packages
// associated with the specified files or directories, plus additional information such
// as the names of any tests within them.
func Packages(PackagesParams) (Packages, error)

type PackagesParams struct {
	File []protocol.URI // file or directory (directories are recursively enumerated)
}

type PackageTree struct {
        Packages []Package
}

// Package describes a directory that is a Go package (not an empty parent).
type Package struct {
	Path string // Go package path
	Dir protocol.URI // package directory
        Root protocol.URI // root of view folder (an enclosing directory)

	// TestFiles describes each file that defines a test, and the names of those tests.
        TestFiles []TestFile  // may be empty
}

type TestFile struct {
	File  protocol.URI // a *_test.go file
	Tests []Test
}

// Test represents a tree of test suites and test cases,
// preserving the structure of Go subtests, where possible.
type Test struct {
	// Name is the complete name of the test as it appears in the output of go test.
	// The server may attempt to infer names of subtests by static
        // analysis; if so, it should aim to simulate the actual computed
        // name of the test, including any disambiguating suffix such as "#01".
        // Clients should quote this name using "^" + regexp.QuoteMeta(Name) + "$"
        // before passing it to go test -test.run=...
	Name string // e.g. "TestToplevel/Inner"

	// Range enclosing the entire test. This is used to place
        // the gutter marker and group tests based on location.
        // For subtests whose test names can be determined statically,
        // this can be either t.Run or the test data table
        // for table-driven setup.
	Range protocol.Range

	// Nested subtests.
	// Note that this is not an exhaustive list.
	// For example, if subtests are dynamically generated or
	// their names cannot be statically determined, those tests
	// may not be included in this list.
	Subtests []Test
}

@adonovan @hyangah I think recursion should be optional. There are scenarios where recursion is helpful, and others where it is not. Especially for large workspaces (https://github.com/golang/go/issues/59445#issuecomment-1499488301, https://github.com/golang/vscode-go/issues/2504), the user may not want all tests to appear, even if gopls's performance is acceptable. The scenarios I see are:

Small workspace/user wants tests to be eagerly loaded:

Call Packages with the list of workspace roots, with recursion.

Medium workspace/user wants tests to be lazily loaded:

List workspace roots as test items, but do not populate.
When the user expands a workspace root test item, call Packages with that root, with recursion.

Large workspace/user only wants tests loaded for packages they're working on:

Do not populate the test view at all, until the user opens a file.
When the user opens a file, call Packages with that file or its directory ("show tests for open files" vs "show tests for packages with open files"), without recursion.

It might be better to use modules as the roots, instead of workspace folders. That would require some "find all Go modules given these workspace folders" logic, which would be easier with gopls than with JavaScript. But it might also make sense just to use workspace folders, since that will mirror the folder structure.

Thanks @firelizzard18
Here is a little bit of the background behind the API sketch @adonovan shared.
The upcoming zero-config gopls work (https://github.com/golang/go/issues/57979) will introduce a significant change in how gopls models a workspace. Basically, gopls won't create a view until users open a certain go-related files (go.* files, *.go files, go template files). If a go-related file opens, then gopls attempts to find the appropriate workspace boundary for it. That can be a module root, or a workspace root, or a gopath directory or the current directory. It solves many scalability and performance issues. A similar model had been used in other editors and we think the experience will be better in a large workspace with multiple modules, or a workspace with go code in nested modules.
With this new model, we consider to adjust vscode-go test explorer's UX a bit. Gopls doesn't analyze modules or packages until the corresponding views about them are created. Test explorer needs to honor that too, instead of attempting to eagerly load all modules and packages in the workspace. (Otherwise, either gopls won't return anything, or trigger creating views unnecessarily and cancelling out the benefit of zero config design).
We originally attempted to create package trees with multiple view roots. But we are afraid that leaks too much of the gopls's internal data structure. The API in https://github.com/golang/go/issues/59445#issuecomment-1856911780 instead attempts to simplify the problem by returning a flat package list. If the client wants hierarchical presentation, it can still reconstruct it based on the package paths and the package directory paths & root (the view folder).
The three cases you listed were also discussed, and we think a client can implement the three cases with the API.
PackagesParams's File is meant to be all the open files.

Eagerly loading everything under the editor's workspace folder: this involves the editor plugin (vscode-go) to walk and find all go.mod files and opens them. That will let gopls load corresponding views. Then, query with the list of all open files.

Lazily package loading: that's the default behavior. If the client wants to show only the tests in the subfolders of the currently open files, it can still do so by just scanning the returned Package list and filtering out based on the package directory and root info when creating Test UI API TestItems.

Lazy test loading: the client can do filtering based on the package directory info and show only what users want.

For subtests, we try to preserve the hierarchies found from the source code. However, as you also pointed out, "/" in the test name is not escaped, so reconstructing the hierarchy for dynamically discovered subtests reliably is still difficult. I think for now, vscode-go test explorer can place all the dynamically discovered subtests under the test function.
I was originally afraid of the message sizes (and client-side overhead). But from a quick analysis of some large repos, the volume of returned results won't be too bad compared to other LSP messages. I hope we can try with this minimal API, and then if that becomes an issue we can consider to include some filters for optimization.

@hyangah The API sketch @adonovan shared looks good to me. I only have two very minor reservations:

Recursive subtests (Test.Subtests). I think it might be more natural to expose subtests as just another test and let the caller/vscode-go decide how to present them. Especially for weird cases like stretchr test suites, my gut feeling is that the recursive structure could get in the way. But I might be totally wrong, and we can always change it.
Modules. The proposed API (and currently my implementation) pretty much totally ignores modules. That might be fine, but it might not be the desired UX in some cases. For example, if a directory is loaded that contains a nested submodule, I believe the current implementation will ignore that and treat it as just another package. In that case, the user may wish the nested module to be displayed as a separate root in the test explorer. We can make that the client's problem, but it would be easier for the client to do if Package included the module path.

For visibility, here's a WIP CL for integrating this into vscode-go: https://go-review.googlesource.com/c/vscode-go/+/550555. It's the minimum viable change to switch to using gopls for test discovery. I plan to do more, but the full integration will require a lot of refactoring so I thought it would be best to do the basic integration in a small CL.

Thanks for the investigation and prototype @firelizzard18

Recursive subtests: I still think placing all subtests under a test under their enclosing test function (func Test*) seems natural. Whether to lay out subtests in a test function hierarchically is questionable, and I think it's ok to flatten them as a list. A nice thing about stretchr is that its framework is now built around go's standard subtest structures. So, instead of handling it to invoke -testify.m flags, I think we should construct the test items just using the go's standard subtest naming scheme. That way, we can avoid a dangling test item issue (see the picture).

Modules: the idea is the Dir and the Root in the result Package provides the information about which module the package belongs too. Currently, we are still refining the meaning of the 'Root' though. So the protocol proposal we sketched may change.

Currently all of our team members are busy finishing up planned work for 2023. We are planning to work on testing UX in 2024 H1 (as part of the planning discussion we revived this thread).

@hyangah I agree that subtests should be displayed nested - I was thinking purely about the backend implementation. vscode-go needs a function for "find the parent of this subtest given the subtest name" for dynamic subtests. So vscode-go could easily handle receiving a flattened list of subtests from gopls; it would just look for a slash in the name and then run it through that function.
From a backend implementation POV, I am thinking flattening the gopls.packages command response might make it easier to implement, since it reduces the data structure complexity.

A nice thing about stretchr is that its framework is now built around go's standard subtest structures. So, instead of handling it to invoke -testify.m flags, I think we should construct the test items just using the go's standard subtest naming scheme.

I didn't realize that was an option, though I guess I just assumed I had to use -testify.m since that's what vscode-go was doing before.

That way, we can avoid a dangling test item issue (see the picture).

This has been a continual irritation to me and is definitely something I intend to eliminate. The existing code is supposed to realize that the dynamically discovered test matches the static one, but that obviously isn't working.

Modules: the idea is the Dir and the Root in the result Package provides the information about which module the package belongs too. Currently, we are still refining the meaning of the 'Root' though. So the protocol proposal we sketched may change.

I'll ignore it for now; we can always add a new field later if it makes sense.

We are planning to work on testing UX in 2024 H1 (as part of the planning discussion we revived this thread).

@hyangah @adonovan Any ETA on when you'll be ready to look at this issue again?

@hyangah @adonovan?

Sorry for the delay. As you noticed our team was busy closing gopls v0.15.x bugs and handling regressions related to the latest vscode release. I think we need more time to think about the problems 1 and 2 in your problem description  https://github.com/golang/go/issues/59445#issuecomment-1497972638 (affected by gopls's zero config change, and it's very vscode-specific). However, the problem 3 is concrete enough. I will get back to you early next week.

@firelizzard18 @adonovan @findleyr
I have the updated API sketch in cl/579438

https://github.com/golang/go/issues/59445#issuecomment-1857070266 discusses the need for recursion.
→ Added Recursive bool field in PackagesArgs

https://github.com/golang/go/issues/59445#issuecomment-1858588187 commented about the subtest hierarchy and the module information.
→ Eliminated the hierarchy.  I hoped this could help mitigate issues like https://github.com/golang/vscode-go/issues/3286 but if this cannot help if the subtests cannot be statically determined anyway.
→ Added Module field in PackagesResults. Also clarified that Packages command cannot go past a module boundary.  The client has to make a separate Packages call for each module.
→ Added the Modules command to help module discovery. This can return all modules in a multi-module repo even when they are not in gopls's memory yet. The implementation should make sure not to trigger new modules loading.

Testify case is a bit more complicated. I hope we can eliminate the need for use of -testify.m, but we will learn more. The use of RunCmd protocol.Command in @firelizzard18 's cl/579438 is interesting. If we are open to the idea of teaching gopls to understand testify's flag structure, we can potentially encode the info in RunCmd.
Let's discuss details in cl/579438.


[Server] memory mismanagement leads to crashing after placing blocks.

Issue
When placing blocks, the WorldQL server crashes immediately.
Video
https://discord.com/channels/701873003679449108/701876824732598414/885880438978138112
GDB Output
GNU gdb (Ubuntu 9.2-0ubuntu1~20.04) 9.2
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./WorldQLServerUbuntu...
(No debugging symbols found in ./WorldQLServerUbuntu)
(gdb) set args -server
(gdb) run
Starting program: /home/myokan/worldql/server/WorldQLServerUbuntu -server
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
WorldQL Server 0.0.1 initializing...
[New Thread 0x7ffff6817700 (LWP 11328)]
Connecting to PostgreSQL...
[New Thread 0x7ffff6016700 (LWP 11329)]
[New Thread 0x7ffff5815700 (LWP 11330)]
[New Thread 0x7ffff5014700 (LWP 11331)]
[New Thread 0x7ffff4813700 (LWP 11332)]
[New Thread 0x7fffeffff700 (LWP 11333)]
[HANDSHAKE] Server thread starting...
[HANDSHAKE] Server thread listening...
Connection to database succeeded.
[MAIN] Server thread starting...
[HANDSHAKE] Got handshake from new client <IP_ADDRESS>
[HANDSHAKE] Successfully assigned client tcp://<IP_ADDRESS>:29900
[HANDSHAKE] Server thread listening...
[HANDSHAKE] Got handshake from new client <IP_ADDRESS>
[HANDSHAKE] Successfully assigned client tcp://<IP_ADDRESS>:29901
[HANDSHAKE] Server thread listening...
Block place
terminate called after throwing an instance of 'std::length_error'
  what():  basic_string::_M_create

Thread 1 "WorldQLServerUb" received signal SIGABRT, Aborted.
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
50      ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) backtrace
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x00007ffff7ad0859 in __GI_abort () at abort.c:79
#2  0x00007ffff7d56911 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#3  0x00007ffff7d6238c in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#4  0x00007ffff7d623f7 in std::terminate() () from /lib/x86_64-linux-gnu/libstdc++.so.6
#5  0x00007ffff7d626a9 in __cxa_throw () from /lib/x86_64-linux-gnu/libstdc++.so.6
#6  0x00007ffff7d59326 in std::__throw_length_error(char const*) () from /lib/x86_64-linux-gnu/libstdc++.so.6
#7  0x0000555555565322 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_create(unsigned long&, unsigned long) ()
#8  0x0000555555567154 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_mutate(unsigned long, unsigned long, char const*, unsigned long) ()
#9  0x000055555556506a in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_append(char const*, unsigned long) ()
#10 0x0000555555562bfd in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::append(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) ()
#11 0x0000555555561bdc in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > std::operator+<char, std::char_traits<char>, std::allocator<char> >(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) ()
#12 0x00005555555597c0 in MinecraftBlockPlace::handleMinecraftBlockPlace(WorldQLFB::StandardEvents::Update const*, std::vector<zmq::socket_t, std::allocator<zmq::socket_t> >&, std::vector<int, std::allocator<int> >&, pg_conn*) ()
#13 0x000055555555ad10 in MainServer::runServer(zmq::context_t&, std::vector<zmq::socket_t, std::allocator<zmq::socket_t> >&, std::vector<int, std::allocator<int> >&, pg_conn*) ()
#14 0x000055555555cab9 in main ()

It’s an invalid string error in C++, length error. Potentially part of the code attempting to accessing a character in a string that does not exist or is out of range.
I’ll take a look at the code later today, it’s probably just a typo.

@Jaxkr Where is the repository for WorldQL? This is the mammoth repo.

After further investigation, this particular issue only happens when the worldql_record_blocks table does not exist. When the table exists, this is replaced by a bad_alloc crash:
[HANDSHAKE] Server thread listening...
Block break
NoRepeat.BlockBreak
From: 29900
Instruction not repeat? 1
Block place
terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc

Thread 1 "WorldQLServerUb" received signal SIGABRT, Aborted.
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
50      ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) backtrace
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x00007ffff7ad0859 in __GI_abort () at abort.c:79
#2  0x00007ffff7d56911 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#3  0x00007ffff7d6238c in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#4  0x00007ffff7d623f7 in std::terminate() () from /lib/x86_64-linux-gnu/libstdc++.so.6
#5  0x00007ffff7d626a9 in __cxa_throw () from /lib/x86_64-linux-gnu/libstdc++.so.6
#6  0x00007ffff7d56522 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#7  0x0000555555568672 in __gnu_cxx::new_allocator<char>::allocate(unsigned long, void const*) ()
#8  0x0000555555567318 in std::allocator_traits<std::allocator<char> >::allocate(std::allocator<char>&, unsigned long) ()
#9  0x00005555555653a3 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_create(unsigned long&, unsigned long) ()
#10 0x0000555555567154 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_mutate(unsigned long, unsigned long, char const*, unsigned long) ()
#11 0x000055555556506a in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_append(char const*, unsigned long) ()
#12 0x0000555555562bfd in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::append(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) ()
#13 0x0000555555561bdc in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > std::operator+<char, std::char_traits<char>, std::allocator<char> >(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) ()
#14 0x00005555555597c0 in MinecraftBlockPlace::handleMinecraftBlockPlace(WorldQLFB::StandardEvents::Update const*, std::vector<zmq::socket_t, std::allocator<zmq::socket_t> >&, std::vector<int, std::allocator<int> >&, pg_conn*) ()
#15 0x000055555555ad10 in MainServer::runServer(zmq::context_t&, std::vector<zmq::socket_t, std::allocator<zmq::socket_t> >&, std::vector<int, std::allocator<int> >&, pg_conn*) ()
#16 0x000055555555cab9 in main ()

@Jaxkr Where is the repository for WorldQL? This is the mammoth repo.

Unfortunately, the worldql server is closed-source, making this significantly more infuriating.

@Jaxkr Where is the repository for WorldQL? This is the mammoth repo.

Unfortunately, the worldql server is closed-source, making this significantly more infuriating.

How are we supposed to contribute to WorldQL then? The plugin being open source means nothing, it’s just a connector. It’s small beans compared to WorldQL.
We can’t modify Mammoth without knowing the underlying architecture of WorldQL.

WorldQL has been completely rewritten in Rust to avoid memory safety issues like this from ever occuring again.


AF when requesting help on keyword

Right click on 'while' in the editor and choose 'Help on while' (eval calls ?while). appears to be ignorable.

Please verify

verified


Explicitly set the OptiX pipeline stack size.

Description
testrender and testshade were both relying on the automatic pipeline stack size computation in OptiX 7. This was causing a handful of tests to fail for reasons that are still not clear. Computing and manually setting the required stack size clears up the failures.
Tests
I didn't add any tests, but all existing tests pass with this change.
Checklist:

[x] I have read the contribution guidelines.
[x] I have previously submitted a Contributor License Agreement.
[x] I have updated the documentation, if applicable.
[x] I have ensured that the change is tested somewhere in the testsuite (adding new test cases if necessary).
[x] My code follows the prevailing code style of this project.

I've tried out this patch locally and it fixes the issues I was seeing with recent drivers.
LGTM!


documentReference.setData to assign a random ID to the document reference

It would be great upon documentRefeference.setData, the document reference has a random String ID within the collection.
My expectation in the following test:
import 'package:cloud_firestore_mocks/cloud_firestore_mocks.dart';
import 'package:flutter_test/flutter_test.dart';

void main() {
  testWidgets('documentRefeference.setData should set documentId',
      (WidgetTester tester) async {
    final fakeFirestore = MockFirestoreInstance()..setupFieldValueFactory();

    final documentRef1 = fakeFirestore
        .collection('users')
        .document();
    await documentRef1.setData(<String, dynamic>{
      'content': 'MockFirestoreInstance is great',
      'createdBy': 'Tomo',
    });

    expect(documentRef1.documentID, isNotNull); // This fails.

    final documentRef2 = fakeFirestore
        .collection('users')
        .document();
    await documentRef2.setData(<String, dynamic>{
      'content': 'Mock tests are good',
      'createdBy': 'Joichiro',
    });
    expect(documentRef2.documentID, isNotNull);

    expect(documentRef2.documentID, isNot(documentRef1.documentID));
  });
}

But sometimes there isn't a meaningful ID for the document, and it's more convenient to let Cloud Firestore auto-generate an ID for you. You can do this by calling add()

https://firebase.google.com/docs/firestore/manage-data/add-data

You're right, a semi-random id is created in one of two cases:

collectionRef.add(): already roughly supported at https://github.com/atn832/cloud_firestore_mocks/blob/878187ec0d94c26f921196b4f53317305db781f4/lib/src/mock_collection_reference.dart#L43
collectionRef.document(): not supported yet

https://pub.dev/documentation/cloud_firestore/latest/cloud_firestore/CollectionReference/document.html:

DocumentReference document (
[String path]
)
If no path is provided, an auto-generated ID is used.

To fix this issue, we should make it so that mockCollectionRef.document() generates some ID, then returns a MockDocumentRef for that ID, perhaps something like:
  @override
  DocumentReference document([String path]) {
    if (path == null) {
      path = // make some id
    }
    return MockDocumentReference(path, getSubpath(root, path), root,
        getSubpath(snapshotStreamControllerRoot, path));
  }

Do you have such a use case in your projects? I have never run into it because I always use add().

The test case reflects my use case. I don’t call “add” when setData is enough.
While it’s feasible to change my code, I prefer cloud_firestore_mocks behaves the same as real firestore as much as possible.

In real Firestore, document('nonexistent key').get returns a future of documentSnapshot but the document is not in Firebase Database.
    final x = await _personsCollection.document('nonexistentkey').get();


SXL 1.2

The draft version can be viewed here: https://rsmp-nordic.github.io/rsmp_specifications/rsmp_sxl_traffic_lights/1.1.1

[x] https://github.com/rsmp-nordic/rsmp_sxl_traffic_lights/pull/165
[x] #170
[x] #145 and https://github.com/rsmp-nordic/rsmp_sxl_traffic_lights/issues/110
[x] https://github.com/rsmp-nordic/rsmp_schema/pull/68
[x] https://github.com/rsmp-nordic/rsmp_sxl_traffic_lights/issues/111
[x] https://github.com/rsmp-nordic/rsmp_sxl_traffic_lights/issues/117
[x] https://github.com/rsmp-nordic/rsmp_sxl_traffic_lights/issues/104
[x] #171
[x] JSON examples needs to be updated

Renamed to SXL 1.2 as decided by https://github.com/orgs/rsmp-nordic/discussions/26

Branch renamed to 1.2, so this pull request needs to closed and a new one created to replace this one.


[dvsim] ip_template sec_cm testplan error

In this PR we have an CI failure: https://github.com/lowRISC/opentitan/pull/13446
I directly edited the sec_cm_testplan to match the actual test.
I think when I use make hw, the sec_cm_testplan will be override to original template again.
Maybe a better solution is to auto-generate the testplan under ip_template/alert_handler/data, then use topgen.py to copy this file over to top_earlgrey/ip_autogen/alert_handler/data.

Yeah - that would be the right way to fix the CI failure.

great! Thanks for confirming! PR should be out soon!


Numbers in the paper

Hi, thanks for your great work!
Is it possible to release the numbers of your R-D curve in your paper, Figure 6, like what CompressAI does? It would help us a lot to study and learn from your work.

Sorry for late, the RD points have been updated just now.


bam_vs_bed: split alignments

bam_vs_bed currently treats alignments as starting at the start location and continuing to the end location, irrespective of if the alignment is split (/spliced).
Adding split to the bedtools intersect call will treat the two halfs of aligment as seperate intervals, and thus will not count towards anything in the spliced out region. However, this would mean the read might be counted twice.
Suggestions?

Thorough counting in an RNA-seq context will be hard without redesigning the counting from scratch.
In the meantime, we can enable the option to "--split" - and add to the documentation that this might cause one read to be counted multiple times.


Use GitHub CI for x86_64 build

GitHub now offers free CI service via Actions. It's faster and has better integration with GitHub. As of this time, ubuntu-latest (18.04 LTS) and macos-latest (Catalina 10.15) are both supported. See https://help.github.com/en/actions/reference/virtual-environments-for-github-hosted-runners for details. Free users can run up to 20 concurrent jobs for Ubuntu and 5 for Mac. This is already more than enough. See https://help.github.com/en/actions/getting-started-with-github-actions/about-github-actions#usage-limits for details.
Below are the jobs for this ci.yml:

ubuntu-build-test:
OS: ubuntu-latest (latest LTS)
Steps:
GCC build
make test
make test_tls
macos-build-test
OS: macos-latest
Steps:
Clang build
make test || true (intentional || true. Currently test is failing on Mac)
make test_tls || true (intentional || true. Currently test is failing on Mac)
debian-build (Build check against latest GCC and deps)
OS: debian:testing container
Steps:
GCC build
No Tests. Build Only.

Advantages:

Faster and better integration with GitHub.
Devs can immediately check for build errors on a clean CI environment after push without waiting for PR. Results can be checked on their GitHub Actions page or the commit status.

Disdvantages (as of this time):

No support for arm64 that's why Travis is retained.
Cannot manually start build without action (e.g. push, PR).

See https://github.com/jefyt/memcached-windows/actions/runs/81668653 for the runner result of this PR's push. Please take note that test (e.g. t/expirations.t) is failing on Mac and OK status is intentionally set. This needs separate defect investigation and not CI issue itself.

Thanks! Having mac is neat.. but I really don't want three builder systems :/ Most of my bugs have been an issue of the builders not being wide enough.

I can close this PR or continue with just the macOS build. What do you think?

why no tests for the debian-testing build?
I'm getting pretty far along with my buildbot redo (https://build.memcached.org/) - it's structured to run a "fast" test (~5m at most) on a host (ubuntu 20.04) with compile/test parallelism, and if that passes it runs a full gamut of platforms... which I will have to build out over time. This will include an ARM, but I don't have an OS X one unless someone sets one up or donates.
The problem is I can't securely (not yet, anyway) throw tests at random PR's immediately. I'll build out a tool for me to label/mark/something on PR's that'll run the buildbots after a quick code review and then report back via comment/status, but it won't be immediate feedback.
So, thinking of trying this out; disabling travis and using just the latest + OS X builds from this one. Travis is currently 30+ minute delay and the ARM platform is broken on and off. If this is faster people can still get quickish feedback.
I strongly dislike the idea that people don't run tests before pushing code (and therefore full tests after a quick review would be fine), but what can I do :P

why no tests for the debian-testing build?

I just thought you prefer only Ubuntu build-test based on the current Travis setup. Adding tests for debian-testing is not a problem although based on #659, the TLS tests will surely fail.

I strongly dislike the idea that people don't run tests before pushing code (and therefore full tests after a quick review would be fine), but what can I do :P

If I understood your comments correctly, you prefer this:

Completely disable Travis both for ARM and x86_64
Enable Ubuntu build+tests
Enable Debian build+tests (currently failing for TLS)
Enable macOS build+tests (currently failing)

Please confirm. Thanks!

Completely disable Travis both for ARM and x86_64
yes
Enable Ubuntu build+tests
yes
Enable Debian build+tests (currently failing for TLS)
no
Enable macOS build+tests (currently failing)
yes

bonus points for fixing the mac issue :)

Please confirm again the latest branch. Travis CI is now removed. See https://github.com/jefyt/memcached-windows/actions/runs/92925715 for the result.
Please take note of the following test failures on macOS:

testapp non-TLS: test stopped after ok 54 - binary_illegal, most likely caused by test_binary_pipeline_hickup?
testapp TLS: OK
t/idle-timeout.t and t/ssl_cert_refresh.t: NG

OS X is always clang? There're a few people who've been reporting test issues on OS X (@devnexen ?) but I haven't heard anything about those tests failing. Makes me a bit concerned about the platform.

You can usee gcc from homebrew too. Finally I still get the issue I mentioned the other day but it is not consistent seemingly, I also get abort raised but not the same test as this travis build.
f 5 frame #5: 0x0000000100001a9e testappcache_redzone_test at testapp.c:228:5
225 	    char *p = cache_alloc(cache);
226 	    char old = *(p - 1);
227 	    *(p - 1) = 0;
-> 228 	    cache_free(cache, p);
229 	    assert(cache_error == -1);
230 	    *(p - 1) = old;
231
`
`ok 25 - binary_set
Failed to read: Connection reset by peer
Process 26772 stopped

thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGABRT
...
f 3
frame #3: 0x0000000100006cc9 testappsafe_recv(buf=0x00007ffeefbfecb8, len=24) at testapp.c:955:17  952 	        if (nr == -1) {  953 	            if (errno != EINTR) {  954 	                fprintf(stderr, "Failed to read: %s\n", strerror(errno)); -> 955 	                abort();  956 	            }  957 	        } else {  958 	            if (nr == 0 && allow_closed_read) { (lldb) f 4 frame #4: 0x0000000100005d6b testappsafe_recv_packet(buf=0x00007ffeefbfecb8, size=1024) at testapp.c:972:10
969 	static bool safe_recv_packet(void *buf, size_t size) {
970 	    protocol_binary_response_no_extras *response = buf;
971 	    assert(size > sizeof(*response));
-> 972 	    if (!safe_recv(response, sizeof(*response))) {
973 	        return false;
974 	    }
975 	    response->message.header.response.keylen = ntohs(response->message.header.response.keylen);
`

@devnexen do you get any of these other failures jefyt mentions though?

nope I do not get these.

not sure I'm thinking straight but maybe kill the OS X bits (comment them out?). Seems like it has a delta with real users and I want this github bit to be an immediate sanity check for pull requests more than anything else right now.
On that note, you might try setting PARALLEL=5 or so for the make test suite. If there isn't much RAM available 2-4 might work, as that cuts the test suite time down nearly linearly.

On that note, you might try setting PARALLEL=5 or so for the make test suite. If there isn't much RAM available 2-4 might work, as that cuts the test suite time down nearly linearly.

Okay, I'll try PARALLEL=5 first then will just lower if needed.

https://github.com/jefyt/memcached-windows/actions/runs/93396403 contains latest build.

gcc --version outputs Apple clang version 11.0.3 (clang-1<IP_ADDRESS>).
macOS test is inconsistent compared with the last run, now it stopped after ok 42 - binary_incr

Please confirm again the modifications.

did you try OS X without TLS at all? on new enough systems the TLS bits are causing it to hang forever in testapp sometimes.
just one last sanity check for that... but I think we should drop OS X and just have the one quick test.

did you try OS X without TLS at all? on new enough systems the TLS bits are causing it to hang forever in testapp sometimes.

memcached no TLS support: NG.
memcached with TLS support: NG. It test stops before executing with TLS.

just one last sanity check for that... but I think we should drop OS X and just have the one quick test.

Should we drop OS X build and/or test check?

Yeah drop the whole OS X tester. I don't want to debug that as it doesn't seem to match what users have. Can look at adding it back, perhaps if we get a user who's invested enough in ensuring it works :) I don't have the spare time.

Please confirm again the modifications. See https://github.com/jefyt/memcached-windows/actions/runs/94352395 for the latest log. It took just < 4 mins to complete the job.

With PARALLEL=5 this test failed here on my ARM64 VM:
===(   81975;131  18088/?  24/?   28/130  4/?  1/? )====================
#   Failed test 'client cert is verified'
#   at t/ssl_verify_modes.t line 20.
#          got: undef
#     expected: '1'
# Looks like you failed 1 test of 1.
t/ssl_verify_modes.t ........ Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/1 subtests

Same error with PARALLEL=4.
It works fine with PARALLEL=3!

Same error with PARALLEL=4.
It works fine with PARALLEL=3!

@martin-g I also confirmed this error on GitHub CI environment. See https://github.com/jefyt/memcached-windows/runs/643275527?check_suite_focus=true. PARALLEL=5 make test_tls failed. t/ssl_session_resumption.t (Wstat: 256 Tests: 8 Failed: 1).
As requested, this PR's latest branch removed make test_tls that's why no error.

Excellent, thanks! Sorry I basically cut this down to nothing. Getting a result in 4 minutes instead of 30+ will be a big help.

Out in 1.6.6


[Fix]REST API errors while opening API Docs

Added additional check of if not self.request.user.is_anonymous so that opening API Docs doesn't throw error.
Fixes #291

Coverage decreased (-0.1%) to 97.634% when pulling be4daeff1e5a0da9d7ade3367ec3b171f6117a69 on aagman945:issue291 into 554086cabd902a45433e4f32fa155c5566a3268e on openwisp:master.


longRunningRecognize support

I have a project that is suitable for longRunningRecognize, for example, transcribe the whole class lesson.
How difficult is it to add this feature?  I think streaming will work for my purpose but I wonder if I can use longRunningRecognize in the future.

Since I wanted to complete the package bit by bit, this feature is of course on my list. Now that there is a real use for it, I will try to include this feature next. I can't estimate how long it will take, though.

Thats a real need too felix

I could really use the longRunningRecognize for a project I am working on as well. It would be great if you could add it @felixjunghans.

I'm hoping #26 is all that's needed to get this working.

Thanks @spenceralbrecht for your work. I released a new version (v.2.1.0) with longRunningRecognize support

Thanks @spenceralbrecht for your work. I released a new version (v.2.1.0) with longRunningRecognize support

Thank you! 💪

@felixjunghans amazing job at checking off the roadmap items 💪
thanks to @spenceralbrecht too
I wanted to try out the longRunningRecognize feature and I am getting a bit stuck on reading the results of the transcription.
 final responseLongRunning = await speechToText
        .longRunningRecognize(
            RecognitionConfigBeta(
              encoding: AudioEncoding.LINEAR16,
              model: RecognitionModel.basic,
              enableAutomaticPunctuation: true,
              sampleRateHertz: 16000,
              languageCode: 'en-US',
              useEnhanced: false,
            ),
            'gs://testFile/New Recording 6.m4a')

now this essentially returns an operation. and I can get the name of the operation by
responseLongRunning.name
but is it now expected to make a get request using this name or is there a more obvious way of accessing the alternatives.first.transcript value ?

@felixjunghans amazing job at checking off the roadmap items 💪 thanks to @spenceralbrecht too I wanted to try out the longRunningRecognize feature and I am getting a bit stuck on reading the results of the transcription.
 final responseLongRunning = await speechToText
        .longRunningRecognize(
            RecognitionConfigBeta(
              encoding: AudioEncoding.LINEAR16,
              model: RecognitionModel.basic,
              enableAutomaticPunctuation: true,
              sampleRateHertz: 16000,
              languageCode: 'en-US',
              useEnhanced: false,
            ),
            'gs://testFile/New Recording 6.m4a')

now this essentially returns an operation. and I can get the name of the operation by responseLongRunning.name
but is it now expected to make a get request using this name or is there a more obvious way of accessing the alternatives.first.transcript value ?

Your request willl return an operation object that will have a property "done" which you will need to check in order to access the "response" property of the operation properly. See https://pub.dev/documentation/google_speech/latest/generated_google_longrunning_operations.pb/Operation-class.html.


Deepstack model error

Love the new rewrite!  I'd been using v1 for some time and just moved over to v2 container recently.  Great stuff!
The recent addition of the deepstack models does not seem to be working.  I see in issue #28 after it was closed that there's a report that trying to get a deepstack model working resulted in a error.  I'm getting the same error with two different .pt models
Error:
2022-03-12 21:30:35,452 - uvicorn.access - INFO - <IP_ADDRESS>:36888 - "POST /detect HTTP/1.1" 500
2022-03-12 21:30:35,452 - uvicorn.error - ERROR - Exception in ASGI application
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/uvicorn/protocols/http/h11_impl.py", line 373, in run_asgi
    result = await app(self.scope, self.receive, self.send)
  File "/usr/local/lib/python3.8/dist-packages/uvicorn/middleware/proxy_headers.py", line 75, in __call__
    return await self.app(scope, receive, send)
  File "/usr/local/lib/python3.8/dist-packages/fastapi/applications.py", line 208, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/local/lib/python3.8/dist-packages/starlette/applications.py", line 112, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.8/dist-packages/starlette/middleware/errors.py", line 181, in __call__
    raise exc
  File "/usr/local/lib/python3.8/dist-packages/starlette/middleware/errors.py", line 159, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.8/dist-packages/starlette/middleware/base.py", line 57, in __call__
    task_group.cancel_scope.cancel()
  File "/usr/local/lib/python3.8/dist-packages/anyio/_backends/_asyncio.py", line 574, in __aexit__
    raise exceptions[0]
  File "/usr/local/lib/python3.8/dist-packages/starlette/middleware/base.py", line 30, in coro
    await self.app(scope, request.receive, send_stream.send)
  File "/usr/local/lib/python3.8/dist-packages/starlette/exceptions.py", line 82, in __call__
    raise exc
  File "/usr/local/lib/python3.8/dist-packages/starlette/exceptions.py", line 71, in __call__
    await self.app(scope, receive, sender)
  File "/usr/local/lib/python3.8/dist-packages/starlette/routing.py", line 656, in __call__
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.8/dist-packages/starlette/routing.py", line 259, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.8/dist-packages/starlette/routing.py", line 61, in app
    response = await func(request)
  File "/usr/local/lib/python3.8/dist-packages/fastapi/routing.py", line 226, in app
    raw_response = await run_endpoint_function(
  File "/usr/local/lib/python3.8/dist-packages/fastapi/routing.py", line 159, in run_endpoint_function
    return await dependant.call(**values)
  File "/opt/doods/api.py", line 40, in detect
    detect_response = self.doods.detect(detect_request)
  File "/opt/doods/doods.py", line 127, in detect
    ret = detector.detect(image)
  File "/opt/doods/detectors/deepstack.py", line 45, in detect
    results = self.torch_model(image, augment=False)[0]
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/root/.cache/torch/hub/ultralytics_yolov5_master/models/yolo.py", line 126, in forward
    return self._forward_once(x, profile, visualize)  # single-scale inference, train
  File "/root/.cache/torch/hub/ultralytics_yolov5_master/models/yolo.py", line 149, in _forward_once
    x = m(x)  # run
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/root/.cache/torch/hub/ultralytics_yolov5_master/models/yolo.py", line 61, in forward
    if self.inplace:
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1177, in __getattr__
    raise AttributeError("'{}' object has no attribute '{}'".format(
AttributeError: 'Detect' object has no attribute 'inplace'

Both of the models work with the deepstack service itself, and doods properly reports the labels embedded in the models
    {
      "name": "dark",
      "type": "deepstack",
      "model": "external/models/dark.pt",
      "labels": [
        "Bicycle",
        "Boat",
        "Bottle",
        "Bus",
        "Car",
        "Cat",
        "Chair",
        "Cup",
        "Dog",
        "Motorbike",
        "People",
        "Table"
      ],
      "width": 0,
      "height": 0
    },
    {
      "name": "combined",
      "type": "deepstack",
      "model": "external/models/combined.pt",
      "labels": [
        "person",
        "bicycle",
        "car",
        "motorcycle",
        "bus",
        "truck",
        "bird",
        "cat",
        "dog",
        "horse",
        "sheep",
        "cow",
        "bear",
        "deer",
        "rabbit",
        "raccoon",
        "fox",
        "coyote",
        "possum",
        "skunk",
        "squirrel",
        "pig",
        ""
      ],
      "width": 0,
      "height": 0
    }

so the models seem intact.
My config for the two models is minimal:
    - name: dark
      type: deepstack
      modelFile: external/models/dark.pt
    - name: combined
      type: deepstack
      modelFile: external/models/combined.pt

do I need more than that or is there some issue with the deepstack integration at the moment?

maybe needs to investigate it its a compatible-bug between models trained on older yolo versions?
see -> https://github.com/ultralytics/yolov5/issues/5234#issuecomment-946854334
or my comment on openlogo.pt-model -> https://github.com/snowzach/doods2/issues/28#issuecomment-1072968647

I am wondering if this is the same issue where it's trying to use Yolo6 and I need to figure out how to make it use yolo5

Okay.... I just looked into this more.. the OpenLogo detector is working fine for me. I suspect that this is an issue where it cannot properly convert whatever image type you are working with into the one that Deepstack expects. Where are you getting your image data?

does this image features survive here?
as i upload it, i see that i looks like alpha-channel image... maybe thats it?

maybe an interference with some models from my config?
root@ub-doods2:/usr/src# cat example.yaml
doods:
  detectors:
    - name: default-org
      type: tflite
      modelFile: models/coco_ssd_mobilenet_v1_1.0_quant.tflite
      labelFile: models/coco_labels0.txt
      numThreads: 0
      numConcurrent: 4
    - name: edgetpu
      type: tflite
      modelFile: models/mobilenet_ssd_v2_coco_quant_postprocess_edgetpu.tflite
      labelFile: models/coco_labels0.txt
      numThreads: 0
      numConcurrent: 4
      hwAccel: false
    - name: tensorflow
      type: tensorflow
      modelFile: models/faster_rcnn_inception_v2_coco_2018_01_28.pb
      labelFile: models/coco_labels1.txt
      width: 224
      height: 224
      numThreads: 0
      numConcurrent: 4
    - name: default
      type: tflite
      modelFile: models/lite-model_aiy_vision_classifier_birds_V1_3.tflite
      labelFile: models/aiy_birds_V1_labelmap.csv
      width: 224
      height: 224
      numThreads: 0
      numConcurrent: 4
      hwAccel: false
    - name: pybirds
      type: pytorch
      modelFile: nicolalandro/ntsnet-cub200,ntsnet
      width: 600
      height: 600
      numThreads: 0
      numConcurrent: 4
    - name: yolo5
      type: pytorch
      modelFile: ultralytics/yolov5,yolov5s
      width: 224
      height: 224
      numThreads: 0
      numConcurrent: 4
    - name: logodeep
      type: deepstack
      modelFile: models/openlogo.pt
      width: 640
      height: 640
      numThreads: 0
      numConcurrent: 4

Oops, whats this?
i left only openlogo in the config alone:

my config:
doods:
  detectors:
    - name: yolo5
      type: pytorch
      modelFile: ultralytics/yolov5,yolov5s
      width: 224
      height: 224
      numThreads: 0
      numConcurrent: 4
    - name: logodeep
      type: pytorch
      modelFile: models/openlogo.pt
      width: 640
      height: 640
      numThreads: 0
      numConcurrent: 4

now this:

i have to have yolo5 in the config,
than no error on startup. but still processing error

config (now openlogo as deepstack-type):
doods:
  detectors:
    - name: yolo5
      type: pytorch
      modelFile: ultralytics/yolov5,yolov5s
      width: 224
      height: 224
      numThreads: 0
      numConcurrent: 4
    - name: logodeep
      type: deepstack
      modelFile: models/openlogo.pt
      width: 640
      height: 640
      numThreads: 0
      numConcurrent: 4

i test with the image from the openlogo-site, but still that error
Testimage:
https://github.com/OlafenwaMoses/DeepStack_OpenLogo/blob/main/images/fedex_detected.jpg?raw=true
Error:

Uhhhh, i was behind the "latest" docker...
PULL!

and test now...

did not help:

testimage still not work.
seems to be interference with yolo5 (verstion 6= yolo5s6 did also not help)
TESTIMAGE:
https://raw.githubusercontent.com/OlafenwaMoses/DeepStack_OpenLogo/main/images/fedex.jpg

I get the same failure of the doods deepstack models no matter what image I use.
*doods2 (tensorflow model) -> success
*doods2 (pytorch model) -> success
*deepstack instance -> success
*doods2 (deepstack model) -> error
Occasionally the error is different, so maybe there is more than one issue here.  Most of the time I get the error in my original post
AttributeError: 'Detect' object has no attribute 'inplace'

but on some images I get the same error ozett has just reported in several posts
RuntimeError: The expanded size of the tensor (1) must match the existing size (48) at non-singleton dimension 3.  Target sizes: [1, 3, 1, 1, 2].  Tensor sizes: [3, 80, 48, 2]

Here's one of my example images

Soooo this is interesting... It appears to be some sort of collision with having a pytorch detector enabled along with the deepstack detector. If I disable pytorch the logo detector works fine. That gives me something reproducible to look into at least.

single config of deepstack-type with openlogo throws error on my install.
cannot confirm that it works. see this error:

I can confirm that deepstack models fail to load if I remove pytorch from my configuration.
2022-03-20 16:41:30,151 - doods.doods - ERROR - Could not create detector deepstack/dark: No module named 'models.yolo'
2022-03-20 16:41:30,151 - doods.doods - ERROR - Could not create detector deepstack/combined: No module named 'models.yolo'

Are you able to share a model?

My deepstack/dark model is based on the pt file that can be downloaded here: https://github.com/OlafenwaMoses/DeepStack_ExDark/releases/tag/v1
Given that I do get it to work with the deepstack server, I assume the file I downloaded is complete and intact.  But here's my local copy as well:
https://drive.google.com/file/d/1lM4QAEQx83Tb7j1Hxqz4Yv9mGW3mYq16

Okay, I think I fixed part of the issue. It was missing some of the deepstack trainer files it needed to start the dark model. I pushed an image if you want to try that. The issue still remains if you also load the latest yolo model as well, it will print the other error about mismatched tensor sizes. I am still trying to figure that one out.

The new image works as advertised.  As long as I don't load up pytorch, both my deepstack based models now work with errors so that's big step in the right direction.

i got the downloaded openlogo.pt model to work
with the yolo5 "detect.py" script. in my first experiments:
https://github.com/snowzach/doods2/issues/28#issuecomment-1024890510
so i assume, i could be working like yolo5 in the doods2-envrionment . without changing the model.pt file.
but i am not capable to analyse the "detect.py " loading process.
but maybe that hint helps you

Okay, I hacked together something. I don't love it but it works. It basically tries the new model way of doing it and if fails it reverts back to the old way. It shouldn't add too much delay. Let me know if it works for you.

nope. not really....

works with openlogo config only....

not with more config-options like normal -> https://github.com/snowzach/doods2/issues/38#issuecomment-1073274642

I had to fix an issue with no detections but it seems to work fine for me with this config
logger:
  level: debug
doods:
  detectors:
    - name: default
      type: tflite
      modelFile: models/coco_ssd_mobilenet_v1_1.0_quant.tflite
      labelFile: models/coco_labels0.txt
    - name: logos
      type: deepstack
      modelFile: models/openlogo.pt
    - name: dark
      type: deepstack
      modelFile: models/dark.pt
    - name: lp
      type: deepstack
      modelFile: models/licence-plate.pt
    - name: yolo5
      type: pytorch
      modelFile: ultralytics/yolov5,yolov5s

Make sure you don't have the yolo source files sitting around anywhere, especially in your models directory as it can inadvertently import that.

So this is interesting... If you load pytorch/yolo before deepstack models, it will fail. If you load deepstack models first, it works fine...

So this is interesting... If you load pytorch/yolo before deepstack models, it will fail. If you load deepstack models first, it works fine...

i will try this order

looks like a hit.
then i would like to thank you very much for this. i am so glad that you found time to look for it. maybe you will also find the real error. then just post here. i like to test .
now i'll try to build a node-red flow that will be fed with the logos. thanks again so much!

Thank you!  That works for me too (as long as the config order is correct).  I'll consider this issue closed, even if this is only a temporary fix till you find a solution you like better.

testimage still works:
https://raw.githubusercontent.com/OlafenwaMoses/DeepStack_OpenLogo/main/images/fedex.jpg
new testimage also works:
https://assets.entrepreneur.com/content/3x2/2000/20160210172912-branding-logos-companies-businesses-identity.jpeg
myimage fails:
https://user-images.githubusercontent.com/5937811/159519375-f4d7ece2-6b81-49a4-b8f5-2a909661e752.jpg

its not because of the tiny size.

this is 10px smaller than the tumbnail.jpg, but still works.
must be something else

sometimes its working,
but somehow i miss the marked image in the response.
do i miss an option in the request?

@ozett make sure you are pulling the latest image. I made a change that should fix the NoneType error you see. @JustinGeorgi it should automatically reorder the detectors so you don't need to worry about the order you put in your config.

i will try next 60 minutes... thx.

pulled in milliseconds, running fine now with my first failure-image.
will let my new flow run over it for today for test.
really GREAT, thanks a lot, appreciate this!
doods2 is getting more and more multifunctional and failsafe!

flow is running, needs fine-tuning and for croping the region
i was wondering where is your image origin ?
where is top and bottom of the image? like on opencv?
i need to provide region-coordinates for detections of x,y,w,h and cant fiddle it right
TESTIMAGE
https://www.clickedstudios.com/wp-content/uploads/2019/02/2019-luxury-top-brands-feature.png

figured it out, activated lost  memory.
coordinates are nomalized.. -> https://www.tensorflow.org/api_docs/python/tf/image/draw_bounding_boxes

Okay, closing this issue.


Check for previous/existing GitHub issues/module proposals

[X] I have checked for previous/existing GitHub issues/module proposals.

Check this module doesn't already exist in the module indexes

[X] I have checked for that this module doesn't already exist in the module indexes.

Bicep or Terraform?
Terraform
Module Classification?
Pattern Module
Module Name
avm-ptn-openai-cognitivesearch
Module Details
This is a pattern module proposal for a Corporate Line of Business (LoB) ChatBot application using Azure OpenAI and Azure Cognitive Search Service. It includes:

Azure App Gateway
Azure App Service Premium v3 (Linux): Web App + API App
Azure BLOB Storage
Azure SQL Database
Azure Cache for REDIS
Azure App Insights
Private VNET

Note This is taking over from https://github.com/Azure/Azure-Verified-Modules/issues/1000
Do you want to be the owner of this module?
Yes
Module Owner's GitHub Username (handle)
mikestiers
(Optional) Secondary Module Owner's GitHub Username (handle)
andbron

Hi @mikestiers
since this is request for pattern module, we would need few more things to check:

Where is it coming from/what’s backing it - e.g., Azure Architecture Center (AAC), community request, customer example.
Provide an architectural diagram and related documentation if possible - or a pointer to these if they are publicly available.

Tnx!

I'm working on the architecture diagram. I thought we had it already, but I just checked and it's not there so I'll reply on Thursday hopefully with it.
In the meantime, this architecture is coming from a generic customer example that has been provided by Architect @mariocon. We have quite a few patterns that we will be contributing that are a result of customer examples gathered by @mariocon from a number of customer engagements.

@mikestiers
Thanks for requesting/proposing to be an AVM module owner!
We just want to confirm you agree to the below pages that define what module ownership means:

Team Definitions & RACI
Shared Specification (Bicep & Terraform)
Module Support

Any questions or clarifications needed, let us know!
If you agree, please just reply to this issue with the exact sentence below (as this helps with our automation 👍):
"I CONFIRM I WISH TO OWN THIS AVM MODULE AND UNDERSTAND THE REQUIREMENTS AND DEFINITION OF A MODULE OWNER"
Thanks,
The AVM Core Team
#RR

I CONFIRM I WISH TO OWN THIS AVM MODULE AND UNDERSTAND THE REQUIREMENTS AND DEFINITION OF A MODULE OWNER

@mikestiers
Thanks for confirming that you wish to own this AVM module and understand the related requirements and responsibilities!
Before starting development, please ensure ALL the following requirements are met.
Please use the following values explicitly as provided in the module index page:

For your module:

ModuleName - for naming your module
TelemetryIdPrefix - for your module's telemetry

For your module's repository:

Repo name and folder path are defined in RepoURL
Create GitHub teams for module owners and contributors and grant them permissions as outlined here.
Grant permissions for the AVM core team and PG teams on your GitHub repo as described here.

Check if this module exists in the other IaC language. If so, collaborate with the other owner for consistency. 👍
You can now start the development of this module! ✅ Happy coding! 🎉
Please respond to this comment and request a review from the AVM core team once your module is ready to be published! Please include a link pointing to your PR, once available. 🙏
Any further questions or clarifications needed, let us know!
Thanks,
The AVM Core Team


Working on the user authentication in the sign up page

Currently the sign up page is not working. It is showing 'error occurred'.

I would like to work on this issue. Please assign this to me.

Currently the sign up page is not working. It is showing 'error occurred'.  I would like to work on this issue. Please assign this to me.

Hello @Lanaanvar, Go for it
All the Best


what is the lowest loss we can achieve with this architecture?

I tried to overfitting the model but it seems so hard to achieve
I've minimize the data into 100 image  with 1 captions per image
and the lowest loss I can achieve when training in 500 epochs is around 0.9xxx and the loss value is stagnant till 1000 epoch
what is the lowest loss you've achieve @sgrvinod ?

Hey, sorry, I've been really busy and I'll respond to your email (and here) in the next day or two.
I don't remember the loss values, especially the training loss - but I also don't recall noticing that it was stagnant or slowing down. I wouldn't think that the model couldn't overfit.
To eliminate any errors in the code, were you able to get it to work on MSCOCO?

the data is from flickr8k images and karpathy's json file but I cut it only 100 training image and 20 validation image here are the json in my gist

Hi,
When train on coco, mine minimum is about 3.x. Is it normal? My generated captions seems very bad also...


Added some elements from Gian "code reviews, UX, and tests" email.

Description
@gianm 's email from dev mailing list made some suggestions about how to write PRs, with emphasis on UX and testing. I've added a few paragraphs to the PR template to facilitate creation of better PRs and therefore ease the PR review process.
This PR has:

[X] been self-reviewed.

For reference, here's Gian's dev list email:
Hey Druids,
I am writing to you all to ask for your help 🙂
In particular, your help in ensuring that potential code contributions are
reviewed in a timely fashion. Right now we have 72 open PRs, which due to
stalebot are mostly opened pretty recently. That's a lot of people that
want to contribute stuff!
First, I wanted to point out that a lot of the reviews are done by a
relatively small number of people. If you are one of those people, thank
you. If you aren't, please consider becoming one.
Second, I've been thinking about how to make reviewing a little easier.
Maybe this will help more people become reviewers. I'd like to suggest
focusing on user experience and tests.

"User experience" means any new properties being introduced, any new
features being introduced, any behavior changes to existing features.

"Tests" are ideally automated tests that run as part of Travis CI. But
there are other kinds of tests too.

It's important to get user experience right, because if we release a
feature with poor UX, then it's hard to change later without breaking
compatibility. We'd like the master branch to be releasable at all times,
which means we should figure this stuff out before committing a patch.
It's also important to get tests right, because they ensure high quality
releases, and they also ensure that future changes can be made with low
risk.
It's less important to get other code-level things right, because we can
always change them later, and if there aren't UX or quality impacts then no
harm was done.
For contributors, focusing on UX and tests means writing out (in natural
language) how your patch changes user experience, and why you think this
change is a good idea. It also means having good testing of the new stuff
you're adding, and writing out (in natural language) why you think your
tests cover all the important cases. Speaking as a person that has reviewed
a lot of code: these natural language descriptions are very helpful,
especially when they add context to the patch. Don't make reviewers
reverse-engineer your code to guess what you were thinking.
For reviewers, it means that if we are confident in the UX and testing, we
don't need to spend a ton of time poring over every line of code. (I'd
still take the time to review key classes and interfaces, but this doesn't
take as much time as reviewing every line.) It also means that if we get a
PR that isn't set up to enable quick review, then we should write that to
the contributor and invite them to improve their PR, rather than ignoring
it or spending too much time on it. (Of course, we should only ask for this
if we can actually follow up with a review when the PR is improved later
on.)
I'd love to hear what people think.
Gian

@sergioferragut, great ideas. Can we add this to the Contributing to Druid page?
Maybe also add the premise of Gian's e-mail: that there are a few reviewers and here are the things that will help those reviewers approved you changes: (Then list the items.)


mobile: change Envoy::Platform::LogLevel to an enum class

They seem to have a number of advantages over "plain" enums according to https://stackoverflow.com/a/18335862, and this will allow them to be imported properly into Swift.
Commit Message:
Additional Description:
Risk Level: Low, should be a source-compatible change.
Testing: Existing tests.
Docs Changes: None.
Release Notes: None.
Platform Specific Features:
[Optional Runtime guard:]
[Optional Fixes #Issue]
[Optional Fixes commit #PR or SHA]
[Optional Deprecated:]
[Optional API Considerations:]

/assign @RyanTheOptimist

Sorry Ryan, this raced with https://github.com/envoyproxy/envoy/pull/26037 so I had to merge main into this branch and that dismissed your review.


Barco ClickShare is not installing

Hi There,
I'm using installomator version VERSION="10.4beta" VERSIONDATE="2023-02-24" with macOS 13.2.1
All computer giving the same error "ERROR: Error mounting "
Also tested with installomator version 10.2
Maybe it's just a defect in ClickShare version 4.28.0-b11, never seen this issue.
Script result: 2023-04-06 09:59:48 : REQ   :  : shifting arguments for Jamf
2023-04-06 09:59:48 : REQ   : clickshare : ################## Start Installomator v. 10.2, date 2023-01-06
2023-04-06 09:59:48 : INFO  : clickshare : ################## Version: 10.2
2023-04-06 09:59:48 : INFO  : clickshare : ################## Date: 2023-01-06
2023-04-06 09:59:48 : INFO  : clickshare : ################## clickshare
2023-04-06 09:59:49 : INFO  : clickshare : BLOCKING_PROCESS_ACTION=prompt_user_then_kill
2023-04-06 09:59:49 : INFO  : clickshare : NOTIFY=silent
2023-04-06 09:59:49 : INFO  : clickshare : LOGGING=INFO
2023-04-06 09:59:49 : INFO  : clickshare : LOGO=/System/Applications/App Store.app/Contents/Resources/AppIcon.icns
2023-04-06 09:59:49 : INFO  : clickshare : Label type: appInDmgInZip
2023-04-06 09:59:49 : INFO  : clickshare : archiveName: ClickShare.zip
2023-04-06 09:59:49 : INFO  : clickshare : no blocking processes defined, using ClickShare as default
2023-04-06 09:59:49 : INFO  : clickshare : App(s) found: /Applications/ClickShare.app
2023-04-06 09:59:49 : INFO  : clickshare : found app at /Applications/ClickShare.app, version 4.27.2-b4, on versionKey CFBundleShortVersionString
2023-04-06 09:59:49 : INFO  : clickshare : appversion: 4.27.2-b4
2023-04-06 09:59:49 : INFO  : clickshare : Latest version of ClickShare is 4.28.0-b11
2023-04-06 09:59:49 : REQ   : clickshare : Downloading https://www.barco.com/services/website/en/TdeFiles/Download?FileNumber=R3306192&TdeType=3&MajorVersion=04&MinorVersion=28&PatchVersion=00&BuildVersion=011&ShowDownloadPage=False to ClickShare.zip
2023-04-06 09:59:59 : REQ   : clickshare : no more blocking processes, continue with update
2023-04-06 09:59:59 : REQ   : clickshare : Installing ClickShare
2023-04-06 09:59:59 : INFO  : clickshare : Unzipping ClickShare.zip
2023-04-06 09:59:59 : INFO  : clickshare : found dmg: /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/tmp.JFfgKQcJ/ClickShare_Setup.dmg
2023-04-06 09:59:59 : INFO  : clickshare : Mounting /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/tmp.JFfgKQcJ/ClickShare_Setup.dmg
hdiutil: attach failed - Bestand of directory bestaat niet
2023-04-06 09:59:59 : INFO  : clickshare : App not closed, so no reopen.
2023-04-06 09:59:59 : ERROR : clickshare : ERROR: Error mounting /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/tmp.JFfgKQcJ/ClickShare_Setup.dmg error:
Log
2023-04-06 09:59:59 : REQ   : clickshare : ################## End Installomator, exit code 3
====
2023-04-07 09:35:24 : REQ   :  : shifting arguments for Jamf
2023-04-07 09:35:24 : REQ   : clickshare : ################## Start Installomator v. 10.4beta, date 2023-02-24
2023-04-07 09:35:24 : INFO  : clickshare : ################## Version: 10.4beta
2023-04-07 09:35:24 : INFO  : clickshare : ################## Date: 2023-02-24
2023-04-07 09:35:24 : INFO  : clickshare : ################## clickshare
2023-04-07 09:35:24 : INFO  : clickshare : BLOCKING_PROCESS_ACTION=prompt_user_then_kill
2023-04-07 09:35:24 : INFO  : clickshare : NOTIFY=silent
2023-04-07 09:35:24 : INFO  : clickshare : LOGGING=INFO
2023-04-07 09:35:24 : INFO  : clickshare : LOGO=/System/Applications/App Store.app/Contents/Resources/AppIcon.icns
2023-04-07 09:35:24 : INFO  : clickshare : Label type: appInDmgInZip
2023-04-07 09:35:24 : INFO  : clickshare : archiveName: ClickShare.zip
2023-04-07 09:35:24 : INFO  : clickshare : no blocking processes defined, using ClickShare as default
2023-04-07 09:35:24 : INFO  : clickshare : name: ClickShare, appName: ClickShare.app
2023-04-07 09:35:24.824 mdfind[7847:290986] [UserQueryParser] Loading keywords and predicates for locale "en_US"
2023-04-07 09:35:24.824 mdfind[7847:290986] [UserQueryParser] Loading keywords and predicates for locale "en"
2023-04-07 09:35:24.871 mdfind[7847:290986] Couldn't determine the mapping between prefab keywords and predicates.
2023-04-07 09:35:24 : WARN  : clickshare : No previous app found
2023-04-07 09:35:24 : WARN  : clickshare : could not find ClickShare.app
2023-04-07 09:35:24 : INFO  : clickshare : appversion:
2023-04-07 09:35:24 : INFO  : clickshare : Latest version of ClickShare is 4.28.0-b11
2023-04-07 09:35:24 : REQ   : clickshare : Downloading https://www.barco.com/services/website/en/TdeFiles/Download?FileNumber=R3306192&TdeType=3&MajorVersion=04&MinorVersion=28&PatchVersion=00&BuildVersion=011&ShowDownloadPage=False to ClickShare.zip
2023-04-07 09:35:35 : REQ   : clickshare : no more blocking processes, continue with update
2023-04-07 09:35:35 : REQ   : clickshare : Installing ClickShare
2023-04-07 09:35:35 : INFO  : clickshare : Unzipping ClickShare.zip
2023-04-07 09:35:35 : INFO  : clickshare : found dmg: /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/tmp.ZFvScVUC/ClickShare_Setup.dmg
2023-04-07 09:35:35 : INFO  : clickshare : Mounting /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/tmp.ZFvScVUC/ClickShare_Setup.dmg
hdiutil: attach failed - No such file or directory
2023-04-07 09:35:35 : INFO  : clickshare : App not closed, so no reopen.
2023-04-07 09:35:35 : ERROR : clickshare : ERROR: Error mounting /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/tmp.ZFvScVUC/ClickShare_Setup.dmg error:
Log
2023-04-07 09:35:35 : REQ   : clickshare : ################## End Installomator, exit code 3

yes, I can confirm. The ClickShare bundle now comes as down as a zip with a dmg inside ...
🤦
We don't have a dmginzip type yet... So this would require some major re-building. Maybe you can file complaint with the vendor and we can wait if they change it back. Can you file an issue, so we can keep track of this?

Done, but I think that's not sufficient to let it change the way they offering software...

To add to this, I was able to apply a slight workaround by replacing the existing tar -xf in the installAppInDmgInZip with the below and it completes successfully.
 if [ "$name" = "ClickShare" ];then         tar --strip-components=1 -xf "$archiveName"     else;         tar -xf "$archiveName"     fi

just looking at browser console, it looks like the download URL can be extracted from https://www.barco.com/bin/barco/tde/downloadUrl.json?fileNumber=R3306192&tdeType=3
{"downloadUrl":"https://barcoprdwebsitefs.azureedge.net/barcoprdfs/Data/secure/downloads/tde/Active/SoftwareFiles/ApplicationSoftware/R3306192_48_ApplicationSw.zip?Tkq7KOrFzaCMeZ78C7dHkkX8toAfIzqOCSzthMXyA1eIYj9nBupaLmvyR9K4WnPR8wGy6PRLe-TYT5-p97kkccS6V1V5","isSuccess":true,"isNotFound":false,"isUnauthorized":false}
Some curl and text manipulation should allow extraction of downloadURL

I've got this for version:
appNewVersion=$(curl -fs "https://www.barco.com/content/dxp/regions-countries/global/en/support/software/R3306192/jcr:content/root/main_section/supportdownloadversi.asynccomponent.html" | grep -ioE 'v[0-9]+.[0-9]+.[0-9]+.[0-9]+' | awk -F'[v.]' '{print $2"."$3"."$4"-b"$5}' | head -n 1)
However for latest release my installed version is '4.31.0-b6' but they list '<IP_ADDRESS>'
https://www.barco.com/en/support/software/R3306192
Using the above matches all other versions I can see in my environment, did they just make a mistake?

I am getting this error with the label ClickShare:
2023-08-29 11:24:06 : REQ   : clickshare : no more blocking processes, continue with update
2023-08-29 11:24:06 : REQ   : clickshare : Installing ClickShare
2023-08-29 11:24:06 : INFO  : clickshare : Unzipping ClickShare.zip
tar: Error opening archive: Unrecognized archive format
2023-08-29 11:24:06 : DEBUG : clickshare : Deleting /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/tmp.qgm4coat
2023-08-29 11:24:06 : DEBUG : clickshare : Debugging enabled, Deleting tmpDir output was:
/var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/tmp.qgm4coat/ClickShare.zip
2023-08-29 11:24:06 : DEBUG : clickshare : /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/tmp.qgm4coat
2023-08-29 11:24:06 : INFO  : clickshare : App not closed, so no reopen.
2023-08-29 11:24:06 : ERROR : clickshare : ERROR: couldn't find dmg in zip ClickShare.zip
2023-08-29 11:24:06 : REQ   : clickshare : ################## End Installomator, exit code 22

I've modified the download URL to pull the correct / latest version. Hope this helps.
downloadURL=$(curl -fs "https://www.barco.com/bin/barco/tde/downloadUrl.json?fileNumber=$(curl -fs "https://www.barco.com/en/products/clickshare-conferencing-collaboration/app" | grep -B2 -i "mac installer" | awk -F'"' '/href=/{print $4}' | awk -F'/' '{print $NF}' | awk -F'.' '{print $1}')&tdeType=3" | grep -i "downloadurl" | awk -F'"' '{print $4}' | sed 's/\\//g')


Corrections to RectHelper and SizeHelper's definition of "empty"

RectHelper and SizeHelper use a definition of "Empty" which matches .NET's System.Windows.Rect.Empty. A Rect or Size with Width == 0 or Height == 0 is not "empty" because it is not the Empty value.

RectHelper.Empty and SizeHelper.Empty actually return structs where the Width and Height are initialized to negative infinity, not 0.
For GetLeft/GetTop/GetRight/GetBottom, the implementation actually calls GetIsEmpty rather than checking for positive Width or Height.
In the Remarks section for RectHelper.GetIsEmpty and SizeHelper.GetIsEmpty, add the same caveat that is in the .NET docs: "this method should not be used to check for zero area".

Note that this information also applies to the Microsoft.UI.Xaml version of these methods. I assume I'll need to open a separate PR against winapps-winrt-api for that; please let me know if there's a different process which is preferred.

@jwmsft
IMPORTANT: When the changes are ready for publication, add a #sign-off comment to signal that the PR is ready for the review team to merge.
#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team

#sign-off


SQLite test trace fix

Modify sqlite testing script to work with jenkins configuration and fetch a built version of the testing framework instead of the source code.

Coverage remained the same at 75.159% when pulling 23cf8e751526a0226050df51ea418c0c107830c5 on tli2:tianyu-sqlite-trace into 570cc8ecf2e8b7b9b342e581da6b466d0ef6f1ac on cmu-db:master.


Added Custom Delimiter Support

Implements GH-1 - this time taking style tips from the coerce branch.
This will allow one additional single character delimiter to be used in conjunction with the existing syntax, allowing support for querystrings such as:
foo.bar[0].boing[he.llo].world.rawr=rawk.txt

To be parsed correctly with the . delimiter as:
{ foo: { bar: [ { boing: { 'he.llo': { world: { rawr: 'rawk.txt' } } } } ] } }

Found this behavior mentioned on mcavage/node-restify#94 which lead me to GH-1, figured I'd give it a whirl; feedback welcome.

personally I dont see the point, everyone else on the planet uses foo[bar]=baz and it's really not that bad

I agree that [] isn't that bad, but that's not the same as no room for improvement. After all it was in the issues list, so at some point its inclusion must have made sense.

the main reason was because the original node one supported it before they took out nesting support, aside from disliking some of the characters there's no real reason to support it IMO

I see, well it's not on by default, so it shouldn't go breaking any existing use cases. Maybe given the option people will prefer and migrate back to the dot notation (personally I think it suits node better), or maybe it won't get used at all. I'll leave its inclusion to your best guess at what the people want :)

for better or for worse the only thing I've actually seen used in practice (other than paypal haha) is the [] notation. Not saying that's my favorite or anything but it's definitely what people have adopted


adjust embeddable rendering with context menu flyout

Description
Adjusts the flyout to use the EmbeddableRenderer, because it displays the embeddable without a border as needed. Before, we were relying on props within the EmbeddablePanel, which required changes to OpenDashboards to work.
Issues Resolved
Related to discoveries in https://github.com/opensearch-project/OpenSearch-Dashboards/pull/3924
Check List

[ ] New functionality includes testing.

[ ] All tests pass

[ ] New functionality has been documented.

[ ] New functionality has javadoc added

[ ] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@lezzago please review—this is related to the feedback from Ashwin in the OpenDashboards PR.

Closing as the feature has been launched


THis page is outdated

[Enter feedback here]
This document is outdated, following this tutorial does not create sample app. For example:
Msal.Module.forRoot expects 3 arguments but in your tutorial only 2 are provided. If you move area that containts "consentScopes" as a third argument (probably type on your end) different error comes up. In the end this sample is a struggle for anyone attempting to use MSAL with Angular. I hope this becomes better over time.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 1889fcfe-437f-f8cb-2de9-f4eb605f5430
Version Independent ID: 6e6d0777-15c3-4002-ce38-64365ba01436
Content: Tutorial: Create an Angular app that uses the Microsoft identity platform for authentication - Microsoft identity platform
Content Source: articles/active-directory/develop/tutorial-v2-angular.md
Service: active-directory
Sub-service: develop
GitHub Login: @hamiltonha
Microsoft Alias: hahamil

@zuboje Thanks for your comment! We will review and provide an update as appropriate.

@zuboje This is work in progress and will be updated once PG have reviewed and sign-off the change  .

@zuboje  The change has been submitted on this one and it will be updated by first week of June. We will currently close this issue .


Launching Karabiner-Elements at login

In System Preferences I added Karabiner-Elements as a login item, and it does indeed launch. However, it also opens the Karabiner window when I log into my computer. This is undesirable, as I don’t want to modify my keyboard settings every login, I just want Karabiner to start functioning in the background.
Is there a way to have Karabiner Elements automatically launch at login, without opening its window every time?

Same thing.

Same here too. Also looking for a solution.

me too

Karabiner-Elements starts automatically at login without login items.
Please remove Karabiner-Elements from login items.

Karabiner-Elements starts automatically at login without login items.
Please remove Karabiner-Elements from login items.

In my experience, Karabiner-Elements will *restart* automatically at login, if it was running when I logged off. However, if for any reason it was *not* running at the time I logged out, then it will not launch at login. I would rather have it launch at login regardless of whether it was running previously.
Additionally, from a purely user-experience perspective, I greatly prefer to have all my login items listed and managed from one place—namely the Login Items pane for my account. Thus, all else being equal, I *want* to put Karabiner-Elements as a login item, and I do *not* want it (or any other program I install) to launch automatically if it isn’t there.

Do we still need "Karabiner" as a Login-item with Karabiner Elements? I've removed it but wondering if it was supposed to be uninstalled somehow...

Takezo's comment from Dec 9th 2017 solved the issue.
Just remove Karabiner-Elements from login items.

I'm still confused about this. Shouldn't there be an option "Launch on startup" in the GUI?

AppleScript workaround

Open a new Script Editor document, set the language to AppleScript if it isn't already, and insert this snippet into the script body:

do shell script "launchctl load -w /Library/LaunchAgents/org.pqrs.karabiner.karabiner_console_user_server.plist"

Save the script into any directory you like; maybe the same directory as Karabiner-Elements and Karabiner- EventViewer, with a name like, “Karabiner-Loader”. I think these options are optimal:

“File Format: Application”
“Show startup screen” unchecked
“Stay open after run handler” unchecked

Go to System Preferences → Users & Groups → Login Items, and add Karabiner-Loader to the list.

“File Format: Application”
“Show startup screen” unchecked
“Stay open after run handler” unchecked

@habibalamin  Where can the above settings be found?


Rpi build references g++-4.7 directly instead of testing for a C++11 compiler

I am running a Raspberry Pi with GCC 4.9 instead of GCC 4.7, and had to manually switch g++-4.7 in toolchains/raspberrypi.toolchain.cmake to just g++.
Ideally you would use some sort of C++11 detection in CMake, or at least tell people that they'll need to do this to build with anything other than g++ 4.7.

Hi,
Thanks for reporting,

That hits the spot, thanks.


Verify AWS EKS module in Terraform Module Registry

I have issues
I'm submitting a...

[x] feature request

What is the current behavior?
Currently this AWS EKS module is not a verified module, therefore the module is harder to find in the Terraform module registry.
What's the expected behavior?
The provision count of this module is above 40_000, has active maintainers and a community around it. It would be nice to verify this module in the Terraform module registry. This way it is easier to distinguish this well maintained module from the many other EKS modules in the Terraform registry. In addition, it is just easier to find.
Any other relevant information?
Another module which reached a verified status is the VPC module. The VPC module is an important building block and can be used with this AWS EKS module. It should be possible to contact the VPC module maintainer and ask about the module verification process.

Great! Looking forward to it.

Thanks for submitting this. To tell the full story, I tried to get this module and my ALB module verified early on and received a response (3 months later) with some uncertainty and redirection process-wise. My attempts to follow up didn't get anything further. Out of the blue, the verified badge showed up on the ALB module... totally unclear to me how, when, or why 🤷♀ . I somewhat expect the same thing will happen here.
Let's keep this closed for the moment to declutter the issues a bit as there's really no further action to take on our end. Feel free to chime back in with any updates.


GitHub actions, no big endian test

I added all Travis CI tests to github actions, with the exception of the big endian test.  There is no straight forward support for big endian architecture in github actions, so will be done in a separate workflow.  There are two known failures, the Numpy Dev and Pre-Release Dependencies tests.

For the Windows test, it looks like the test is expecting an IOError but it is actually raising a ValueError.  Strange.

For the Windows test, it looks like the test is expecting an IOError but it is actually raising a ValueError.  Strange.


Implement :set command

Should be able to i.e. dynamically change the width of the sidebar.

Can be closed as of 540888a


test: use gotests to generates table driven tests framework of cri-utils

Signed-off-by: Starnop<EMAIL_ADDRESS>
Ⅰ. Describe what this PR did
For issue #1756 , we need add more unit-test for CRI，most of them locate at cri/v1alpha1/cri_utils.go & cri/v1alpha2/cri_utils.go. So I generates table driven tests framework for future works with gotests.
Ⅱ. Does this pull request fix one issue?

fixes part of #1756
Ⅲ. Describe how you did it
use the tool gotests and run command as follows:
gotests --all -w cri_utils.go cri_utils_test.go
Ⅳ. Describe how to verify it
Ⅴ. Special notes for reviews

Codecov Report

Merging #1894 into master will decrease coverage by 0.36%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #1894      +/-   ##
==========================================
- Coverage   58.68%   58.32%   -0.37%
==========================================
  Files         200      200
  Lines       15565    15565
==========================================
- Hits         9135     9078      -57
- Misses       5143     5211      +68
+ Partials     1287     1276      -11

Flag
Coverage Δ

#criv1alpha1test
33.28% <ø> (+0.07%)
:arrow_up:

#criv1alpha2test
33.8% <ø> (ø)
:arrow_up:

#integrationtest
37.88% <ø> (-0.03%)
:arrow_down:

#unittest
13.43% <ø> (ø)
:arrow_up:

Impacted Files
Coverage Δ

cri/stream/httpstream/spdy/upgrade.go
54.28% <0%> (-5.72%)
:arrow_down:

daemon/mgr/container.go
48.87% <0%> (-4.44%)
:arrow_down:

ctrd/watch.go
78.12% <0%> (+3.12%)
:arrow_up:

The generated test framework will help a lot. Well done!


healthcheck filling logs (again)

I get alot of :
INFO    postfix/smtpd[225]: improper command pipelining after CONNECT from localhost[<IP_ADDRESS>]: EHLO healthcheck\n
It seems that at some point the CONNECT in that string have changed casing :) I just tested and this patch fixes the issue.
 if $syslogseverity <= '6' then {
     # Do not log healthchecks
-    if ($msg contains "connect from localhost[<IP_ADDRESS>]") then { stop }
-    if ($msg contains "lost connection after EHLO from localhost[<IP_ADDRESS>]") then { stop }
-    if ($msg contains "disconnect from localhost[<IP_ADDRESS>] ehlo=1 commands=1") then { stop }
+    if ($msg contains_i "connect from localhost[<IP_ADDRESS>]") then { stop }
+    if ($msg contains_i "lost connection after EHLO from localhost[<IP_ADDRESS>]") then { stop }
+    if ($msg contains_i "disconnect from localhost[<IP_ADDRESS>] ehlo=1 commands=1") then { stop }
     #<email-anonymizer>
     #action(type="mmexternal" binary="/scripts/email-anonymizer.sh <anon-email-format>" interface.input="msg")
     #</email-anonymizer>

Perfect, I will add the change, as I am about to release a new version soon.

Hi,
Forgive me if I'm wrong but I'm still encountering the same issue. Is it possible that the change hasn't been applied to a release yet?

That is interesting, as the code is there: https://github.com/bokysan/docker-postfix/blob/master/configs/rsyslog.conf#L63
This also appears in the image:
docker run --entrypoint /bin/bash --pull always -e ALLOW_EMPTY_SENDER_DOMAINS=yes --rm -it boky/postfix:v4.0.2-ubuntu
v4.0.2-ubuntu: Pulling from boky/postfix
Digest: sha256:f27431bb4501ab793f7b1adc1b86867d7bf134606f09b6bb5c4dc0120058c690
Status: Image is up to date for boky/postfix:v4.0.2-ubuntu
root@ea27ac36e3f4:/tmp# tail -n 20 /etc/rsyslog.conf
#   7 – debug messages (Appears during debugging only)

include(file="/etc/rsyslog.d-before/*.conf" mode="optional")

if $syslogseverity <= '6' then {
    # Do not log healthchecks
    if ($msg contains_i "connect from localhost[<IP_ADDRESS>]") then { stop }
    if ($msg contains_i "lost connection after EHLO from localhost[<IP_ADDRESS>]") then { stop }
    if ($msg contains_i "disconnect from localhost[<IP_ADDRESS>] ehlo=1 commands=1") then { stop }
}

#<email-anonymizer>
#action(type="mmexternal" binary="/scripts/email-anonymizer.sh <anon-email-format>" interface.input="msg")
#</email-anonymizer>
action(name="stdout" type="omfile" DynaFile="devicelog" template="<log-format>" DirCreateMode="0755" FileCreateMode="0644")

include(file="/etc/rsyslog.d-metrics/*.conf" mode="optional")
include(file="/etc/rsyslog.d/*.conf" mode="optional")

stop

Can you see it in the image?

I see it in both images and now the additional warning doesn't show anymore in the Debian image, or at least I see it only sometimes, not at every healthchecks, I'm a bit confused. I'm not too bothered by it. Thank you for your quick response.