[BUG]: Internal compiler error with MSVC 2022 when building nb_func.h

Problem description
We have recently started building our library Basix on Windows using MSVC 2022 using GitHub Actions. We are using scikit-build-core to orchestrate the CMake build and we build nanobind statically. We use vcpkg to bring in LAPACK and BLAS.
After switching to nanobind 2.0.0 we are consistently triggering an internal compiler error when building nanobind statically. The error points to L42 in nb_func.cpp, but that line looks relatively innocent.
https://github.com/FEniCS/basix/actions/runs/9404421938/job/25903298628#step:5:763
https://github.com/wjakob/nanobind/blob/master/src/nb_func.cpp#L42
We don't have Windows machines locally so we haven't tried reproducing this locally. Your CI seems to be running green on MSVC 2022. Any ideas?
Reproducible example code
No response

We have been getting the same internal compiler error pointing to L42 in nb_func.cpp since about a day ago.

That's quite mysterious since it doesn't seem to happen in the nanobind CI. Two requests:

Could you please open an issue with Microsoft? The true source of these issues is never fixed unless escalated to the compiler vendor.
The failure is either happening because of a change in the compiler (e.g. update) or because of a change in nanobind. Do you know which one it is? If the latter, can you bisect it to a specific commit? (That said, even in that case, the issue should be reported to Microsoft)

Regarding your second point @wjakob
We are using the 2.0.0 tag point release from wjacob/nanobind. We started noticing this error roughly from yesterday. I think it's safe to say that something upstream, either in the MSVC compiler or in the GitHub Windows Server runner image, has changed.
Possibly related issue: https://github.com/actions/runner-images/issues/10004

@wjakob, is it possible for you to trigger a windows-latest build job for your nanobind Tests job? From looking around a bit over at actions/runner-images, there seems to be a bit of talk going on that something has happened to that image within the last 24 hours. I'm curious if your test build can trigger this internal compiler error.

To throw another datapoint into the mix:
I maintain nanobind-bazel, a Bazel ruleset for Python bindings builds with nanobind, and my CI (which consists of building @wjakob 's nanobind_example across multiple Python versions with the respective installed C++ toolchain on every OS) is still entirely green on 20240603.1.0: https://github.com/nicholasjng/nanobind-bazel/actions/runs/9416682418/job/25940329652#step:1:9

Tried on a fork https://github.com/jhale/nanobind, doesn't trigger the issue  on either a slightly tweaked v2.0.0 or a current main.

Could you tell us more about your project @miklhh ?

https://developercommunity.visualstudio.com/t/Internal-compiler-error-w-nanobind-v19/10662883?sort=newest
@jhale The issue has been reported and fixed, but they haven't released the patch.

Thanks @metab0t - Even though this is not an issue with nanobind itself, I propose we leave this issue open until the fix is rolled out @wjakob?

@jhale I found out that windows-2019 image can be used instead of windows-latest as a temporary solution to compile nanobind.

Also see https://github.com/pypa/cibuildwheel/issues/1875 for another.

It's quite a strange bug. Since I could build my lib successfully on my win10 laptop with vs2022. But on github action, it just failed. I don't quite understand what's the difference.

Do you have version 16.10 locally? That’s the buggy one that came to GHA a week or two ago. It’s breaking other things too, I’ve seen three unrelated packages broken so far. Older versions are fine.

I got Microsoft Visual Studio Community 2022 (64-bit) - Current Version 17.9.5 locally.

Yes, got the first number wrong. The buggy one is 17.10. 17.9 is fine.

(If you check the link above, apparently it has been fixed internally, but isn’t in a released version yet, as of 17.10.2)

Download here old version of msvc :
https://learn.microsoft.com/fr-fr/visualstudio/releases/2022/release-history#release-dates-and-build-numbers

I found a solution to make nanobind compile on the windows-2022 image on github actions -- using clang-cl. (because Visual Studio 2019 does not support c++20 standard well, I have to use the windows-2022 image)
Although the bug is still not fixed in Visual Studio 17.10.4, the clang-cl in Visual Studio does not have this bug.
So I just add a new CMakeLists-VS.txt file and set(CMAKE_GENERATOR_TOOLSET "ClangCl") before calling project.
cmake_minimum_required(VERSION 3.20)
set(CMAKE_GENERATOR_TOOLSET "ClangCl") # newly added
project(symusic)
set(CMAKE_CXX_STANDARD 20)
set(CMAKE_POSITION_INDEPENDENT_CODE ON)

And in the github actions' workflow, I add the following step to specify which CMakeLists to use:
      - name: Force ClangCL on Windows
        if: ${{ matrix.buildplat[0] == 'windows-2022' }}
        # remove the default CMakeLists.txt and replace it with CMakeLists-VS.txt
        run: |
          rm CMakeLists.txt
          mv CMakeLists-VS.txt CMakeLists.txt

I know this is not an elegant and general way, but this does make my github actions work. Now I can precompile the new version of my lib and  upload them to pypi.

I found a way to use clang-cl that doesn't involve overwriting CMakeLists.txt: use the following if statement before project().
# https://github.com/wjakob/nanobind/issues/613
#
# Visual Studio 17.10.4's MSVC compiler gives an ICE in nanobind's nb_func.h,
# but clang-cl does not.
if(WIN32)
    set(CMAKE_GENERATOR_TOOLSET "ClangCl")
endif()

clang-cl is compatible with MSVC's commandline, so flags in if(MSVC) blocks work. The only flag I use that wasn't recognized was /Zf.

If you wrap the CMAKE_GENERATOR_TOOLSET variable in if(WIN32), you can avoid needing to overwrite CMakeLists.txt:

I got it. However, because CMakeLists are packaged with the source code and uploaded to pypi, this can cause the installation to fail when installing from source for users on Win who do not have an additional LLVM installation for Visual Studio.
And I found that when using ninja, set(CMAKE_GENERATOR_TOOLSET "ClangCl") will make cmake fail.
So if we write these settings to the same file, more if is needed.

I'd do things like this from the command line, -DCMAKE_GENERATOR_TOOLSET=ClangCL, or make a toolset file. You aren't supposed to hard code situation specific things like this in CMakeLists.txt.

I take it this is still not fixed? It was supposed to be in the latest images...

I'd do things like this from the command line, -DCMAKE_GENERATOR_TOOLSET=ClangCL, or make a toolset file. You aren't supposed to hard code situation specific things like this in CMakeLists.txt.

Well, I'm using scikit build, and the only way I find (maybe there are other ways) to pass cmake arguments is to put them in the pyproject.toml, like this:
[tool.scikit-build.cmake.define]
CMAKE_GENERATOR_TOOLSET="ClangCL"

I also don't like hard code things like this, but I only find this work. So I choose to create a temporary file, and it will be removed once Microsoft fix that bug.

I take it this is still not fixed? It was supposed to be in the latest images...

Yes, in the latest 17.10.4, the bug is still there, waiting to be fixed.

Config settings lets you do this, pip install -Ccmake.define.CMAKE_GENERATOR_TOOLSET=ClangCL . Or if you are using cibuildwheel, that can set config-settings too. You can also use environment variables, such as:
[tool.scikit-build.cmake.define]
CMAKE_GENERATOR_TOOLSET = { env="CMAKE_GENERATOR_TOOLSET" }

Now you can set CMAKE_GENERATOR_TOOLSET as an environment variable, and it will get passed in as a define.

Thanks for the tips on clang-cl!
Unfortunately the main purpose of our Windows CI testing is to make sure that our packages are buildable within Conda, which uses cl as standard, so this isn't really a suitable workaround for us.

Hi. I also noticed this issue on my own project. I notice that if I add the /permissive- to nanobind's CMake I am also able to reproduce the issue locally on nanobind's repo itself:
diff --git i/CMakeLists.txt w/CMakeLists.txt
index 82f9b28..7d9a03e 100644
--- i/CMakeLists.txt
+++ w/CMakeLists.txt
@@ -22,6 +22,8 @@ if (NOT MSVC)
   option(NB_TEST_SANITZE    "Build tests with address and undefined behavior sanitizers?" OFF)
 endif()

+add_compile_options("/permissive-")
+
 # ---------------------------------------------------------------------------
 # Do a release build if nothing was specified
 # ---------------------------------------------------------------------------

Which result in the following error:
  inter_module.vcxproj -> C:\Users\jedumas\workspace\github\nanobind\build\tests\Debug\inter_module.dll
C:\Users\jedumas\workspace\github\nanobind\include\nanobind\nb_func.h(42,1): fatal  error C1001: Internal compiler error. [C:\Users\jedumas\workspace\github\nanobind\build\tests\nanobind-static-abi3.vcxproj]

This with the latest version of Visual Studio 17.10.4 / MSVC 19.40.33812.0.

I just tried to compile with VS 2022 and C++ 20, it gives internal compiler error in nb_func.h due to some template stuff around 'func_create'
However, compiling with C++17 works without error

I have applied the following patch, which resolves the Internal compiler error. I believe this patch will become unnecessary once the MSVC version is updated and applied to GitHub Actions in the future.
diff --git a/include/nanobind/nb_func.h b/include/nanobind/nb_func.h
index e53e1a4..428ca81 100644
--- a/include/nanobind/nb_func.h
+++ b/include/nanobind/nb_func.h
@@ -199,14 +199,24 @@ NB_INLINE PyObject *func_create(Func &&func, Return (*)(Args...),

         PyObject *result;
         if constexpr (std::is_void_v<Return>) {
+#if defined(_WIN32)
+            cap->func(static_cast<cast_t<Args>>(in.template get<Is>())...);
+#else
             cap->func(in.template get<Is>().operator cast_t<Args>()...);
+#endif
             result = Py_None;
             Py_INCREF(result);
         } else {
+#if defined(_WIN32)
+            result = cast_out::from_cpp(
+                       cap->func(static_cast<cast_t<Args>>(in.template get<Is>())...),
+                       policy, cleanup).ptr();
+#else
             result = cast_out::from_cpp(
                        cap->func((in.template get<Is>())
                                      .operator cast_t<Args>()...),
                        policy, cleanup).ptr();
+#endif
         }

         if constexpr (Info::keep_alive)

A fork of nanobind with the above patch applied to v2.0.0 is available at: https://github.com/jhale/nanobind/tree/jhale/msvc2022-workaround

@jhale I'm wondering if it makes sense to temporarily merge this patch into the master branch. Does GHA still serve the broken version of MSVC?

Yea, MSVC from the current windows-2022 image still gives an ICE: https://github.com/calcmogul/Sleipnir/actions/runs/10070019011/job/27837879240

C:/Program Files/Microsoft Visual Studio/2022/Enterprise/VC/Tools/MSVC/14.40.33807/bin/Hostx64/x64/cl.exe

D:\a\Sleipnir\Sleipnir.py-build-cmake_cache\cp39-cp39-win_amd64_deps\nanobind-src\include\nanobind\nb_func.h(42,1): fatal  error C1001: Internal compiler error. [D:\a\Sleipnir\Sleipnir.py-build-cmake_cache\cp39-cp39-win_amd64_jormungandr.vcxproj]
(compiler file 'msc1.cpp', line 1593)

@melpon @jhale I'm wondering if it makes sense to temporarily merge this patch into the master branch. Does GHA still serve the broken version of MSVC?

It's still broken in GHA. It's up to you on master, I can see pros (it fixes the issue) and cons (best not to workaround compiler bugs). I will stick with patched v2.0.0 in any case.

I've merged a temporary fix for this issue but plan to remove it once GHA is functional again.

Currently the workaround causes compile errors for me (described in this discussion post), but when removing it and building nanobind myself, everything works.
Is it possible that the problem was fixed?

Currently the workaround causes compile errors for me (described in this discussion post), but when removing it and building nanobind myself, everything works. Is it possible that the problem was fixed?

Could you clarify which version of nanobind you are working with? It's fixed on main, yes.


How to integrate slight changes in the upstream packages?

Hi,
I am trying to use org-ref package in Doom-emacs, but with a slight modifications in the upstream code. When I specify (package! org-ref)  in ~/.doom.d/package.el, it directly pulls the package from upstream which does not reflect the changes I intend to make.
Unlike in Scimax, I could not figure out the location of the byte-compiled source code inside .emacs.d/ as well as .doom.d folders so that I can make my changes directly.
Did I miss something?
Cheers, and stay safe,
/z

You can fork it and use the fork. See https://github.com/hlissner/doom-emacs/blob/develop/docs/getting_started.org#installing-packages-from-external-sources.
Alternatively, you can use el-patch to modify the package.

Wedens method will work, but it makes it a little hard to iterate on, because you have to keep re-compiling the package (running doom sync) every time you make a change for it to be reflcted. Here are two suggestions:

What wedens suggested, forking the project and installing the fork:
(package! org-ref :recipe (:host github :repo "zenny/org-ref" :no-byte-compile t))

But I added :no-byte-compile t to the recipe. This way you can modify the package files without bothering to doom sync all the time.

Install the package from a local directory:
(package! org-ref (:local-repo "~/projects/org-ref" :no-byte-compile t))

Again, remember to :no-byte-compile t to save yourself that hassle.

You may need to specify :files if you need more than the top-level *.el files in a package.

Unlike in Scimax, I could not figure out the location of the byte-compiled source code inside .emacs.d/ as well as .doom.d folders so that I can make my changes directly.

For posterity, packages are cloned to ~/.emacs.d/.local/straight/repos/ and then "built" to ~/.emacs.d/.local/straight/build/. Building means symlinking the *.el files then byte-compiling them. You can quickly access a package's files by pressing SPC h P then searching for the package by name. E.g. SPC h P org-ref should open ~/.emacs.d/.local/straights/repos/org.ref.el.
Hope that helps!


Issue429

See #429 .
This is my first kotlin experience, pls check carefully for common mistakes, thx :)

@maxemann96 are you still interested in having this merged?

Currently im heavy loaded with work and have no time to fix the open points and rethink this pull request. I would consider closing this and reopen later


Improve snippet storage

At the moment, snippets are stored in a json file.
This should be probably be changed to a series of text files in a flat directory.

Any chance of moving this to something like SQLite to help it scale better and stay fast as your catalog grows?

Sounds like a great idea!
If you are willing to contribute a pull request, I'd love it.
If not, I'll try to find time in the next few weeks to work on it, but it might take a while.
-------- Original Message --------
On May 13, 2019, 18:48, Ben Lanning wrote:

Any chance of moving this to something like SQLite to help it scale better and stay fast as your catalog grows?
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub, or mute the thread.


Service Name is Incorrect

Hello,
Step 8 of this document states "Restart the Azure AD Connect Synchronization Service under Windows Service Control Manager. This is to ensure that any reference to the old password is removed from the memory cache."
The name of the service is "Microsoft Azure AD Sync".  I do not know if the name of this service is different based on the version of AD Connect that is installed.
Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: e63af3e3-677e-32d8-ca7e-c9b253ca4afb
Version Independent ID: fe8effb3-cd62-cc65-2dd9-3306a63d9e33
Content: Azure AD Connect sync:  Changing the AD DS account password - Microsoft Entra
Content Source: articles/active-directory/hybrid/how-to-connect-sync-change-addsacct-pass.md
Service: active-directory
Sub-service: hybrid
GitHub Login: @billmath
Microsoft Alias: billmath

@MatthewBrown0
Thanks for your feedback! We will investigate and update as appropriate.

@MatthewBrown0
Thank you for pointing this out! I've created a PR to update the documentation and it's currently waiting for the author to review and sign off.
I will now close out this issue. Please allow some time for the author to review and the changes to reflect.
Thank you for your time and patience!


Pattern file name text is partially duplicated if you use "component" in the name

If you generate a new component with a name of for example "Test Component" the resulting file name will be testtestcomponent.{hbs|scss|etc}, i.e. everything except "component" is duplicated.
Testing results:

? What's the name of your pattern? Test Component -> testtestcomponent.hbs
? What's the name of your pattern? SomeComponent -> somesomecomponent.hbs
? What's the name of your pattern? thirdcomponent -> thirdthirdcomponent.hbs
? What's the name of your pattern? Test Component Pattern -> testtestcomponentpatternpattern.hbs
? What's the name of your pattern? somecomponentpattern -> somesomecomponentpatternpattern.hbs

This appears to happen only with the file name, any other place the pattern name is correct (T.Module.SomeComponent, .m-some-component {}).

This is because pattern, component, modifier and decorator have a special meaning. These strings are replaced in filenames of the blueprints with desired pattern name (or decorator, modifier name)
Maybe I can improve this.

Solved with additional validation for pattern names - published in release 1.2.2


Add requirements and Python version in README

Can you please write the requirements and Python version in the README and requirements.txt the code?

Great idea! Added. Should be able to activate with:
conda activate food2vec```

Great idea! Added. Should be able to activate with:
conda activate food2vec```

Great idea! Added. Should be able to activate with:
conda activate food2vec```

Great idea! Added. Should be able to activate with:
conda activate food2vec```


KAFKA-15306 - Integrating committed offsets for updating fetch positions

Support for using committed offsets to update fetch positions.
This PR includes:

movingrefreshCommittedOffsets function out of the existing ConsumerCoordinator so it can be reused (no code changes)
using the above refreshCommittedOffsets for updating fetch positions if the consumer has enabled the Kafka-based offsets management by defining a groupId

Thanks for the comments @junrao , all addressed.

Latest comment addressed, thanks @junrao ! I merged latest changes from trunk and updated some tests accordingly. Build completed with 4 unrelated test failures:
Build / JDK 17 and Scala 2.13 / testCreateTokenForOtherUserFails(String).quorum=kraft – kafka.api.DelegationTokenEndToEndAuthorizationWithOwnerTest
Build / JDK 11 and Scala 2.13 / testFencingOnSend(String).quorum=zk – org.apache.kafka.tiered.storage.integration.TransactionsWithTieredStoreTest
Build / JDK 11 and Scala 2.13 / shouldRemoveOneNamedTopologyWhileAnotherContinuesProcessing() – org.apache.kafka.streams.integration.NamedTopologyIntegrationTest
Build / JDK 20 and Scala 2.13 / testReplicateSourceDefault() – org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationSSLTest


Problem when getting a list of commits associated with a specified branch

Hi,
In the readme is given a piece of code that gets a list of commits associated with a branch
List<Commit> commits = gitLabApi.getCommitsApi().getCommits(1234, "new-feature");
the code doesn't seem to work because the method signature (int, String) is not found.
Here's the eclipse error
The method getCommits(int, int) in the type CommitsApi is not applicable for the arguments (int, String)
Note : Im' using the latest version at the time of writing this issue (4.4.1)

Very well, thank you for your answer Greg.


how to train

hello, i am wodering how can i train the model on my own datasets

Hello,
thank you for your interest in our work. Our work is a training-free approach which utilizes pre-trained foundational models to complement Mask-RCNN.
To use your custom dataset, I would recommend you to read the guide for detectron2 on using custom dataset (https://detectron2.readthedocs.io/en/latest/tutorials/datasets.html)
You may also refer to other docs on the above website for some insights and help on how to change the data loader used in the script.
Further, the actual data loader in the main script is in this line: https://github.com/rohit901/cooperative-foundational-models/blob/fb45630c46d75f7a1856654665191aeddeb3eb98/scripts/novel_object_detection/main.py#L69

Hello,
thank you for your interest in our work. Our work is a training-free approach which utilizes pre-trained foundational models to complement Mask-RCNN.
To use your custom dataset, I would recommend you to read the guide for detectron2 on using custom dataset (https://detectron2.readthedocs.io/en/latest/tutorials/datasets.html) You may also refer to other docs on the above website for some insights and help on how to change the data loader used in the script.
Further, the actual data loader in the main script is in this line:
https://github.com/rohit901/cooperative-foundational-models/blob/fb45630c46d75f7a1856654665191aeddeb3eb98/scripts/novel_object_detection/main.py#L69

may i finetune on my custom dataset in this model

You can use our approach on your custom dataset, you can refer to the linked docs above, and also read the paper once. Our approach does not require any training, and its a training-free method.


editorIndentGuide.activeBackground is being cut by preprocessor directives

Line coloring enabled by editorIndentGuide.activeBackground loses its continuity when there are preprocessor directives along the way. In C/C++ codebases where preprocessor flags are used often, this feature loses its usability.
Yes, it's possible that preprocessor directive might be expanded into something that really breaks the current scope and defines a newer, but I think this happens less often enough to assume by default it's not the case.

Does this issue occur when all extensions are disabled?: Yes

VS Code Version: Version: 1.91.1 Commit: f1e16e1e6214d7c44d078b1f0607b2388f
OS Version: Darwin arm64

Steps to Reproduce:

Expected behavior:
Red line should have continued between and after the #ifdef scope.

I can add that also goto labels break the connection.
Seems like any block that has some mis-indented lines breaks the editorIndentGuide algorithm.


"Cicir, Horia, Mândruloc" are villages in "Vladimirescu" common, "ARAD" county

Precondition
Empty the search bar after every step

Steps 1
Acces to the link https://peviitor-ro.github.io/ui.orase/
Expected result
The page opens

Steps 2
Write "Cicir"
Expected result
In the drop down menu you find "Cicir, ARAD(Vladimirescu)"
Test Data
Cicir

Steps 3
Write "Horia"
Expected result
In the drop down menu you find "Horia, ARAD(Vladimirescu)"
Test Data
Horia

Steps 4
Write "Mândruloc"
Expected result
In the drop down menu you find "Mândruloc, ARAD(Vladimirescu)"
Test Data
Mândruloc


Discovery retry

When started at boot as a service discovery may fail due to it trying to bind to interfaces before the network is actually up. This could be prevented in systemd by starting the service after network-online.target but it requires that a wait-online.service is also enabled which is not always the case since a wait-online.service can potentially hang the boot process until it times out in certain situations. This allows for discovery to retry every 10 secs in the 1st 60 secs of uptime before giving up thus papering over the issue and not holding up the boot process.

This is another of those fixes that's been in Raspotify for months. It's helps to prevent librespot from spaming the logs at boot when it's used as a service.

The functionality seems good to me, however I recommend that you use the sysinfo crate to get the current uptime. We're already using that crate, so can avoid adding another one (or, if it depends on uptime itself, have a common facade).

The functionality seems good to me, however I recommend that you use the sysinfo crate to get the current uptime. We're already using that crate, so can avoid adding another one (or, if it depends on uptime itself, have a common facade).

Are we using sysinfo in main at all? Is there a point to pulling in sysinfo just for uptime when uptime is a super, super tiny crate comapred to sysinfo when all we'd be using is the uptime functionality?

Don't know about main but it's used in librespot-core, so no main without it.

Ok if I can pull it in in main for free it makes since to use it then.

OK let me know when you have and please also add a changelog entry so we can merge.

Don't know about main but it's used in librespot-core, so no main without it.

   |
11 | use sysinfo::System;
   |     ^^^^^^^ use of undeclared crate or module `sysinfo`

OK let me know when you have and please also add a changelog entry so we can merge.

Done.


Runtime Error

[E ProcessGroupNCCL.cpp:587] [Rank 2] Watchdog caught collective operation timeout: WorkNCCL(OpType=ALLREDUCE, Timeout(ms)=1800000) ran for 1801171 milliseconds before timing out.
[E ProcessGroupNCCL.cpp:587] [Rank 1] Watchdog caught collective operation timeout: WorkNCCL(OpType=ALLREDUCE, Timeout(ms)=1800000) ran for 1801172 milliseconds before timing out.
Why is this?
This seems to indicate that a timeout error occurred while performing some kind of parallel computing task. Specifically, when using NCCL (NVIDIA Collective Communications Library) for communication, a certain operation took longer than the set threshold (1800000 milliseconds, or 30 minutes) and was therefore interrupted.
Have you ever encountered such a problem?

Hi, you may use "--set_timeout" flag. This will extend the time limit to 1 day.


Update plugin version strings for 1.2.0

$ git grep '1\.1\.3'
jib-gradle-plugin/gradle.properties:version = 1.1.3-SNAPSHOT
jib-maven-plugin/pom.xml:  <version>1.1.3-SNAPSHOT</version>

Those are changed automatically by the release scripts, which weren't merged yet when I put up this PR.


Document user facing api changes introduced in 3280 and 3641

motivation: document manifest API / behavior changes
changes: add reference to product package name lookup done in #3280

@swift-ci please smoke test

@swift-ci please smoke test


Refactor MAGIC Calib to DL1

This pull request refactors the magic_calib_to_dl1.py script. Here is a brief summary of the changes:

Use the CameraFrame stored in a subarray instead of creating it
Modified the treatment of the find_hotpixels option
Changed the logger information display

Any comments are very welcome.

I also modified the magic_cal_to_dl1.py script as following suggestions by flake8 (max-line-length = 90), and run isort and black. @aleberti just in case could you please have a look and merge this if it looks OK to you? Thanks in advance!

Hi @YoshikiOhtani, I checked and seems fine to me. I will merge in master.

Molte grazie Alessio, I'm now doing for the other scripts now


Ordered policy engine.

Note: There is a HackMD for this PR with the diagrams fully rendered as well as allowing interactive editing: https://hackmd.io/@squeed/ordered-policy/edit

I still need to respond to Joe's feedback -- this isn't ready to go just yet.

@squeed we now have statuses for CFPs https://github.com/cilium/design-cfps#status. Please add the one that you think fits here


ci: use Chrome addon

We can probably switch to the Chrome addon for Travis CI, see https://docs.travis-ci.com/user/chrome

Hmm build not starting on this PR. Not sure but I tried to use travis chrome addon first, but ended up with chrome 62. That's why I am installing google chrome latest manually.

Works now, see https://travis-ci.org/justadudewhohacks/face-api.js/builds/448281544

Ok great, seems to be latest chrome indeed. Thanks for the PR.


Src from array of string or objects instead of http

is it possible to have the source from a static array of strings or objects instead of making an http call?

Hi @sonnny!
You could override the fetch() method and return a Promise like this:
methods: {
  fetch () {
    let items = [
      { name: 'abc' },
      { name: 'aabc' },
      { name: 'aaabc' },
      { name: 'aaaabc' }
    ]

    return Promise.resolve({
      data: items.filter((item) => {
        return item.name.startsWith(this.query)
      })
    })
  }
}


env declared in cc_binary is ignored by itest

It appears that any env values declared in a cc_binary are silently ignored when running via itest_service. The itest service is also defining an env map but for different values.
cc_binary(
    name = "binary"
    ...
    env = {"ENV1": "VALUE"},
)

itest_service(
    name = "binary_itest",
    exe = "binary",
    env = {"ENV2", "VALUE"},
)

bazel run //:binary_itest does not pass ENV1 to binary.

Good catch, we were already merging the env for tests, but not for binaries. I just pushed https://github.com/dzbarsky/rules_itest/commit/9572fa97380b6cd78a186179858f8aa434f0d726 to fix this, but unfortunately this will not work for sh_binary because it doesn't provide the RunEnvironmentInfo provider. I did add a test verifying that cc_binary works, so hopefull you should be good to go!

Awesome, thanks!


[bullet3]Update to 3.17

Bump bullet3 to 3.17

What does your PR fix?

Which triplets are supported/not supported? Have you updated the [CI baseline]

all (no change)
Note: I had to remove some of the patch, as the verbatim change breaks CMake. It appears to target APPLE; I will amend the change if the CI breaks, I have no way to check.

Does your PR follow the [maintainer guide]

I strongly hope so

If you have added/updated a port: Have you run ./vcpkg x-add-version --all and committed the result?

yes

@stephenjust I don't understand the changes to the install targets and really don't want to break something. If you can give me a quick look over the patch file.

Have you tested the feature multithreading? @rioki

I privately tested x86-windows and x64-windows with multithreading. x64-windows multithreading is even in a shipped product.

/azp run

Can you run  vcpkg x-add-version bullet3 to update versions? @rioki

LGTM now, thanks for your update @rioki.

Thanks @rioki :)


Provide dataset with analysis results as community resource and create dashboard

It may be nice to provide a dataset with our per-site crawl results as community resource. We would need to add a disclaimer that the results are not manually checked, that they can contain errors, and give sites an option to submit corrected findings to us.
More a note to self for the time being, but thoughts welcome ... :)

@katehausladen will explore this a bit along the lines of the CCPA Metrics.

I put the crawl data from December into a Google sheet. This is in our google drive, and anyone who has the link can view it. The data is added via the data analysis colab. I was thinking we would have one Google sheet, and each crawl we run will have its own tab. For future crawls, the same code can be used, we just have to specify a new tab name.

That is a good plan!

The GPP strings, both encoded and decoded, are now in the sheet. I'll link the readme for most of the columns in the Legend tab and provide a description for the others.

Highlighting the sites that are compliant, could be a nice touch.

Having some dashboard with stats, leader board, etc., could be nice. We would need to decide to which extent to build this out.

@zatchliu, if you have time, you can start thinking about how to surface the crawl data to the public so that it is digestible, easy to understand, maintainable and extendable on our end ...

I can try to create some graphs and visualizations for key metrics and summary statistics from the data in the Google Sheet organized by the data of the crawl. Then perhaps display this on a new page in the repo's Wiki or in another tab in the Google Sheet?  This should be relativity easy to extend as we could just run the analysis scripts with new crawl data as it comes out and adjust accordingly.
For now I will clean and organize the data from the previous crawls and create some potential graphs. Do you have and suggestions for how the graphs/visualizations should be organized or what key statistics should be included? Or if you could expand on what should be included in the dashboard mentioned above?

Sounds good, @zatchliu!
For now the focus can be more on the how (as opposed to the what). We can discuss more in today's call ...

As an example, here is a simple way of surfacing analysis results via a GitHub repo + Google Sheet.


Add support for reading .env files

After reading through the chat the past few days, people have wanted a different solution to dealing with config variables and one of the solutions is to use .env files.
Special thanks to @cyphactor for his awesome regex skills on this one. :+1:

I really like the idea of having the ability to load an external config file (I am already doing this manually myself) ... just not sure the public interface should a) limit to .env files and b) not liking the method names very much ...

I thought about naming them .vapor files instead of .env, however, after talking in the #development channel, it might be worth writing a custom encoder/decoder for this, but after thinking about it, is it really needed for something this trivial? Maybe @tanner0101 can weigh in on it

this is much better, hope this makes it in!

I started working on a custom decoder for this, but felt like it was just way too much overhead for such a small thing. I'll go back and work on it then. I guess it would be a more ideal and safer solution. This leads me to wonder if it's even best to use this type of config file. Is there something better, or just stick with a plain text key=value system? I'm thinking the latter just cause it's a well known format for things like this

@anthonycastelli coders can seem a bit verbose but they are really not so bad once you learn the tricks. The default fix-it that xcode gives you adds way more methods than you actually need. You should be able to implement this INI code with just ~10 methods total (since you don't need to support arrays or single value).
I know there's already a Codable impl for YAML: https://github.com/jpsim/Yams. I think one for the .ini/.conf file format would be a greatly appreciated package.
On top of actual coder you just need a thin layer that does the actual file loading. You could make this generic to work with any coder which I think would be ideal. That way people can store their config in whichever file format they want. You could probably get away with using FileManager for this since it's almost always going to happen during boot and performance isn't a huge deal. If you want to make it performant enough to use inside a route closure (not really sure why that would be useful, but who knows what people might want to do) then you should use NIO's NonBlockingFileIO type. Vapor wraps it with FileIO but you could build off of NIO's to make this code more general. It would still be quite easy to hook up to Vapor.

How does something like this work?
.env
# This is a test comment
string="Coding Test" # Another Comment
float=0.3
double=0.2
int=1
uint=12
bool=true

Config Struct
struct TestConfigs: EnvironmentConfig {
    let string: String
    let float: Float
    let double: Double
    let int: Int
    let uint: UInt
    let bool: Bool
}

Decoding
let configs = TestConfigs.load()
print(configs.string) // Coding Test

There are different readings to the 12factor manifesto. In my opinion, this line: "The twelve-factor app stores config in environment variables." does not imply that internally the application can only read environment variables.

@vzsg I am less interested in the various interpretations of 12factor manifesto and more interested in the fact that it is relatively common to configure an application via environment variables in 12factor based PaaS systems. Therefore, whatever path is chosen shouldn't prevent that from happening and in my opninion should make it easy if not trivial to support environment variables based configurations.

We're on the same opinion then.

The ability to decode to a custom struct and then set the variables to the env is still there.
confine.setEnvironmentVariables() @cyphactor
The functionality is there, just not set by default.

I think you should drop the headers :) ...
Also, please make a separate package with that decoder :))) @anthonycastelli

The whole point of this PR was to incorperate it into Vapor, a standardized way to load config variables. Unless one of the @core members vote no on implementing such thing into Vapor itself, I think it’s best to keep it here.

I don't think I have permission to create a repository on the vapor organization so you'll have to do that @tanner0101. I've gone ahead and closed this in vapor of moving it to vapor/vapor.

Feel free to rename that repo btw. I'm not sure which exact conf file format we should support. Based of this it seems .init and .conf are commonly referred to as "configuration file" format.

Since it's part of vapor-community, should it be left out of the core Vapor framework then?

Once that package is complete we can add the vapor/vapor dep and move it over to the vapor org :)

What I meant by that is once the package is complete, we can move it to the official vapor organization.


[SCB-715]working in thread that do not have context class loader

Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Make sure there is a JIRA issue filed for the change (usually before you start working on it).  Trivial changes like typos do not require a JIRA issue.  Your pull request should address just this issue, without pulling in other changes.
[ ] Each commit in the pull request should have a meaningful subject line and body.
[ ] Format the pull request title like [SCB-XXX] Fixes bug in ApproximateQuantiles, where you replace SCB-XXX with the appropriate JIRA issue.
[ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
[ ] Run mvn clean install to make sure basic checks pass. A more thorough check will be performed on your pull request automatically.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

Coverage increased (+0.005%) to 87.05% when pulling 7a1538f089b9d5456f5158d45e3e5b189844d687 on liubao68:concurrent into c78c08cebdc84fc73971b4d04966b060c51321a1 on apache:master.


Add format examples to isLatLong

I wanted to point express-validator users to here, but I found the isLatLong validator lacking an example of what the supported format is.

Thanks!


synching reports

https://github.com/CMSCompOps/wtc-console/issues/11#issuecomment-463145757
@sharad1126 has noticed a discrepancy between the report and what appears in the console.
https://github.com/CMSCompOps/WmAgentScripts/issues/374 would help preventing this as a side product

Is this the same as issue #9? If so, please check  this comment

this is about having the report being made by unified (as it already is) so that it can never be out of sync


Automated cherry pick of #2458: Use ChainExists to reduce memory footprint

Cherry pick of #2458 on release-1.2.
#2458: Use ChainExists to reduce memory footprint
For details on the cherry pick process, see the cherry pick requests page.

Codecov Report

Merging #2501 (184bc2c) into release-1.2 (e99f409) will decrease coverage by 17.66%.
The diff coverage is 0.00%.

@@               Coverage Diff                @@
##           release-1.2    #2501       +/-   ##
================================================
- Coverage        59.67%   42.01%   -17.67%
================================================
  Files              284      146      -138
  Lines            22171    18115     -4056
================================================
- Hits             13231     7611     -5620
- Misses            7524     9820     +2296
+ Partials          1416      684      -732

Flag
Coverage Δ

kind-e2e-tests
?

unit-tests
42.01% <0.00%> (-0.01%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.


Scanning packages should not claim the same file for different packages (of the same type)

Currently, scanning in ORT is package-based, and (project-)packages are identified by "definition files" (like "pom.xml", "build.gradle", "package.json" etc.) in the directory tree. So all files and directories below a definition file are regarded as belonging to the project that the definition file defines. See:
ROOTDIR
|
+-SUBPROJDIR_A
| |
| +-pom-a.xml
| |
| +-license-a.txt
|
+-SUBPROJDIR_B
| |
| +-pom-b.xml
| |
| +-license-b.txt
|
+-WEBDIR
| |
| +-package.json
| |
| +-license-w.txt
|
+-pom.xml
|
+-license.txt

So, the project spanned by pom.xml in the root directory is considered to "own" all files below the root directory, including SUBPROJDIR_A/license-a.txt, SUBPROJDIR_B/license-b.txt and WEBDIR/license-w.txt. This means that scanner findings in those file will get associated to the roo project.
However, when the scanner's view shifts to the (project-)packages in the subdirectories, the project spanned by SUBPROJDIR_A/pom-a.xml also gets the scan result for SUBPROJDIR_A/license-a.txt assigned (similar for the other subprojects).
This is historically so because ORT not really understand the semantics of a project's directory tree. However, the result can be really confusing, as scan findings (and potential violations) might show up multiple times in the reports, although they all stem from the same single file.
As a solution to this, one idea is to associate files always only to the nearest enclosing project when walking up the directory tree to the root. Maybe this logic should be limited to projects of the same type / package manager only; however, in the example above this would result in the scanner findings from WEBDIR/license-w.txt to still be associated to the root project spanned by pom.xml.
I have some hopes that the required filtering logic would be easier to implement once https://github.com/oss-review-toolkit/ort/issues/2668 is merged, as it implements some similar filtering to associate provenance-based scan results to individual packages IIUC.

And as mentioned by @mnonnenmacher over here, in the context of this it should also be ensured that Unmanaged project only get the "left-overs" associated that cannot be accosicated to any other project according to the logic.


Add example of using modals

Add example of using modals to KitchenSink.
This example should then be followed by other sample apps (but that's another issue in these apps).

Example added:

http://localhost:8080/KitchenSink/Dialog


Sent ada, txn confirmed but ada is not showing in wallet. shows on pool.pm

hi i sent ada from coinbase and eternl wallet to my nami wallet, none of th 3 transactions are appearing. been several hours now and cardanoscan shows it was compete pool.pm shows correct balance but the wallet itself does not.

@ryan993420 Talk to Cardano Support on the Live support page so that your issue can be looked into.

Same thing happened to me but thanks to Blackhathacker111 who helped me message him on telegram his telegram username is https://t.me/Blackhathacker111


Delete old/empty folders

When a note is moved, the attachments are automatically renamed/moved to the new file path. This leaves the folders of the old filepath empty - and continually growing.
Would it be possible to recursively check if the attachment's folder (and parent folders) are empty and, if so, delete them?

Try the latest version

If I set "Attachments" as the attachment folder and don't use the subfolders, when I rename/delete any file, it will delete all empty folders in the main folder. My plugin setting is as follows:

And it's the problem:

when I delete "test.md" file, it deletes all the empty folders and becomes like this:

@sans-125 Tks for your report, it's fixed in latest version.


Fix Naumann demo

Fix setup step! Still have bug on run step and hope will figure out soon

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

24 of 25 (96.0%)  changed or added relevant lines in 3 files are covered.
30 unchanged lines in 3 files lost coverage.
Overall coverage increased (+0.4%) to 87.21%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

improv/actor.py
2
3
66.67%

Files with Coverage Reduction
New Missed Lines
%

improv/link.py
2
69.17%

improv/actor.py
5
76.12%

improv/store.py
23
71.11%

Totals

Change from base Build<PHONE_NUMBER>:
0.4%

Covered Lines:
1016

Relevant Lines:
1165

💛  - Coveralls


Fix list recursive for object store (#8491)

Fix list recursive for object store

Improve lsr testing

Use normalize to add the path separator

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/2251/
Test FAILed.

Jenkins, test this please

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/2252/
Test PASSed.


added manifest to include wiki.md s

Installation fails when pip installed without edit mode, added MANIFEST.in to include .md so that the wikis can be imported.

Thank you!


Prefer acquiring reference to DeltaManager directly from Container in end-to-end tests

Currently, the DeltaManager that is created by and primarily managed by the Container is plumbed through most of the runtime layers.  For the most part, the full interface is not needed in those other layers -- we only really use the minimumSequenceNumber, lastSequenceNumber, and active properties in the data-object and DDS layers.  This change plumbs those specific properties through so they can be referenced directly, and drops the DeltaManager usages from those layers.  This will help facilitate experimentation with and changes to the DeltaManager and ContainerRuntime.
However this change does stop short of actually removing the DeltaManager from the IFluidDataStoreContext and IFluidDataStoreRuntime, since it is still used for a variety of purposes in the Fluid app.  I'll need to investigate those usages in more detail before I can make a recommendation there.
After PR feedback, scoping this change to only reduce the usage of this plumbing in our end-to-end tests (and instead prefer to retrieve the DeltaManager from the Container rather than through the data-object when possible) and not adding any new public API surface.  I'll continue thinking about options for how to scope DeltaManager access in the future.

However this change does stop short of actually removing the DeltaManager from the IFluidDataStoreContext and IFluidDataStoreRuntime, since it is still used for a variety of purposes in the Fluid app. I'll need to investigate those usages in more detail before I can make a recommendation there.

I'd prefer this to be resolved before we move with this direction.
Mainly because DDS should not differ in terms of what it has access, compared to data store.
We are planning to support data stores on channels, and thus if data store needs deltaManager, then DDS should receive it as well. Basically we want to standardize context & channel interfaces (as much as it's possible) across all layers (container runtime is the only channel in Container, channel can be data store or DDS, and data store has channels)

I'd prefer this to be resolved before we move with this direction.

Agreed, I didn't remove the deltaManager members in this change since I don't have a comprehensive answer here yet and don't intend to actually remove it until I do.
One alternative option if you might prefer leaving the interfaces untouched until I have a full plan, most of these references can still be cleaned up without the extra members (would only be leaving out the 3 DDS references and agent-scheduler).

I'd prefer not to sprinkle code base with members on various levels to quorum, audience, msn, sequence number, etc.
I believe we need to define subset of what is exposed from delta manager (and maybe ContainerContext), stick it on some interface and give access to all layers. Current interface is too large and exposes too much control, so it should be cleaned, but I'm not sure adding members here and there is the answer.
I also do not like it from composition POV - once we allow data stores to be on channels, getting something simple as sequence number may result in a ton of calls across layers until we actually reach delta manager.
For example, Scriptor -> table -> scriptor cell (I think we will organize these as "child" data stores) will go through 3 data stores and 3 contexts + container runtime + context - that's 8 functions on stack!)

Updated PR to go with the scoped option that adds no new API surface.

A note of caution here -
Container.deltaManager returns the real DeltaManager whereas all the layers below return DeltaManagerProxy. So, we need to be careful how we use them in the end-to-end tests.
For instance, in a real use case, only the app layer would have access to the real delta manager whereas layer below such as data stores / components would access to the proxy. The tests should be modeled similarly.
Also, I have noticed that when a test uses a combination of these, they sometime hang because of how pause / resume works in delta manager. This was quite a while back though and the pause / resume functionality may have gotten better. But I still think it needs to be revisited.

Container.deltaManager returns the real DeltaManager whereas all the layers below return DeltaManagerProxy. So, we need to be careful how we use them in the end-to-end tests (mostly because of the pause / resume functionality).
For instance, in a real use case, only the app layer would have access to the real delta manager whereas layer below such as data stores / components would access to the proxy. The tests should be modeled similarly.

At a high level my argument here is that the lower levels should not be given implicit access to the DeltaManager, proxy or real.    I wasn't able to remove the access in this PR but that's where I'm hoping to take it.

Container.deltaManager returns the real DeltaManager whereas all the layers below return DeltaManagerProxy. So, we need to be careful how we use them in the end-to-end tests (mostly because of the pause / resume functionality).
For instance, in a real use case, only the app layer would have access to the real delta manager whereas layer below such as data stores / components would access to the proxy. The tests should be modeled similarly.

At a high level my argument here is that the lower levels should not be given implicit access to the DeltaManager, proxy or real. I wasn't able to remove the access in this PR but that's where I'm hoping to take it.

Sounds good. Just something to keep in mind since I ran into troubles in tests :-).


Expose the make-key, make-token, and make-user-data commands

Expose these commands, now that we have back-end support for JWT auth.

LGTM


Python SSHNPD: Thread Cleanup

Is your feature request related to a problem? Please describe.
Current implementation has very poor thread management.
Describe the solution you'd like
Handle the threads better.
Describe alternatives you've considered
No response
Additional context
No response

Closed by #425


[EPIC][TEST] Implement harvester/tests Repo On ci.harvesterhci.io For #2096 To Allow For Testing

What's the current flow for the ci.harvesterhci.io:
Currently, ci.harvesterhci.io provisions based on the harvester/ipxe-examples repo:

N # of Harvester Nodes
With its only validation being the harvester-dashboard ui is accessible
There are no testing phases that actually implement any tests whatsoever based around our tests repo

What are the goals for the ci.harvesterhci.io:

To allow Jenkins to run air-gapped based tests:

originally the intent was purely for Harvester but it now encompasses a Rancher single instance to provide on the fly air-gapped integration testing

The items involved:

Dockerizing the harvester/tests repo, as it would most likely be the approach to have a Dockerfile that bundled up the tests repo and then various stages in the Jenkins pipeline could actually be triggered to run different subsets of tests
Jenkins:

we are currently over a year behind in much-needed LTS releases for Jenkins since the site is publicly facing, 2.303.1 was released in: 2021-08-25, the latest version of LTS Jenkins is 2.346.3, which was released 2022-08-10, planning the upgrade path and mitigation of issues we can leverage the 'support' plugin for Jenkins to download a Dockerfile that would contain all the information about the current loadout and cross-reference upgrading to one version at a time https://www.jenkins.io/doc/upgrade-guide/
currently the groovy script located: https://github.com/harvester/tests/blob/main/CI/harvester-installer/roles/jenkins/templates/harvester_vagrant_installation_test.groovy.j2 - has no way of if it is "updated" being automatically deployed, thus making edits to the pipeline require someone to have access to admin to the box and update it that way:

offer re-build-ability, the ability to rebuild with different parameters, if there's additional tests for instance that are being set up on a feature branch to test against new harvester-installer features that are being reviewed - or implementing testing on rc based versions of Rancher for comparing integration points, etc.
an approach may be to try to mimic what the openstack jenkins job builder does: https://docs.openstack.org/infra/jenkins-job-builder/attic/ ,  traditionally following elements from: https://github.com/SUSE-Cloud/automation , and having a seperate git repo that would contain the jenkins job yamls- where the pipeline script can live within the harvester/installer possibly
another approach may be to try to continue down the jenkins job dsl based groovy way, but creating a copy of the groovy script within the harvester-installer repo, and using the job-dsl (the .groovy) being called within a separate Jenkinsfile: https://github.com/jenkinsci/job-dsl-plugin/wiki/User-Power-Moves#use-job-dsl-in-pipeline-scripts, so harvester-installer could be the sole source of pipeline edits, be able to seed the pipeline with new changes (parameters, choice options, stages etc.) then also having a job built from the seeder that would then have set stages that switch off the given PR's comment like /run-api-tests or /run-integration-tests
we would want Jenkins pipeline to be configured with a default testing portion, some bare minimum smoke tests to run, but allow for github pr based comments via gitops to trigger other stages running other sets of tests as desired

Tentative Subtasks:

Determine Upgrade Path Compatibility to Mitigate CVE's As Well As Much-Needed Jenkins LTS Admin
Determine Automation Of Jenkins Admin Behaviors
Determine Feasibility Of Implementing OpenStack Jenkins Job Builder Instead of Jenkins Job DSL Pipeline
Determine Feasibility Of Implementing Job DSL Groovy Job Being Seeded At GitOps Comment That Also Provides All Pipeline Job Details For Running The Job As Well
Create Dockerfile That Implements All Needed Dependencies For harvester/tests Repo
Investigate A Way To Automate Plugin Version Maintenance and Plugin Additions
Determine Best Way To Get Bare-Metal Node Information Funneled To Slack With Alerting On Elements Like (disk-space, cve's with packages installed, hung jobs in Jenkins, inode availability, etc.)
Have A Testing Environment To Preform Iterations (Ideally Group Based)
Determine If We Should Be Dockerizing Jenkins Instead Of Running The JAR/WAR Directly On The Nodes

In terms of:

Determine If We Should Be Dockerizing Jenkins Instead Of Running The JAR/WAR Directly On The Nodes

We can not run it Dockerized, due to:

it needing to use Docker -> https://github.com/harvester/harvester-installer/blob/master/Dockerfile.dapper -> https://github.com/harvester/harvester-installer/blob/master/Makefile that uses a Docker build wrapper ( https://github.com/rancher/dapper ) ->  that is using luet https://github.com/mudler/luet to build the resources

Not applicable now, we are not implementing Harvester tests on ci.harvester.


[FEAT] install and build github actions

Add CI to github repo in order to patch version of the package.json.

All good to me


A serious error has ocurred

Hello good day, this error only happens when i have Morrowind Zombie Apocalypse loaded in the game.
If i deactivate said mod, tes3merge will work normally.
TES3Merge.log
Additional information:
I don't use OpenMW and i am on Windows 10 OS, i use MWSE though.

It got fixed by the newest dev build.
I will close this thread as solved.


[Feature Request]: Multiple embeddings per document

Describe the problem
Is there a way to use multiple embeddings functions?
My understanding is that each collection has a single embedding function. I would like to be able to have several and then specify which one to use when querying.
Is there some way to do this with already with multiple collections? If so ideally I would be able to have one collection with NO embeddings and JUST documents and ids. And multiple other collections with NO documents and JUST embeddings and ids.
Describe the proposed solution
I should be able to create a collection with multiple embedding functions. Ideally I should be able to add more functions later as well.
OR
There should be a clear way to create a collection with NO embeddings, and then a way to create a collection that sources its docs from another collection.
Alternatives considered
No response
Importance
would make my life easier
Additional Information
No response

@eebmagic can you tell me more about your use case? and the motivations behind this? we have thought about this but currently don't have a super clean way of doing this internally.

@eebmagic can you tell me more about your use case? and the motivations behind this? we have thought about this but currently don't have a super clean way of doing this internally.

@jeffchuber
I'd like to be able to do some comparisons of embedding models for one large set of docs.  Also this would be nice if I had specializer models and wanted to query different aspects of documents with those embeddings without having to manage parallel collections.
I'm assuming that creating several collections with the same docs but different embedding functions would result in redundantly storing the docs for all N collections (and would also just be a bit more work to manage N collections) which would require alot more storage than I can afford given my document set size.
I think the ideal setup for me would be having one collection with the docs and multiple embeddings for each doc and then when I run a query I would just specify which embedding to use. I think Milvus lets you do this, but I was having other problems with Milvus.

@eebmagic currently our recommended path here is to write a small util to manage the "duplicated" index. should be pretty easy - ofc a bit less efficient to store the metadata and document twice, but I think the simplicity and making it "easy to reason about" is worth it

So if say, for example, you want to parse a directory of documents and define multiple collections because you want different embeddings, do you need to separately add the documents to each collection with its .add_documents method?
But if so, each collection will have its own doc_id for the same retrieved document, so how will you be able to combine them together?

@namp I'm imagining if the solution were to require multiple collections, then I should be able to create the second collection as a "bound" or "slave" collection to the first that already has all the docs. Then ideally when ever the doc set changes in the first collection the embeddings would be built in both collections.
I don't think doc ids don't have to be unique across collections, so you could be able to have a single id for a doc, the actual doc text stored in a single collection, but multiple embeddings associated with the doc id in separate collections.
The major point is to avoid redundant storing of a large set of texts.

I have a use case. I have used openAI as embedding model for my current database setup, but I would like to start migrating to a different provider to cut down on costs. Thank would mean I have to backfill my DB once again with different embeddings. Having both version ready in production would help cut down on costs on more heavily used elements of my app, while retaining the richer output for other parts of it. Having both versions available in one document would leave me with a lot cleaner database and save me time having to use different database routes for specific use cases (I imagine this might get messy real quick).

@namp @eebmagic @Jkense while we don't currently support this, there are a couple of workarounds to try.

Have a 'master' collection which acts as the document store, with original embeddings. 'slave' collections can then contain just the new embeddings, and metadata which references the document ids in the master collection. When querying a 'slave' collection, after results are retrieved, extract ids from the result's metadata, and call get(ids=[...]) on the master collection.

This approach avoids redundant storage of documents, and lets you link them in this fashion, though it requires some additional logic.

Write a DataLoader for text like the one we have for images (we would be interested in this contribution). Both collections can then just refer to the same uri. In this setup you'd be maintaining your own document storage.

Neither is perfect but we don't currently have specific plans to implement this feature, though we may in future.

Use case:
chunks tend to be longer, but include summaries.
As a user, I would like to be able to find chunks be the vector similarity one of/either/both the summary and/or the raw content itself.
Motivation:
As small LLMs become better they will become a part of the data ingest process more and more. It's likely I'm the future that chunk size will grow and we will need semantic search both on the original chunk content as well as the summary, and let the LLM further down in the RAG process pick which part of the chunk to include in the context window.


OpenSSL extension is required.

The picture shows the error message.I search the php.ini and sure openssl extension is ok.
In the first time,I load the index.php.Everything is fine.
However, load again the page.The problem comes.
If I clear the cookie in the browser,it is ok.
Please help me!!

restart your apache when you have changed your php.ini.

The problem is solved.

Close this please?


Literals are valid LeftHandSideExpressions

So turns out that null and other literals are valid LeftHandSideExpressions and through the use of early error conditions disallows Literals in AssignmentExpression etc.
References: kangax/compat-table#380, #185, https://twitter.com/webbedspace/status/550898215130640384

weird.

Yeah it's bizzare and weird but what else can you expect from JavaScript. 😕

So this might mean we have to revert 0b59fc198b (#185) again, since LeftHandSideExpression does not actually mean the same as 'is assignable to'?

@marijnh That commit is fine, it's just that checkLVal behaves incorrectly. checkLVal isn't synonymous with "is assignable to", so another method will have to be made like checkAssignable and it used in those scenarios.

As far as I can see, checkLVal encodes exactly the tests for being assignable to, rather than being a LeftHandSideExpression.

@marijnh Woops, you're right. I naively jumped to the conclusion that lval stood for lefthand value or something like that.

It does, but not in the way the JS grammar uses lefthand value ('lval' is traditionally also used to mean 'can be assigned to').

Any expression is a LeftHandSideExpression through grouping parentheses.

So then this is simply about precedence, and patch 0b59fc1 should definitely be reverted (and replaced by making a call to parseExprSubscripts instead of parseExpression, maybe?)

Yes. And add parentheses around the class expression in the test.

Those should take care of this.


[Internals] Polymorphish data model for sat transfers

The only sat transfer mechanism we really have currently is the vote and boost (which is in the same db table but differentiated with a boolean). Even the dust cost of posting and commenting is kind of recorded as a vote.
We want to support many types of sat transfers on the site eventually. To name a few immediate ideas:

tips
referrals

e.g. every user who signs up with your referral link gets 20 sats and you get 20 sats

faucets/rewards

e.g. 10 free sats when you sign up, 100 sats if you visit every day this week, 1000 free sats if you post a story that gets to the front page, etc.

Eventually:

gating replies to your posts/comments with custom sat amounts

e.g. every reply to this thread must 'stake' 10 sats rather than 1 sat

Q&A bounties

e.g. the best answer to my question by Saturday gets 1000 sats and if I don't pick a winner it's distributed to all answerers

sub-stacker (read subreddit) economies, i.e. create subs with different sat-based unit economics

e.g. every post in this sub-stacker costs 100 sats (X amount goes to the sub mod, Y amount is airdropped on the sub users, Z goes to the site), comments cost 25 sats, etc.

Let users set a sat cost for mentioning them , i.e. charge users for consuming your attention
DM's gated with a sat cost

e.g. pay per message or pay to unlock a conversation with me

The sky is the limit IMO and we are going to the moon my plebs! These are naive examples. The incentives will have to be carefully examined before deploying of course.

Anyway, we don't want a DB table for all of these so we should probably have something more generic. The trick is making something intended to be generic also performant/correct so we should deliberately design such a system.
Off the top of my head it's probably best to have an xfer model with a sender id then we have a polymorphic receiver (e.g. an item (post/comment), a user for direct tips, a message for message hurdles) ... I'm not sure how well all of these will fit into this model so I'll probably attempt an exhaustive experiment to make sure any particular design fits.

As I implement this, I'm coming to the conclusion that it's over-engineered. I should perhaps just rename and adapt the Vote model as features and use cases arise rather than attempt to anticipate all future needs. e.g. There are far future scenarios that I'd have to design into this if I want it to be truly generic, and it's questionable whether the intermediate generic forms (mostly supporting tips and referrals) would be better than iterating on Vote. Certainly I can get tips done faster by iterating on Vote.
Also, upon examination referrals fit poorly into this model.
On top of it all, stuffing all sat transfers into a single model is mixing too many concerns, let alone wholly anticipated concerns.


Save map for offline use, gets stuck while downloading.

Describe the bug
While downloading a map to save for offline use, the download gets stuck.
Are you able to reproduce the problem in a new project after restarting the app?
Yes (As per the drop box comments)
Screenshots

Fixed issue with commit b5aca6b164aee0f6b3d418dcb40f0e942065c23a which is a merge of MapChanges branch.


Verify the payload file is greater than 0

Is this checking that the tar/tgz file isn't zero-byte, or is it looking at the payload Word Document within.  This is to cover for a valid tgz / JSON etc, but where the .doc file uploaded by the transferrer is unexpectedly empty.  This isn't something we'd notice without an explicit test.

Actually ignore my previous comment as it's all fine. I'm just not very good at reading Scala but I can see it's all good.


Documentation

Is there any reason why the doc is in .rst? It doesn't render well on GitHub :/

Mostly because I'm used writing rst for docs :) However, I'm fine with using markdown.


Make deploy reliable

create-deploy-branch creates a branch derived from the current one, but including the files in .gitdeploy that are normally ignored. So if you're on master, it creates deploy/master, with those generated files included.
This means the built monolith will get included.
It might make sense to add a check for a clean repo in there, and bail if not.

gulp monolith does not create a new commit, but it is the build that yields the ignored build/ directory that the deploy actually uses

(basically, create-deploy-branch is my compromise between committing build products for deploy and ignoring them for a 'clean' repo.)


Update/Add binutils-arm-none-eabi

Tool information

Tool name: binutils-arm-none-eabi
Add or update? Add
Desired version: 2.25-5+5 (latest)
Approximate size:
If this is an add request:

Brief description of tool: GNU assembler, linker and binary utilities for ARM Cortex-A/R/M processors
URL for tool's homepage: https://packages.debian.org/source/jessie/binutils-arm-none-eabi

Area for Triage:
Packages
Question, Bug, or Feature?:
Feature
Virtual environments affected

[x] macOS 10.15
[x] Ubuntu 16.04 LTS
[x] Ubuntu 18.04 LTS
[ ] Windows Server 2016 R2
[ ] Windows Server 2019

Can this tool be installed during the build?
Unlikely.
Are you willing to submit a PR?
No.

Tool can be installed in runtime via the following code (installation takes about 30 seconds):
# Ubuntu
sudo apt-get install gcc-arm-none-eabi
# macOS
brew tap ArmMbed/homebrew-formulae
brew install arm-none-eabi-gcc

Hi @martmists, considering that it's fairly easy to install on runtime we won't be baking this into the image at this time. Depending on popularity and interest we might reconsider later. Thanks for the suggestion.


Hacky attempt at integrating this with FaRL LAION checkpoints

Hi there! First of all thank you for the work on this awesome project.
I'm looking for an easy way to run the FaRL LAION checkpoints on a directory of images. The code structure in the main FaRL repo is not user friendly for inference uses so I tried to get it to work in here. However, since the LAION trained checkpoints are not JIT compiled I had to manually copy-paste the torch modules over and integrate this with the old inference code you had written in this repo.
My implementation is likely wrong because it does not output correct looking predictions. Maybe you can tell me what I'm doing wrong :) Or we can brainstorm a better solution to getting this repo working with the FaRL LAION checkpoints (maybe JIT compile them?). It could be very valuable for the research community!
https://gist.github.com/evancasey/eea749d7186e92670fca728ddb384212
Sample output:

Hello， Do you find the bug in your code implementation?


"Added new NEAMS logo" #238

Added NEAMS logo that works in light and dark mode

Job VTB Documentation on 10c509e wanted to post the following:
View the site here
This comment will be updated on new commits.


RFC: Brainstorming single-page documents

Building off discussions in #113, I've been thinking about what a single-page document could look like in the context of Jupyter (Note)Book. Specifically, I'm trying to take a step back and think a little on the life cycle of such a document, from authoring and collaborating to rendering.
It'd be great to get feedback on what folks think of this workflow so far, if there's anything I've missed, or if you have any ideas on next steps !
Authoring
Jupytext allows researchers to create unexecuted Jupyter Notebooks in Markdown, RMarkdown, and several other formats. This may be useful in improving the authoring experience with Jupyter Notebooks, since it allows for readable version control under Git or other Version Control Systems and interacting with the content in a standard text editor.
The distinction between authoring a Jupyter Notebook in RMarkdown and authoring a standard RMarkdown report, then, is that Jupyter Notebooks enable the option to version control generated outputs -- if the author does not require a legible diff. This can be especially useful in situations where the outputs take substantial time and computational resources to generate, as in many science-oriented use cases.
For scientific publishing, several features are still missing from the Jupyter authoring experience. Among these are in-text formatted citations and auto-generated reference lists à la BibTeX, numbered theorems, as well as options to include footnotes and informational sidebars. All of these features are available either natively in Pandoc or as pandoc-filters. Pandoc recently announced it would natively support the Jupyter Notebook format (#94), enabling the possibility of a smooth experience for converting between .ipynb files and other file types.
Open questions about authoring:

Is there a way to track outputs with legible diffs ?
Should bibliography style (e.g., APA) be added as a metadata field for Jupyter, or at least Jupytext ?
What would the ideal UI look like for this authoring process ? A GUI where users can switch between the Markdown and Jupyter Notebook views ?

Collaborating
When authors want to share a Jupyter Notebook publication with their co-author, it is unclear what object is best to share. Ideally, collaborators would have access to both the inputs and outputs, meaning that Jupytext Markdown is unlikely to be sufficient unless all authors are working in the same environment (e.g., a shared JupyterLab).
Commenting features such as those available in Microsoft Word are often highly desirable for scientists; excitingly, there is ongoing work to render Jupyter Notebooks as Word documents. Another option is to use ReviewNB, a proprietary tool developed to enable cell-level notebook comments. Nonetheless, it would be preferable to avoid tying this workflow to proprietary solutions, and it is therefore advisable that other collaborative workflows are established. At this point, however, such a workflow seems to necessitate reverting to the Jupytext Markdown and using tools such as e.g. HackMD, hackpad, or equivalent to collaborate online.
Open questions about collaborating:

Should co-authors review the Jupytext paired .md / .ipynb or the executed Jupyter Notebook ?
If the latter, how would we recommend co-authors to leave comments directly on Jupyter Notebooks ?

Rendering
Generating PDFs would provide traditional, static representations of the Jupyter Notebook that are familiar to most academic readers. This would be in addition to the HTML outputs currently created with Jupyter Book.
Radix, developed by the RStudio team and since renamed to Distill, uses pandoc to convert RMarkdown into a Distill-style HTML template. Using pandoc as a conversion engine enables it to capture many of the desired features outlined above in Authoring. A similar template could be adopted for .ipynb files. We might like to add additional features, such as “show/hide” code cells.
For both PDF and HTML, we should continue to link directly to the underlying notebook, both as an optional download (#231) and as an executable object in an associated Binder environment.
Open questions about rendering:

What barriers are associated with moving to pandoc rendering of Jupyter Books ?

This is great! here are some thoughts from me on a few of the questions here:

Is there a way to track outputs with legible diffs ?

Not that I know of - you can use something like nbdime for diffing notebook outputs (at least, I think it diffs the outputs), but I doubt that'll be a solution for most authors because they want something that works on GitHub. I wonder how far we could push the "dual markdown + ipynb" workflow...then you could use the github diff-er for outputs.

Should bibliography style (e.g., APA) be added as a metadata field for Jupyter, or at least Jupytext ?

That seems reasonable to me - we don't necessarily need this to be a change to the notebook format or anything, we could just choose what kind of metadata to treat as "special". What about something like a field called "bibliography" that would contain either:

a list of bibtex-style entries in the metadata of the notebook
a path to a bibtex file

?

What would the ideal UI look like for this authoring process ? A GUI where users can switch between the Markdown and Jupyter Notebook views ?

Yeah, something like Jupytext would be great for this if we could get the notebook to automatically re-render whenever the markdown file is updated (and vice-versa). I bet that'd be possible. Then you could just open them side by side if you wished...maybe even create some UI elements that gave 'hackmd'-style buttons (that automatically open them side by side).
Another cool thing would be if you could run code on the markdown file and it'd execute in the corresponding notebook file...hmmmm

Should co-authors review the Jupytext paired .md / .ipynb or the executed Jupyter Notebook ?

IMO it'll require less machinery if we want people to review .md files since we've already got a common pattern of using GitHub for this

If the latter, how would we recommend co-authors to leave comments directly on Jupyter Notebooks ?

That's a good question - I agree this might add a (non-trivial) amount of code and customization.

What barriers are associated with moving to pandoc rendering of Jupyter Books ?

The main things that I can think of:

Pandoc is a dependency, which may or may not be a big problem for folks (I'm inclined to think it's fine)
Pandoc output is slightly different from nbconvert output, which can create some annoying tasks when it comes to writing CSS rules etc. This wouldn't be a big deal if we switched entirely to one or another, but might be tricky if we use both.
In my experience, nbconvert does some more intelligent things about retaining jupyter-specific metadata etc in the outputs of cells. I haven't tested this rigorously, but I suspect that nbconvert will be better at keeping different kinds of HTML outputs if they're non-standard. No idea if that's really true though.
The biggest challenge, to me, is that any changes we'd need in pandoc will be much harder to implement than if they were in nbconvert - pandoc is written in Haskell and I don't know anybody that knows Haskell lol

Lemme know what you think!

There's a ton of great ideas going on here.  I'm glad you all are thinking hard about this.
Here's a go at @emdupre's idea surrounding Distill.pub: https://github.com/roualdes/nbdistill.  I've only spent a few hours on it.  It's definitely incomplete, but thought you all might be interested anyway.

holy cow that is beautiful 🤩🤩🤩
I'd love to find a way to build this into jupyter-book, maybe as a separate theme? I'd love to be able to standardize the design elements across themes as much as possible (e.g. a right sidebar, figures+captions, etc).
Related to that: how much of your distill example depends on modifications to the HTML template? I see there are a bunch of classes etc added in there, are those specific things that distill looks for? I'm wondering if we can achieve the same effects, but using the same basic HTML structure that nbconvert uses...

Thanks.  Building this into jupyter-book would be great and I'm happy to help.
Re modifications to the HTML template: While I didn't add Jinja2 blocks or modify any of the blocks that nbconvert defines, I did use many of the HTML tags defined by Distill.
nbconvert side: I started with nbconvert's template html/full.tpl, and then used some of the Jinja2 blocks from the templates in the folder skeleton as well.
distill side: any HTML tag that is prefixed with d- comes from distill's javascript.  Distill also has a number of HTML classes to use for specifying the layout.  These classes are all prefixed with l-.  For instance, in my report.tpl file you can find <div class="l-gutter">.  There are other layout options I'd like to use, but haven't gotten to this yet.

OK cool - maybe I don't have a super clear understanding of how Distill works. It sounds like it inserts HTML into the page itself using the distill javascript library? Is that right?
I think for some elements that'd be fine. Do you think that we'd need to have a distill-specific jinja template in that case? Or could we get away with just CSS differences + the updates from the JS library?
My order of preference here is:

We use a single HTML template for Jupyter Book.
We use two HTML templates, one "base" and one "distill" that inherits from base and makes just a few updates
We use two totally separate HTML templates

Yep, that's it.  Distill inserts HTML into the DOM itself.
What we'd need to have depends on where it is we're trying to go, and I'm not super clear on the specifics of the goal yet.  Nonetheless, here's an attempt at a well thought out answer with only HTML output in mind.
I think we could use a single HTML template for Jupyter Book if we selectively picked structural elements from Distill to use.  For instance, if we only wanted to borrow Distill's citations support.  Selecting the right elements would just take some care.
Having two HTML templates, a base and a more specific one, offers a lot of flexibility at the expense of some maintenance.  I feel like this option is a reasonable path towards themes, re #262. I think both Tufte CSS and Distill would somewhat naturally allow for theme specific templates.
I worry that option 3 would be too much of a maintenance burden.

After #278 lands, I think that the template should be relatively stable for a bit (potentially...:-) ). I think that PR will also lay the foundation for lots of distill-like elements (e.g. sidebar, etc). Perhaps we can prototype what would be needed to get a distill theme for Jupyter Book? I'm definitely happy to review a PR! :-)

Closing as this should be superceded by beta.jupyterbook.org, specifically with the jupyter-book page command


Review BigInt usage

Inspired by https://github.com/ChainSafe/lodestar/issues/5892 and https://github.com/ipfs/protons/issues/112
cc: @wemeetagain

For the most part we only use BigInts where a protobuf definitions means we need to handle values encoded as 64 bit numbers - we cannot use the js number type here because it can only handle integers up to 53 bits.
If the incoming message specifies things as 64 numbers I think we need to support the full range otherwise we could open ourselves up to attack or introduce unpredictable behaviour since js nodes will act differently compared to other implementations based on the data they are sent.

I think it depends on the field. For example, in https://github.com/ChainSafe/js-libp2p-gossipsub/pull/327 we are looking at decoding the backoff field in the prune control message as a number. The backoff field is the time in seconds to wait before re-grafting. If a huge backoff value is provided, it doesn't really matter if we wait 2 ** 53 + 5 or 2 ** 53 + 1 seconds because we'll all be dead anyways.
This is the impetus for this issue. Inspect which uses of bigint are needed in practical terms.

Related #2000
Another place where BigInt is used where the range provided by number is beyond adequate.


refactor: 💡 Eliminate RPC specific conversion functions

Description
Modify the schema object to make its types compatible with the rest of
the api. This elminates snake case conversion functions to DRY the code
base
Breaking Changes
JIRA Link
DA-328
Checklist

[ ] Updated the Readme.md (if required) ?

:tada: This PR is included in version 14.0.0-alpha.16 :tada:
The release is available on:

npm package (@alpha dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 15.0.0-alpha.1 :tada:
The release is available on:

npm package (@alpha dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 15.0.0-beta.1 :tada:
The release is available on:

npm package (@beta dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 15.0.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Update build.js

I've updated the build.js file so the classes are initialized with 'window' instead of 'this' . I didn't build /dist cause I assume you prefer to do that according to the discussion in issue #28 (I wasn't able to get the jake build task to run without error anyhow).
thanks,
Greg

Thanks


Sign internals of EXE package so that it works correctly when signed

PR Summary
Sign internals of EXE package so that it works correctly when signed

Refactor exe package creation into separate function
Add functions to enable signing of engine of the exe package
Move signing of MSI to package creation Job

PR Context
The extracted EXE and MSI of the wrapper must be signed if the EXE is signed or install will fail.
PR Checklist

[x] PR has a meaningful title

Use the present tense and imperative mood when describing your changes

[x] Summarized changes
[x] Make sure all .h, .cpp, .cs, .ps1 and .psm1 files have the correct copyright header
[x] This PR is ready to merge and is not Work in Progress.

If the PR is work in progress, please add the prefix WIP: or [ WIP ] to the beginning of the title (the WIP bot will keep its status check at Pending while the prefix is present) and remove the prefix when the PR is ready.

Breaking changes

[x] None
OR
[ ] Experimental feature(s) needed

[ ] Experimental feature name(s):

User-facing changes

[x] Not Applicable
OR
[ ] Documentation needed

[ ] Issue filed:

Testing - New and feature

[x] N/A or can only be tested interactively
OR
[ ] Make sure you've added a new test if existing tests do not effectively test the code changed

Tooling

[x] I have considered the user experience from a tooling perspective and don't believe tooling will be impacted.
OR
[ ] I have considered the user experience from a tooling perspective and opened an issue in the relevant tool repository. This may include:

[ ] Impact on PowerShell Editor Services which is used in the PowerShell extension for VSCode
(which runs in a different PS Host).

[ ] Issue filed:

[ ] Impact on Completions (both in the console and in editors) - one of PowerShell's most powerful features.

[ ] Issue filed:

[ ] Impact on PSScriptAnalyzer (which provides linting & formatting in the editor extensions).

[ ] Issue filed:

[ ] Impact on EditorSyntax (which provides syntax highlighting with in VSCode, GitHub, and many other editors).

[ ] Issue filed:

disconnect/connect are not data verbs


hyperparameters for resnet50 training

I am wondering if you can share the hyper-parameters for resnet50 training. I have trained a model but it is 2% worse than yours.

@pengleigithub The default settings are the hyper-parameters of the paper. If you directly run the train_alignedreid.py, you will get similar results shown in README.

In the paper, it says 'mini-batch size is set to160,in which each identity has 4 images.  Each epoch includes 2000 mini-batches'. I am wondering if there is a typo. Each mini-batch has 40 identities and there are 2000 mini-batches. So there are 40 * 2000 = 80000 identities in each epoch. But none of the datasets has 80000 identities.

@pengleigithub Please wait our new paper Alignedreid++. The experiments of AlignedReID exist some mistakes.

Can you share the details since it could be a long time before the paper is out?

Can you share the details since it might be a long time before the paper is posted? Thx.

AlignedReID++ trains and tests on each datasets, and add some theory explanation.

Thx. Can you share the training details on market1501?

All details is in the train_alignedreid.py. There are no any tricks.

Adam
300 epoch
lr:0.0002 (0~150); 0.00002(150~300)
loss: softmax + triplet loss


v3.0.0

Here is a running list of things to be merged into master for v3. There is a working branch named three-master where these breaking PRs will be integrated and tested and ultimately merged into master.
Changes:

[x] Replace runnable-hermes (PR #35)
[x] Update error-cat to v2 (PR #33)

Sanity:

[x] Take existing Ponos project and change it to use three-master

Documentation:

[x] Changelog
[x] Breaking changes
[x] Migration guide

Done!


Support for Web Components

Clear and concise description of the problem
I work with Lit Element and web components and I have had problems when integrating the library mainly with shadow dom, I have had to do some hacks with light dom, if the library had support for web components natively it would be of great help
Suggested solution
Creation of an official web component of the library
Alternative
No response
Additional context
No response
Validations

[X] Follow our Code of Conduct
[X] Read the docs.
[X] Check that there isn't already an issue that request the same feature to avoid creating a duplicate.

Would you like to open a PR for this feature?

[ ] I'm willing to open a PR

I have had problems when integrating the library mainly with shadow dom

What problems? Should we guess?

if the library had support for web components natively it would be of great help

Can't imagine what it can do better that is can't be done with vanilla JS library


Scrolling RecyclerView is not smooth

Scrolling normally on the RecyclerView, the Handle is not moving until the next item is reached and so it changes position suddenly: basically the movement is not linear.
This effect is more visible with few elements while with a lot of elements it might be considered acceptable, even if it is not nice to have/see.

So I've reviewed the code.
Now IMO, it is much much better, scrolling RecyclerView is smooth in all cases!
In my code there's a bug with high numbers, proportion is still not ok (should be smaller than now), so I had to add a new factor, called C, in the calculation (experimented while watching that proportion was too big, but now it is a bit too big with high numbers).
Also it's critic the calculation when the difference (itemCount - visibleRange) is 1 or less.
I removed all the old calculation. Now I use dy (delta y) and I put it in the calculation with the new proportion. Please have a look and let me know, this is the new code that should be fixed however:
	@Override
	public void onScrolled(RecyclerView rv, int dx, int dy) {
		//Only react on scroll events when not done by moving the handle by touch
		// prevents nervous jumping of the handle
		if (handle.isSelected()) return;

		float itemCount = recyclerView.getAdapter().getItemCount();
		float visibleRange = recyclerView.getChildCount();

//		Log.d(TAG, "------------------------");
//		Log.d(TAG, "onScrolled itemCount=" + itemCount);
//		Log.d(TAG, "onScrolled visibleRange=" + visibleRange);
//		Log.d(TAG, "onScrolled dy=" + dy);

		//This decreases the proportion to adjust the final multiplication with dy.
		float c = itemCount / visibleRange;
//		Log.d(TAG, "onScrolled factor c=" + c);

		//Subtracting visibleRange value: these items must removed from calculation
		// (increase proportion and scroll faster with few items)
		float proportion = (itemCount - visibleRange < 1) ? height :
				visibleRange / (itemCount - visibleRange + c);
//		Log.d(TAG, "onScrolled proportion=" + proportion);

		float y = handle.getY() + dy * proportion;
		//dy could be bigger then necessary (due to fling) fix min and max value
		if (y < 0) y = 0;
		else if (y > height - handle.getHeight())
			y = height - handle.getHeight();

//		Log.d(TAG,  "onScrolled Y=" + y);
		handle.setY(y);
	}

Can you show it in the sample app perhaps?

I tried with 15 elements. With your code:
https://www.dropbox.com/s/ggs4fh6mxamdl1i/capture_Fri_Jul_17_19.32.00.mkv?dl=0
With my code handle movement is smooth during the scroll, it takes the "delta y":
https://www.dropbox.com/s/7fle9f4eb2xldwh/capture_Fri_Jul_17_19.30.00.mkv?dl=0
But there are 2 bugs, code needs improvements:

When handle is arrived at bottom/top there's still some scroll to do.
When handle is dragged and released, position is not relative to the effective position Y. Especially visible if you scroll the RecyclerView again: handle can't move because scroll is finished (hande y got different y caused by manual drag).

I hope it's clear my explanation and videos as well.
I really would like this library works well :-)

oh, it's not that bad. This reminds of what happens to the normal fast-scroller of a ListView, where you choose "setSmoothScrollbarEnabled(false)" :
http://developer.android.com/reference/android/widget/AbsListView.html#setSmoothScrollbarEnabled(boolean)
I hope to see how to fix it too.

Any news here?

@Rainer-Lang No. Otherwise I would have closed this issue. :)

I have some questions，But the demo did not.

@haogefeifei I don't understand.

anyway issue was fixed by @dimbslmh .

I'm not sure, but it might be that adding decorators ruin the fast-scroller, as they can appear and cause computeVerticalScrollOffset and computeVerticalScrollRange jump to different values.
I've found it when adding this library : https://github.com/eowise/recyclerview-stickyheaders


[Pico W] Vendor specific host implementation

Hello,
I have three interconnected hubs and on each hub I have connected an STM32 whose USB is in vendor-specific device mode. I want to plug it to a pico w to establish wifi communication through usb(see image above).
Is there an example on vendor specific host implementation or tell me how can i build it?
Thanks.

Details of the tinyusb examples are in the readme here https://github.com/raspberrypi/pico-examples/blob/master/README.md
and https://github.com/hathach/tinyusb/tree/master/examples/host
This sounds complicated, I doubt there's an example that does exactly what you want.


Feature Suggestion: allow upload/create Prebid JSON config file like fontello

The prebid.org website is great! One thing that would make it even greater is allowing users to skip the configuration of the build of Prebid.js from http://prebid.org/download.html every time they want to update the version of Prebid on their website.
You could achieve this by using the http://fontello.com/ model.

 Also in that ZIP file is a config.json file that indicates all of the icons you've selected.
The next time you go to fontello.com it:
2a. remembers that config by reading out of the browser localStorage where it wrote the config at last download button click
2b. and allows you to drag/drop that config.json file from your ZIP file onto the page to auto-select all of the fonts you've already selected

This makes re-downloading or simply updating an existing configuration of font icons really easy, and would make downloading/updating prebid.js from prebid.org/download.html super easy as well!
Just a suggestion - one that seems like it would be easy to implement on prebid.org/download.html! :)

@Wade-McDaniel - thanks for the supportive idea.
We don't have time to implement this kind of feature, but happy to take community contributions. Adding the "help wanted" label here to see if anyone's willing to take up this kind of task.
Due to privacy concerns, I would mandate that such a feature is opt-in -- we can't be writing to user's devices without very clear notice and approval.

I'm glad this feature request sounds appealing!
But I wanted to clarify one aspect of my suggestion: the JSON config file
could only be part of the downloaded prebid.org Zip file that contains the
prebid.js file if also writing to the browsers local storage is a security
concern.
Thanks!
On Thu, Jun 11, 2020, 9:42 AM bretg<EMAIL_ADDRESS>wrote:

@Wade-McDaniel https://github.com/Wade-McDaniel - thanks for the
supportive idea.
We don't have time to implement this kind of feature, but happy to take
community contributions. Adding the "help wanted" label here to see if
anyone's willing to take up this kind of task.
Due to privacy concerns, I would mandate that such a feature is opt-in --
we can't be writing to user's devices without very clear notice and
approval.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/prebid/prebid.github.io/issues/1780#issuecomment-642799577,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AEJHMV6X2IAR4U75GCAMMITRWECOXANCNFSM4KNKMJ7Q
.

Going to give this one another 6 months -- if someone doesn't pick this up by mid-2023, going to close this minor enhancement request


Emergent Abilities of Large Language Models

집현전 최신반 스터디

2022년 7월 3일 일요일 10시
김유빈님 오수지님
논문 링크: https://arxiv.org/abs/2206.07682

Abstract
Scaling up language models has been shown to predictably improve performance and sample efficiency on a wide range of downstream tasks. This paper instead discusses an unpredictable phenomenon that we refer to as emergent abilities of large language models. We consider an ability to be emergent if it is not present in smaller models but is present in larger models. Thus, emergent abilities cannot be predicted simply by extrapolating the performance of smaller models. The existence of such emergence implies that additional scaling could further expand the range of capabilities of language models.

발표자료 첨부합니다~!
집현전_7조_220703.pdf

발표링크: https://www.youtube.com/watch?v=xHbOZC79Ag4&list=PL2tRglRS_GqivYoDuzLNo_HezOvh4Z1XN&index=13