ENH: import ANT Neuro .CNT files (some code available)

ANT Neuro has a special fileformat (.cnt and .trg files). I could not find any support to import them in MNE.
They do have a c-library libeep which you can compile for python. They were so nice to send me a precompiled 64bit linux version (thus I did not really try to compile myself, but should be straight forward) and I did an early sketch of an importer script for MNE-python. You can find this script here. For me this script seems to work nicely. So far without any preload capacities, but using libeep should be straight forward.
As I don't have much experience in mne-python I did not really know any guides to follow. I adapted a mix between the read_eeglab and the read_cnt functions.
If you do use the libeep-tool be sure to include a link to the source somewhere.
Best, Benedikt

Looks really nice.
Would you be able to share some of these files? The CNT reader is a quite recent addition and I remember there being some variation in the file format. It would be an interesting project to try and see if we can make it work without the dependencies to libeep.

I'm not sure if you mean script files or .cnt files. You can use the read_antcnt file if you want. The compiled version of libeep I don't really know and you would have to contact ANT / Robert Smies (the current author of libeep).
If you want a .cnt file & trigger files you can find examples here. These are only bipolar luminance sensor channels, I can try to find a small .cnt EEG file if you need it.

Yeah, I meant the data files, thanks. These should be fine. Maybe I'll have some time next week to see if I can make any sense of it.

Might be worth asking the author of the library for source plus license
permission

Hi,
Maintainer of libeep here. Our project is located on sourceforge: https://sourceforge.net/projects/libeep/
We use the LGPL license which means you can include a copy of the code in your project, provided you submit back changes you make and keep a reference to the original code.
That said, go ahead and include our project into MNE. If it's possible in git I would recommend pulling the sources directly from our subversion repo at sourceforge to prevent having old copies when we make updates.
I have not looked in detail in your project yet, so I can't comment on the feasibility of it, but I recommend to not use binaries, but instead compile from source each time a user installs your project. From my own experience I know it can be quite a handful to support binaries on multiple platforms(Windows, Linux, etc).
If you need any information on how to build the sources(simple python-dev suite + cmake should suffice), let me know. You can contact me on my github account or at: rsmies(at)ant-neuro(dot)com
Yours, Robert

thanks Robert
mne-python is meant to be a pure python package to work out of the box on
all systems.
to support these files we need to rewrite the readers in pure python

I wonder if file specs for these ANT files are available somewhere. @smeeze @behinger ?

We have a document that describes our riff format. You can find it here: http://eeprobe.ant-neuro.com/doc/cnt_riff.txt
We have made same modifications recently that have not been updated in this document yet. Mainly the adjustment to use 64-bit chunk sizes to allow very large files to be written. Files with these offsets can be recognized with the Riff tag 'RF64' instead of 'RIFF'. This deviates from the riff standard though.
apart from using riff as container format, we also use a variant of huffman coding to compress our sample data. This is described in the document above under the 'data' chunk. There is more to our file format, such as legacy file support, but I doubt you will come across such old files.
Hope this helps.

Great, thanks. Let's see if I can make this work.

Has there been any progress on this? I have a .cnt file that fails on load with read_raw_ctn and I am guessing it is because they are from an ANT Neuro system.

no progress AFAIK. We need volunteers to look into it. @jaeilepp is not
working with us anymore.

Ok, then I guess I can volunteer.

Great

Hi all,
I'm really sorry to open up this issue again after one year, but I'm still struggling with importing ANT Neuro .cnt files with MNE. I started a conversation with @behinger and Robert a long time ago, but I didn't manage to solve the issue. In my specific case, I might need to compile the libeep library on my own since I'm using MacOS, but I was wondering whether the problem has been solved in any other way?
Many thanks.

we don't still yet such a reader. Someone has to take a stab at it.
in the meantime you can convert your files to edf or .set and then use mne?

Since you are on macOS you might want to try the Python bindings of Biosig. I don't know if it supports ANT CNT, but it's worth a try.

Hi,
@agramfort : oh, that's a pity :( I think that @behinger created one, but his solution was specifically designed for ubuntu 64bit if remember correctly.
Btw, yes, that's what I'm doing at the moment: using .set to work with MNE. I think I'll also check whether our ANT recording software is able to export the raw files directly into edf or some other mne-supported format.
@cbrnr : thank you, I'll take a look at them and see whether they can help me sort the issue out
Many thanks to both of you!

@agramfort and I looked into possible solutions to this issue.
At the moment, the fix involves compiling the libeep library linked above yourself (or using binaries such as those provided by @behinger) and using the Python bindings by copying files to you site-packages. Unfortunately, this solution is Python 2.7 only. Moving to Python 3 would involve providing our own bindings for this library and this is not a clean solution.
We concluded that a better option and long term solution is to contact ANT and see if they can provide support for their file formats.

ANT recording system(eego) could export edf, .eeg(brainvision) or .cnt(neuroscan).

Hello, are there any advantages to the ANT neuro .cnt format over the neuroscan .cnt format? As mentioned above our ANT system has the ability to export to .cnt(neuroscan) so I'm wondering what would be the motivation to ever export to ANT's format. Is there increased precision?

I have no idea

I emailed someone from ANT Neuro who responded that

It mostly just has a higher bit-depth and more detailed information for events (triggers that have names). With lower bit depth file formats, you may need to apply a high-pass filter first in order to reduce DC offsets.

So it seems the data does not exactly match between ANT Neuro .cnt format and Neuroscan .cnt format

I have no objection to add a reader for ANT data but it just needs to
be implemented
by someone. Maybe suggest this to ANT engineer? we have received help
from different
companies in the past to support their file format.

Since our group work extensively with both the ANT-Neuro system as well as MNE-Python, I just want to chime in here with some updates and tentative paths forward.
Status quo: We still don't have a native reader of ANT-Neuro EEProbe Continuous Data (.cnt) files in MNE-Python. We cannot use the mne.io.read_raw_cnt() method that imports Neuroscan .cnt files for this purpose. They are different file formats that just unfortunately happen to coincide on picking the same .cnt extension.
Tentative solutions:

Import EEProbe .cnt files in MATLAB with the EEGLAB importer provided by ANT-Neuro here and save as .set files. Then use mne.io.read_raw_eeglab() to read into MNE-Python as a raw object.
Download the source code of the data I/O library behind the EEGLAB importer (called libeep) here and use the provided python wrapper (import libeep) to read such EEProbe .cnt files. One would then manually create the MNE-Python raw object by grabbing all relevant information from the "cnt" object instance created by libeep.
When exporting recordings from the ANT-Neuro laptop, select a different format (BrainVision, EDF+, Neuroscan) from the EEProbe .cnt file format. You can then use appropriate mne.io.read_raw() functions to read them into MNE-Python. This is acceptable except slight differences on bit depth and loss of certain trigger events. For instance the impedance check events are ignored in these different formats and you don't get the impedance values at the beginning and end of recording. Amplifier disconnect/reconnect events will also be omitted in these other formats (but EEProbe .cnt file do include them)!

p.s. I'm personally going with option 1 simply because I do some stuff in MATLAB too. But all are equally viable. Note that the BrainVision files exported by ANT-Neuro all have a missing space between 'Brain' and 'Vision', so you would need to either fix that before reading or adjust the byte length in mne.io.read_raw_brainvision (found by @proloyd).
Next step: The obstacle right now is that most of the source code in libeep is written in C. To conform to MNE-Python's goal of being a standalone Python package, someone needs to completely rewrite these C functions in Python instead of simply adding a Python wrapper. This will likely take a while, and not sure if anyone from ANT-Neuro is going to do it. When I find the time I might just take a stab at this myself, but very unlikely until late 2023.

Next step: The obstacle right now is that most of the source code in libeep is written in C. To conform to MNE-Python's goal of being a standalone Python package, someone needs to completely rewrite these C functions in Python instead of simply adding a Python wrapper. This will likely take a while, and not sure if anyone from ANT-Neuro is going to do it. When I find the time I might just take a stab at this myself, but very unlikely until late 2023.

And as another blocker the license on SourceForge is listed as GPLv2, which is incompatible with MNE-Python's BSD clause. So these functions cannot be used as a reference / looked at while writing equivalent Python code. To be able to do that, you'd have to contact the original authors and get permission to relicense/adapt their code as BSD in MNE-Python.

Download the source code of the data I/O library behind the EEGLAB importer (called libeep) here and use the provided python wrapper (import libeep) to read such EEProbe .cnt files. One would then manually create the MNE-Python raw object by grabbing all relevant information from the "cnt" object instance created by libeep.

One option would be to improve the libeep Python bindings so that it import libeep could have a libeep.read_raw_ant or whatever that returned a RawANT instance that subclasses BaseRaw. The BaseRaw interface has been established for long enough now that I think as long as the _read_segment_file is implemented properly and __init__ creates info properly, it wouldn't be too fragile. The tough part would be creating pip- and conda-installable libraries, but nowadays with cibuildwheel and conda-forge it's not insurmountable.

@larsoner great suggestion... We'd also need the license permission from ANT-Neuro to update their Python bindings of libeep, is that right?

You only need to relicense if you want to include code in MNE-Python. My "One option" / second paragraph above if you license your libeep Python library as GPLv2 as well everything should be okay. No new code is added to MNE-Python so BSD only matters in your pylibeep (or whatever you call it) in that you would subclass BaseRaw (which should be fine -- GPL code can use BSD but not vice-versa).

@mh105 I would consider contacting ANT support to ask them for a Python reader. We managed to get this from EGI/Philips who released mffpy. But it tool sometime to get this...

@larsoner I realized what @behinger provided is exactly doing the subclassing mne.io.BaseRaw route. I've updated my comment above so that people know that option 2 above

Download the source code of the data I/O library behind the EEGLAB importer (called libeep) here and use the provided python wrapper (import libeep) to read such EEProbe .cnt files...

is already pretty much set. I agree @agramfort that we should just have a pip installable package from ANT instead of managing the c library ourselves... I'll give it a try to persuade ANT, might cc you guys on the email.

I met the same problem, @agramfort  Is there any progress in the contacting ANT support to Python

a customer of ANT should reach out to them. I don't have any contact with
them

I contacted the support this morning for a python reader. We are using a lot this system in Geneva, let's wait for their reply.

From what I heard is that ANT Neuro is working on a pip installable package, and it will likely be released later this year.

I had a chat with Eshwar Ghumare, product manager @ ANT. They have an updated version of the SDK which will come out early next year (march / april) with alpha versions by the end of this year. It will include a python interface (he mentionned a pure python implementation, but I have some doubts based on the interaction with the SDK he mentionned). He'll keep me in the loop and will send my way the alpha/beta versions for testing.
Overall, I'm +1 to remove this issue from the sprint-2023 and have it tackled through a distributed python package in 2024.

any update on this from Eshwar Ghumare, @mscheltienne?  I skimmed the ANT-Neuro website and don't see anything about Python in their downloads area (where their MATLAB stuff is).

No news.. not on this or on 2 other matters transfered to him by ANT's support.

A bit of an update as I had to play with cnt files this week, on Windows and Linux. I have little hope that ANT will provide a pure python reader or will handle the maintenance of a pip and conda installable libeep package. The last update was 4 years ago. That said:

One option would be to improve the libeep Python bindings so that it import libeep could have a libeep.read_raw_ant or whatever that returned a libeep.RawANT instance that subclasses mne.io.BaseRaw. The BaseRaw interface has been established for long enough now that I think as long as the _read_segment_file is implemented properly and init creates info properly, it wouldn't be too fragile. The tough part would be creating pip- and conda-installable libraries, but nowadays with cibuildwheel and conda-forge it's not insurmountable.

I never packaged python library which require C extension and which need to be build differently on all platforms, and I'd like to learn. I started playing @ https://github.com/mscheltienne/antio where I have 3 branches:

libeep_gitlab: a mirror of the gitlab branch
libeep: a step-down simpler version which generates the binaries we need in the python package with cmake
main: a python package within which a module libeep exist

The CI build is split in 2 parts:
(1) Build the binaries required by the module libeep -> works on all platforms.
(2) Copy those binaries to the module libeep and build wheels with cibuildwheel.
And that's where I'm stuck at the moment. Even building locally, the wheel generated with cibuildwheel are broken and are not installable (I actually got a wheel for antio which depends on... antio).
On the other hand, running:
pip install build
python -m build

Does create working wheels.
I'll keep looking into it, but if anyone has the time to look at it and point me in the right direction, I'd love to learn the best packaging practice here 😉

I think this looks good, but it's difficult to debug, because the actions fail in the testing step.
That said, have you looked at build tools that might make this task a bit easier out of the box? I'm thinking of scikit-build-core, which does have native cmake support and also includes an example.

I think this looks good, but it's difficult to debug, because the actions fail in the testing step.

Locally, disabling testing, the generated wheels can't even be installed. Purely broken..

I'm thinking of scikit-build-core, which does have native cmake support and also includes an example.

I'll have a look, I'm really looking for what might be the best practice here.
Probably compared to my previous message, I was thinking that I will move the libeep branch as a folder within the main branch, and have this folder included in the sdist through MANIFEST.in. Then the cmake commands would be invoked in setup.py, this way the sdist would actually include everything needed to build the package.

Probably compared to my previous message, I was thinking that I will move the libeep branch as a folder within the main branch, and have this folder included in the sdist through MANIFEST.in. Then the cmake commands would be invoked in setup.py, this way the sdist would actually include everything needed to build the package.

Yes, it is certainly better to have everything in one place. However, I'd still take a look at scikit-build-core, which should make it easier to build (no messing around with MANIFEST.in etc).

Update of the day:

Rework the structure to have src/antio containing the python code and src/libeep containing the dependency, in the main branch.
scikit-build-core: I gave it a shot, but I'm getting further away from getting a working wheel with it. It looks like I would need to rewrite the cmake files, and my limited experience in code compiling makes it difficult. The less I change libeep, the better.
Still, the structure src/antio and src/libeep, inspired from the documentation of scikit-build-core led me to include everything you need in the sdist and a scripting build in setup.py which invokes cmake and moves the built files to the correct location before building the wheel. The sdist now includes everything you need and will work as long as you have cmake installed.
python -m build works and give me working wheels on all ubuntu-latest, windows-latest, macos-13 (intel) and macos-14 (arm), on python 3.9 to 3.12. c.f. https://github.com/mscheltienne/antio/actions/runs/10353511600
cibuildwheel is evading me. I spent hours trying to figure out how to get it to work on the manylinux docker image locally, still no luck.

@mscheltienne I could look into the cibuildwheel to see if I have any quick ideas to try. Should I just open a PR to https://github.com/mscheltienne/antio if I have anything useful?

Sure! :pray:

If you find anything, can you please ping me here (or in the relevant PR)? I've been struggling with cibuildwheel in the past as well, so I'm curious what the problem is.

For now, what I have are a mix of cibuildwheel and setuptools issues:

(1) I need to install python3-devel with yum because the cmake commands needs it to run.
(2) the quay.io/pypa/manylinux2014_x86_64 runs old versions (as expected), and I can't get setuptools to a high enough version to use pyproject.toml entirely, except if I install a more recent python version through what I figure is the equivalent of a ppa on Ubuntu.
(3) This older version of setuptools (59.6.0) behaves differently to the recent one, the variable self.build_lib does not contain the path to the python package being built as in the recent version. Overall, the way I run the extension build with build_ext in setup.py is flawed.

(1) makes sense because you probably need Python.h. Re (2), why do you have to use such an old version? Can you not install e.g. 2_28 with pipx (https://cibuildwheel.pypa.io/en/stable/setup/)? This would also solve (3), and I really think it doesn't make sense to build with such ancient versions.

I don't think you should need to do (1) or (2) (and hence not deal with (3)) manually, cibuildwheel should do this for you. But yes if you use the docker image directly yourself rather than invoking python -m cibuildwheel locally you'll need to replicate however they manage to install Python in cibuildwheel.

... in other words, I'm testing by doing:
git clone ... && cd antpy
CIBW_BUILD=cp312-* python -m cibuildwheel --archs native

I removed the stuff about before-all and I get to:
[ 90%] Built target EepObjects
[ 95%] Linking C static library libEepStatic.a
[ 95%] Built target EepStatic
[100%] Linking C shared library libEep.so
[100%] Built target Eep
error: [Errno 2] No such file or directory: '/tmp/tmp3ljxt5i0/python/pyeep.so'

which seems like progress?

Great job @mscheltienne for making the python wrapper working! Our lab stumbled upon this a longtime ago, and we had to use their matlab importer as a work around.
However, their libeep package does not expose a lot of header information, i.e., patient id, date of birth, machine info, channel types, start time etc etc. I am working on including that via libeep, so that those fields can be populated in the rawANT object. I will make a pull request shortly with my changes.
Also, I would like to add the impedance values to the rawANT.info, since it is something we look at in the lab. Any idea how we shall approach this? @larsoner I remember mne.Raw object supporting only single set of values for them.

.info won't have support for it, would be better maybe to have a utility function in mne.io.ant.<something> that allowed you to get extra information that can't safely/properly be stored in raw.info yet.

@larsoner Does MNE have a built-in format to store impedance values from electrodes? Given that impedances is one of the BIDS-EEG recommended fields for electrodes, I wonder whether there is ongoing efforts to create a space for storing such information?

@proloyd Thanks a lot for the interest and addition! For information, the RawANT object and the associated reader function mne.io.read_raw_ant are being added in this PR #12792. MNE 1.8 will be shortly out and will NOT include the reader. Once MNE 1.8 will be out, the version will be bumped to 1.9.devXXXX at which point #12792 will be merged and the development version will be available to use the reader. The reader will be available in the stable version 1.9 in a couple of months. That also gives us time to parse all the metadata you want, to add tests files, and so on :wink:
For the metadata (patient id, date of birth, machine info, channel types, start time etc etc.), my approach would be to extract everything we can in antio.libeep and eventually in parsing functions in antio.parser. Then, we can use those parsing function in MNE as a PR after #12792 to move those informations into appropriate fields of mne.Info.
@proloyd @mh105 For the impedances, in #12792 I parse them in raw.impedances, similarly to what mne.io.read_raw_brainvision does. It don't think it will survive an I/O roundtrip (meaning raw.save("fname.fif") followed by read_raw("fname.fif") will drop the field), but at least you get access to them after reading a .cnt file.
As ANT does multiple impedances measurements, raw.impedances contains a list[dict[str, float]]. mne.io.read_raw_brainvision impedance field stores a dict[str, float] with the key as channel name and the float as impedance value, thus I re-used this structure for ANT.
@larsoner Am I missing an I/O roundtrip compatible impedance field?

@mscheltienne thanks a lot for your great work on this. One quick question, what happened to @agramfort 's initial concern that we want to keep MNE-Python a pure Python source code and don't distribute c libraries along with MNE that would make it OS dependent? Are you using some workaround for this or are you shipping the entire libeep as part of MNE now?

@mh105 The compiled binaries are distributed in antio, at the moment on Pypi and soon on conda-forge. antio  becomes an optional dependency of mne required to run mne.io.read_raw_ant.


⚠ ClamAV has degraded performance

In 1b49fd2, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 90 ms

Resolved: ClamAV performance has improved in 86edae3.


Pare down logging for the docker-compose reference implementation.

We need to enable the Docker-in-Docker solution to deploy and function properly in a containerized environment.
For this, we need to pare down logging for the docker-compose reference implementation of the system management executor (aka sys-mgmt-executor) because, while the facilities for logging are available to edgex-go services in general, sys-mgmt-executor is a standalone, bare-bones executable for which it does not make sense to pull in the machinery for initializing full-blown logging as we do for other services proper; instead, we are paring down to lean (built-in golang) logging.
Specifically, when an attempt is made to use full-blown logging, the following (as an example) ensues, with main.go:74 referring to the first instance of (un-initialized) instance of LoggingClient:

bash-4.4# ./sys-mgmt-executor
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0x85b13b]

goroutine 1 [running]:
main.main()
        /go/src/github.com/edgexfoundry/edgex-go/cmd/sys-mgmt-executor/main.go:74 +0x7b
bash-4.4# ./sys-mgmt-executor edgex-core-data stop
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0x85b13b]

The other way you could solve this is by initializing LoggingClient, no?

The other way you could solve this is by initializing LoggingClient, no?

@anonymouse64 Yep, thanks, that is indeed the approach I'm currently implementing.


update async-* dependencies

Async-Channel 2 breaks some public API we reexport so we cannot bump it before async-std 2


TypeError: Cannot read property 'case sensitive routing' of undefined

TypeError: Cannot read property 'case sensitive routing' of undefined

exports=modules.exports=route  //error
correct:
exports=modules.exports={
route
}

The answer is worse than the question. Great!


Pretraining of StructCoder

Hi, Thanks for the work! I want to know that if you will release the pre-training code of StructCoder?

Hi, We just updated the repo and also released the pretraining code. Please check README.md


Little guidance for agora-test

I noticed agora-test is quote poorly documented... While it will
receive a restructure when we migrate to plutarch-test, these are
small notes in the meantime.

@emiflake It's good to go

Thanks!


improvement(sdlafvideorender): fix multi thread problem when window s…

…ize change

Codecov Report

Merging #311 into develop will decrease coverage by 3.05%.
The diff coverage is n/a.

@@             Coverage Diff             @@
##           develop     #311      +/-   ##
===========================================
- Coverage    66.52%   63.47%   -3.06%
===========================================
  Files          209      209
  Lines        11440    11440
===========================================
- Hits          7611     7262     -349
- Misses        3829     4178     +349

Impacted Files
Coverage Δ

framework/tests/demuxer/demuxerUnitTest.cpp
0.00% <0.00%> (-100.00%)
:arrow_down:

framework/tests/demuxer/demuxerUtils.cpp
0.00% <0.00%> (-76.00%)
:arrow_down:

framework/demuxer/AVBSF.cpp
25.27% <0.00%> (-42.86%)
:arrow_down:

framework/demuxer/play_list/HLSManager.cpp
61.26% <0.00%> (-14.23%)
:arrow_down:

mediaPlayer/media_packet_queue.cpp
67.20% <0.00%> (-8.80%)
:arrow_down:

framework/demuxer/play_list/SegmentList.cpp
87.71% <0.00%> (-8.78%)
:arrow_down:

framework/demuxer/play_list/playList_demuxer.cpp
74.02% <0.00%> (-6.50%)
:arrow_down:

framework/demuxer/play_list/HLSStream.cpp
64.50% <0.00%> (-4.91%)
:arrow_down:

mediaPlayer/buffer_controller.cpp
48.30% <0.00%> (-3.39%)
:arrow_down:

framework/data_source/curl/curl_data_source.cpp
83.26% <0.00%> (-3.35%)
:arrow_down:

... and 12 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9204219...7183677. Read the comment docs.


Feature: Option to ignore "Unlisted Dependencies" for ESLint/Prettier

Hello, and thanks for the awesome tool. I have a feature request relating to meta-packages.
The Problem
Some JavaScript/TypeScript projects use npm meta-packages in order to bundle dependencies into a single package.
 This kind of thing makes reading the dependencies in a project's "package.json" file easier, because the "real" dependencies can easily be seen and are not drowned out by 20+ different ESLint/Prettier dependencies.
Unfortunately, having an npm meta-package makes working with Knip frustrating. With the default config, Knip will complain that ".eslintrc.cjs" and "prettier.config.mjs" are unused. The naïve solution is to exclude them, like this:
{
  "$schema": "https://unpkg.com/knip/schema.json",
  "ignore": [".eslintrc.cjs", "prettier.config.mjs"]
}

This solution works fine to start with, but as our projects get more complicated, we run into more problems. Say for example we want to add eslint-plugin-sort-exports to our linting setup. Now, Knip will complain that "eslint-plugin-sort-exports" is an unused devDependency. We could also include that in the ignoreDependencies array, but this strategy is not going to scale in the long-term.
What we really want to do is to have Knip actually parse our ESLint config. Handily, we can force Knip into knowing that our project does indeed use ESLint by doing this:
{
  "$schema": "https://unpkg.com/knip/schema.json",
  "eslint": true
}

This config gets rid of the "unused .eslintrc.cjs" error, but it then throws errors for "unlisted dependencies of eslint" (and other related errors). So we would have to ignore the unlisted dependencies with this config, which is potentially less hacky than what we started with, but not by much (as there can be a ton of different ESLint plugins and configs).
Proposed Solution 1
It might be nice to have the ability to specify to the ESLint plugin that the dependencies really are there, like this:
{
  "$schema": "https://unpkg.com/knip/schema.json",
  "eslint": {
    "ignoreUnlistedDependencies": true
  }
}

With this flag set to true, it would parse ".eslintrc.cjs" as before, but wouldn't complain about unlisted dependencies.
Note that the Prettier plugin would also need to have this behavior, as the exact same problem applies to that.
Proposed Solution 2
As an end-user, I should probably be able to tell Knip that I am using meta-packages in my project, like this:
{
  "$schema": "https://unpkg.com/knip/schema.json",
  "usingMetaPackages": true
}

With this option set to true, instead of just looking 1 level deep for dependencies, Knip would instead look 2 levels deep for dependencies. In other words, it would see isaacscript-lint as a dependency, and then look through all of the isaacscript-lint dependencies to see if eslint exists.

This exact phenomenon is definitely something I would love to tackle properly, it's a major concern (basically since day 1). So thanks a lot for raising this and thinking along here.
Regarding your proposed solution, I'm not targeting to ignore such issues, but rather address them as good as possible.
The usingMetaPackages option is an interesting option, definitely something to consider.
Another angle I'm considering is to have some kind of "transfer" mechanism that can be configured at root level like so (not sure yet about the meta name):
{
  "eslint": { "meta": "@internal/eslint-config" },
  "jest": { "meta": "@scope/shared-configurations", "config": "config/jest.config.ts" },
  "prettier": "@internal/shared"
}

Knip can then "transfer" the results of a plugin to the provided meta package.
Ideally it would require no config it all, but this seems near impossible (how to recognize the meta package, etc).
Curious to what you (and others) think :)

Yes, the transfer mechanism seems like a better solution than my Proposed Solution 1.
Interestingly, the transfer mechanism would also fix an semi-related bug having to do with monorepos. Like most monorepos, I have the following setup:
project/project.json
project/packages/foo/package.json

Knip is intended to be run at the root of the monorepo, but I cannot get it working properly there, and find it easier to use Knip on individual packages at a time. However, when I run Knip from the "foo" directory, it complains that ESLint is not found, because ESLint is actually located in the root "package.json" file as a monorepo-wide dependency (which is the exact same problem in the OP, just because of monorepos instead of meta-packages).
So, if a transfer solution existed, then I could specify a config like this:
{
  "eslint": { "meta": "../.." },
}

Ideally it would require no config it all, but this seems near impossible (how to recognize the meta package, etc).

I think it is possible in principle to detect a meta-package. For example, here is the package.json file for isaacscript-lint. It doesn't have any "main" field or "exports" field (for code) or "types" field (for TypeScript types).
If it has a "dependencies" field and does not have main/exports/types, then it is a meta-package.

Maybe there are some other fields to consider other than main/exports/types? Not sure.

Knip is intended to be run at the root of the monorepo, but I cannot get it working properly there, and I find it easier to use Knip on individual packages one at a time.

You can use knip --workspace [dir] to focus on individual packages. The logic behind this flag has been refactored/improved around v2.32 (it includes ancestor + dependent workspaces), so you may want to (re)try that.
If that's not working well, besides what's being discussed in this thread, then please file an issue.

Ideally it would require no config it all, but this seems near impossible (how to recognize the meta package, etc).

I think it is possible in principle to detect a meta-package. For example, here is the package.json file for isaacscript-lint. It doesn't have any "main" field or "exports" field (for code) or "types" field (for TypeScript types). So in this case, we can assume that the package "isn't doing anything" other than just being a meta-package.
More formally, the algorithm would be:

Look through each dep package.json file.
If it has a "dependencies" field and does not have main/exports/types, then it is a meta-package.

Maybe there are some other fields to consider other than main/exports/types? Not sure.

Not sure if approaching it this way, taking into account the myriad of ways people using such packages, is something Knip should rely on. Granted, Knip is full of heuristics and such, but this feels a bit far fetched. There can also be more than of such packages, e.g. I've seen @org/eslint-config and then some package(s) with one or more (shared) configuration files (sometimes without dependencies). Not easy to link plugins to their "meta" package(s). The main field in package.json is not mandatory (default is index.js I think), etc.
Maybe for some plugins it could be derived from extends fields and such (like ESLint and Typescript configs allow), but not all tools have this.
By the way, the isaacscript monorepo is a great repository for Knip to work on this feature :)

You can use knip --workspace [dir] to focus on individual packages. The logic behind this flag has been refactored/improved around v2.32 (it includes ancestor + dependent workspaces), so you may want to (re)try that.

The isaacscript monorepo does not use workspaces. Sorry, I know that's a bit non-idiomatic, but I think that the "correct" way for monorepos to work is to not use workspaces at all and just install all the deps at the root of the monorepo. (I have wrote about this strategy here.)

Knip works best from root and if it knows about the packages. Idea for the root config knip.json:
{
  "workspaces": {
    "packages/lib-*": {},
    "packages/package-with-different-config": {}
  }
}

I'd recommend this, because something like "meta": "../.." ("external internals" to the workspace) is something I currently feel is not likely to make it into the final solution for what the original issue is about.

Idea for the root config knip.json

Ok, that seems reasonable to me! Let me know when it's ready and I will give it a whirl. : )

This already works :)

Thanks webpro. To continue the discussion, I tried to run knip from the root of my monorepo, but I am running into some problems.

I have a monorepo package that is an npm library. It exports helper functions in the "index.ts" file.
These functions are otherwise unused, but obviously this is intentional because it is an npm library that other people are expected to consume.
When running knip from the subpackage directory, it seems to work fine with no errors.
When running knip from the monorepo directory, it complains that all of the exports are unused.

I figured that maybe I need to explicitly tell knip about the entry point, so I tried this config, based on what you said in the previous comment:
{
  "$schema"<EMAIL_ADDRESS>  "workspaces": {
    "packages/foo": {
      "entry": ["./packages/foo/src/index.ts"]
    }
  }
}

But that didn't work. Furthermore, I changed "./packages/foo/src/index.ts" to "BOGUS_VALUE", and knip didn't even throw any errors. Is that intended? Shouldn't knip be throwing runtime errors if I put in bogus values for the purposes of failing-fast on typos? Regardless, I'm not quite sure how to make this work, any guidance would be appreciated.

I would really like to just be able ignore these issues, at least interim.
At the moment, there is no nice workaround.
If I am getting an error:
Unused devDependencies (7)
@graphql-codegen/add                    package.json
@graphql-codegen/typed-document-node    package.json
@graphql-codegen/typescript             package.json
@graphql-codegen/typescript-operations  package.json
@graphql-codegen/typescript-resolvers   package.json
@graphql-eslint/eslint-plugin           package.json
@roarr/cli                              package.json
Unlisted dependencies (3)
eslint-config-canonical/graphql    .eslintrc.cjs
eslint-config-canonical/zod        .eslintrc.cjs
eslint-import-resolver-typescript  .eslintrc.cjs

My options seem to be:

add eslint-config-canonical as a direct dependency (don't want to do that)
ignore .eslintrc.cjs

We usually do #2, but then that causes its own new issues, because now if .eslintrc.cjs was importing something, that suddenly gets merged as unused. So now we have to ignore these dependencies that are actually used. :-(

Yes, this has been awful since early days. Unfortunately I don't see a good generic solution.
The real solution is the upcoming ESLint flat config format, this will eliminate all these issues at once (already using it myself in https://github.com/webpro/knip/blob/v3/packages/eslint-config/index.js). But it will take some time before this is actually useful.
Until then, maybe it helps if Knip will support wildcards in ignoreDependencies. This in combination with also disabling the eslint plugin (btw this is possible today with eslint: false) and ignoring the ESLint config files might be somewhat bearable.

The problem with {eslint: false} is that it also starts reporting imports that were referenced in the config files as unused.

This one is the most annoying:
const path = require('node:path');

module.exports = {
  extends: ['@contra/eslint-config'],
  root: true,
  settings: {
    'import/resolver': {
      typescript: {
        project: path.resolve(__dirname, 'tsconfig.json'),
      },
    },
  },
};

Knip reports:
$ knip; knip --production --strict
Unlisted dependencies (1)
eslint-import-resolver-typescript  .eslintrc.cjs
✂  Excellent, Knip found no issues.

Now if I ignore eslint-import-resolver-typescript:
$ at 16:24:53 ❯ knip; knip --production --strict
✂  Excellent, Knip found no issues.
Configuration issues (1)
Unused item in ignoreDependencies: eslint-import-resolver-typescript
✂  Excellent, Knip found no issues.

so cannot do neither that or that.

I did not resolve the actual underlying issue, but by improving a few things the workaround should help a bit here. We can now use regexes in ignoreBinaries and ignoreDependencies, and the minimal version of the workaround is explained here: https://knip.dev/guides/handling-issues/#eslint--jest
Let mew know if you encounter any issues.


CMAEntry supporting Metadata and tags?

Hi,
we've seen that the current implementation of CMAEntry does not support attribute "metadata" however we've seen that it is return from the API when getting an entry.
There's any plan to add Metadata and so tags support to the CMA SDK?
Thanks,
Elena C.

Hey @elenacusenza yes, I will add it to our backlog, should be in production in a couple of weeks.

Hey @elenacusenza this now should be available in v.3.4.9


Thal University Bhakkar Email

added file

@ahmadrazacdx file/lib/domains/pk/edu/tu/scholar.txt: this request has an incorrect file path format, please ensure that you are familiar with     this repository rules located at the bottom of this page and change     your request accordingly. If you don't know how to change request, close it and open a new one.     If you are not able to make the pull request according the instructions, please send us an email to<EMAIL_ADDRESS>Please mention this request URL in the message.

@ahmadrazacdx Thank you for your patience while waiting for a review and processing your request. Pull request merged.
Please start requesting the licenses in about an hour, to let the changes to propagate through our system.


Finger names etc

Remember #3 from way back? I think the only problem was that I couldn't figure out how to calculate the height of a character. Fortunately the width of the character 'M' is extremely close to the real height of a number character so I went ahead and added a Height property to the object returned by MeasureString function.
I think it looks pretty good now:

Awesome!

was the double digit issue from the original issue solved?
i'm still running into it, going to try to figure it out tomorrow, any advice?

@mwcm For the double digit issue we'd have to change the convention of the positions attribute as it's currently expected to have 6 non-separated single digits. Ideally we'd keep backwards-compatibility by still allowing stuff like positions="320003", but then also allow using a separator like this positions="3 2 0 0 0 3" and positions="9 11 11 10 9 9".

@T-vK  thanks for the quick reply! just finished going through the script, like you said it expects 6 non separated digits wherever position is mentioned. for sure, don't want to break anything
going to take another look tomorrow afternoon

ended up combining VexChords with ChordJS ChordSVG and adding the space separated positions as an option

Cool, are there other improvements besides the space separated positioning?

Yeah there are a couple changes, thanks for asking!
The code structure is based on ChordJS, but the drawing is all based on the Vexchords SVG rendering, which means that nearly all the Vexchords customizations work in ChordSVG (save Barre chords, still deciding how to go about it)
Also changed a couple things for my use case:

the # of shown frets is auto adjusted based on the frets used in the positions array.
reworked the Vexchord position variable to be interpreted from the positions array
added chord name drawing to Vexchords


Adjusting chunk size for streaming datasets

Is your feature request related to a problem? Please describe.
I want to use mc4 which I cannot save locally, so I stream it. However, I want to process the entire dataset and filter some documents from it. With the current chunk size of around 1000 documents (right?) I hit a performance bottleneck because of the frequent decompressing.
Describe the solution you'd like
I would appreciate a parameter in the load_dataset function, that allows me to set the chunksize myself (to a value like 100'000 in my case). Like that, I hope to improve the processing time.

Hi ! Data streaming uses fsspec to read the data files progressively. IIRC the block size for buffering is 5MiB by default. So every time you finish iterating over a block, it downloads the next one. You can still try to increase the fsspec block size for buffering if it can help. To do so you just need to increase fsspec.spec.AbstractBufferedFile.DEFAULT_BLOCK_SIZE
Currently this is unfortunately done in a single thread, so it blocks the processing to download and uncompress the next block. At one point it would be nice to be able to do that in parallel !

Hi! Thanks for the help, I will try it :)


Two types of Norg verbatim blocks

Not sure why Norg does this, but it supports two types of verbatim block syntaxes:
|code
style 1
|end

@code
style 2
@end

I guess the latter is ment for embedding code, which I assume headlines.nvim is aiming for.
This PR add support for both.

README is now updated.


Question - adbClient.Connect(ip) outcome (string result)?

adbClient.Connect used to be able to return a string to confirm that the connection had been successful (or not) as:
string result = adbClient.Connect(ip)
string response = adbClient.Connect(new DnsEndPoint(ip, port));
But it seems no longer available. Has something changed?
Is this the most efficient to check if connection was successful?
var device = adbClient.GetDevices().FirstOrDefault(x => x.Serial.Contains(ip));

Not really an issue, and since no one replies, closing. Thanks anyway.. :)


Question: How is the right way to use bitwise and operator &

Can you help me with this?
I have this piece of code
BITS   = [0x10, 0x08, 0x04, 0x02, 0x01]
BASE32 = "0123456789bcdefghjkmnpqrstuvwxyz"

"c216ne".scan(/./) do |c|
    BITS.each do |mask|
        puts "#{BASE32.index(c[0]) & mask}"
    end
end

When I run the code I get this exception:
undefined method '&' for Nil (compile-time type is (Int32 | Nil))

        puts "#{BASE32.index(c[0]) & mask}"

If I try to compare with && instead of & it will work, but will not provide the result I want.
I don't really know if it's a but or I am doing something wrong.
Obs: I run the same piece of code in Ruby and it's ok.

String#index returns Int32 or Nil.  Please read this: https://crystal-lang.org/api/0.23.1/Nil.html
And please don't ask these questions here, use StackOverflow. Thank you!

All right, sorry for that!


Is there a plan to build zig from source in flake?

Curious about how much work will it take?

No there isn’t. Upstream Nixpkgs is good for that. The goal of this one is to provide easy accesss to the Zig official binaries.


[SPARK-19975][Python][SQL] Add map_keys and map_values functions to Python

What changes were proposed in this pull request?
This fix tries to address the issue in SPARK-19975 where we
have map_keys and map_values functions in SQL yet there
is no Python equivalent functions.
This fix adds map_keys and map_values functions to Python.
How was this patch tested?
This fix is tested manually (See Python docs for examples).

Can one of the admins verify this patch?

Looks good :)

jenkins, ok to test.
Does someone on the SQL side have a chance to look at this to say if its something they want added to the DataFrame API? Maybe @marmbrus ? I'm a little hesistant with adding it to functions in this way since the map_values has a different meaning than mapValues in RDD land and it seems like that could cause some confusion.

ok to test

Test build #78242 has started for PR 17328 at commit 021b551.

LGTM except the comments about versions.

+1 for this PR.

Test build #78242 has finished for PR 17328 at commit 021b551.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78242/
Test PASSed.

Thanks for the review. The PR has been updated with version string fixed. Please take a look.

Test build #78260 has started for PR 17328 at commit 8e920e4.

Test build #78260 has finished for PR 17328 at commit 8e920e4.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78260/
Test PASSed.

LGTM, merging to master.
Since the correctness has been covered by the Python test cases, we can merge it now. If possible, could you also submit a follow-up PR to add a test case? Thanks!


whats the browser connection url when running under docker compose ?

Its does not fill it in, but not sure form code and docs ( does not say ).
https://pionion.github.io/docs/quickstart/quickstart#3-chat-1

Should add deps to setup caddy and mkcert, so uses can run under https.


Add Client version when agent register instance

Please answer these questions before submitting your issue.

Why do you submit this issue?
[ ] Question or discussion
[ ] Bug
[x] Requirement
[ ] Feature or performance improvement

Question

What do you want to know?

Bug

Which version of SkyWalking, OS and JRE?

Which company or project?

What happen?
If possible, provide a way for reproducing the error. e.g. demo application, component version.
sometimes in a company  may have hundreds of applications and they may use different agent version, if we fix some agent bug in current version, we want to know who is using this version to notify them to upgrade.

Requirement or improvement

Please describe about your requirements or improvement suggestions.

ok,I read current code,we should add a field agent_version in instance index when instance registered,
ui show in this place

ok,I read current code,we should add a field agent_version in instance index when instance registered,
ui show in this place

That is an easy way. Even you do so, you have to face there are so many agents in the cluster. Let's say at least thousands of them. That is why we should add a new group, named Settings，an Agent status page under that group. By that, you could see the status of all, and group them by application codes, IP, or version.

Show in Running Server page

I propose recall why show agent version to user? what problem need to by solved?
The keywords  hundreds of applications in your description. User will hardly to find which agent version is different from each others.


Add parallelization feature

The dispersion analysis, basically a monte-carlo simulation, is predestinated for parallelization.
How could one implement this within this python/java framework?
Maybe https://docs.python.org/3/library/multiprocessing.html would do the job.

I played a little bit with Multiprocessing and concurrent.futures.
I got to the point

TypeError: cannot pickle '_jpype._JPackage' object

which means that I cannot run more than one simulation with the same orh instance. Trying to create more than one instance of orhelper.OpenRocketInstance() gives

File "....\ortools\lib\site-packages\orhelper_orhelper.py", line 50, in enter
jpype.startJVM(jvm_path, "-ea", f"-Djava.class.path={self.jar_path}")
File "....\ortools\lib\site-packages\jpype_core.py", line 177, in startJVM
raise OSError('JVM cannot be restarted')

It seems to be impossible to launch the JVM (i.e., orhelper.OpenRocketInstance()) more than once within the same python script: https://github.com/jpype-project/jpype/issues/959
@PatrickKa Any ideas how to overcome this?
For creating more than one OR-jar file we would have to adapt orhelper, because the jar file is hardcoded:
https://github.com/SilentSys/orhelper/blob/bc240170b8fde9077c7e4ab6cd2aa01ace142a36/orhelper/_orhelper.py#L14


[Build] Patch maven's wagon-http library to fix issues in artifact downloads

Motivation

Fixes "IllegalStateException: Entry [] has not been leased from this pool" errors which are causing a lot of build failures in Pulsar CI.

Upgrade maven's wagon-http library to version 3.4.3 to get fix for
https://issues.apache.org/jira/browse/HTTPCORE-634

Flink CI has had a similar problem, https://issues.apache.org/jira/browse/FLINK-16947

Modifications

Patch maven's wagon-http library to version 3.4.3
Add -Dmaven.wagon.http.retryHandler.class=standard to MAVEN_OPTS

/pulsarbot run-failure-checks

Example of a problem that this PR fixes:
Error:  Failed to execute goal org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process (process-resource-bundles) on project pulsar-functions-instance: Failed to resolve dependencies for one or more projects in the reactor. Reason: Could not transfer artifact io.sundr:sundr-codegen:jar:0.22.0 from/to central (https://repo1.maven.org/maven2): Entry [id:4166][route:{s}->https://repo1.maven.org:443][state:null] has not been leased from this pool
Error:    io.sundr:sundr-codegen:jar:0.22.0
Error:
Error:  from the specified remote repositories:
Error:    central (https://repo1.maven.org/maven2, releases=true, snapshots=false),
Error:    bintray-yahoo-maven (https://yahoo.bintray.com/maven, releases=true, snapshots=false),
Error:    confluent (http://packages.confluent.io/maven/, releases=true, snapshots=false),
Error:    apache.snapshots (https://repository.apache.org/snapshots, releases=false, snapshots=true),
Error:    sonatype-nexus-snapshots (https://oss.sonatype.org/content/repositories/snapshots, releases=false, snapshots=true)
Error:  Path to dependency:
Error:  	1) org.apache.pulsar:pulsar-functions-instance:jar:2.8.0-SNAPSHOT
Error:  	2) org.apache.pulsar:pulsar-functions-secrets:jar:2.8.0-SNAPSHOT
Error:  	3) io.kubernetes:client-java:jar:10.0.0
Error:  	4) io.kubernetes:client-java-api:jar:10.0.0
Error:  	5) io.sundr:builder-annotations:jar:0.22.0
Error:  	6) io.sundr:sundr-codegen:jar:0.22.0
Error:  -> [Help 1]
Error:
Error:  To see the full stack trace of the errors, re-run Maven with the -e switch.
Error:  Re-run Maven using the -X switch to enable full debug logging.
Error:
Error:  For more information about the errors and possible solutions, please read the following articles:
Error:  [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
Error:
Error:  After correcting the problems, you can resume the build with the command
Error:    mvn <goals> -rf :pulsar-functions-instance
Error: Process completed with exit code 1.

These have been very frequent in the last few days for some reason.

Flink CI is using a similar approach in https://github.com/apache/flink/commit/3279c85233a7d696b344d64d804fe473599bdd01


I've already installed react-chartjs-2 and chart.js but it's still webpackMissingModule.

Hello,
I try to install by npm install react-chartjs-2 chart.js --save.
and I got this log below.
+<EMAIL_ADDRESS>+<EMAIL_ADDRESS>updated 2 packages in 13.283s

I try to use it like
import { Doughnut } from 'react-chartjs-2';

But when I try to run web I got this below...
Uncaught Error: Cannot find module "react-chartjs-2"
    at webpackMissingModule (MyPage.js:2)
    at Object.<anonymous> (MyPage.js:2)
    at __webpack_require__ (bootstrap 3fe36b36664fce345e8a:653)
    at fn (bootstrap 3fe36b36664fce345e8a:83)
    at Object.<anonymous> (RoutesWrapper.js:18)
    at __webpack_require__ (bootstrap 3fe36b36664fce345e8a:653)
    at fn (bootstrap 3fe36b36664fce345e8a:83)
    at Object.<anonymous> (Root.dev.js:4)
    at __webpack_require__ (bootstrap 3fe36b36664fce345e8a:653)
    at fn (bootstrap 3fe36b36664fce345e8a:83)

OK. It's my fail. I use docker. So, I need to install in virtual.
docker-compose exec client /bin/sh then npm install react-chartjs-2 chart.js --save


Invisible Minimap clickable part

Issue Type: Bug

Open a code file with long lines of text so that the drop shadow of the code minimap appears.
Click at the end of the line, around 5px before the visible shadow appears.

Expected result: cursor is placed in the long line of text at the clicked position.
Actual Result: page scrolls down, because the dropshadow extends invisibly a few pixels you click the minimap unintentionally.
Possible solution: make the dropshadow not clickable at all or click through if possible. It would even be preferable to not register any click there, since then one can easily scroll to the right and click again, because the current position in code is not lost as it is now.
VS Code version: Code 1.58.2 (c3f126316369cd610563c75b1b1725e0679adfb3, 2021-07-14T22:10:15.214Z)
OS version: Windows_NT x64 10.0.18363
Restricted Mode: No

System Info

Extension
Author (truncated)
Version

voice-comments
4a1
0.2.3

vscode-sqlite
ale
0.13.0

vscode-angular2-files
ale
1.6.4

nugetpackagemanagergui
ali
1.1.6

ng-template
Ang
12.1.3

path-intellisense
chr
2.4.0

bracket-pair-colorizer-2
Coe
0.2.1

vscode-eslint
dba
2.1.23

githistory
don
0.6.17

tsl-problem-matcher
eam
0.4.0

vscode-html-css
ecm
1.10.2

vscode-nupkg
eri
1.0.1

prettier-vscode
esb
8.1.0

git-project-manager
fel
1.7.1

vscode-nuget-package-manager
jmr
1.1.6

Angular2
joh
12.0.0

vscode-peacock
joh
3.9.1

vscode-csharp-snippets
jor
1.1.0

csharpextensions
kre
1.4.0

vscode-scss
mrm
0.10.0

vscode-docker
ms-
1.15.0

csharp
ms-
1.23.14

python
ms-
2021.7.1060902895

vscode-pylance
ms-
2021.7.7

jupyter
ms-
2021.8.1054968649

powershell
ms-
2021.6.2

debugger-for-chrome
msj
4.12.12

debugger-for-edge
msj
1.0.15

vscode-nuget-gallery
pat
0.0.24

material-icon-theme
PKi
4.8.0

nuget-extensions-vscode
ran
0.1.2

JavaScriptSnippets
xab
1.8.0

json
Zai
2.0.2

@begrs Could you please post also a screenshot with maybe an arrow that highlights the problematic click area?

Clicking the letters "in" in the first line, "te" in the second line and "o" in the last line scrolls to the end of the page instead of allowing edit there.


add merge workflow between load raw api model and validation

What this PR does / why we need it:
Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #1269
Special notes for your reviewer:
Release note:

This change is

@seanknox , the package below is under BSD 3-clause. It is also used in acsrp. So if it is not registered, please help.
github.com/imdario/mergo

ginkgo tests failed with a known issue that should be fixed with #1292


Is there a way to prevent CefSharp/CEF from sending server name during TLS client hello or is there a way to capture and modify it?

I know this problem sounds weird but I do have this requirement for some special purpose, and I didn't found any way yet

Already asked at https://stackoverflow.com/questions/64562078/is-there-a-way-to-prevent-cefsharp-cef-from-sending-server-name-during-tls-clien


Auto verify placements can be correctly created/displayed/deleted

Hi @Phaow, @kasturinarra, @xingxingxia, I added a test case for testing the basic operations for KCP placements. Please take a look if you have time. Thanks!
Test:
passed: (57.3s) 2022-11-29T02:13:52 "[area/transparent-multi-cluster] Author:zxiao-Critical-Verify placements can be correctly created/displayed/deleted for shared compute [Suite:kcp/smoke/parallel]"

1 pass, 0 skip (57.3s)

/cc @Phaow @kasturinarra @xingxingxia

@zimo-xiao may i know the use case that you are trying to solve here ? Thanks !!

Hi @kasturinarra, this is the case that I will be proposing after the automation. It's a basic task that touches all the common operations of placements.

Hi @kasturinarra @Phaow @xingxingxia, I updated the case based on your comments. PTAL. Thanks!
passed: (33.7s) 2022-12-08T03:18:23 "[area/transparent-multi-cluster] Author:zxiao-Critical-Verify placements can be correctly created/displayed/deleted for shared compute [Suite:kcp/smoke/parallel]"

1 pass, 0 skip (33.7s)

Hi @kasturinarra, this is the case that I will be proposing after the automation. It's a basic task that touches all the common operations of placements.

@zimo-xiao I think we better not do that. We propose the test case first, get it reviewed and approved then go for automation. Also i have automated similar cases, i am just wondering why do we have to create things in loop ? This case i am not sure if suitable for functional/regression tests. And deletion part you could add to the existing cases it self may be ?


[SR-4073] Query making no progress for a long time is expired

Changes
Add a feature that a query that makes no progress for a long time(i.e. 300s) is considered to be expired, so its drivers shall be cancel and QueryContext, FragmentContext shall be cleaned. here making progress means that some drivers belonging to the query is ready to be executed by execution thread pool continuously. In a corner case, some Fragment instances is delivered to BE successfuly, while some is missing (this scenario results from FE restart), partial drivers on BE have no predecessor to produce input chunks or no sucessor to consume output chunks, then these drivers will be blocked and stay in PipelineDriverPoller forever, in such a situation, query makes no progress and their drivers never ready, so query expiration mechanism is introduced to resolve this problem.
The expiration mechanism behaviors as following:

QueryContext uses _deadline to keep lifetime.
extends the lifetime of the QueryContext before a ready driver is put on execution thread to execute.
QueryContext is expired if its lifetime terminates.
In PipelineDriverPoller thread, shutdown drivers belongs to the expired QueryContext.
An expired QueryContext is cleaned after all drivers shutdown.

run all

run starrocks_be_unittest

run starrocks_be_unittest


Add Deque::{get, get_mut, get_unchecked, get_unchecked_mut}

Adds direct indexing methods to Deque, equivalent to their VecDeque counterparts.

TODO: depending on which is merged first, either add the methods to #459 or add the methods to DequeView.


Update lexicons fetched from 9879ca9 committed 2023-09-14T20:24:48Z

on hold because of question https://github.com/bluesky-social/atproto/issues/1615

close #152


Docker pull image has issue

The pull command always failed with these log even with the http proxy setting. But the latter build command can be successful.


misleading error in at least promtool

if you provide:
static_configs:
  - files:
    - /path/to/files*.json

in prometheus.yml (which is wrong, it should be file_sd_configs instead of static_configs) at least promtool complains about:
FAILED: unknown fields in target_group: files
This error is quite misleading and makes it harder to locate the actual problem.
My short digging into the topic revealed that something might be calling checkOverflow with a wrongly labeled context, but I haven't digged too far, I must admit.

This error is correct, and I'm not sure we can do much about it. Knowing that you meant a difference SD mechanism one level up in the config would require quite a bit of sophistication, which I'm not sure is possible.

although I think I see how the error comes into place technically, could you line out how you mean "correct" here with regard of promtool complaining about an error in a field that is not specified?
To be more specific: I do not see yet how this is correctly labeled as an unknown field in target_group, as the latter is not specified there.
Maybe a change in formulation could already clarify things.

static_configs is a list of target_groups, files is not a field of target_group.

so the error reads as files not being of type target_group, correct?

No, that files is not a field of the type target_group.

ok, that's what I meant.
For my reading/understanding of the message, the error is wrong then. At least the colon has to go imo, because to me it looks like a separator with files being the unknown field in a target_group somewhere in the config file.

That is what the message means.

In different news:
target_group is outdated, it should read static_config.
Perhaps that clears a bit of the misunderstanding.

Changed issue title accordingly.

Can we get the documentation fixed -> https://prometheus.io/docs/operating/configuration/#<static_config>


Add JSON output

Is your feature request related to a problem? Please describe.
Some search engines (eg searxng) have the ability to format the output in JSON format.  This makes it very useful for integration in to programmatic pipelines (eg, LLM agents).
Describe the solution you'd like
A query to Perplexica using the /?q=%s feature would take an extra argument, format, eg: /?q=%s&format=json.  This would run the perplexica query and instead of HTML, generate a JSON object:
{
  "query": "stuff",
  "answer": "Hmm, sorry I couldn't find any relevant information on this topic with the provided context.",
  "references": [],
  "related": []
}

same #141

Same as https://github.com/ItzCrazyKns/Perplexica/issues/141

great !  But How does this work?
I tried the latest ver .. and
http://localhost:3000/?q=what+is+the+abramelin+ritual&format=json
just came back with the html  not json

I think ItzCrazyKns means that it's a duplicate of #141.  I'm playing around with adding an api endpoint to src/routes and following the logic of handleMessage in src/websocket/messageHandler.ts but my typescript isn't great so it's not very successful so far.

I think ItzCrazyKns means it's a duplicate of #141 and closed on that basis.  I'm playing around with adding an api endpoint to src/routes and following the logic of handleMessage in src/websocket/messageHandler.ts but my typescript is not great so it's not very successful so far.

Would be great if someone post an example of how to use this


updates to get CI passing again

adds Postgres and redis-server to CI config
adds some additional setup to script/bootstrap to apply the scriptz submodule

@patricksmithdds @richard-dds I think we're hitting issues with the installation order here (one passes, one fails). Since each service has script/setup run as a child of the main setup in atat-stack, it's hard to debug the output in CI. We could lock the build order, but that seems like a fragile solution.
I think a better longterm solution for atat-stack will be to wait for each service to be dockerized and then run integration tests against the docker containers. That way their builds are isolated as much as possible.


Disable Global Routing in all applications using WebHooks

not really a #318 fix, especially because it shouldn't be necessary to globally disable this feature

also:

Upgrade dependencies and KoreBuild to enable local testing of #318 problems
Revert "Disable flakey tests"

not necessary with Global Routing disabled
This reverts commit fad82fd.

FYI only due to number of tests failing on the CI. I'll merge once checks complete.

FYI @rynowak, @kichalla, @JamesNK


Provide support for bcast_hw op in0 and output height sharding, as well as the option to execute the op in place. Add fused softmax variant for attention masks of shape (1, 1, seq_len, seq_len)

Bcast op (HW version only) got support for in0/out height sharding. Other variants did not receive that support, as I'm not aware of the use case. One more limitation is that either both input0 and output must be dram-interleaved or height_sharded (one can't be sharded, while the other is dram-interleaved). Will fix this when I get time.
Provide option to perform op in place, without allocating additional memory.
Height shard attention sequence from falcon7b prefill, and provide a test for it.
This is the sequence that is now fully height-sharded: matmul -> bcast -> add -> softmax -> matmul.
In original implementation, at 2k seq length, this took around 45 mil ns, with height sharding (and running 16 slices of the tensor consecutively), this got down to ~14.75 mil ns.
Furthermore, attention sequence was optimised even further by providing a new softmax implementation (height sharded input, attn_mask of shape (1, 1, seq_len, seq_len) (Currently existing softmax implementation requires attn_mask shape to be == input shape (which is (1, 71, seq_len, seq_len). That got the attention sequence down to ~11.4 mil ns.

Looks great, are you integrating these changes into falcon7b in another PR?

Looks great, are you integrating these changes into falcon7b in another PR?

Yes, that is the plan

Passing post commit pipeline:
https://github.com/tenstorrent-metal/tt-metal/actions/runs/8629447397


Backwards/Forward compatibility

Overview
We'd still like the ability to make major revisions, so we'll communicate via semver: Major changes indicate that the format is completely incompatible with old clients. Minor changes indicate that both new clients can use old vg files and old clients can use new vg files. At the time of a major version release, we can increment the version number and possibly consider bundling the old decoder as a fallback - but this is out of scope at the time.
Background
The VG binary format is divided into sections which compose a topologically sorted description of a Flutter picture. The intention is that we can trivially convert the binary format into a Flutter picture "online" without creating and traversing a secondary tree structure. First, we describe shaders and gradients. Then we describe paints, which may reference those shaders/gradients. Then we describe paths. Then we describe text. Then finally we describe commands which may reference any of those objects.
Each section is composed of objects which are indicated by an integer identifier that is specific to that section. Each object is then composed of a fixed size number of fields that are implicit in the type of that object.
Adding backwards/forwards compatibility.
There are a few dimensions to backwards compatibility. One would be the ability to add new fields to existing objects.
Consider the encoded paint. Suppose we want to add the ability to provide a ColorFilter object. Today we would expand the size of the object, breaking old clients. An alternative to allow objects to consider growing is to include the size of the object as the first field after the type. This would allow old clients to ignore and read past unknown new fields, while new clients could fill in default values. This does not allow re-arranging of existing fields. Newly added values would need to be nullable in the decoder, or have default values provided. This would need to handled on a case by case basis.
Another dimension of backwards compatibility is new command types. This can be handled by making the command decode tolerate and skip over these commands with an embedded size tag.
The next dimension would be the addition of new sections. Because the sections are in a reasonable topological order, we need to be more careful. Instead of a more generic solution, let us consider some known missing features: filters and _. We know that these will need to be defined before paint, so we will simply insert an empty section that could be filled in later on. We also give each command section a header size and update the decoder to skip over them.
This does not address all possible backwards compatibility problems, but does allow continued iteration. A major backwards compatibility change could allow us to make further changes, but none are known at this time

One thing that might get sad:
Let's say we come up with some way to significantly compress the size of the binary (for example, when we dropped precision by at least half a ways back).
That will always be a major break. Our Google based customers won't really care that we're using semver unless we completely fork the thing internally, which I don't think we or they will want.
I guess the main thing is: if and once we do this, we've created a format we're committed to supporting. That might be ok but it's contrary to our initial design goals.
In particular, there will be little to no reason to run the compiler as part of the build anymore, and destructive optimizations may make it nearly impossible to recover any meaningful vector data once compilation has occurred. And customers will have no way to know if they got the latest optimizations or not, or even necessarily what optimizations could still be applied.

I guess the main thing is: if and once we do this, we've created a format we're committed to supporting. That might be ok but it's contrary to our initial design goals.

Yes this is true, but at the time I think we were a bit more hopeful this could be solved with infra on the client teams side. If thats less possible than we initially thought then we should change course instead.

That will always be a major break. Our Google based customers won't really care that we're using semver unless we completely fork the thing internally, which I don't think we or they will want.

We could techinically bundle both old and new. The decoders are quite small. If we did this, say, once a year for 10 years. We'd end up with 10 decoders to maintian forever. If we assume they can share a lot of code, its not the worst but its definitely more complicated than having a single active version.

In particular, there will be little to no reason to run the compiler as part of the build anymore, and destructive optimizations may make it nearly impossible to recover any meaningful vector data once compilation has occurred. And customers will have no way to know if they got the latest optimizations or not, or even necessarily what optimizations could still be applied

I'm not following this at all

Customers will stop rebuilding using the latest version of the compiler and miss out on optimizations we add.

for network assets, yes that is true. Also worth considering that impeller will have completely different performance characteristics, so we may want to actually make more intrusive changes based on expected renderer as well. Though its not quite far enough along for extensive benchmarking

Breaking changes only need a migration path between two successive versions. A totally cromulent migration path is, "Over the course of the next N months, you must stop using old assets." Client teams can deploy updates to achieve that.

That could also be achieved by bundling more than one version of the decoder at a time. i.e. we have 1 major version per year, and we support the last major version, which gives folks a 2 year migration window.

That is, the policy + bundling multiple deocders could allow us to have "major versions"


The examples can't be built

Because of stainless, the examples can't be built.

I noticed this yesterday too and I started to move away from all features that requires the nightly build of rust. And the biggest change is of course the use of stainless in the unit tests. So I hope to push the changes this night, if not the next one.

This should build now, using any rust release channel.


Issue with AllAsync

AllAsync implementation always returns "True" when using an async Predicate:
        private static async Task MainAsync()
        {
            var array = new[] { 0, 1 };
            var res = await array.AllAsync(async x => await DoWork(x) == 0);
            Debug.WriteLine(res);
        }

        private static Task<int> DoWork(int x)
        {
            return Task.FromResult(x - 1);
        }

Hello @ikaros88 ,
Are you sure this method is coming from our library? The code seems not compiling.
I do not remember supporting async predicate, but I may be wrong since I have not touched this library for a long time.
Best Regards,
Jonathan

Hi Jonathan,
I am sure that the method is from your library, the implementation can be found here:
All.cs
I am currently using FirstOrDefaultAsync, AnyAsync, WhereAsync, SingleAsync from your library, they all work perfectly, both in a real world project and also in small test cases such as the one above. Only AllAsync is always true as soon as the predicate is a Task.
Greetings

Hello @ikaros88 ,
You are right. I was probably still sleeping and opened a very old project.
We can successfully reproduce the issue. The problem happens because we treat the predicate in the All function like a filter. All result that returns false is filtered and skipped.
We will try to release a fix for Monday.
Best Regards,
Jonathan

Hello @ikaros88 ,
The v1.0.1 has been released
https://www.nuget.org/packages/Z.Linq.Async/
Let me know if the AllAsync method is now working correctly.
Best Regards,
Jonathan

Hi Jonathan,
everything works perfectly now, thanks a lot!
Greetings


Get Theremometer to clean up after the test

Thermometer should delete the temp hdfs directory and hive db, etc. It should also be possible to prevent that easily for debugging.

I’ve implemented a branch with optional support for deleting the thermometer-test* tmp directories (created by HadoopSupport) upon JVM shutdown. This can be enabled with environment variable THERMOMETER_HADOOP_SUPPORT_OPTS=cleanup:shutdown. A reporting option is also supported to print the sizes. The motivation for this is to prevent shared CI build agents from exhausting their /tmp space/inodes. Will raise a PR after writing tests.

This is resolved in 1.5.9 as an opt-in feature. @toddmowen has suggested that it should be opt-out in a future version bump.


Show hints for next n

basically have !n x show the hints for the next x elements

not sure if this is possible because i have to rework the entire paging system, and there isn't much point in this
also each of the next x elements have multiple hints, how do I display them?

just show as separate messages
the only reason I want this is a qol feature so I can grind elements faster

Closing because formatting doesn't work out and separate messages would be a rate limit hell.


Reverts kubernetes/frakti#234

Reverts kubernetes/frakti#234
dep does not work well during upgrade k8s

This change is


⚠ [Warning] : Missing Country for Locale endpoint -- User Inquiry Regarding Addition of Taiwan to Countries List

Is your warning request related to a problem? Please describe.
A user has reported the inability to select Taiwan as their country within the app, as it is currently not listed among the available countries. This creates inconvenience for users from Taiwan who wish to accurately represent their location.
Describe the solution you'd like
We would like Taiwan to be added to the list of available countries within the app, allowing users to select it as their location.
Describe alternatives you've considered
One alternative solution could be manually adding Taiwan to the list of available countries as a temporary measure until it is officially included in the app. However, the preferred solution is to have Taiwan included in the app's country selection options.
Additional context
The user has expressed their understanding of the complexities involved in adding a new country to the app and has kindly requested updates or insights into the progress of this issue. They are eager to see Taiwan included in the countries list.
Additional context
discord support link with appwrite
If you want speed up process of this issue, please send 🚀 as a reaction.

Demanded feature.
https://bsky.app/profile/miizombie.bsky.social/post/3kn6brknjzx2w

Linked with external issue: https://github.com/appwrite/appwrite/issues/7868

Done with this database upgrade to appwrite v1.5.5 https://status.langx.io/maintenance/361204


Run examples on CI; improve requirement management

CI now runs the tests under examples
Organize the dependencies into three groups

release is a core group of dependencies required for running PyMTL
dev includes release and other tools that facilitate PyMTL development (graphviz, formatters, etc.)
CI includes dev and tools required for performing CI (mainly code coverage plugins)
PyMTL package on PyPI will also have updated dependencies: pytest-xdist, pyparsing, and graphviz are removed

Does this change whatever we need to do currently? Can we still pip install pymtl3? Does it affect travis? How do we specify when we install it as dev?

Does this change whatever we need to do currently? Can we still pip install pymtl3? Does it affect travis? How do we specify when we install it as dev?

We can still pip install pymtl3. After this PR and another PyPI release, pymtl3 will not longer depend on graphviz, pytest-xdist, pyparsing
Travis now uses dependencies specified in CI.txt which is the same as before
After we git clone the repo, do pip install -r requirements.txt && pip install --editable .

Great. I think the last thing we need to decide for this PR is whether we want to drop 3.6 or 3.7 in travis.

Great. I think the last thing we need to decide for this PR is whether we want to drop 3.6 or 3.7 in travis.

Seems to me that having both 3.6 and 3.7 in CI is really helpful because we are actively using 3.7, and officially support 3.6. If the only concern for CI is speed, I'd say we should probably look at other ways...

Great. I think the last thing we need to decide for this PR is whether we want to drop 3.6 or 3.7 in travis.

Seems to me that having both 3.6 and 3.7 in CI is really helpful because we are actively using 3.7, and officially support 3.6. If the only concern for CI is speed, I'd say we should probably look at other ways...

ok. I put pypy as the first launch, but it seems like the pypy run is failing ,..weird

Man, adding example tests to CI revealed a lot of bugs introduced by the bit struct PR ......

We should create better "real" tests in the pymtl kernel then? Can we hypothesis to help stress test bitstructs? It is ok to catch bugs with examples, but that should really only be a last resort ... we should try and get pymtl kernel tests that would have failed too right?


Add a quick-start section to readme

[ ] Include sample config files
[ ] Include link to small dataset
[ ] Include link to pretrained model for testing inference

[ ] Make relevant jupyter notebooks?


[VM] Determine Stack Size Limits for VM Execution

Pull Request #574 introduces function and execution stack size limits that are checked in push operations. This limit should be derived based on the gas limit that transactions have.
Currently the execution stack and function stack size limits are set with the constants EXECUTION_STACK_SIZE_LIMIT and FUNCTION_STACK_SIZE_LIMIT defined in vm_runtime/src/execution_stack.rs and both set to 1024.

Dario, when you refactored the interpreter in #979, you replaced those limits with different names:
const OPERAND_STACK_SIZE_LIMIT: usize = 1024;
const CALL_STACK_SIZE_LIMIT: usize = 1024;

Are these the right values to use for now?

Dario, when you refactored the interpreter in #979, you replaced those limits with different names:
const OPERAND_STACK_SIZE_LIMIT: usize = 1024;
const CALL_STACK_SIZE_LIMIT: usize = 1024;

Are these the right values to use for now?

they are certainly good enough but we need to do a final review on 2 things:

check that we have the limit everywhere we need
ensure the limits are something we like
should we have something tracking all the limits we want?
@vgao1996 and I are following up on that but we don't have a list or doc.
Thoughts?


Expose an API to consume FileDescriptorProtos

I have a collection of FileDescriptorProtos that I have decoded from a protobuf stream. I want to generate Rust code capable of parsing them, but there doesn't seem to be any public API for this.
I think I want the prost_build::Config::generate method in the public api.

Hi @Plecra! You can use the prost-types crate for this - it has a FileDescriptorProto type which can be parsed in the normal way.

I'd actually already managed to decode a prost_type::FileDescriptorProto :) My problem is that I'm not really able to do anything with it.
Ultimately, I want to generate a rust crate capable of consuming the types in the  FileDescriptorProtos, but I'd have to rewrite the code from the generate method to do that (or practically, fork the crate)

Ah, I see! Sorry, I misunderstood the aim here.

I don't understand the request, @Plecra could you add more details about what functionality you would like?  Is this something you could do in your own crate?

prost_build::Config::compile_protos can be used to compile .proto files to rust code that can consume them - in my crate, I want to compile FileDescriptorProtos to rust code. For this, I need to call the prost_build::Config::generate function directly with the FileDescriptorProtos I have. My request is for it to be made a pub fn

Ah I see.  At this time prost-build is exclusively meant to be used from build.rs's.  I'd like to have a more thought out public API for niche usecases to build off, but it hasn't been top priority.

I think this is possible with the current prost_build API by:

Create a FileDescriptorSet from the FileDescriptorProto instances
Write it to a temporary file
Set file_descriptor_set_path and skip_proto_run to have prost_build use the file descriptors


Support disabling rules

I did a bit of work to support configuration, but I'd like to get it far enough so that rules can be disabled. I think that'll be important for real users.

I think this is now supported with 0dfcb70501012043e1a67c397b0cc475df8006fb


[Feature Request] Card for Home Assistant

Hello AlexxIt,
i'm very pleased you are working oin this project. Is possible to add a card in Home Assistant that uses go2rtc as a source?
I didn't find any new card after installation. Should we use Glance card maybe? What's the address for various go2rtc stream to point at?
Thank you

Oops, i've found the instruction, thank you anyway

I've looked at the process, which is not so clear. A dedicated card would be the best user friendly option for HASSIO, so I'm reopening the issue

You may use generic camera and Lovelace card as usual. As you using it before. But after install this add-on and setup buildin integration you will get WebRTC stream.

Is there any possibility of 2 way audio within a Home Assistant card? Would this require a brand new card?
Would need ability to mute & unmute microphone etc also

Well. Someone needs to extend default Lovelace card or write new one. I plan to spend my time on advanced server features in near futures.

For all of your information, you can feed the Go2rtc RTSP feed into https://github.com/dermotduffy/frigate-hass-card
This is very useful if you use frigate

Thank you but the fact here is that we passed from AlexxIT WebRTC Camera, with lovelace card with PTZ support (1 product, no config needed) to:

go2rtc
RTSPtoWebRTC
Generic Camera
Glance card
which is a step back in user friendlyness: need to configure everything, add single generic camera, etc

For all of your information, you can feed the Go2rtc RTSP feed into https://github.com/dermotduffy/frigate-hass-card
This is very useful if you use frigate

Can you quickly explain how to do that? Where do I reference the stream from go2rtc in the frigate card? I think it will just use the default RTSP stream and not from go2rtc?

If you install go2rtc and RTSPtoWebRTC, you can use instructions "Native WebRTC".

I haven't tested it completely, because I don't have a microphone on my PC. But the browser asks for microphone permissions, so it seems to work.
Can anyone test it?

I just tested it and it works in a browser and in the companion app.  I'll likely use it as a card-mod popup card.  It's a good temporary solution but it lacks a mute button which makes it risky. :)

I've been using the glance card and then added PTZ control via a separate integration (foscam in my case, ONVIF etc would also work).
Here's an example:
type: picture-glance
camera_view: live
title: Front Door
camera_image: camera.front_door
tap_action:
  action: more-info
entities:
  - entity: camera.front_door_camera_control
    name: Home
    icon: mdi:home
    tap_action:
      action: call-service
      service: foscam.ptz_preset
      service_data:
        entity_id: camera.front_door_camera_control
        preset_name: Home
  - entity: camera.front_door_camera_control
    name: Pan Left
    icon: mdi:arrow-left
    tap_action:
      action: call-service
      service: foscam.ptz
      service_data:
        entity_id: camera.front_door_camera_control
        movement: left
        travel_time: 0.75
  - entity: camera.front_door_camera_control
    name: Tilt Up
    icon: mdi:arrow-up
    tap_action:
      action: call-service
      service: foscam.ptz
      service_data:
        entity_id: camera.front_door_camera_control
        movement: up
        travel_time: 0.75
  - entity: camera.front_door_camera_control
    name: Tilt Down
    icon: mdi:arrow-down
    tap_action:
      action: call-service
      service: foscam.ptz
      service_data:
        entity_id: camera.front_door_camera_control
        movement: down
        travel_time: 0.75
  - entity: camera.front_door_camera_control
    name: Pan Right
    icon: mdi:arrow-right
    tap_action:
      action: call-service
      service: foscam.ptz
      service_data:
        entity_id: camera.front_door_camera_control
        movement: right
        travel_time: 0.75

camera.front_door is my go2rtc stream, amera.front_door_camera_control is the foscam integration used solely for PTZ control.

Supported in WebRTC Camera v3 (beta)
https://github.com/AlexxIT/WebRTC/releases


KLT cert-manager installation type fails

Referring to this particular section: https://lifecycle.keptn.sh/docs/getting-started/#install-version-060-and-earlier
The two commands given throws separate errors:

kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.11.0/cert-manager.yaml -> This particular command throws this error:

kubectl wait --for=condition=Available deployment/cert-manager-webhook -n cert-manager --timeout=60s -> This particular command throws error of timout which I think is due to the first command not executing completely.

To dig more into the origin of the issue refer to this issue: https://github.com/SuperAayush/KLT-Killercoda/issues/11
cc - @agardnerIT

Now that KLT is on v0.7.0 I'd be in favour of removing any older instructions. It clearly a) doesn't work and b) causes confusion.

I agree.
CC @StackScribe

Removal of the outdated installation docs will be solved in #1168


Review and improve readability of code in g.part1()

I am creating this issue as a placeholder for work by @l-k- to review and improve readability of g.part1() and its subfunctions.
We will tag this issue for key commits and PRs.

Closing this now as main work has been done and merged.


[SPARK-LLAP-3] Use Apache Trade Mark explicitly.

This resolves #3.

Hi, @bikassaha .
Could you review this PR? This adds Apache and the trade mark explicitly for Apache Spark, Apache Hive, and Apache Ambari first.

lgtm. Before committing please consider adding a title like SHC has - e.g. Apache Spark connector for Apache Hive LLAP.

I see. Thank you for review.


Healthcheck failed Tesla Model S

Hi
I am not able to get my car to communicate to the app. It was working yesterday but it doesn't connect anymore. I am seeing the token refresh fail from the logs.  Log snippet below.
Try this to see if the server is live at a different location.
https://www.isitdownrightnow.com/owner-api.teslamotors.com.html

That only shows the DNS is responding.
Try this to see if the server is live at a different location.
https://www.isitdownrightnow.com/owner-api.teslamotors.com.html


iDeep F.relu retains input array unnecessarily

If iDeep is used, F.relu retains both of input and output during training, even though iDeep implementation of F.leaky_relu retains outputs only. This can cause memory exhaust.
Apart from the issue itself, it is questionable whether iDeep improves F.relu performance, because it is band-width limited and trivially written in NumPy. So it might be better to change
intel64.should_use_ideep('>=auto')

to
intel64.should_use_ideep('==always')

Sorry, the second paragraph is wrong. iDeep can improve backprop speed.
I'll create a PR for the issue.

#5871


Fix schema definition to match code

This turns out to have had an incorrect value for a while, however the version of jsonschema being used didn't detect this as it didn't support Draft 6 of JSON Schema. Update the package to a more recent range which does support that.

Coverage remained the same at 99.453% when pulling ee283d7a168beb074f3350a0c51d8ff63c10569a on fix-schema-definition into a1f7e78f21ab17340f8bb533de645c19ce9716ae on master.


Park and Unload boxes blank

Hi i have installed your plug in but the 2 boxes Park and Unload are blank with no name or color to them. I have not tried to change filament eyt as i have only now managd to find out what these 2 buttons do.
Hope you can help resolve the small problem.
I am using a Anycubic i3 mega with Octoprint
Terry

What other plugins do you have installed? Anything that might change the theme/style of OctoPrint?

Hi Jim
Many thanks for your prompt reply to a newbie to Octoprint.
I have the following plugs in installed :-
Action Command Prompt
Action Trigger
Change Filament
Enclosure Plug in
Filament manager
Navbar temperature
Preheat Button
Printtime genius
Tab order
Themeifly
Hope it helps
Best regards
Terry Warner
KB Agency
Tel          (44) 1604 404862
Mobile   (44) 7899 668079
DISCLAIMER:
The information in this E-Mail message is confidential and may be legally privileged. It is intended solely for the addressee and access to this message by anyone else is unauthorised. If you have received the message in error, please notify us immediately and delete it. If you are not the intended recipient, any disclosure, copying, distribution or any other use is prohibited and is be unlawful.
From: Jim P<EMAIL_ADDRESS>Sent: Tuesday, July 23, 2019 2:39 PM
To: jim-p/Change_Filament<EMAIL_ADDRESS>Cc: Terry w<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>Subject: Re: [jim-p/Change_Filament] Park and Unload boxes blank (#8)
What other plugins do you have installed? Anything that might change the theme/style of OctoPrint?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/jim-p/Change_Filament/issues/8?email_source=notifications&email_token=AJQA6EMPFIWNIVAHH6QG3S3QA4CXZA5CNFSM4IGASDWKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2TERJI#issuecomment-514214053, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AJQA6ELCXR5N56Z27GPZX4LQA4CXZANCNFSM4IGASDWA.

This almost certainly a duplicate of #4 then -- There is a known issue with Themify.


⚠ Bofrost.lu has degraded performance

In fefd9d9, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 1173 ms

Resolved: Bofrost.lu performance has improved in c2abe84.


Why all_fea = torch.cat((all_fea, torch.ones(all_fea.size(0), 1)), 1) ?

Hello, after read the previous answers of this question, I am still confused about this operation. Why should we explicitly add 1 as bias? As far as I know, the linear transformation with bias on will not change the size of our feature, so I feel it is unnecessary to add a bias manually.
Would you please further explain this for me?
Thx

@yyaaa1 for the last classifier[w, b], if we add 1, then the new features will be close to the [w_k, b_k] via the cosine distance, which is adopted in this paper.

@tim-learn
        nn.init.zeros_(m.bias)
I see that bias is initialized to 0 in your code, which is equivalent to no bias, isn’t it?

@tim-learn
 nn.init.zeros_(m.bias)
I see that bias is initialized to 0 in your code, which is equivalent to no bias, isn’t it?

I think the bias would have a non-zero value after learning, maybe you can check that value.

@tim-learn
 nn.init.zeros_(m.bias)
I see that bias is initialized to 0 in your code, which is equivalent to no bias, isn’t it?

I think the bias would have a non-zero value after learning, maybe you can check that value.


Error while uploading a base64 encoded file using a Meteor Method.

Here is the code of a method used in server:
export const insertImage = new ValidatedMethod({
  name: 'images.insert',
  validate: new SimpleSchema({
    file: { type: String },
  }).validator(),
  run({ file }) {
    Images.write(new Buffer(file, 'base64'), {
      fileName: 'ImageName.png',
      type: 'image/png',
    }, function (error, fileRef) {
      if (error) {
        throw error;
      } else {
        console.log(`${fileRef.name} is successfully saved to FS. _id: ${fileRef._id}`);
        return fileRef;
      }
    });
  },
});

I am receiving file encode in base64 from frontend correctly.
The document is created in mongo database but the image saved in FS is corrupted.(it's not an image).
https://imgur.com/a/yCE2A

I am not able to receive fileRef, or information of saved document to client( frontend).

Well for the first point I have made a progress on fixing it using this hack: https://gist.github.com/madhums/e749dca107e26d72b64d
Now I am able to save image correctly. here my new Method.
export const insertImage = new ValidatedMethod({
  name: 'images.insert',
  validate: new SimpleSchema({
    file: { type: String },
  }).validator(),
  run({ file }) {
    Images.write(new Buffer(file.replace(/^data:image\/\w+;base64,/, ''), 'base64'), {
      fileName: 'ImageName.png',
      type: 'image/png',
    }, (error, fileRef) => {
      if (error) {
        throw error;
      } else {
        console.log(`${fileRef.name} is successfully saved to FS. _id: ${fileRef._id}`);
        return fileRef;
      }
    });
  },
});

I am intending to use this method for all types of files and I am not sure if it will work (videos, pdf ... etc).
Please let me know what you think !

@badis I didn't get it. Have you solved it?

Yes, I solved the first part.
Now i need to return information to client.
I am not able to receive fileRef, or information of saved document to client( frontend).

.wrapAsync sounds good ! because it's a meteor function.
Can you please help me implementing it with my code.
Images.write returns a FilesCollection instance. I want to return fileRef to my client.

Sure, you just wrapping it up, regardless what this function returns, as we interested in a callback of this function:
const asyncWrite = Meteor.wrapAsync(Images.write);
// That's it
// Now when you need fileRef:
const fileRef = asyncWrite(buffer, opts);

Thank you !


fetchart: Minimum width option

Hi,
The fetchart plugin is finding tiny images that are useless for my albums.  Would it be possible to set a minimum image size eith in width/height or in KB.

Is this still being worked on? I'd like this feature too.

Oh right! I was/am waiting on PR #1277 to be merged, considering some work was done there on ArtResizer, but I have to admit I totally forgot about this issue (woops).


Please rename csv2bin to a less generic name

csv2bin binary name is too generic. It looks as it it's one of the 2 tools that converts generic one format to another (e.g. ps2ascii  ps2pdf  ps2pdf13  ps2pdfwr  ps2ps  ps2txt  ps2epsi  ps2pdf12 ps2pdf14  ps2pk  ps2ps2, etc.).
Is this actually a csv2vpd tool? Or it is specifically a genwqe-csv2vpd tool (only for genwqe cards)?
Please rename csv2bin to a more descriptive name.

Reading sample spec file:
genwqe_csv2vpd
Would be a better name for this tool, similar to genwqe_vpdconf & genwqe_vpdupdate.

Agreed, genwqe_csv2vpd is appropriate.

After renaming the tool now twice ;-), I hopefully have now the correct name.


Warnings helper

Summary
As we are stabilizing experiments API, it becomes important to implement proper deprecation logic (especially in experiments since we should consider back port for previously db-saved experiments). This PR adds several deprecation decorator and some supports such logic.
Details and comments

Thanks @coruscating


Use published ostree-ext 0.10

I just pushed this out, should fix the cargo deny failures.

@travier I invited you to be a maintainer here so the approval works :smile:

🤔

Hum, weird.