BUG: Combining SciPy 1.14.1 and statsmodels 0.14.2 on OSX causes "symbol not found in flat namespace '_npy_cabs'"

Describe your issue.
When importing statsmodels.api, the following error appears:
ImportError: dlopen(/private/var/folders/7v/pg90myg55bs218ff8dpkrfrc0000gn/T/tmp.8CjfzJEYnh/venv/lib/python3.12/site-packages/statsmodels/tsa/statespace/_representation.cpython-312-darwin.so, 0x0002): symbol not found in flat namespace '_npy_cabs'

This issue was originally reported on StackOverflow. I have independently reproduced the failure on OSX, but failed to reproduce it on Linux.
Curiously, this issue only shows up when SciPy 1.14.1 is installed. If SciPy 1.14.0 is installed, this error does not show up. Similarly, if statsmodels 0.14.1 is installed, the issue also goes away.
I'm unsure if this is a bug in statsmodels or SciPy - it seems very strange that a change in another package could cause this kind of error.
pip freeze output:
Failing:
numpy==2.1.0
packaging==24.1
pandas==2.2.2
patsy==0.5.6
python-dateutil==2.9.0.post0
pytz==2024.1
scipy==1.14.1
six==1.16.0
statsmodels==0.14.2
tzdata==2024.1

Working:
numpy==2.1.0
packaging==24.1
pandas==2.2.2
patsy==0.5.6
python-dateutil==2.9.0.post0
pytz==2024.1
scipy==1.14.0
six==1.16.0
statsmodels==0.14.2
tzdata==2024.1

Reproducing Code Example
# My minimized reproducer
import statsmodels.api as sm

# Original reproducer
import pandas as pd
import plotly.express as px

if __name__ == "__main__":
    data = {
        "Date": [0, 7, 14, 21, 28],
        "Value": [100, 110, 120, 115, 122]
    }
    df = pd.DataFrame(data)
    px.scatter(df, x="Date", y="Value", trendline="ols")

Detailed Reproduction Instructions

Create new virtual environment.
Run pip install scipy==1.14.1 statsmodels==0.14.2, downloading pre-built binaries. For me this is scipy-1.14.1-cp312-cp312-macosx_14_0_arm64.whl.
Run either of the reproducers. The reproducer should fail.
Run pip install scipy==1.14.0 statsmodels==0.14.2
Run either of the reproducers. The reproducer should succeed.

Error message
Traceback (most recent call last):
  File "/private/var/folders/7v/pg90myg55bs218ff8dpkrfrc0000gn/T/tmp.8CjfzJEYnh/test.py", line 1, in <module>
    import statsmodels.api as sm
  File "/private/var/folders/7v/pg90myg55bs218ff8dpkrfrc0000gn/T/tmp.8CjfzJEYnh/venv/lib/python3.12/site-packages/statsmodels/api.py", line 136, in <module>
    from .regression.recursive_ls import RecursiveLS
  File "/private/var/folders/7v/pg90myg55bs218ff8dpkrfrc0000gn/T/tmp.8CjfzJEYnh/venv/lib/python3.12/site-packages/statsmodels/regression/recursive_ls.py", line 14, in <module>
    from statsmodels.tsa.statespace.mlemodel import (
  File "/private/var/folders/7v/pg90myg55bs218ff8dpkrfrc0000gn/T/tmp.8CjfzJEYnh/venv/lib/python3.12/site-packages/statsmodels/tsa/statespace/mlemodel.py", line 33, in <module>
    from .simulation_smoother import SimulationSmoother
  File "/private/var/folders/7v/pg90myg55bs218ff8dpkrfrc0000gn/T/tmp.8CjfzJEYnh/venv/lib/python3.12/site-packages/statsmodels/tsa/statespace/simulation_smoother.py", line 11, in <module>
    from .kalman_smoother import KalmanSmoother
  File "/private/var/folders/7v/pg90myg55bs218ff8dpkrfrc0000gn/T/tmp.8CjfzJEYnh/venv/lib/python3.12/site-packages/statsmodels/tsa/statespace/kalman_smoother.py", line 11, in <module>
    from statsmodels.tsa.statespace.representation import OptionWrapper
  File "/private/var/folders/7v/pg90myg55bs218ff8dpkrfrc0000gn/T/tmp.8CjfzJEYnh/venv/lib/python3.12/site-packages/statsmodels/tsa/statespace/representation.py", line 10, in <module>
    from .tools import (
  File "/private/var/folders/7v/pg90myg55bs218ff8dpkrfrc0000gn/T/tmp.8CjfzJEYnh/venv/lib/python3.12/site-packages/statsmodels/tsa/statespace/tools.py", line 14, in <module>
    from . import (_initialization, _representation, _kalman_filter,
  File "statsmodels/tsa/statespace/_initialization.pyx", line 1, in init statsmodels.tsa.statespace._initialization
ImportError: dlopen(/private/var/folders/7v/pg90myg55bs218ff8dpkrfrc0000gn/T/tmp.8CjfzJEYnh/venv/lib/python3.12/site-packages/statsmodels/tsa/statespace/_representation.cpython-312-darwin.so, 0x0002): symbol not found in flat namespace '_npy_cabs'

SciPy/NumPy/Python version and system information
1.14.1 2.1.0 sys.version_info(major=3, minor=12, micro=5, releaselevel='final', serial=0)
/private/var/folders/7v/pg90myg55bs218ff8dpkrfrc0000gn/T/tmp.8CjfzJEYnh/venv/lib/python3.12/site-packages/scipy/__config__.py:154: UserWarning: Install `pyyaml` for better output
  warnings.warn("Install `pyyaml` for better output", stacklevel=1)
{
  "Compilers": {
    "c": {
      "name": "clang",
      "linker": "ld64",
      "version": "15.0.0",
      "commands": "cc"
    },
    "cython": {
      "name": "cython",
      "linker": "cython",
      "version": "3.0.11",
      "commands": "cython"
    },
    "c++": {
      "name": "clang",
      "linker": "ld64",
      "version": "15.0.0",
      "commands": "c++"
    },
    "fortran": {
      "name": "gcc",
      "linker": "ld64",
      "version": "13.3.0",
      "commands": "gfortran"
    },
    "pythran": {
      "version": "0.16.1",
      "include directory": "../../../../../../private/var/folders/hw/1f0gcr8d6kn9ms0_wn0_57qc0000gn/T/pip-build-env-bjescfad/overlay/lib/python3.12/site-packages/pythran"
    }
  },
  "Machine Information": {
    "host": {
      "cpu": "aarch64",
      "family": "aarch64",
      "endian": "little",
      "system": "darwin"
    },
    "build": {
      "cpu": "aarch64",
      "family": "aarch64",
      "endian": "little",
      "system": "darwin"
    },
    "cross-compiled": false
  },
  "Build Dependencies": {
    "blas": {
      "name": "accelerate",
      "found": true,
      "version": "unknown",
      "detection method": "extraframeworks",
      "include directory": "unknown",
      "lib directory": "unknown",
      "openblas configuration": "unknown",
      "pc file directory": "unknown"
    },
    "lapack": {
      "name": "accelerate",
      "found": true,
      "version": "unknown",
      "detection method": "extraframeworks",
      "include directory": "unknown",
      "lib directory": "unknown",
      "openblas configuration": "unknown",
      "pc file directory": "unknown"
    },
    "pybind11": {
      "name": "pybind11",
      "version": "2.12.0",
      "detection method": "config-tool",
      "include directory": "unknown"
    }
  },
  "Python Information": {
    "path": "/private/var/folders/hw/1f0gcr8d6kn9ms0_wn0_57qc0000gn/T/cibw-run-oogm4_8g/cp312-macosx_arm64/build/venv/bin/python",
    "version": "3.12"
  }
}

Architecture
I don't know if this is relevant, but both the original reporter and I are using ARM-based hardware. (An Apple M2 in his case, and the M3 in mine.)

The statsmodels maintainers suggested trying to downgrade NumPy to 2.0.1. The result works, meaning that this issue only happens for the latest NumPy plus the latest SciPy plus the latest statsmodels.
See https://github.com/statsmodels/statsmodels/issues/9333#issuecomment-2305438605 for a detailed list of the configurations I tried.

From the stacktrace I'm pretty sure that the issue lies in the statsmodels space, scipy doesn't appear anywhere there.
scipy.special does use npy_cabs, it's defined in <numpy/npy_math.h>.

From the investigation done by the statsmodels maintainer, this seems like either a statsmodels issue (in that it depends on the _npy_cabs symbol being exported by another library) or a numpy bug (in that it stops exporting _npy_cabs.) I'm not sure which, at this point.
However, SciPy's involvement seems pretty incidental. Although it does stop exporting the symbol _npy_cabs in 1.14.1, that doesn't really seem like part of our public API. For that reason, I'm closing in favor of the statsmodel/numpy issues.

However, SciPy's involvement seems pretty incidental. Although it does stop exporting the symbol _npy_cabs in 1.14.1, that doesn't really seem like part of our public API

This was on purpose at least, I tried hard to clean up symbol visibility issues and only export a single PyInit_* symbol from each extension module. I agreed with your conclusion here - not part of our API. I'll comment more on the numpy/statsmodels issues as needed, after looking into the root cause.

The statsmodels maintainers suggested trying to downgrade NumPy to 2.0.1. The result works, meaning that this issue only happens for the latest NumPy plus the latest SciPy plus the latest statsmodels.
See statsmodels/statsmodels#9333 (comment) for a detailed list of the configurations I tried.

Temporary solution for me was pip install --force-reinstall numpy==2.0.0


Minimize the member cluster resources tracked by the control plane

The federated control plane can be scoped to all namespaces or a single namespace, and will then watch all resources in the selected namespace(s) for enabled types (in all member clusters) whether or not those resources are federated.  Since each tracked resource imposes a memory cost on the federation control plane, it would be desirable to minimize memory usage by not tracking resources that were not federated.
Possible solutions:

Where federated resources existed in a subset of available namespaces, support limiting the scope of the control plane to a subset of available namespaces (via a label selector) to limit caching to the selected namespaces.

Implementation note: Simple to implement, informers already support filtering by label.

Where federated resources potentially existed in all namespaces, support limiting the scope of the control plane to resources identified as being federated.  Resources could be labeled on propagation and the control plane would only track resources with the expected labeling. (this was suggested by Anders Janmyr on slack)

Implementation note: Propagation logic would have to be updated to handle the case where an un-labeled resource existed in a member cluster and needed to be updated to be federated.  Since the resource wouldn't exist in the informer, the federated updater would need to catch AlreadyExists on a call to create, retrieve the resource to retain fields, and then perform an update.

/kind feature

@marun Have we measured what that memory cost is?


UpdateFromQueryAsync with IDictionary throws Argument types do not match

Description
Hi pals!
First of all thanks for this amazing lib, but currently I'm getting an exception when I try to set a null into a nullable field (FieldB) using the dictionary approach:
        var dict = new Dictionary<string, object>();

        dict.Add("FieldA", 3);
        dict.Add("FieldB", null);

        return Context.SomeTable.Where(p => true).UpdateFromQueryAsync(dict);

Exception
System.ArgumentException: Argument types do not match
   at System.Linq.Expressions.Expression.Bind(MemberInfo member, Expression expression)
   at BatchUpdateExtensions.UpdateFromQuery[T](IQueryable`1 query, IDictionary`2 dictionary, Action`1 batchUpdateBuilder)
   at BatchUpdateExtensions.UpdateFromQuery[T](IQueryable`1 query, IDictionary`2 dictionary)
   at BatchUpdateExtensions.`1.()

Further technical details

EF version: "Microsoft.EntityFrameworkCore.SqlServer" Version="3.1.0"
EF Plus version: "Z.EntityFramework.Plus.EFCore" Version="3.0.43"
Database Provider: SqlServer

Hello @jomendoza84 ,
Thank you for reporting.
We will look at it.
Best Regards,
Jon

Performance Libraries
context.BulkInsert(list, options => options.BatchSize = 1000);
Entity Framework Extensions • Entity Framework Classic • Bulk Operations •  Dapper Plus
Runtime Evaluation
Eval.Execute("x + y", new {x = 1, y = 2}); // return 3
C# Eval Function • SQL Eval Function

Hello @jomendoza84 ,
The v3.0.45 has been released.
Could you try it and let us know if everything now works as expected?
Best Regards,
Jon

Hi Jon, the new version is working as expected :)
Thanks!!


[BUG] Github watcher stat incorrect

Description
GitHub stats for watchers is incorrect (but actually what we want is to add subscribers)
https://github.com/Microsoft/Recommenders/blob/c4b522cec7f4f38784bc7d2250b719fbee51b810/scripts/repo_metrics/git_stats.py#L117
Related info on api here: https://developer.github.com/changes/2012-09-05-watcher-api/
Expected behavior (i.e. solution)
want to capture watchers and subscribers counts separately from stars

resolved


KMLexport: Where does the exporter query the geometry (envelop) information from the database?

Hello
Currently I am trying to identify the location of the code where the KMLexporter queries the envelop information of a building from the database.
I was able to locate the execution of the following statement in KmlSplitter.java:

As you can see that the exported KML file contains some coordinates:

I wonder how and where the exporter exactly queries this information from the database in the source code. Thank you for any help.

All SQL queries are put here:
https://github.com/3dcitydb/importer-exporter/blob/master/impexp-kml-collada-plugin/src/main/java/org/citydb/modules/kml/database/Queries.java

Thank you very much for your response. I have seen this Queries class and the sqlbuilder library.
I am trying to translating the SQL statement to SPARQL statement for graph database, the provided sqlbuilder is quite useful for the dynamic translation in generic way (SQLstatement object --> SPARQLstatement object). But I also noticed that the queries class generates only sql query string which requires additional parser (from string to SQLstatement) for the translation.
I wonder if there is SQLparser library can help me for dynamic translation or any exclusive library which can translate directly from SQL to SPARQL. Thank you very much in advance.

Hi @lisha992610, the Queries class used by the KML/COLLADA/glTF exporter in fact just uses plain strings for the SQL statements. Our internal sqlbuilder library is tailored to the purpose and requirements of the Importer/Exporter. It does not provide SPARQL support. Personally, I have not worked with SPARQL a lot so far, so I am not aware of a generic library that could translate between SQL and SPARQL.
I am closing this issue now. If you have found a library or solution, it would be great if you could post your experience here.


Lyrics Support

Hi, first of all thanks for the awesome app and the fact that you're actively maintaining it !
Second is, I don't know much about audio tags, I think I learned a few things while trying to understand what was wrong.
So here's the story :
I'm using Oto Music on Android to listen to my .flac songs, and also to fetch the synced lyrics. I thought the lyrics would be shown afterwards on Tagger, but it's not the case.
I tried to understand where the lyrics were stored, and apparently they're stored in a lyrics custom tag. I saw that you did an update 2 weeks ago to add support for custom tags, and indeed they're here.
Still, the lyrics tag doesn't show up. I found out while using Ex Falso (another tag editor) that apparently the issue could be that the lyrics are (obviously) stored in some sort of "multi-line" field, because on Ex Falso there's a checkbox to "unsee multi-lines tags", which makes the "lyrics" tag disappear.
It may not be the issue, but if this is indeed the issue, is there any way to add support for "Multi-Lines" custom tags ?

Yes I'll look into multi-line custom tags...probably just need to let the row expand vertically to show it.
Also, I'm thinking we could also add proper lyrics support to Tagger as they have some special features (could probably download lyrics from web too)

Wow, thanks for the fast answer.

add proper lyrics support to Tagger

I wasn't thinking about that in the first place ... that would be awesome !
Maybe some sort of helper for editing .LRC data as well ? :thinking:
That's not necessary though, just being able to edit the lyrics would be cool.

Please try out the feature in this release: https://github.com/NickvisionApps/Tagger/releases/tag/2023.8.2-rc1

Hi !
Sorry I only saw that yesterday before I went sleeping.
I already installed it through flathub, I'm gonna do some tests to provide a feedback.
The synced lyrics editor looks cool though, and the update arrived quite quickly, that's very nice !

@nlogozzo Hi again !
I tried with 2 .flac files and one blank .mp3 (by "blank" I mean a song which does not currently have embedded lyrics).
Unfortunately I have some issues.
The first one may not be Tagger related, it's just that my synced lyrics are stored in the LYRICS tag, where Tagger gets the unsynced lyrics, so it's displayed as plain text.
(which is still better than no lyrics at all though)
I got another issue where the lyrics were just not displaying on one of my .flac song, even though they were on other apps (such as NTag or Ex Falso).
The file was a .flac file and the lyrics were in the LYRICS tag. As they were displayed on the first file, I opened the file in NTag and tried some things, such as shortening the lyrics to match the first files' length (I thought that it may be a string length issue ...). I even tried to replace the lyrics with the ones from the other working song, but they weren't displaying when I opened the file in tagger.
It could be a pretty bad issue because when I edit some things in tagger with this file (like the song name), tagger overwrites the lyrics tag, and delete it since it sees it as empty.
I tried setting the lyrics with tagger on this file, and they display properly on other softwares (as NTag and Ex Falso). However, when I edit them with NTag and close every apps (Tagger, NTag, Ex Falso), when I open the file with NTag and Ex Falso, the lyrics are as they were when I edited them, but with Tagger they stayed as they were before the edit. Which is really weird.
If I do other changes to the file with tagger, the lyrics are overwritten to the before-edit-state.
I tried with the other "lyrics-working-at-first" file as well, and it's the same issue :thinking:. And yes, I tried both refreshing and closing Tagger, as well as closing Tagger before editing with another software.
I tried copying more music files to my Tagger folder and the lyrics aren't displaying on some of them. It's like 2 or 3 lyrics-missing files on 19 songs.
As for the other tabs, I don't seem to see where the informations are stored. When I try to add synced lyrics there does not seem to be any change to the file when I lookup on other softwares. Same goes for the "language" and the "description" fields in the configure tab.
The datas are obviously there, because I tried doing some md5sum on the files after each edit, and checking the sum after reverting the edits to see if it matches the previous one. I I just don't know where they are x)

@nlogozzo 2 other things I just realized :

I think users should not be allowed to create a lyrics custom tag since that's what the "Unsynced lyrics" tab is basically for.
In the "Synced lyrics" tab it's said that "lyrics are stored as a Dictionnary of strings identified by their timestamp in milliseconds.
I don't know if there's some sort of specification for this type of embedded lyrics, but - for me using Oto Music at least - the lyrics are in .lrc format (the same as the file format, just embedded). (not sure if it should be stored in the lyrics or if this is some Oto Music specific thing.

Here's a link about LRC : https://en.wikipedia.org/wiki/LRC_(file_format)

Thanks for all the information!! It's sounds more like an issue in the tagging backend Tagger uses (atldotnet)...so I ask if you can send me an mp3 file and a flac file with lyrics that were NOT edited in Tagger....I'll pass them along to atldotnet's developer so we can see how this other software stores tags and then parse them correctly so they work in Tagger.
Feel free to email them to me<EMAIL_ADDRESS>

I can easily send you some different .flac file in a few hours.

Let me know here once you send the email as I'll be working all day and I'll see the github notification before the email one 😅

I sent you an email.
The .flac files were too heavy to be sent alongside the email, so there's a Proton Drive link at the end.
The rest of the email contains some infos about what I did with each file (where does the lyrics got applied, etc...).
By the way, I didn't mentionned it earlier but whenever I open the Lyrics dialog, even though I didn't changed anything Tagger considers the file as edited, and will prompt to apply the changes.

Fine !
I'll check for the new beta version. However, I'm going to go on vacation this sunday, I'l probably be able to try the new version but I'll also probably be less there.

Fine ! I'll check for the new beta version. However, I'm going to go on vacation this sunday, I'l probably be able to try the new version but I'll also probably be less there.

Alrighty no worries...I should be able to test it with the files you sent me as well
Enjoy the vacation 🙂

Please try the beta and let me know how it goes: https://github.com/NickvisionApps/Tagger/releases/tag/2023.8.3-beta1 :)

Hi again !
I tried. It's really nice ! I like the fact that it doesn't overwrites all the lyrics, but keeps the existing tags even though tagger don't use them.
Still some issues though :
First of all, you can't just edit the synced lyrics. If you only edit the synced lyrics - not talking about adding a line but rather editing an existing one - the "apply" button will still greyed-out. I checked with another software as well : not edited. You need to change another metadata in order to enable the "apply button", then apply the changes. Then, the lyrics are edited.
Can't save language and description, but I don't have any idea what those values are. I just know I can't edit them, like if I add a value it disappears after an apply.
Weird behaviour when having both synced and unsynced lyrics. Which is understandable since it's just not possible lol. Basically, adding unsynced lyrics wipes out the synced lyrics. Adding synced lyrics is not possible if there's already unsynced lyrics. Not a bug, but it would be nice to ... I don't really know, maybe greying out one of the tabs if the other one is the one currently in use. But we should find a way to explain to the user why the tab is greyed out and how to switch. Or maybe making both tabs overwrite the lyrics field if and only if it's not the current used tab, and adding a modal dialog to warn the user that this change will be destructive.

Here's an image if the second "modal" suggestion is not clear enough. It's really late so I don't really know.

Good night c:

First of all, you can't just edit the synced lyrics. If you only edit the synced lyrics - not talking about adding a line but rather editing an existing one - the "apply" button will still greyed-out. I checked with another software as well : not edited. You need to change another metadata in order to enable the "apply button", then apply the changes. Then, the lyrics are edited.

Ah this issue came when we refactored the code...I know the fix (see #269)

Can't save language and description, but I don't have any idea what those values are. I just know I can't edit them, like if I add a value it disappears after an apply.

Will look into

Weird behaviour when having both synced and unsynced lyrics. Which is understandable since it's just not possible lol. Basically, adding unsynced lyrics wipes out the synced lyrics. Adding synced lyrics is not possible if there's already unsynced lyrics. Not a bug, but it would be nice to ... I don't really know, maybe greying out one of the tabs if the other one is the one currently in use. But we should find a way to explain to the user why the tab is greyed out and how to switch. Or maybe making both tabs overwrite the lyrics field if and only if it's not the current used tab, and adding a modal dialog to warn the user that this change will be destructive.

It should be possible to have both unsync and sync lyrics actually...will investigate

It should be possible to have both unsync and sync lyrics actually...will investigate

It works for me...can't reproduce:
Screencast from 2023-08-22 00-37-05.webm
Could you email me the file that's giving you trouble and see if it happens with different files and different file types (flac vs mp3 vs m4a)

Can't save language and description, but I don't have any idea what those values are. I just know I can't edit them, like if I add a value it disappears after an apply.

From our backend's dev:

I simply assumed setting a description or a language code without setting the lyrics themselves is pointless. Plus in ID3v2, language code and description cannot float in outer space - they need to be linked to other metadata.

Which makes sense. Therefore I'll probably gray those fields out in the UI until unsync or sync lyrics are set.

Ok got all of this fixed, just waiting for an updated package of our backend and I'll make a new release

(sorry for the late reply as well)

It should be possible to have both unsync and sync lyrics actually...will investigate

Oh ... I just saw your screencast. How does it work ? :sob: I mean, both of them are stored in the "lyrics" field :thinking:

Could you email me the file that's giving you trouble and see if it happens with different files and different file types (.flac vs .mp3 vs .m4a)

According to your screencast you're using an mp3 file. I just tried with an .mp3 file and it works, but not with my .flac ones.
I think your guess is correct. The .flac files I sent you a few days ago should work. (or, "not work" more precisely. I mean ... it should work to not work.).
*tbh I guess you probably already did some tests since it's been 3 days and apparently "you got all of this fixed", but I can still provide you other files if you want.

not really an "edit" but more of a "I'm adding something" since I tend to do testings along writing my comments.
I just managed to add a language code and a description to the lyrics with a .flac file. I didn't manage to reproduce with the other one I'm doing my tests with. I don't have any idea why. Same goes for the ability to have both types of lyrics. Maybe it's just an issue with this file ... I don't have the time to try more intensively right now and since we're waiting for a future release it may have been fixed. Anyways, ask me if you still want that file.

Anyways, ask me if you still want that file.

Yeah please send me that specific flac file that's giving you issues...

I emailed you the files (still a Proton Drive link, one file is working, the other one isn't).
Here's a video of me trying to edit both files as well.
Capture vidéo du 2023-08-25 21-03-40.webm

I emailed you the files (still a Proton Drive link, one file is working, the other one isn't).
Here's a video of me trying to edit both files as well.
Capture vidéo du 2023-08-25 21-03-40.webm

Thank you!

Alright so everything should be fixed in https://github.com/NickvisionApps/Tagger/releases/tag/2023.8.3 expect having both unsync and sync for flac files...going to work on that for next version :)

Just note for lang code and description will be disabled now until you add an unsync or sync lyrics as tags required for it to be saved...might move the config tab to the last one as i think that'll make more sense

Use #279 to track further

https://github.com/NickvisionApps/Tagger/releases/tag/2023.9.0-beta1


Correcting resource type in CODEOWNERS file, and adding a metric explorer workbook

PR Checklist

[x] Explain your changes, so people looking at the PR know what and why, the code changes are the how.
Discovered and corrected an erroneous resourceType string in the prior CODEOWNERS PR.
[x] Validate your changes using one or more of the testing methods.

If adding or updating templates:

[x] post a screenshot of templates and/or gallery changes
[x] ensure your template has a corresponding gallery entry in the gallery folder
[x] If you are adding a new template, add your team and template/gallery file(s) to the CODEOWNERS file. CODEOWNERS entries should be teams, not individuals
[x] ensure all steps have meaningful names
[x] ensure all parameters and grid columns have display names set so they can be localized
[x] ensure that parameters id values are unique or they will fail PR validation (parameter ids are used for localization)
[x] ensure that steps names are unique or they will fail PR validation (step names are used for localization)
[x] grep /subscription/ and ensure that your parameters don't have any hardcoded resourceIds or they will fail PR validation
[x] remove fallbackResourceIds and fromTemplateId fields from your template workbook or they will fail PR validation

CODEOWNERS change is to correct an invalid name for our resource type. My apologies.

[Screenshot of Variant comparison workbook]

[Screenshot of Metric explorer workbook]


Intermediate checkpoints outperforming final checkpoint?

Hi -- this isn't an issue with the code, but I'm encountering some difficulty in training these models that I figured you might be able to help with.
I am applying DINOv2 to a specialized domain (which I unfortunately can't describe in any more detail). We have ~500M images for training, and we have implemented 16 validation tasks, some with small datasets (<10k) and some with large datasets (>100k).
We are running into something strange where when we evaluate the model on the validation tasks, a checkpoint taken 50% through training is outperforming the final model checkpoint at the end of training. We have tried tuning the learning rate, and it has helped a little bit, but we really expect the final checkpoint to have the best performance, and across all learning rates we have tried it does not.
Have you encountered this ever? Is there some hyperparameter(s) other than learning rate that might help fix this issue? Is our assumption correct that if we train the model well, then the 100% checkpoint should outperform all previous checkpoints?
Any suggestions or feedback would be much appreciated. Thank you!

how bad ? can you plot the loss and eval curves ?

The loss curves seem relatively normal, continuing to decrease throughout training, but here are plots of the results of 16 evaluation tasks. As you can see, very often the models at 50% do better than the models at 100%. These are ViT-S models. For some of the tasks (eg, the bottom middle two), the curves are extremely irregular, and so these might not be great tasks, but the aforementioned behavior seems to be present in many of the others.

I had the same experience, not too surprised. we have plots showing similar behaviour here: https://arxiv.org/abs/2401.04720


Tried one last time.

I have all necessary dependencies installed and running as necessary.
michael@megatron:~/OpenDevin$ docker run     -e LLM_API_KEY     -e WORKSPACE_MOUNT_PATH=$WORKSPACE_DIR     -v $WORKSPACE_DIR:/opt/workspace_base     -v /var/run/docker.sock:/var/run/docker.sock     -p 3000:3000     ghcr.io/opendevin/opendevin:latest
INFO:     Started server process [1]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://<IP_ADDRESS>:3000 (Press CTRL+C to quit)
INFO:     <IP_ADDRESS>:45932 - "GET / HTTP/1.1" 307 Temporary Redirect
INFO:     <IP_ADDRESS>:45932 - "GET /index.html HTTP/1.1" 304 Not Modified
INFO:     <IP_ADDRESS>:45932 - "GET /assets/index-C089YNx-.js HTTP/1.1" 304 Not Modified
INFO:     <IP_ADDRESS>:45946 - "GET /assets/index-C3qOtGcN.css HTTP/1.1" 304 Not Modified
INFO:     ('<IP_ADDRESS>', 45948) - "WebSocket /ws?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzaWQiOiJmZjI4ZTZlNy1iMTQ0LTRiNGYtOWU5NS1hYjk0N2VmNjNiMDEifQ.xtBvYb6dwokdpUAoUftSwgQnwLD1_gyjnKL-PK8CFSQ" [accepted]
INFO:     connection open
Starting loop_recv for sid: ff28e6e7-b144-4b4f-9e95-ab947ef63b01
INFO:     <IP_ADDRESS>:45946 - "GET /locales/en/translation.json HTTP/1.1" 304 Not Modified
INFO:     <IP_ADDRESS>:45946 - "GET /api/refresh-files HTTP/1.1" 200 OK
INFO:     <IP_ADDRESS>:45946 - "GET /api/configurations HTTP/1.1" 200 OK
INFO:     <IP_ADDRESS>:45932 - "GET /api/messages/total HTTP/1.1" 200 OK
00:18:27 - opendevin:INFO: ssh_box.py:271 - Container stopped
00:18:27 - opendevin:WARNING: ssh_box.py:283 - Using port forwarding for Mac OS. Server started by OpenDevin will not be accessible from the host machine at the moment. See https://github.com/OpenDevin/OpenDevin/issues/897 for more information.
00:18:28 - opendevin:ERROR: ssh_box.py:311 - Failed to start container
00:18:28 - opendevin:ERROR: agent.py:127 - Error creating controller: 400 Client Error for http+docker://localhost/v1.43/containers/a03a0fd2249fe296fb143e0e70eea3bf159093031a1c71004f45082e281d39c7/start: Bad Request ("failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "/usr/sbin/sshd": stat /usr/sbin/sshd: no such file or directory: unknown")
Traceback (most recent call last):
File "/app/.venv/lib/python3.12/site-packages/docker/api/client.py", line 265, in _raise_for_status
response.raise_for_status()
File "/app/.venv/lib/python3.12/site-packages/requests/models.py", line 1021, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: http+docker://localhost/v1.43/containers/a03a0fd2249fe296fb143e0e70eea3bf159093031a1c71004f45082e281d39c7/start
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/app/opendevin/server/agent/agent.py", line 118, in create_controller
self.controller = AgentController(
^^^^^^^^^^^^^^^^
File "/app/opendevin/controller/agent_controller.py", line 91, in init
self.command_manager = CommandManager(self.id, container_image)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/opendevin/controller/command_manager.py", line 26, in init
self.shell = DockerSSHBox(
^^^^^^^^^^^^^
File "/app/opendevin/sandbox/ssh_box.py", line 87, in init
self.restart_docker_container()
File "/app/opendevin/sandbox/ssh_box.py", line 312, in restart_docker_container
raise ex
File "/app/opendevin/sandbox/ssh_box.py", line 293, in restart_docker_container
self.container = self.docker_client.containers.run(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/docker/models/containers.py", line 880, in run
container.start()
File "/app/.venv/lib/python3.12/site-packages/docker/models/containers.py", line 417, in start
return self.client.api.start(self.id, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/docker/utils/decorators.py", line 19, in wrapped
return f(self, resource_id, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/docker/api/container.py", line 1135, in start
self._raise_for_status(res)
File "/app/.venv/lib/python3.12/site-packages/docker/api/client.py", line 267, in _raise_for_status
raise create_api_error_from_http_exception(e) from e
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/docker/errors.py", line 39, in create_api_error_from_http_exception
raise cls(e, response=response, explanation=explanation) from e
docker.errors.APIError: 400 Client Error for http+docker://localhost/v1.43/containers/a03a0fd2249fe296fb143e0e70eea3bf159093031a1c71004f45082e281d39c7/start: Bad Request ("failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "/usr/sbin/sshd": stat /usr/sbin/sshd: no such file or directory: unknown")
What could the issue be? I checked the docker container with f6321c673c4e:/app# ls -l /usr/sbin/sshd
-rwxr-xr-x 1 root root 1261336 Dec 19 14:51 /usr/sbin/sshd
root@f6321c673c4e:/app#
the /usr/sbin/sshd exists in the right area and has the correct permissions.

Docker container is running. (base) michael@megatron:~/OpenDevin$ docker ps
CONTAINER ID   IMAGE                                COMMAND                  CREATED         STATUS                PORTS                                           NAMES
eef5651de99a   ghcr.io/opendevin/opendevin:latest   "uvicorn opendevin.s…"   5 minutes ago   Up 5 minutes          <IP_ADDRESS>:3000->3000/tcp, :::3000->3000/tcp       sad_jackson

Getting the same issue on mac, not able to run it at all. Any quick resolution will help

@cardoza1991 thanks for the report. Can you tell us more about your operating system?
In the future, filling out the bug template is very helpful

@cardoza1991 thanks for the report. Can you tell us more about your operating system?
In the future, filling out the bug template is very helpful

Yeah I am using Ubuntu 22.04 LTS desktop not the server

Is it possible there are some security controls in place that might prevent access to sshd?

For more clarity, the problem is on this line: https://github.com/OpenDevin/OpenDevin/blob/7710112ae284ae9b988f2ac4729194fbb21ecefc/opendevin/sandbox/ssh_box.py#L293
We are trying to start a container running sshd. This usually works, but for some reason not in your env...

For now, adding -e SANDBOX_TYPE=exec should fix it

Is it possible there are some security controls in place that might prevent access to sshd?

not that I am aware of.

For now, adding -e SANDBOX_TYPE=exec should fix it

good news is it ran successfully.
docker run
-e LLM_API_KEY
-e WORKSPACE_MOUNT_PATH=/home/michael/the-reserves
-v $WORKSPACE_DIR:/opt/workspace_base
-v /var/run/docker.sock:/var/run/docker.sock
-p 3001:3000
ghcr.io/opendevin/opendevin:latest
I added the -e SANDBOX_TYPE=exec as shown above.
it runs on my system successfully. Ubuntu 22.04 lts, Ubuntu Desktop

Sounds like this is fixed!


INTERFACE REVIEW: Endpoint interface

Built atop #13277 as it requires the Pollable concept from that change
Major changes:

Pollable allows us to remove three virtual methods to manage this objects participation in various polling_interface entities

Looks good to me.

Reviewed 7 of 7 files at r1.
Review status: all files reviewed at latest revision, 1 unresolved discussion, some commit checks broke.

src/core/lib/iomgr/endpoint_interface.h, line 39 at r1 (raw file):

  // \a slices even when the callback is invoked with error != GRPC_ERROR_NONE.
  virtual void Read(grpc_exec_ctx* exec_ctx, grpc_slice_buffer* slices,
                    grpc_closure* cb) = 0;

Do we need to get the ClosureRef change in first, or will it be easy to update this once that happens?

Comments from Reviewable


Nervos sudt destroy status not updating

I have cross-chained back to BSC and Ethereum from Nervos many times. Although the cross-chain was successful in the end, the susd destruction status display in the first step of cross-chain execution has always been stuck. Although the block has been confirmed successfully, the status display has always been It is pending, which causes these cross-chain records to be stuck there. Please solve this problem. The following is the cross-chain I executed on 2022.1.28. Until now, the records are still there.

Did you fail to cross-chain or it's just a display error?
If you click the link, could you see the transaction on Nervos Explorer? What's the status of the transaction, success, pending or revert?

1.cross-chain success, only display error, this problem appeared in the last month.
2.the transaction on Nervos Explorer is exists and it success
so i think the logic of checking for destruction needs to be checked.

1.cross-chain success, only display error, this problem appeared in the last month. 2.the transaction on Nervos Explorer is exists and it success
so i think the logic of checking for destruction needs to be checked.

Sorry for the inconvenience. We are grateful if you could disclose the pending burn tx hash to help us to address the problem.

Hi mkxbl I just had a very similar issue. Below is my TX hash. Your help is greatly appreciated
0x732c6a384a506c8576a902a51e34f3a2806616949739be07be1e15b5aea28e0d


(BOLT-198) Add 'bolt task show', 'bolt plan show' to error messages

Adds a note on viewing available tasks and plans when one cannot be
found.

CLA signed by all contributors.


Android: Is call method @Subscribe after 20 sec?

Suppose I have Activity with button.
After click button I open fragment.
Before open fragment I sent eventBus message.
Something like this (on Activity)
    public class DetailsActivity extends AppCompatActivity {

    @OnClick(R.id.imageViewr)
    public void onClick() {
      EventBus.getDefault().post(new CategoryFilterEvent(categoryId));
      // open fragment
      FragmentManager fragmentManager = getSupportFragmentManager();
      fragmentManager.beginTransaction().replace(R.id.mainActivity, currentFragment).commit();
    }

    }

Here snippet on fragment:
public class MapFragmentTab extends Fragment {

    @Subscribe
    public void onChangeFilter(CategoryFilterEvent categoryFilterEvent) {

    }
}

OK. It's work.
When fragment is create the method onChangeFilter() is call. Nice.
But suppose that fragment will be create not immediately after click button on Activity, but after 20 seconds.
In this case is method onChangeFilter() is call in fragment (after 20 sec)?

The event will only be posted to subscribers that are currently registered. So in your case the answer is: no.
You might want to try using sticky events to solve this problem.
-ut

No response. Closing. -ut


[Bot Services] Web chat channel creation

Hi,
After running across the documentation of the Azure CLI and more specifically the Bot Services part I noticed that the create and delete commands were not available for the Web Chat channels.
I am looking forward for this feature as this would allow me to automate this part of the process on my side.
Is there any development planned on that? A potential release date approximation?
Thank you for taking into consideration my request.

Bot Services

@huski1017 thanks for creating this issue, we will investigate this request.

@huski1017 this looks to be doable with the current Swagger-generated Python SDK. I'm going to tentatively mark it for S170 and see if I can get it via https://github.com/Azure/azure-cli/pull/13462.

@stevengum That would be great, thank you for taking care of my request!

@stevengum Any update about my feature request?

Could it be possible to get some updates about my request?
It's been 2 months that someone is supposed to take care of it on the S170 we're now about to reach the end of the S173 and still nothing changed.

Hi,
Is there any update about this request?
I am still looking forward this feature that would allow me to automate a painful process.
Regards,
Huski1017

@huski1017 Apologies for the late reply. For the webchat only show command is currently available in CLI. Refer this. Please let us know if you have any further questions on this. Awaiting your reply.
Refer similar issue here https://github.com/microsoft/BotFramework-WebChat/issues/2913


Release v2.1.0

🎉 😆 🎉

:tada: This PR is included in version 2.1.1 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Gazeable Canvas Button

Hi guys
I've created a quite simple component that you can add to an existing Canvas Button (from Unity UI) and that makes it works nicely with gaze (similarly to how it works with mouse).
Was wondering if I could keep it in my local repo or make it available here.
Anyone interested ?
Happy to upload a PR so you can give it a try.

Yes please!  This is going to be a very common need in HL apps.

On Jul 13, 2016, at 5:11 AM, Stephane Bertout<EMAIL_ADDRESS>wrote:
Hi guys
I've created a quite simple component that you can add to an existing Canvas Button (from Unity UI) and that makes it works nicely with gaze (similarly to how it works with mouse).
Was wondering if I could keep it in my local repo or make it available here.
Anyone interested ?
Happy to upload a PR so you can give it a try.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

I believe Unity is providing support for this, via addition of the HoloLens Input Module. It's currently available in the latest HTP previews. It's the HoloLens equivalent of the Standalone Input Module, which I believe is what handles mouse/touch for Canvas elements. This should make Canvas work without any additional components.
I'll dig into it a bit further.

That would be even better indeed !

Closing as supported by the platform via HoloLens Input Module. If there are issues with that implementation we can reopen this.

So this was added to the lates beta from the unity side is that what you mean ?


Split React.podspec into separate podspecs for each Xcode project

Summary
This PR implements the first part of RFC0004: CocoaPods Support Improvements, splitting the React.podspec into separate podspecs to more closely match the structure of Xcode projects.
The new structure aims to have one to one mapping between Xcode projects and podspecs. The only places where we differ from this mapping are:

React/React-DevSupport.podspec: DevSupport is a part of React.xcodeproj, which corresponds to the React-Core pod. However, we can't include it in the React-Core pod because DevSupport depends on React-RCTWebSocket, which depends on React-Core. Pods may not have circular dependencies.
The new pods under ReactCommon/ don't have a corresponding xcodeproj because there are no xcodproj files in ReactCommon/. Those C++ modules are included in React.xcodeproj.

Next steps (not in scope of this PR):

Start submitting the Podspecs to CocoaPods on a deploy (or turn the React Native repo into a spec repo): this is important in order to make the experience nicer for library consumers, so that it's not necessary to specify the local path of each Podspec in Podfile, you can just add pod 'React', <version>.
Add Podfile to the default project template (I have a PR ready for this, but because of bugs related to subspecs, it's blocked on this PR)

Changelog
[iOS] [Changed] - Split React.podspec into separate podspecs for each Xcode project
Test Plan
Ran cd RNTester; pod install; open RNTesterPods.xcworkspace and made sure the app builds and runs without errors.

iOS platform is locked to 10.0, doesn't RN support minimal version 9.0 anymore?

Yeah RN supports iOS 9, can we update that before landing?

Target set to 9.0.


circle markers not being added to drawn layers

Seems to fix issue #3888
Basically a newbie so let me know if I'm doing this wrong ;)

Merged as a part of #4072, thanks!


Question: how do I open a window

Once I closed a window, is it possible to reopen?

Get the container and set it to open again.
mu_Container *cnt = mu_get_container(ctx, "Window Title");
cnt->open = 1;

Thank you!


frontend/docu: Inconsistant naming downstream-tripper-config

Is your proposal related to a problem?
Naming about downstream-tripper-config is inconsistent.
thanos: error: unknown long flag '--query-range.downstream-tripper-config'

Describe the solution you'd like
Replace query-range with query-frontend.
https://github.com/thanos-io/thanos/blob/main/docs/components/query-frontend.md?plain=1#L108
https://github.com/thanos-io/thanos/blob/main/CHANGELOG.md?plain=1#L50

Nice find. Help wanted.


fix extract command

Had cut-and-paste error
Signed-off-by: Larry Steck<EMAIL_ADDRESS>

Just a minor change I'll override approval


Expose Reflection classes to be able to create custom theme

Hi,
I've been trying to find a theme to produce a Markdown version of the docs. Found references to https://github.com/kimamula/typedoc-markdown-theme/ . And found that it's not working anymore as it has been mentioned in another issue. (If memory serves me well it stopped working after 0.6)
The issue I'm having is that I want it to produce .md files (instead of .html) and for this I need it to create a theme that goes beyond simply replacing the template files of the Default Theme. I need to actually include a theme.js file with a subclass of DefaultTheme.
This leads to my problem. The Reflection classes are not exported by the typedoc project. And a lot of the logic of the DefaultTheme (which I'm using as a base) depends on "instanceof" logic. Without these classes being exposed I'm finding that I can only use imports of the type:
import { DefaultTheme, TemplateMapping } from 'typedoc/dist/lib/output/themes/DefaultTheme';
but this doesn't really work, as I then experience issues on runtime where the DefaultTheme class loaded by typedoc is different than the DefaultTheme class loaded by the template and instanceof checks fail.
It's not a big change, and I'll submit a pull request with the classes I think would make sense to expose for the development of custom themes, but thought it would be good to ask whether there was any rationale behind why these classes are not exposed or an alternative way to tackle this issue.

@pragone Thanks for investigating that. Could you explain how this relates to #382? With #504 you should be able to use a theme.js file.
I think I follow you basic premise but I'm not understanding why the instanceof on the deep imports aren't working. Shouldn't it be pointing to the same instances?

Hi @aciccarello.
Indeed I had looked at that PR. What it does is to include the theme.js file and load it properly. Which is OK. That's not the issue I was having. This is the issue.
Now, my theme.js file must export a class that extends DefaultTheme. But this class is not exposed by the typedoc package. Also, in its internal logic it needs other classes like DeclarationReflection for parts like:
            entryPoint.children.forEach((child: Reflection) => {
                if (child instanceof DeclarationReflection) {
                    TypedocMarkdownTheme.buildUrls(child, urls);
                }
            });

That DeclarationReflection is not exposed by the typedoc package either. This means that the only way I can access it is by doing a require like:
import { DeclarationReflection } from 'typedoc/dist/lib/models/reflections/declaration';

Now... doing imports this way caused issues with the instanceof checks. It seems that when typedoc itself loads the reflection information from the JSON file, and creates objects of type DeclarationReflection the class it uses is loaded from the typedoc codebase directly. When the theme loads the class it is loading it from it's typedoc dependency inside its own node_modules folder. It turns out that even if these two classes are called the same, they come from different files and therefore fail in instanceof checks.
If in the theme I could import:
import { DeclarationReflection } from 'typedoc'

Then it uses the cached module and the class is the same and it all works. It's quite an obscure issue TBH.
Does that make it clearer? It's a bit convoluted I agree

Going to track this in #784, it's not just reflections that ought to be exported and aren't.


Automated cherry pick of #2181: cpu core bugfix

Cherry pick of #2181 on release/2.11.
#2181: cpu core bugfix

Codecov Report

:exclamation: No coverage uploaded for pull request base (release/2.11@3031a71). Click here to learn what that means.
The diff coverage is 0%.

@@              Coverage Diff               @@
##             release/2.11   #2182   +/-   ##
==============================================
  Coverage                ?   7.09%
==============================================
  Files                   ?     451
  Lines                   ?   70882
  Branches                ?       0
==============================================
  Hits                    ?    5026
  Misses                  ?   65443
  Partials                ?     413

Flag
Coverage Δ

#aFlag
7.09% <0%> (?)

Impacted Files
Coverage Δ

pkg/multicloud/aws/instance.go
0% <0%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3031a71...9f97d3a. Read the comment docs.

/lgtm

/approve


Documentation is out of date

The documentation for "How-to for IOT" seems to be out of date.
Executing the following commands as indicated in the instructions throws the following errors:
flogo install activity github.com/TIBCOSoftware/flogo-contrib/activity/log
Error: Too many arguments given
Usage:
flogo  install [-v version][-p] contribution
flogo add flow reqcounter.json
FATAL: unknown command "add"
Usage:
flogo  [arguments]

Documentation has been updated via PR #79


Update common_corporate_passwords.lst

I have seen this password to be used actively in one of the corporate environment during my pentest engagements

Thanks @njmulsqb


Agreement on how to write connection parameters

There is a "small" issue that makes the UpJet AWS provider unusable in conjunction with the SQL provider.
This is because there is a discrepancy between the way the UpJet provider writes credentials and how the SQL provider reads them.
Could the maintainers of both providers discuss and find a solution to unblock all users who would like to create databases and easily execute queries?
List of issues to help you analyze the problem:

https://github.com/crossplane-contrib/provider-upjet-aws/issues/778
https://github.com/crossplane-contrib/provider-upjet-aws/pull/798
https://github.com/crossplane-contrib/provider-upjet-aws/issues/1189
https://github.com/crossplane-contrib/provider-upjet-aws/issues/1578
https://github.com/crossplane-contrib/provider-sql/pull/154
https://github.com/crossplane-contrib/provider-sql/issues/191
https://github.com/crossplane-contrib/provider-sql/issues/203

The follow-up discussion can also be found here: https://crossplane.slack.com/archives/C01718T2476/p1732801643245679.

For ref, my response on the provider-sql issue: https://github.com/crossplane-contrib/provider-sql/issues/203#issuecomment-2506836814


fix #68 labels should be dict of string to string

:green_heart: Build Succeeded

Expand to view the summary

Build stats

Build Cause: [Pull request #84 opened]

Start Time: 2020-06-02T19:26:47.456+0000

Duration: 9 min 26 sec

Test stats :test_tube:

Test
Results

Failed
0

Passed
25

Skipped
4

Total
29

@gregkalapos @russcam @sgryphon this one has to go in 1.6.0 since we a have minor non breaking clause in https://github.com/elastic/ecs-dotnet#versioning

Wondering if labels should be Dictionary<string, string> or whether it should be a Dictionary constrained to scalar values. For example, the APM agent labels can be any of string, number or boolean, and are modelled internally with a Label type:
https://github.com/elastic/apm-agent-dotnet/blob/2c433d31c45444e026081721e9c4190dadc383b8/src/Elastic.Apm/Api/Label.cs
As far as I understand, APM agent labels map to the ECS labels field. Should Elastic Common Schema .NET have a similar type to constrain label values?

@russcam this fix needs to be in 1.6 (if it is not there already... although I guess you just filter on tag = 1.6)


Fix the list recommendation functionality

Codecov Report

Merging #135 (67eb38a) into main (fb2eae0) will decrease coverage by 1.13%.
The diff coverage is 63.63%.

@@            Coverage Diff             @@
##             main     #135      +/-   ##
==========================================
- Coverage   95.27%   94.13%   -1.14%
==========================================
  Files           5        5
  Lines         296      307      +11
==========================================
+ Hits          282      289       +7
- Misses         14       18       +4

Impacted Files
Coverage Δ

service/routes.py
94.89% <63.63%> (-3.96%)
:arrow_down:

:mega: Codecov can now indicate which changes are the most critical in Pull Requests. Learn more


When/how often does the Progress report the latest info? When is the callback fired?

FluentFTP Version: v38.0.0
Framework: .NET 6
👋🏻 G'day! I'm trying to learn how often/when the Progress callback is actually "fired". Is there any way to "control" the timing of the callback? Like, could I say, "Report" every 5 seconds? or every 1 second?
I'm thinking about when I have some files that are large and could taken a while to upload or download.
Is this possible?
Thank kindly mate!

Every successful socket read or write operation, typically a full buffer, a call to progress is made.
FXP transfer seem to have a config parameter where you can select an interval for reports, but normal FTP transfers do no.
I typically report the change every time the % value changes and have noticed no detrimental effect - other code paths are overwhelmingly more influential in this.

For me, this is a background worker. So the progress changes are text-logging ... not a UI "progress bar" update :(
So to keep the logs from getting "text spam", I was hoping to restrict the amount that is reported - hence why I thought I could do it every x-seconds.
You said:

FXP transfer seem to have a config parameter where you can select an interval for reports, but normal FTP transfers do no.

er... sincere apologies, but what does this mean? Is this a type of FTP connection? or a specific class that is instantiated?

https://github.com/robinrodricks/FluentFTP/wiki/FXP-Server-To-Server
I noticed that whoever wrote that, included a parameter to set the time interval where progress is called.
Seems not to have been done for normal FTP.

But I check a static old percent and compare, in my progress routine, for example. You could check for elapsed time, it makes not real difference where this check is made, does it, only you need 3 lines of your own code.

Hmm.. so you mean to have the Action somehow remember the 'last time' and if it's over x-seconds, then report and reset the time?

Exactly. And that is what that sort of code inside the library would also do. So its not a big performance difference whether you do it or the library does it - only the location of the code IMO. Also, you have more options: By elasped time, by percent change, non linear, etc.

Start a stopwatch at begin. Check the sw.elapsed each progress call. remember each elapsed and check the difference, and so on and on...

@PureKrome  as FanDjango said, you can easily implement event throttling on your end. We give you the best possible accuracy of progress by firing the event as often as possible, and you can reduce the frequency of events by doing this:
lastTime = DateTime.Now;

progressEventHandler(){
  var nowTime = DateTime.Now;
  var duration = nowTime - lastTime;
   if (duration > 1000){                        // 1000 MS is 1 second, change this to make it whatever you need
       lastTime = nowTime;

       // do whatever

   }
}

@robinrodricks thanks for the code! I'll give this a go, next week at work


Support valueLink and onChange in combination.

In the meantime valueLink got deprecated by React, and im planning to remove support for it from this lib entirely with the next major version. Therefore merging this for this short time is undesired. Gonna close it, thanks for the PR though.


Use individual lodash packages

Only two packages from lodash are used. These have been added appropriately.

Is assign from lodash necessary?
Object.assign has been in Node since version 4 and this project's package.json specifies supporting Node >= 4.

This PR will most likely be in conflict with changes in #58 (suggestion to remove lodash completely).

#58 and #166 have the same result, neither have been merged and there are conflicts now.
The full Lodash package is still in master ¯\_(ツ)_/¯


Toggle todoist-shortcuts message pop-up

Is there a way to disable the "Message from todoist-shortcuts" popup upon copying tasks? Is this from the extension or is it part of Todoist itself? I find it a bit intrusive. This is a very niche issue so no problem at all if you don't think it's worth addressing. Thanks!

This can be disabled in the code (see the instructions in the readme for local modifications).  Currently todoist-shortcuts does not have any options (customizing keyboard shortcuts is tracked by #99)
The reason for the pop-up is that nearly all of the shortcuts have a visual indication that something happened.  Modifying the clipboard does not, and users might be surprised to find their clipboard modified after using one of these non-standard (not ctrl-c / ctrl-x) bindings for modifying the clipboard.

Closing as wontfix


Fix: Expose test_from_until filling decorator in ethereum-test-tools package

It looks like test_from_until was forgotten in the ethereum-test-tools init file.

thanks!


latency improvements

Trying to make lineaxis more inferable, and maybe finally get the unnecessary updates under control.

Compile Times benchmark
Note, that these numbers may fluctuate on the CI servers, so take them with a grain of salt.
using time
master:  10.53 < 11.44 > 11.81, 0.47+-
pr:      8.99 < 9.40 > 10.35, 0.49+-
speedup: 1.10 < 1.20 > 1.31, 0.06+-
median:  -17.77% => improvement

This PR improves the using time.
ttfp time
master   26.64 < 29.24 > 29.96, 1.21+-
pr       25.71 < 28.48 > 29.09, 1.48+-
speedup: 0.98 < 1.04 > 1.15, 0.06+-
median:  -2.57% => invariant

This PR does not change the ttfp time.

Somehow the merges became really messy, and this PR should also be rather based on master.
Continued in: https://github.com/JuliaPlots/Makie.jl/pull/1968


Wrong overview cards initialized with RBAC restrictions

Describe the bug
When we apply RBAC restrictions for viewing AWS data, the overview perspective shows that "Amazon Web Services" is selected, but the cards show all cloud filtered by OpenShift cards.
This can be also reproduced with Azure.
To Reproduce
Steps to reproduce the behavior:

Set rbac rights for AWS viewing
Go to overview
See error

Expected behavior
Correct overview cards initialized
Screenshots

Desktop (please complete the following information):

OS: Fedora
Browser Chrome

Additional context
Seen on CI beta

verified


Adding palm pose support

Adds support for the new Palm pose extension (https://www.khronos.org/registry/OpenXR/specs/1.0/html/xrspec.html#XR_EXT_palm_pose)
Note that this extension is currently only available in the latest SteamVR beta (and maybe in the Oculus runtime, haven't tested yet).
If the extension is not available, we skip registering the bindings. It will be up to the end user to check if the extension is available if and react accordingly.

Ok, this is working correctly now. At least with SteamVRs latest beta.

Marked this ready for review seeing the functionality is part of OpenXR 1.0.24 and we're dealing with it being an optional feature.


Run the OVE Server on a VM (just clone this repo for now)

The application launcher page of OVE should be visible and it should be possible to launch new OVE applications.

Some success with this: The OVE homepage can be viewed on the test server (internal/VPN) at https://liionsden.rcs.ic.ac.uk/.
Unfortunately, none of the subpages are visible e.g. https://liionsden.rcs.ic.ac.uk/ui/launcher to launch applications and https://liionsden.rcs.ic.ac.uk/ui/status to see the status of running OVE components.
I strongly suspect this is because we are using Caddy but OVE uses nginx already. We may be able to modify the Caddyfile to fix this, but it might be better overall to look into doing away with Caddy and just using nginx, modifying .config/default.conf accordingly.

I should say that I am also having trouble viewing those subpages locally. I tried to get to the ui launcher and it just didn't work

When running locally, I've found that clearing the browser cache or using an incognito page has worked for viewing the ui launcher. Also running docker volume prune and docker network prune if changing any config.

I needed to destroy all my images and re-build them and it got there

I think we can close this issue now. Basically OVE itself is running fine on the VM: you should be able to navigate to http://liionsden.rcs.ic.ac.uk:8080 and http://liionsden.rcs.ic.ac.uk:8080/ui/launcher. See example screenshots below.
Only nginx is required, not caddy.
After getting nginx itself to output more detailed logs, it turned out that the problems were arising from incorrectly configuring nginx to handle https requests. This will take some fixing, so I'll open a separate issue now with much more detail because ultimately we want to handle https requests. For now though, the server is not configured for https://... with ssl, so only http://... will work.
I will submit a PR before we close the issue, but it'll just be a bit of extra info in the README on the VM configuration.
OVE application launcher page:

Preview of OVE space and section with HTML app:


Implemented constraint equality.

Aiming to address #3616

Although this is technically in progress, it isn't necessarily going to go into Iris 3.2. So to avoid it cluttering the "in progress" column, I will move it to to do. We can then pick it up if we time.

If we do get to it then we'll need a rebase onto main before it can be reviewed I think

@pp-mo are you able to finish this one off before getting back into #5016? Sounds like this is pretty close.

N.B. put in draft temporarily
It is not good to go, since the auto-merged result is still using Coord._as_defn()
And we now want to use common metadata instead.

Code now simplified ...
On closer inspection, I conclude that the 'coord_name' of a _CoordConstraint is only ever intended to be a string.
The only way it can be otherwise is if you construct Constraint(coord_values={something:value}).  But whereas my earlier code suggested that 'something' could be a CoordDefn (or, now, a metadata), it actually can't since that is not a hashable type, so cannot act as a dict key.
It is still just about "possible" that you could pass a Coord instance as the identifier in this case, but we have no tests for it.
I simply don't believe it is useful.

@trexfeathers I think this is good to go.   Stephen's comment with change request is outstanding, but I think you can dismiss that if you think it is now OK.

Thanks @pp-mo! I'll let @stephenworsley take it over the line.

Thanks @stephenworsley
I hope this latest addresses all the points raised.
Sorry for the delay !

Thanks @stephenworsley
I think I've now fixed them all to be the same.

I forgot to mention before, but this still needs a whatsnew entry.

Whoops sorry forgot.
Doing it now ...

Ok, see if that suits @stephenworsley  ?

Hang on now I need to rebase/resolve...

Ok, rebased with added whatsnew

please see if that suits, @stephenworsley

Thanks @stephenworsley for going the distance !

Thanks guys! 😀


GeoJSON sort and join

Since geojson2nd exists, it should be able to:

Extract a property to the start of the line so you can sort on that property, or just offer sorting internally
Join sorted CSV to sorted GeoJSON

Alternately, maybe the sorting could be factored out of the Tippecanoe input stage.

"Sorted" is complicated, because on both the JSON and CSV side it would be best to have the same ordering that the Unix sort tool can produce, but neither JSON nor CSV strings naturally sort this way.
I think on the JSON side this means writing everything in \u form, because if not everything is escaped, " will sort with \ instead of between ! and #. But is there any form at all that will put surrogate pairs in proper UTF-8 sequence?
On the CSV side, it's only going to work if the CSV is consistent about quoting. If everything is quoted, "" will sort like " and that's fine. If not everything is quoted, the quoted strings will sort first. The sort command knows nothing about CSV conventions.

I guess the thing to do with surrogate pairs is to represent them, and any other Unicode that follows the surrogate ranges, as UTF-8, since then they will sort after \\, in order.

Working on this in https://github.com/mapbox/tippecanoe/pull/473

Merged


permission denied

Please describe the issue in as much detail as possible, including any errors and traces
when using sudo, give out permission error
➜   sudo bash
python3: can't open file '/lib/security/howdy/compare.py': [Errno 13] Permission denied
Unknown error: 2

Linux distribution (if applicable): ubuntu 18.04
Howdy version: 2.3.0 from ppa

I workaround this by :

chmod 644 /lib/security/howdy/{compare.py,config.ini}
change /lib/security/howdy/pam.py

status = subprocess.call(["/usr/bin/python3", os.path.dirname(os.path.abspath(__file__)) + "/compare.py", pamh.get_user()])

PAM is running as root by default, but i've seen a few of similar issues where PAM is running Howdy without root rights. Is Howdy fully functional now you've changed the permissions of only those 2 files?

yeah，looks like its fully functional.
I use it when login and type sudo in gnome terminal only, however.

➜  ~ sudo howdy test

Opening a window with a test feed

Press ctrl+C in this terminal to quit
Click on the image to enable or disable slow mode

Closing window

nothing strange happened

but I see nothing

I use pyenv, so the python path is different, maybe this issue should be split into 2 issues focus on different things

Following up on issues i forgot to reply to, are you still having this problem or did you fix it/uninstall Howdy?

I did my workaround and it works well for a long long long long time.
I think the workaround may be necessary.

A fix is now planned for the next release, main threat is #83


Support for reverse proxy?

There is a need for me to have this web ui placed behind a reverse proxy with a prefix such as "https://example.com/gpt-webui".
Is there any plan to add reverse proxy support?

We will support custom openai request url later, please follow #49


Can we help uncover or explain span reporting lag?

from @dan-tr

@dan-tr Sep 28 23:09
Is timestamp_millis created based on the timestamp field or is created based on the current time when the record is inserted into ES? I'm looking for away to discern if there is any lag time between the message being put on the Kafka queue (client-side) and the record being stored in ES (server-side).

We should add the description to the README here, but no. this timestamp isn't used for lag, it is based on the best timestamp.
https://github.com/openzipkin/zipkin/tree/master/zipkin-storage/elasticsearch
timestamp + duration would be the time a span was recorded and eligible to be sent. In https://github.com/openzipkin/zipkin-reporter-java, and other libraries, there would be up to a second before the span goes on the wire.
We could look at our local endpoint vs one of the endpoints in the span and try to determine roughly the lag, based on limitations of clock skew guessing. If we did, we'd likely record that as a stat vs put it into the span... unless we created a "fake span" which we then recorded against zipkin-server.
Another option would be to accept traces originated by reporters. Ex start a trace when the span was sent. In kafka, we'd have to hijack the message key, which isn't a problem in zipkin topics. https://github.com/openzipkin/brave/pull/212
Thoughts?

Actually, I think all I am really looking for is when did the record get inserted into Elasticsearch. If Zipkin does only inserts, then I am looking for the record creation timestamp. If it does inserts and updates, then I am looking for the last update timestamp.
So I don't think any calculation needs to be done. Zipkin just needs to store the current time on each insert/update.
In production this timestamp can be valuable when something goes wrong and you want to know the last time Zipkin wrote anything to ES.
Does that make sense?

we could add this, but it will be the third timestamp field, which makes me
reluctant.
Would metrics be reasonable way to track the last write attempt?
https://github.com/openzipkin/zipkin/tree/master/zipkin-server#collector

ps @openzipkin/elasticsearch this topic is a pretty opsy one. anyone doing something similar and have opinions?
ex. timestamp(_millis) is a semantic field for the span. If someone wanted to know the created_at, well elasticsearch doesn't do that. The question here, is should we? if so, how can we label such a field so that it isn't confused with the timestamp reported by the apps?

I agree this is pretty opsy. I think logstash does this by adding an @ sign to the front of the field. Maybe Zipkin could use the same convention.

In similar situations (ingesting from a queue-like pipeline) we simply emit a metric as timer("ingestion.latency", time.Now() - span.startTime). Wouldn't this be sufficient for this use case, rather than storing it in ES?

+1 for metrics over adding a meta field to the data model.

Where is the metric emitted to? And how does it correlate that to the record written to Elasticsearch? Seems like we would need to emit the ingest latency along with the span id.

I think the key thing here is that Dan is looking for a per-message
accounting of latency vs an aggregate one (ex via metrics). This is the
first time such a request has come up (ex hasn't happened on any other
storage system). That may be because spans are lossy and so not guaranteed.
Maybe people aren't looking at spans one-at-a-time knowing that. I can
certainly understand that when troubleshooting or setting up the system for
the first time, something like this could be handy!
To answer the question, the server's /metrics endpoint is json and there's
also a prometheus metrics endpoint exposed. Regardless of monitoring system
used, these metrics would need to be collected to be useful in any way.

I think that we should add (and define) latency metrics as an initial step, since this would be useful regardless of the storage backend. Will keep this issue open about the forensic investigation on a span-by-span basis.
ps opened this to help showcase metrics in use https://github.com/openzipkin/docker-zipkin/issues/121

My the insert timestamp could be a optional field turned on either through an environment variable or maybe the Elasticsearch template (not sure about that).
Because we are using async messaging (via Kafka) there is always the possibility that a backlog of message to be processed. Knowing the insert date would be very help in deducing how far behind the server is in processing the backlog.

Since this is not supported in other storage layers, and others don't seem
interested. I'm hesitant about adding an if statement, environment variable
and another hack.
I also don't like how close the variable name is to the other two
timestamps. It really feels like tech debt.
Since this is for manual troubleshooting, what about encoding the stored
timestamp into the id of the document? This is done with cassandra iirc and
wouldn't clutter the json with another similar field. We would assign an id
that encodes the timestamp instead of having elasticsearch generate one.
Would that work for you?

here's a concrete counter-offer :)
since we don't change spans after writing them to ES, we can use a timestamp convention for the document ids.
Ex. something like $epochmillis-$traceId.$spanId. The trace and span id would help ensure against collision. It is unlikely a span will be reported twice in the same millisecond.
So, if I wrote the same span successively (for client and server), it would look like the following rows:
"1475335327601-0000000000000001.0000000000000003"
"1475335439449-0000000000000001.0000000000000003"
This would be a longer id than the normal ones in ES, and not good for any automated analysis, could be helpful to answer your question about when the span was stored without defining another field. Is this worth doing?

Hum... To me that feels like we would be overloading a data field for two different purposes (for timestamping and identification).
I think for now I would prefer to just let this issue "percolate" for a while and see anyone else asks for it.
No point making a change like this just for me :smiley:

Hum... To me that feels like we would be overloading a data field for two
different purposes (for timestamping and identification).

indeed that's what it would be :)

I think for now I would prefer to just let this issue "percolate" for a
while and see anyone else asks for it.
No point making a change like this just for me 😃

yeah usually issues like this end up with multiple requestors, thanks for
deferring until then

+1 for this feature. I was looking through my spans today and was wondering what time they were ingested. I used a crude '/zipkin-/_stats' end point to see when the index was last written to but having a timestamp field on the ingested span would make this process a lot easier.
I think the best way approach here is to add an @timestamp field to every span that is ingested. The value of the @timestamp field would be the time the span is written to ES. This is a standard practice for other logs ingested by logstash, so it is not odd thing to do in ES land. Since this change is localized to the ES module, it shouldn't impact other storage engines or the interface.

Thanks, suman. Can someone cite the practice with a link? busy-work I know,
but I'm fine with doing this, just want to borrow this rationale in code
comments.

@adriancole I looked at the code earlier and had the same feeling. I was looking for a better idea but can't think of one. I like your suggestion. However, I would prefer the name of the field to be collector_timestamp_millis

I like the field name collector_timestamp_millis as well.

here's the fix https://github.com/openzipkin/zipkin/pull/1328

In testing the new ES 5 support, I found that our hypothesis, which is that the new ingest node functionality can address this.. well it is supported.
The following stuff works like a champ and requires no zipkin change except to punch a hole to specify a pipeline id
PUT _ingest/pipeline/my-pipeline-id
{
  "description" : "add collector_timestamp_millis",
  "processors" : [
    {
      "set" : {
        "field": "collector_timestamp_millis",
        "value": "{{_ingest.timestamp}}"
      }
    }
  ]
}


basic keychain login working

Login via keychain implemented.  Very basic, still needs refinement.

Looking great Robert.


The hotreloading does not work inside xp

When you are in devmode and update a page the page is not updated.
This works if you directly access the next frontend server (default port: 3000)
So probably an issue in the next proxy?

Firefox can’t establish a connection to the server at ws://localhost:8080/admin/site/preview/project/draft/sitename/_next/webpack-hmr. [main.js:36:17](http://localhost:8080/admin/site/preview/company-web/draft/sitename/_next/static/chunks/main.js?ts=1657278989876%20line%2071%20%3E%20eval)
GETws://localhost:8080/admin/site/preview/project/draft/site-name/_next/webpack-hmr

When used directly at localhost:3000 nextjs client-side code creates a websocket connection at /_next/webpack-hmr.
When used in XP /_next/webpack-hmr is transformed to /admin/site/inline/next/draft/hmdb/_next/webpack-hmr.
So the websocket connection should be handled by lib-nextjs-proxy now, and tunneled to localhost:3000/_next/webpack-hmr.
The main problem here is that lib-http-client that we use in lib-nextjs-proxy is not able to handle ws requests.
So even though there is a websocket XP lib that can be used to allow ws connection from browser to lib-nextjs-proxy, we won't be able to forward it to the next server.
So we either can try to leave this url out of url rewriting and let it connect directly to nextjs server bypassing the lib-nextjs-proxy.
Or we can disable hot reload when viewed in XP completely, but keep it on when viewed directly on localhost:3000.

I don't mind tunneling it dirrectly to localhost:3000 since this would only be done on development servers.
But it is a bit annoying for developers to have to setup different connections for ws and another one for other traffic.
I see two options here:

Make a feature request to handle websockets in http-lib with our use case
Bypass to localhost:3000 and consider this as an okay way to do it.


[meta] Establish common data model for analytics

Migrated from GitLab: https://gitlab.com/meltano/meltano/-/issues/34
Originally created by @joshlambert on 2017-11-09 23:52:16

Today the vast majority of the fields in SFDC, Zuora, Marketo, another SaaS services are custom. While there are some default fields, these are the exception rather than the rule.
This means that every company has a different data schema, but is calculating largely similar types of metrics. (For example many SaaS companies utilize common metrics for establishing business performance, etc.)
This presents both a problem and an opportunity:

Setting up the integration between these services, and then the analytics to make use of the data is time consuming and expensive. It often involves consultants or dedicated employees. This is expensive and time consuming.
We have an opportunity to try to establish a common "best practices" data model, where more of these types of analytics could "just work" if you followed the conventions. This would dramatically ease downstream analytics and more tools/config/samples could be shared and applied.

To that end, we should do a few things:

Iterate ourselves towards the common "best practice" data model and schema.
Implement a "mapping stage", to map a customers custom fields to the fields in the common data model. This could be manual at first, and more automated/intelligent later.
Evangelize the common data model, it's benefits, and the interim bridge step of the mapping service.

View 15 previous comments from the original issue on GitLab


Federico: Watch studyflight.com

Federico requests the watch of the watch_keyword studyflight.com. See the Metasmoke search here and the Stack Exchange search here.

studyflight.com has been seen in 2 true positives, 0 false positives, and 0 NAAs.

Coverage remained the same at 65.519% when pulling e18397ec2a1d77df281a6a8a276ec6b507213e9b on auto-blacklist-1513576793.66213 into d0569d3c6b81107cb15a54c547b40fa61c89ccc6 on master.

!!/approve


disable AV scanning in build output dir

From https://discourse.llvm.org/t/rfc-future-of-windows-pre-commit-ci/76840.
Hoping disabling the AV scanning in the build directory only will improve reliability.
I could definitely use some help testing this.

as I remember those docker images don't have AV bundled but maybe that will work
to test changes you can either:
a) ssh to one of the running machines with k8s kubectl exec -it <POD_NAME> -- bash and then run the command you want. Downside of that is that will happen on currently executing machines but change seems benigh.
b) update docker container and run on windows workstation (you will need to provide it with buildkite agent token etc.). There should be 'win-dev' stopped machine in cluster for that that you can RPC to.
c) update docker container, upload image to the GCP and either restart kubernetes/buildkite/windows.yaml k8s deployment (again - that will affect "prod" machines) or update windows-test-32 deployment (the issue with that deployment at the moment is that it cannot be run as we don't have enouth machines but scaling down "prod" for some time is fine).
I would recommend testing with a) first and then update prod directly if command seems to work.

This was the wrong approach, closing.


fix: fix memory leak

Please check if the PR fulfills these requirements

[x] The commit message follows our guidelines
[x] Tests for the changes have been added (for bug fixes / features)
[x] Docs have been added / updated (for bug fixes / features)

What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)

Bug fix,

What is the current behavior? (You can also link to an open issue here)

memory leak

What is the new behavior (if this is a feature change)?

fix memory leak

Does this PR introduce a breaking change? (What changes might users need to make in their application due to this PR?)

no

Other information:

:tada: This PR is included in version 5.2.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


coco.yaml file

this looks really useful.
is there an example cooc.yaml file to get started learning this ?

Hi gedw99,
first of all thank you for reaching out!
At the moment unfortunately our UX documentation is still lacking behind and we don't have a nice end-to-end example readily consumable.
I assume, you're referring to the component configuration file and not to the CLI configuration, right?
In that case, component configurations are very limited at the moment.
You can only specify the component's name and the direct dependencies that the component consumes, e.g.
name: my-component
dependencies:
- first-dependency-that-I-consume
- second-dependency-that-I-consume
- ...

Currently, the file is only used for dependency graph generation and evaluation.

However, if you were referring to the CLI configuration, you can inspect that by running
coco inspect

to obtain the current configuration that coco uses. You can either adjust the file (default location in $HOME/.coco, or pass a different configuration file by running coco --config ${my_config_file}.

Feel free to reach out if there are any details we can help you with.

thanks @blackhatcrazy
Yeah i just wanted an example to kick the tires as they say.
can start anywhere. dont need a fancy UX or anything.

You can only specify the component's name and the direct dependencies that the component consumes, e.g.

Its a start :)
I can also help extend things from there too.  If there is a very basic TODO, i can see the holes and start to contrib...

Given we're fairly people-constrained, we're always happy for contributions if you want to chip in. I'll see to fill the issues with some features and fixes we have in mind.

Ok thanks @blackhatcrazy

Sounds good. I will try a few things out


How to distill Flux model

Hi, do you plan to open the distill method for Flux?

https://github.com/RedAIGC/Target-Driven-Distillation/tree/main/train/FLUX @lovejing0306

https://github.com/RedAIGC/Target-Driven-Distillation/tree/main/train/FLUX @lovejing0306

Thanks, TDD is your new work?


Fix a failure in errtest on some platforms

errtest checks that errno remains unchanged across a call to ERR_get_error(). This is required because SSL_get_error() calls ERR_get_error() (well actually ERR_peek_error() but internally they are both handled the same way) and returns SSL_ERROR_SYSCALL if a system call error occurred. In that case the user is supposed to consult errno.
In some cases ERR_get_error() can end up calling various system calls, so we need to make sure errno is preserved across all of those.

Fixes #6953

maybe this code helps:
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

#define NUM_SYS_STR_REASONS 127

int main(int ac, char **av)
{
    int i, sz = 0;
    char *buf = NULL, *p;
    int starts[NUM_SYS_STR_REASONS];

    /* Calculate size. */
    for (i = 0; i < NUM_SYS_STR_REASONS; ++i) {
        const char *cp = strerror(i);
        if (cp != NULL)
            sz += strlen(cp) + 1;
    }
    p = buf = malloc(sz + 1);

    /* Stuff all strings into one buffer. */
    for (i = 0; i < NUM_SYS_STR_REASONS; ++i) {
        const char *cp = strerror(i);

        if (cp == NULL) {
            starts[i] = -1;
            continue;
        }

        starts[i] = (int)(p - buf);
        p += strlen(strcpy(p, cp));
        p++;
    }

    /* Test. */
    while ((p = *++av) != NULL) {
        i = atoi(p);
        if (i < 0 || i > NUM_SYS_STR_REASONS)
            perror(p);
        else
            fprintf(stderr, "%d %s --> %s\n", starts[i], p, &buf[starts[i]]);
    }
    return 0;

}

I dropped the third commit from this since that has now been fixed by @levitte in a different way. Also rebased.

Updated to fix feedback and rebased to resolve merge conflict

👍

Pushed. Thanks.

Looking back at this, I'm surprised by how the actual changes differ from the description...  only build_SYS_str_reasons got the preservation call.  What happened?

Looking back at this, I'm surprised by how the actual changes differ from the description... only build_SYS_str_reasons got the preservation call. What happened?

Huh? dlopen got it too?

@mattcaswell, perhaps re-read the PR description?


Repo Restructuring

Description
Objective: Rework the repo to follow the program of the new course
Changes

[x] Change order of courses
[x] Lint and format all files

Meant to be pushed to https://github.com/artefactory/xhec-mlops-crashcourse-2023


python3 -m pip install -r requirements.txt not working

This is on Mac OS Catalina.
Typing python3 -m pip install -r requirements.txt in the correct directory brings
Collecting argparse==1.4.0   Using cached argparse-1.4.0-py2.py3-none-any.whl (23 kB) Collecting colour==0.1.5   Using cached colour-0.1.5-py2.py3-none-any.whl (23 kB) Collecting numpy==1.15.0   Using cached numpy-1.15.0.zip (4.5 MB) Collecting Pillow==5.2.0   Using cached Pillow-5.2.0.tar.gz (14.5 MB) Collecting progressbar==2.5   Using cached progressbar-2.5.tar.gz (10 kB) Collecting scipy==1.1.0   Using cached scipy-1.1.0.tar.gz (15.6 MB) Collecting tqdm==4.24.0   Using cached tqdm-4.24.0-py2.py3-none-any.whl (43 kB) ERROR: Could not find a version that satisfies the requirement opencv-python==<IP_ADDRESS> (from -r requirements.txt (line 8)) (from versions: <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>) ERROR: No matching distribution found for opencv-python==<IP_ADDRESS> (from -r requirements.txt (line 8))
How do I fix the error?

This error may be caused by the following information.
ERROR: Could not find a version that satisfies the requirement opencv-python==<IP_ADDRESS> (from -r requirements.txt (line 8)) (from versions: <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>)
ERROR: No matching distribution found for opencv-python==<IP_ADDRESS> (from -r requirements.txt (line 8))

So we need to install a specific opencv-python version manually.

pip3 install opencv-python==<IP_ADDRESS>

Have a try!

I tried that, it gives the same error.

I fixed it by changing the requirements.txt to opencv-python==<IP_ADDRESS>.


HTTP Error 404: Not Found

it still does not work even after using the --skipyandex option
==> Analyzing https://scontent-mxp1-1.xx.fbcdn.net/v/t1.0-1/c<IP_ADDRESS>a/p74x74/1379841_1015000455280190...
Traceback (most recent call last):
File "eagle-eye.py", line 261, in <module>
main(skipFB=args.skipfb, skipY=args.skipyandex, FBUrls=[], jsonRep=jsonRepFile,
dockerMode=aDocker, dockerName=aName)
File "eagle-eye.py", line 104, in main
profile_links, profile_imgs = r.getValidLinksAndImg(name)
File "/home/thefox/Desktop/tools/Information/EagleEye/face_recog.py", line 86, in
getValidLinksAndImg
valid_indexes = self.constructIndexes(label)
File "/home/thefox/Desktop/tools/Information/EagleEye/face_recog.py", line 54, in constructIndexes
urlretrieve(i, tmp_path)
File "/usr/lib/python3.7/urllib/request.py", line 247, in urlretrieve
with contextlib.closing(urlopen(url, data)) as fp:
File "/usr/lib/python3.7/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.7/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.7/urllib/request.py", line 641, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib/python3.7/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/usr/lib/python3.7/urllib/request.py", line 503, in _call_chain
result = func(*args)
File "/usr/lib/python3.7/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found

Does that happen with every image and/or name?

Yes, it happens with every image and every name with which I tried

Can you provide the name and image?

I'm sorry but I can not

404 not found meaning it cannot find the image on fb.
Okay you can do the following.
go in line 53 or face_recog.py:
https://github.com/ThoughtfulDev/EagleEye/blob/master/face_recog.py#L53
And change it to:
console.task('Analyzing {0}...'.format(i.strip()))
It should then print the full image link which you can copy and see for yourself if the FB profile image link return a 404.


WIP [skip ci]

Description
Working on removing swagger
Review Instructions
Describe if this ticket needs review and if so, how one may go about it in qa and/or staging environments.
For example, a ticket based on Security Hub, Snyk, or Dependabot may not need review since those services
will generate new warnings if the issue has not been resolved properly. On the other hand, an infrastructure
ticket that results in visible changes to the end-user will definitely require review.
Many tickets will likely be between these two extremes, so some judgement may be required.
Issue
A link to a github issue or SEAB- ticket (using that as a prefix)
Security
If there are any concerns that require extra attention from the security team, highlight them here.
Please make sure that you've checked the following before submitting your pull request. Thanks!

[ ] Check that you pass the basic style checks and unit tests by running mvn clean install
[ ] Ensure that the PR targets the correct branch. Check the milestone or fix version of the ticket.
[ ] Follow the existing JPA patterns for queries, using named parameters, to avoid SQL injection
[ ] If you are changing dependencies, check the Snyk status check or the dashboard to ensure you are not introducing new high/critical vulnerabilities
[ ] Assume that inputs to the API can be malicious, and sanitize and/or check for Denial of Service type values, e.g., massive sizes
[ ] Do not serve user-uploaded binary images through the Dockstore API
[ ] Ensure that endpoints that only allow privileged access enforce that with the @RolesAllowed annotation
[ ] Do not create cookies, although this may change in the future
[ ] If this PR is for a user-facing feature, create and link a documentation ticket for this feature (usually in the same milestone as the linked issue). Style points if you create a documentation PR directly and link that instead.

Note to self: a big problem here is that TRS v2 beta was never exported to openapi 3.
So we have a couple of choices, upgrade TRS v2 beta to openapi, potentially breaking things. Remove tests for it (potentially not knowing if things will break) .... or  ....


it has some issues....

When I first open the app without sending anything, on some devices it behaves like it received 'NULL' intent.
When I send few files, not multiple send, but one after another,  It receives the same file few time.
FlutterSharingIntent.instance.reset(); doesnt help!!!!

On which platform are you?
Are you using the example code?

Try running the example project from inside this repository on your devices and tell us if your observations persist.

Android. Galaxy tab A7 and Galaxy note 8.
It happens more frequently on the A7.
Sometimes, it receives the same file twice with delay of few milliseconds.
So my way to deal with the issue is to save the time of the receiving in a variable, and check if two seconds passed from the last file received. If no,
the app do not process the intent.
Yes, I used the example code.