error: ‘GROOVE_EVENT_DEVICEREOPENED’ was not declared in this scope

Hi there I've run across the following error while node-gyp tries to build groove...
gyp info it worked if it ends with ok
gyp info using<EMAIL_ADDRESS>gyp info using<EMAIL_ADDRESS>| linux | x64
gyp info spawn make
gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
make: Entering directory '.../node_modules/groove/build'
CXX(target) Release/obj.target/groove/src/groove.o
../src/groove.cc: In function ‘void Initialize(v8::Handlev8::Object)’:
../src/groove.cc:86:51: error: ‘GROOVE_EVENT_DEVICEREOPENED’ was not declared in this scope
SetProperty(exports, "_EVENT_DEVICEREOPENED", GROOVE_EVENT_DEVICEREOPENED);
^
groove.target.mk:95: recipe for target 'Release/obj.target/groove/src/groove.o' failed
make: *** [Release/obj.target/groove/src/groove.o] Error 1
make: Leaving directory '.../node_modules/groove/build'
gyp ERR! build error
gyp ERR! stack Error: make failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/usr/local/lib/node_modules/node-gyp/lib/build.js:269:23)
gyp ERR! stack     at ChildProcess.emit (events.js:110:17)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (child_process.js:1074:12)
gyp ERR! System Linux 3.19.0-21-generic
gyp ERR! command "node" "/usr/local/bin/node-gyp" "build"
gyp ERR! cwd .../node_modules/groove
gyp ERR! node -v v0.12.5
gyp ERR! node-gyp -v v2.0.1
gyp ERR! not ok

I believe that the libgroove on your system is older than 4.3.0 and that's why you're getting this issue.
2 ways to solve this:

version-lock node-groove to 2.3.x (see https://github.com/andrewrk/node-groove#versions)
upgrade libgroove on your system to the newest release (currently 4.3.0)

The first way is probably easier for now. libgroove 4.3.0 was just released recently so package maintainers have not yet had a chance to get the newer version into your system.

Thanks it works now, I went with specifying groove 2.3.4 in my package.json file for now.


Title needed when using retriever.retrieve after using GenericDataLoader(...)load_custom(...)

I have noticed an error when using EvaluateRetrieval.retrieve(model) when using the exact_search combined load_custom() method from the GenericDataLoader.
https://github.com/UKPLab/beir/blob/main/beir/datasets/data_loader.py -> GenericDataLoader:
It says in the document that the title is optional, but when feeding a custom corpus without title, it goes through the _load_corpus() method where the following code is executed:
            self.corpus[line.get("_id")] = {
                "text": line.get("text"),
                "title": line.get("title"),
            }

more precisely:
"title": line.get("title"),
By default, if the title is not present, it returns a None.
Then in the following file:
https://github.com/UKPLab/beir/blob/main/beir/retrieval/evaluation.py
It uses the method retrieve, which uses the search method:
return self.retriever.search(corpus, queries, self.top_k, self.score_function, **kwargs)
When using search.dense.exact_search.py as the retriever, the following code is executed:
corpus_ids = sorted(corpus, key=lambda k: len(corpus[k].get("title", "") + corpus[k].get("text", "")), reverse=True)
The problem is the following: When using GenericDataLoader(...)load_custom(...), if there is no title field, It creates a None type as title, then when trying to run this code :
corpus[k].get("title", "") + corpus[k].get("text", "")
It returns a TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'
What I suggest to resolve this problem (It worked in my case) is replacing the code like the following, for the  _load_corpus() method:
            self.corpus[line.get("_id")] = {
                "text": line.get("text"),
                "title": line.get("title", ""),
            }

It creates a string, then it is possible to run the following code:
corpus[k].get("title", "") + corpus[k].get("text", "")
Olivier

I stumbled upon the same problem and totally support this. To make the code work I modified my dataset to have "title": "" for each corpus entry.
However, I find it ambiguous that "title" is used by default in evaluation if present while default behavior with no title will be to use the "text" only (after the proposed fix).
I would also add a parameter to enable or disable title by default. It should probably go in the SentenceBERT.encode_corpus(use_title_if_present)


Incorrect table definition; there can be only one auto column

Hello I am happy to download the newest version of this module but am running into some errors: I get the error Area code is not set during setup:upgrade and some digging around gave me the following error:
Incorrect table definition; there can be only one auto column and it must be defined as a key, query was: ALTER TABLE channable_returns MODIFY COLUMN id bigint UNSIGNED NOT NULL  AUTO_INCREMENT COMMENT "Entity ID", MODIFY COLUMN order_id int UNSIGNED NOT NULL   COMMENT "Order ID", MODIFY COLUMN channel_id int UNSIGNED NOT NULL   COMMENT "Channel ID", ADD CONSTRAINT  PRIMARY KEY (id)
I had a look at the discriminating table and found the following:
<table name="channable_returns">
        <column name="entity_id"
                xsi:type="int"
                padding="20"
                identity="true"   // would the primary key
                nullable="false"
                unsigned="true"
                comment="Entity ID"
                onCreate="migrateDataFrom(id)" // nice
        />
...
<column name="id"  // the old id?
                xsi:type="int"
                padding="20"
                identity="false"  // so no longer the key
                nullable="false"
                unsigned="true"
                comment="Old ID"
        />
<constraint xsi:type="primary" referenceId="PRIMARY">
            <column name="entity_id"/> // defines the new key
        </constraint>

and in the previous version:
 <constraint xsi:type="primary" referenceId="PRIMARY">
            <column name="id"/>
        </constraint>

So my table had id as the primary key and now should have entity_id as the new key. But how do I unassign the old primary key? Magento seems to think it still is the primary key and simply wants to add a new primary key?
Could I be missing something? Or some command? Any help is appreciated :-)

Hi @SolsWebdesign, thank you for opening this issue. We had to change the primary from id to entity_id in channable_returns table in v1.9.0. This will throw an issue if you try and downgrade the module to a previous version.
Was this the case? If yes, you can remove the entity_id field in the channable_returns table  and rerun the setup:upgrade.

Hi @Marvin-Magmodules thank you for your explanation and yes we tried to downgrade due to the area code is not set error. I am going to dig further because it is very well possible the "area code is not set" error is caused by one of the other modules that have been updated after running composer update. You can close this issue for now. Thank you for your quick reply!

Hi @SolsWebdesign thank you for the quick reply, will close this issue now. But feel free to contact me on https://www.magmodules.eu/support-form.html to help dive into this issue so we can try and make sure this was not related to the Channable module.

We also ran into this issue but we didn't do a downgrade.
This is how we fixed it. Run this on the database before running bin/magento set:up:
ALTER TABLE `channable_returns` MODIFY COLUMN `entity_id` bigint(20) UNSIGNED NOT NULL;
ALTER TABLE `channable_returns` MODIFY COLUMN `id` bigint(20) UNSIGNED NOT NULL AUTO_INCREMENT;


Allow the setting of S3ClientOptions on AmazonS3Client object

I am trying to use my own S3 repository to store snapshots and have an SSL enabled endpoint that does not support virtual hosting.  I need to use path style access to access the endpoint.
Ultimately I'd like to define a setting in the elasticsearch.yml file that indicates that it should use the "path style access" option.  Perhaps something like: cloud.aws.s3clientoption.path_style_access: true
Setting the property would result in the S3ClientOption.setPathStyleAccess() method being called with true and then those client options being set on the AmazonS3Client object.
I pulled the code, made a modification locally to the org.elasticsearch.cloud.aws.InternalAwsS3Service.getClient(String, String, String) method to set the path style access to true to make sure this would solve my problems.
Here is the simple hack I implemented in the InternalAwsS3Service class to test it out:

After I made the change the plugin worked as expected and I was able to store the snapshots in my own S3 repository and restore from that repository as well.
Any help in getting this functionality into the plugin would be greatly appreciated.

I added a PR for this. See #159.

Closing in favor of https://github.com/elastic/elasticsearch/issues/15114


kernel driver installation blue screen

installed the driver using three different methods, but all have blue screens.
first
Disable all Windows signatures Install using boot
second
Installing with the DSEFix program
third
Installation using the TDL program
I tried to install it as a test ... I can not install it
How can I let blue screen

I don't know how do you build the driver,and i always get BSD in win7 x64 also before i reinstall a win10.
Maybe you can use a VM to test loading driver.

Did you build driver as unsigned and build options Release x64?

Yes, I built it with 64 bit using the unsigned option
Help me

How are you loading the driver?

I do not understand..
How to load the driver ...
I was on the way above
Three ways ~

Is there any way to install the driver in Windows 7?

Are you having problems installing the driver on windows 7?

@jussihi Hope that the author gives a detailed program instructions please.

I do not have Windows 7, and I'm not going to buy / install Windows 7 machine. Seems like I can't fix this issue so closing it. @hackxx If you will buy me a test machine with Win7 x64, I will gladly test and debug it!


Close reveal modal with button

i have already try this
http://foundation.zurb.com/forum/posts/1542-how-can-i-close-a-reveal-modal-with-a-button
but the result is like this picture
what i want is "X" still in the right top bar and i create new button which have function to close reveal modal.
thus cancel button class seems not working correctly but the function to close reveal modal is working, here is my cancel code,
<input type="button" class="tiny button secondary close-reveal-modal" value="Cancel"></input>

and the second is data-modal-id and
 for modal must contain "myModal" , without it reveal modal not working. because i use to loop reveal modal , like "myModal1" , "myModal2" and so on. dont know if this is reveal modal pattern or bugs. ?

You could try adding your own event handler to the button. Add some unique class or attribute to the button that you can target with jQuery. In this example the class is .close-button.
<input type="button" class="tiny button secondary close-button" value="Cancel">

And now in the JavaScript, trigger an open event on your modal when this close button is clicked.
$('#modal .close-button').click(function() {
  $('#modal').foundation('reveal', 'close');
});

The above solution has been tested and works, so I'm closing this. Let me know if you need any more help!

Hello @gakimball, I am also facing this problem, but in my case when clicking on cross button it just refreshes the page losses the page data.
Event same think happening with new button as he mention in the document. If am using normal and trying to call $('#myModel').foundation('reveal', 'close'); this code doesn't close modal.
Any help? Any idea?

I just encountered this issue & was able to resolve this by just editing foundation.css
The class that trigger the link to close is
.close-reveal-modal
In foundation.css, look for the following class:
.reveal-modal .close-reveal-modal {
   //some css blocks
 }

now remove all of the css blocks inside and create another class, for example:
.reveal-modal .close-x-btn{
   //paste the css blocks here from the .close-reveal-modal
 }

Overall your button should look like this
Cancel

As for the "x" at the top right corner of the modal
×

So now, you can close a modal using a button and the "x" on the top right corner while still keeping the original style of the "x" link

this works for me to do in a more general way

 $('.btn-close-modal').on('click', function () {
            var div = $(this).closest('div.reveal-modal').first();
            $(div).foundation('reveal', 'close');
        })


Not found

Hello, when i try to load remote.js i'm getting 404. Fix it please ;)
https://i.imgur.com/deAyYQF.png

Nope ;)
(no longer supported)


Components digram of the MS

Link to Notion page

ESTIMATED - 2 hours
REALIZED - 1 hour 30 minutes


Change Domain Name

Hi! Quick question, how should I change the configuration of Dokku/Nginx if I'm changing the domain name of the website, and changing it from normal HTTP to HTTPS?
I deployed on DigitalOcean, and yes I know one way is to erase the server and restart everything, but I'm just trying to know if there is a way that this can be done faster.

So wait....are you trying to add SSL or are you changing the domain name of the site?

Change domain. The domain happens to be SSL, while the old one isn't

Use the domains plugin. Check out the docs for more info.

Also if you want nginx to handle SSL, that's in the docs as well. Give it a shot and feel free to comment back here with questions.


Button freezes when quit without saving files.

I have checked the following:

[X] I use the newest version of bruno.
[X] I've searched existing issues and found nothing related to my issue.

Describe the bug
Using the latest version of Bruno in my M1 Apple Silicon.
When I tried quitting the app without saving the files, a pop up appears with 3 options of which the save button is not working as intended and appear to be freezed..
.bru file to reproduce the bug
It's an UI issue I guess
Screenshots/Live demo link
https://github.com/usebruno/bruno/assets/63138398/b312943d-c5a6-4028-913c-8a43115ea6f7

Check out #2130. There are currently two PRs waiting to fix this problem via #2156 and #2157.

The latest version of Bruno has resolved the issue. Thank you for reporting it!


Update readme to reflect recent changes?

TC39 now says decorators are stage 2. Could this plugin's readme be updated to reflect the current status?
My impression (after an hour of reading conflicting info in various readmes) is that decorators are now well and truly on the standards track, and we're now waiting for an official Babel plugin. But I'm not clear on whether the official plugin will be essentially the same as this 'legacy' one, or if they will work differently. In other words I want  some guidance on whether I should start using decorators today via this plugin - will it be easy to switch?
If you know the current situation it would be great to explain it the 'why legacy' section in this readme.

There are pretty big changes between the stage 1 proposal (which this transform implements) and the stage 2 proposal. No decorator written against this transform will work with what is currently on the standards track (afaict at least). So it seems to make sense to keep this as "-legacy" because people who depend on the old decorator interface might still need it, even if babel starts including an updated transform.

We're planning to deprecate this and move it back to babel-plugin-transform-decorators soon, and then iterate on it there for updates to the new spec. Still working out how we want to do versioning, but not worth tracking this in this repo.


Improve the installation of the python library

At the moment the CMAKE installs the python library in the default environment. It would be better to have control over where the lib is installed. For example in the current conda env or in the intelpython libraries if intel oneapi is installed.
I guess this needs to be modified in the CMAKE
https://github.com/NicoRenaud/votca/blob/pybind11_nico_/xtp/src/pyxtp/CMakeLists.txt

Instead of figuring out all the places where the library could be installed, would it be option to package the library and let the user install it where they need it?

One possible solution would be to install cmake in a conda env and use that cmake to install the code and see where the python lib is then installed ...

We also have to cater for different needs:

people who want to install locally using conda to simply experiment with the code
people who want to use the python interface in a HPC env where most dependencies will be in modules

The priority is the 1. since the HPC people can also decide not to use the python interface and directly use the C++ code.

It should be possible for the CMAKE to detect the conda install : https://gitlab.kitware.com/cmake/cmake/-/issues/21322 but it's maybe not totally straightforward

Indeed that works if the python version in the active conda env is higher than the OS one. We can maybe change that  with the Python_FIND_STRATEGY option

we can manually tell cmake to install in a given python using :  cmake -B votca_builddir_conda -S votca -DBUILD_XTP=ON -DPYTHON_INSTDIR=/home/nico/miniconda3/envs/votca2/lib/python3.10/site-packages -DCMAKE_INSTALL_PREFIX=${prefix}
i.e. the -DPYTHON_INSTDIR pointing that the site-packages folder of the desired python version

see PR #13
Tested on ubuntu 22

install MKL see #1
install dependencies through apt-get see #1
create clean conda env : conda create -n votca python==3.10
install python dependencies:  conda install numpy h5py more-itertools pillow -c anaconda there might be more missing
install votca following instruction in #1


Support dicts as Grid items and some finessing

This PR is intended more as a vehicle for discussion than something you might merge as-is (although does work as-is).
Only functionality changes are in these commits:

Support dicts as items - Can provide dicts in the list of items given to the Grid control, i.e. works as long as the dicts have keys matching the expected columns
Fix init call argument - Was accidentally always passing in None

Then there is a copy-paste update to tests to make them run with the new Commands.
For discussion:

Non-functionality impacting changes to make the code more "typical Python" and to use some of the built-in capabilities.
Added beartype to have method name type hints checked automatically instead of manual asserts. Requirements say it is Python >= 3.6, but I saw something in docs about >= 3.9, so might need to check that before adopting, if you decide it looks useful.
Used the black formatter on the code. Would highly recommend it to avoid any discussion and back-and-forth on styling. Also very nice to focus on the functionality and have the editor automatically reformat the code (I use darker with PyCharm, it has good instructions for other editors as well).

Ha, black is on HN home page right now: https://news.ycombinator.com/item?id=30130315 :)

Ha, black is on HN home page right now: https://news.ycombinator.com/item?id=30130315 :)

👍🏻 Opinions differ, but I must say that the fundamentally non-value-adding tab discussion that immediately follows the first post there is, to me, a testament to the value of black.
But, on a more functional note, I realized the PR does not yet take care of the updates to the item values, need to add that.

Using and updating dicts as Grid item list values works now with the Grid sample.

PR looks great, thank you! Working with dictionaries in Grid is definitely a must-have addition. Grid test is be a good example for other tests. Also, I'm going to reformat everything using Black in a separate branch and add it as a requirement to contributing guide.
I'm thinking... should we split Grid classes into separate files under grid directory? Having them all in grid.py is a kind of inconvenient. What does idiomatic Python say?

To my knowledge there is no idiomatic Python rule dictating how many classes there should be per file. Thus it is a matter of preference (and maybe capabilities of your editor) when a single file would be considered unwieldy and in need of being split up. grid.py is still on the smaller side and quite far from my personal threshold.
But the single file can be made much more approachable by placing the top class (Grid) first in the file, and go down in granularity as you read down the file. That is somewhat "Pythonic", as Python has no need for forward declarations in most cases.
To demonstrate, I am pushing a commit where I have just quickly rearranged the order of the classes.

Oh, and I did delve into beartype a bit more, it does support Python >= 3.7, so no concerns in that regard.

Sounds great, let's merge this PR. I'm going to push https://github.com/pglet/pglet-python/pull/61 with Black formatting and annotations later this week.


Compiled from git repo, restart nodes, kubelet fails

Compiled current code base for frakti to pick up changes for default cpu/mem
Copied frakti binary to 1 master, 5 minon nodes
Restart all nodes

Result:
KUBELET
● kubelet.service - kubelet: The Kubernetes Node Agent
Loaded: loaded (/etc/systemd/system/kubelet.service; enabled; vendor preset: disabled)
Drop-In: /etc/systemd/system/kubelet.service.d
└─10-kubeadm.conf
Active: activating (auto-restart) (Result: exit-code) since Mon 2017-06-05 12:52:23 CDT; 1s ago
Docs: http://kubernetes.io/docs/
Process: 7419 ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_SYSTEM_PODS_ARGS $KUBELET_NETWORK_ARGS $KUBELET_DNS_ARGS $KUBELET_AUTHZ_ARGS $KUBELET_CGROUP_ARGS $KUBELET_EXTRA_ARGS (code=exited, status=1/FAILURE)
Main PID: 7419 (code=exited, status=1/FAILURE)
Jun 05 12:52:23 master01 systemd[1]: kubelet.service: main process exited, code=exited, status=1/FAILURE
Jun 05 12:52:23 master01 systemd[1]: Unit kubelet.service entered failed state.
Jun 05 12:52:23 master01 systemd[1]: kubelet.service failed.
HYPERD
● hyperd.service - hyperd
Loaded: loaded (/usr/lib/systemd/system/hyperd.service; enabled; vendor preset: disabled)
Active: active (running) since Mon 2017-06-05 12:50:52 CDT; 4min 46s ago
Docs: http://docs.hypercontainer.io
Main PID: 7193 (hyperd)
CGroup: /system.slice/hyperd.service
└─7193 /usr/bin/hyperd --log_dir=/var/log/hyper
Jun 05 12:50:52 master01 systemd[1]: Started hyperd.
Jun 05 12:50:52 master01 systemd[1]: Starting hyperd...
Jun 05 12:50:53 master01 hyperd[7193]: time="2017-06-05T12:50:53-05:00" level=info msg="Graph migration to content-addressability took 0.00 seconds"
Jun 05 12:50:53 master01 hyperd[7193]: time="2017-06-05T12:50:53-05:00" level=info msg="Firewalld running: false"
Jun 05 12:50:53 master01 hyperd[7193]: time="2017-06-05T12:50:53-05:00" level=info msg="Loading containers: start."
Jun 05 12:50:53 master01 hyperd[7193]: ...
Jun 05 12:50:53 master01 hyperd[7193]: time="2017-06-05T12:50:53-05:00" level=info msg="Loading containers: done."
Jun 05 12:50:53 master01 hyperd[7193]: E0605 12:50:53.215808    7193 persist.go:113] Pod[k8s_POD.0_kube-dns-2612786695-kqqhv_kube-system_4664f72a-409d-11e7-9579-0021ccd48ab5_9acb0442] failed to load sandbox info: leveldb: not found
FRAKTI
● frakti.service - Hypervisor-based container runtime for Kubernetes
Loaded: loaded (/usr/lib/systemd/system/frakti.service; enabled; vendor preset: disabled)
Active: active (running) since Mon 2017-06-05 12:50:31 CDT; 5min ago
Docs: https://github.com/kubernetes/frakti
Main PID: 7126 (frakti)
CGroup: /system.slice/frakti.service
└─7126 /usr/bin/frakti --v=3 --log-dir=/var/log/frakti --logtostderr=false --cgroup-driver=systemd --listen=/var/run/frakti.sock --streaming-server-addr=master01 --hyper-endpoint=<IP_ADDRESS>:22318
Jun 05 12:50:31 master01 systemd[1]: Started Hypervisor-based container runtime for Kubernetes.
Jun 05 12:50:31 master01 systemd[1]: Starting Hypervisor-based container runtime for Kubernetes...
Jun 05 12:50:52 master01 frakti[7126]: 2017/06/05 12:50:52 transport: http2Client.notifyError got notified that the client transport was broken EOF.
Jun 05 12:50:52 master01 frakti[7126]: 2017/06/05 12:50:52 grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp <IP_ADDRESS>:22318: getsockopt: connection refused"; Reconnecting to {<IP_ADDRESS>:22318 }
Jun 05 12:52:22 master01 frakti[7126]: 2017/06/05 12:52:22 transport: http2Server.HandleStreams failed to read frame: read unix /var/run/frakti.sock->@: read: connection reset by peer
Jun 05 12:55:24 master01 frakti[7126]: 2017/06/05 12:55:24 transport: http2Server.HandleStreams failed to read frame: read unix /var/run/frakti.sock->@: read: connection reset by peer
Jun 05 12:55:45 master01 frakti[7126]: 2017/06/05 12:55:45 transport: http2Server.HandleStreams failed to read frame: read unix /var/run/frakti.sock->@: read: connection reset by peer
Another thing I've noticed is that that does not survive reboots all that well.
Thoughts where I should look?
DB

I had noticed that the init pulled in 1.7.0 so reset it with v1.6.4 and still had the same results. At this point the cluster is dead and wont become stable. So I will have to rebuild from OS again to get a clean cluster. This time around I will just stick with 1 master, 1 Minon to simplify troubleshooting and the impact of rebuilds. I choose baremetal over VM to avoid incurring 12% performance hits.
I will check out the various branches and redo this and see where things land. Thanks for the heads up.
DB

I recompiled from the kubernetest 1.6 branch and reapplied all changes to each node and created a new cluster. Everything is operational again and going to move on to bringing rook.io up given the new changes and see what the result is.
Thanks
DB

@DarkBlaez Cool 👍


Mutating a continuation?

I've been trying to figure out how to implement something like Rust's .next() and .peek() explicit-iterator methods effectfully.
In the fake language I've been using to prototype things before implementing them in Effekt, considering continuations as first-class values that are mutated by .resume() and can be assigned to seems to allow for semantics compatible with both effectful, lazy iterators and also explicit .next() and .peek() methods. That linearity / ownership is preserved seems very hard to deal with (but possible?? i saw a recent paper) but also not super relevant for the main conceptual issue I'm hitting.
pub effect Yield[T] {
  yield(T)
}

pub func iter[T](self: List[T]) {
  if let Cons(head, tail) = self {
    do yield(head)
    tail.iter()
  }
}

pub func next[T](self: mut Continuation[()] / Yield[T]): T? {
  // mutates `self`. thus, multiple calls to `.next()` can be chained
  try self.resume() {
    // ownership over `self` is given up upon catching an effect, or evaluating to a value
    yield(x) => some(x)
    // this is matching the value `()` (empty tuple) of type `()` (unit)
    () => none
  }
}

pub func peek[T](self: mut Continuation[()] / Yield[T]): T? {
  // we mutate self here.
  try self.resume() {
    // upon catching a raised effect, `self` is no longer being mutated.
    yield(x) => {
      func wrapped() {
        do yield(x)
        // so we can uniquely mutate `self` again here to resume.
        self.resume()
      }
      // this might not work wrt. ownership and also efficiency
      self = Continuation(wrapped())
      some(x)
    }
    _ => none
  }
}

// An explicit Continuation constructor converts this computation of type () to a Continuation[()].
// Really, this would be implicit, I think - any func taking in a Continuation would wrap its arg in this.
// This is kinda similar to Effekt's blocks??
let iterator = Continuation([1, 2, 3].iter())
assert iterator.next() is some(1)
assert iterator.next() is some(2)
assert iterator.peek() is some(3)
assert iterator.next() is some(3)
assert iterator.next() is none

But this is very far from Effekt's semantics... since there isn't an explicit Continuation type, and instead computations that are treated as zero-arity functions when passed to a function through a {} block, and are called as a standard function instead of with .resume(), with the target of the resume keyword instead implicit in the effect handler. So I'm kind of wondering, is this sort of thing possible to express in Effekt? Are there different features of Effekt's effect-and-handler system that I haven't stumbled across that might be relevant here? Or is this entirely the wrong approach to the issue, conceptually, and does something like .next() and .peek() necessitate operating on some concrete iterator struct?

Does Effekt have some way to express mutable parameters?
I've finally wrapped my head around #108 and was able to implement .zip() using a similar trick. I think that + mutable parameters is all that is needed to implement a .next() and .peek() effectfully... I'd like to be able to write something like this, basically.
def next[T]() {mut self: => Unit / Yield[T]}: Option[T] = {
  try {self(); None()} with Yield[T] {
    def yield(x) = {
      self = fun() { resume(()) }
      Some(x)
    }
  }
}

def peek[T]() {mut self: => Unit / Yield[T]}: Option[T] = {
  try {self(); None()} with Yield[T] {
    def yield(x) = {
      self = fun() { yield(x); resume(()) }
      Some(x)
    }
  }
}

I suppose this doesn't actually make very much sense with "computations" as distinct from "values"... hmm. Unless instead of yielding values directly a .iter() method (or equivalent) instead returned a thunk?

Hey, sorry for the late response. We have multiple exampels of "mutable continuations" hidden somewhere in this repo. For instance here:
https://github.com/effekt-lang/effekt/blob/master/examples/benchmarks/other/generator.effekt
or in a variation also here:
https://github.com/effekt-lang/effekt/blob/a41dde504adbff0af7ab5328acf0f3d847745191/examples/casestudies/scheduler.effekt.md?plain=1#L38-L61
and in the issue that you linked.
Currently computation is always considered immutable, while for values we have vars and the Ref type in the standard library. Your point is interesting, though, and I have to think a bit more about it.

We have just merged #527 (thank you for your comments on that one). It contains a function source, which does what you want (I think), but not in the way you want it (sadly). It has type
def source[A, R] { stream: () => Unit / emit[A] } { reader: () => R / read[A] }: R

and takes a stream that pushes values with emit and a reader that reads values with read and connects the two. You would use it like this:
import stream

def main() = {
  val _ = {
    with boundary
    with source[Int, Unit] { range(0, 100) }
    println(do read[Int]())
    println(do read[Int]())
    println(do read[Int]())
  }
  println("done")
}

The ability to read values is only available in a limited scope, because we allocate the mutable variable next (in your example self) on the stack, and because doing so might use other effects, which are not available in wider scopes.


Order of operation with fixes

@lonegunmanb - First off, I love the idea of this tool, and I'm excited to see where it goes.
I'm trying to use the fix for local_file to create some files in my repos.  However, it appears that the full folder path needs to exist for the local_file fix to work.  In other words, if I'm trying to create a .github/workflows/file.yaml in a repo, and that repo doesn't have a .github folder and a workflows subfolder already, then I get an error from grept.
So, to get around this I tried to create a rule for dir_exist and use it to run a local_shell fix that would run a simple mkdir command.
This works great, but only when grept decides to run the local_shell fix before it runs the local_file fix.  In doing some testing, sometimes it will, and sometimes it won't.
I tried adding the terraform depends_on argument to the fix, but that made grept throw an error.
Is there a way to specify order of operations, or specify dependencies for the fixes?

Thanks @nnellans for your support, I'm glad that you like grept's idea, it means a lot to me.
I like this depends_on idea since I've also met the very similar issue.

However, it appears that the full folder path needs to exist for the local_file fix to work. In other words, if I'm trying to create a .github/workflows/file.yaml in a repo, and that repo doesn't have a .github folder and a workflows subfolder already, then I get an error from grept

The issue you described looks like a bug to me since local_file is designed to create all sub folders for you. We have a corresponding unit test. I'll verify this issue on my side, whilst would you please share the error message with me returned by grept? Thanks!

Hi @nnellans I found that the afero.Memfs I used in the unit test has different behavior than OsFs, I've rewrite the unit test and fixed the bug you've met, would you please install the latest version and try again? Thanks!

Thanks for the response.  The local_file fix is now giving me a "permission denied" error when it tries to create the .github/workflows directory.  GitHub Workflow Run.
Do I need to run sudo grept instead of just grept maybe?

Hi @nnellans could run mkdir .github./workflows in the session you called grept?

I am running grept from GitHub Actions, on ubuntu-latest runner, and using a simple run step.
If I add a command for mkdir -p .github/workflows and put that right before the grept command, then it works fine.  However, if I simply try to run the grep apply --auto command and let the local_file fix attempt to create the full path, then I get the error mentioned above.

Many thanks @nnellans , I'll try to reproduce this issue.

Here's my workflow.

Hi @nnellans I've merged another pr now a toy workflow can work now.
You can change your workflow by installing grept like this:
go install github.com/Azure/grept@ca36e4333ddf2dbc5371c78bf7b0fe51e17e6d83

This latest version should solve your issue.
A big THANKS for reporting this issue to us!

Works like a charm now!  Thank you very much!

Hey @nnellans I'm glad that this tool could help you, if you meet any further issues please feel free to report them here, thanks again for your support!


Sort repository tree entries in natural way

Before:

After:

Depends on go-gitea/git#81

Codecov Report

Merging #2506 into master will increase coverage by 0.22%.
The diff coverage is 88.88%.

@@            Coverage Diff             @@
##           master    #2506      +/-   ##
==========================================
+ Coverage   26.86%   27.09%   +0.22%
==========================================
  Files          84       85       +1
  Lines       16936    16999      +63
==========================================
+ Hits         4550     4606      +56
- Misses      11712    11717       +5
- Partials      674      676       +2

Impacted Files
Coverage Δ

modules/base/natural_sort.go
88.88% <88.88%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 52e11b2...1234640. Read the comment docs.

LGTM

LGTM

Make LG-TM work


Can't rotate video with ffmpeg argv

I'm trying to rotate my stream 90 degrees but when i pass the flag -vf "transpose=1" it closes the stream. I already have tried to set -s 1080x1920 and it does'nt work. Here is my argv array before spawn ffmpeg: let argv = [
'-fflags',
'nobuffer',
'-i',
this.conf.inPath,
// '-vf',
// '"transpose=1"',
// '-s',
// '1080x1920',
'-f',
format,
this.conf.ouPath];

@joaojlf4
Hi dear friend
Did you understand the answer yourself?
If you understand and know, please tell me what the solution is because it is very important to me
Thanks

@joaojlf4
Hi dear friend
Did you understand the answer yourself?
If you understand and know, please tell me what the solution is because it is very important to me
Thanks

Does anyone know how to rotate the transcoding? I have tried every combination and nothing seems to work. Any help would be appreciated! Thanks.
  trans: {     ffmpeg: 'C:\\ffmpeg\\bin\\ffmpeg.exe',     tasks: [       {         app: 'test',         hls: true, 	rtmp: false,         raw: [            '-vf', 	  'transpose=1'         ],         hlsFlags: '[hls_time=2:hls_list_size=3:hls_flags=delete_segments]',       }     ]   },

Does anyone know how to rotate the transcoding? I have tried every combination and nothing seems to work. Any help would be appreciated! Thanks.
  trans: {     ffmpeg: 'C:\\ffmpeg\\bin\\ffmpeg.exe',     tasks: [       {         app: 'test',         hls: true, 	rtmp: false,         raw: [            '-vf', 	  'transpose=1'         ],         hlsFlags: '[hls_time=2:hls_list_size=3:hls_flags=delete_segments]',       }     ]   },


Bug on last batch of first epoch using TPU with pretrained-model on google colab

Hi,
I am trying to train a mobileNetV2 coming from tf.keras but the training stops on the last batch of the first epoch:
Epoch 1/15
1/2 [==============>...............] - ETA: 10s - loss: 2.7432 - acc: 0.1406
Don't mind about the number of batch, adding some more  changes nothing
Here's the error :
Compilation failure: Asked to propagate a dynamic dimension from hlo %convert.283 = f32[8,80,80,32]{3,2,1,0} convert(f32[8,80,80,32]{3,2,1,0} %add.1), metadata={op_type="FusedBatchNorm" op_name="bn_Conv1_2/FusedBatchNorm"}@{}@0 to hlo %clamp.288 = f32[8,80,80,32]{3,2,1,0} clamp(f32[8,80,80,32]{3,2,1,0} %broadcast.286, f32[8,80,80,32]{3,2,1,0} %convert.283, f32[8,80,80,32]{3,2,1,0} %broadcast.287), metadata={op_type="Relu6" op_name="Conv1_relu_2/Relu6"}, which is not implemented.
TPU compilation failed
[[{{node TPUReplicateMetadata_1}}]]
image of the error for more clarity:
https://drive.google.com/open?id=1Gkzow3mrUJPACYfjz6E7V3Efj8eO261F
Here's my configuration below:
Python 3.6.8 (default, Jan 14 2019, 11:02:34)
[GCC 8.0.1 20180414 (experimental) [trunk revision 259383]] on linux
tensorflow=1.14.0
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2018 NVIDIA Corporation
Built on Sat_Aug_25_21:08:01_CDT_2018
Cuda compilation tools, release 10.0, V10.0.130
Here's the link to my ipynb : https://drive.google.com/open?id=1w2VJAH1gIv6-tJms3qrQg8IxSoKS1aw0
Any idea of what's going on ?

Tell me if you have question abbout my code.

@amapic Please provide a standalone code to reproduce the issue. Please use any toy dataset if the data you are using is private. Thanks!

Thanks, I made it works by using cifar10. But do you have any idea of what was going on with my own data ? It's regular image.
With cifar10, I now have this result :
i know have this as a result:
Epoch 1/15
2/2 [==============================] - 15s 8s/step
2/2 [==============================] - 15s 8s/step
2/2 [==============================] - 35s 18s/step - loss: 2.7288 - acc: 0.0781 - val_loss: 2.3025 - val_acc: 0.0781
Why is there several line of the same steps ?
`from google.colab import drive
drive.mount('/content/drive')
import sys
from PIL import Image
import numpy as np
import tensorflow as tf
import os
(x_train, y_train), (x_test, y_test)=tf.keras.datasets.cifar10.load_data()
img_size = 160
EPOCHS = 15
nb_imag=160
input image dimensions
x_train, y_train=x_train[0:128], y_train[0:128]
x_test, y_test=x_test[0:128], y_test[0:128]
from skimage.transform import resize
from sklearn import preprocessing
imgs=[]
imgs_test=[]
for img in x_train:
im = Image.fromarray(img)
imgs.append(resize(np.array(im), (img_size, img_size,3), anti_aliasing=False))
for img in x_test:
im = Image.fromarray(img)
imgs_test.append(resize(np.array(im), (img_size, img_size,3), anti_aliasing=False))
IMG_ROWS, IMG_COLS = img_size, img_size
imgs=np.array(imgs)
imgs_test=np.array(imgs_test)
labels=y_train
labels_test=y_test
input_shape = (IMG_ROWS, IMG_COLS, 3)
x_train = imgs.astype('float32')
x_train /= 255
x_test = imgs_test.astype('float32')
x_test /= 255
print('x_train shape:', imgs.shape)
NUM_CLASSES=np.unique(labels).size
print("NUM_CLASSES",NUM_CLASSES)
convert class vectors to binary class matrices
y_train = tf.keras.utils.to_categorical(labels, NUM_CLASSES)
y_test = tf.keras.utils.to_categorical(labels_test, NUM_CLASSES)
BATCH_SIZE = 64
resolver = tf.distribute.cluster_resolver.TPUClusterResolver('grpc://' + os.environ['COLAB_TPU_ADDR'])
tf.config.experimental_connect_to_host(resolver.master())
tf.tpu.experimental.initialize_tpu_system(resolver)
strategy = tf.distribute.experimental.TPUStrategy(resolver)
with strategy.scope():
model = build_model(IMG_ROWS, NUM_CLASSES, 1, 0.5, 3)
model.compile(
loss=tf.keras.losses.categorical_crossentropy,
optimizer=tf.train.GradientDescentOptimizer(learning_rate=0.05),
metrics=['accuracy'])
model.fit(
x_train,
y_train,
batch_size=BATCH_SIZE,
epochs=EPOCHS,
verbose=1,
validation_data=(x_test, y_test))
`

@amapic As it is working with cifar data, i am sure it is not a bug related to Tensorflow.
This is not Build/Installation or Bug/Performance issue. Please post this kind of support questions at Stackoverflow. There is a big community (including us) to support and learn from your questions. GitHub is mainly for addressing bugs in installation and performance.
When you post it in Stackoverflow, please provide atleast partial data of your to find the root-cause.
Thanks!


fix(LRUCache): copy using deepcopy, and ensure the get_all function always terminates

@aliu39 discovered that under certain circumstances a process can get stuck in an infinite loop.  Andrew fixed this by using deepcopy which prevents the infinite loop and fixes a bug where the LRU returns incorrect results.  Additionally, I've added a terminating loop in case there are any future bugs we've missed.
Closes: https://github.com/getsentry/sentry-python/issues/3862

We found out the process will hang if you copy a LRUCache, then call get_all() on the original. It works as expected if you call get_all() on clones. Will add some coverage in FF tests for this case (fork a scope then read the flags on the original scope)

Because of this, we had to disable flagpole evaluation tracking in sentry.
We want to quickly get this out in the next release so

we can restore flagpole tracking for sentry employees.
fix bugs for users of the SDK integration, which released to beta today.

This has been released with version 2.19.2: https://github.com/getsentry/sentry-python/blob/master/CHANGELOG.md#2192


Aligning some Invasiveness-related concepts with GISIN

There are some technical aspects worth reviewing here. I am bringing here the pertinence of making some Plinian Core terms totally equivalent to some GISIN terms, these are:
route.
impactTarget.
impact Mechanism.
See: https://github.com/pliniancore/documentation/wiki/InvasivenessAtomizedClass
Can we replace these Plinian Core native terms with their GISIN homonyms?

/cc @qgroom

Thanks Peter

It is not possible  to refer the GISIN terms "route", "impactTarget and "impactMechansim" from Plinian Core because their respective schemas  DispersalStatus, and Impact Status are defined but not implemented See here
Documentation has been updated accordingly


Dbeaver 23.0.2 CE not able to find files on windows when specifying a trust store location for SSL connections

Description
When I updated to version 23.0.2 I was not able to specify the sslTrustStoreLocation anymore, if you notice on the below image, when I paste the location of my sslTrustStore file, I keep getting FileNotFoundException, I tried with normal slash and back slash, if you notice in the image, it always tries to find the file using backslashes in the path even though I clearly pasted the path with regular slashes (/).
This didn't happen in the previous version I started to face this after I updated the product.

DBeaver Version
Community Edition 23.0.2
Operating System
Windows 10
Database and driver

Database: DB2 for Luw
Driver: Common IBM Data Server Driver for JDBC 11.5

Steps to reproduce

Try to open a new connection to DB2 for Luw
Fill the connection details por any DB2 database, host, port, database, user and password
Go to driver properties
In driver properties add a new property called ssTrustStoreLocation
put any valid location for a file on a windows machine
specify the property sslConnection equals to true
Click on Test Connection and you will see that the file is never found by dbeaver, even though the file does exist. Also note that no matter if you use normal slashes or backslashes dbeaver tries to use backslashes for the specified path. This didn't happen in previous versions of dbeaver.

Additional context
No response

Hello @lherreranetlink,
Could you please send me the log files? You can find instructions here: https://dbeaver.com/docs/wiki/Log-files/. Thank you.

There has been no update for a long time. If the issue is still actual for you - feel free to reopen it.

I'm having this issue as well.


Update reference to codegangsta/cli to urfave/cli

codegangsta/cli is now renamed to urfave/cli, and they recommend
to update old reference.

Notice: This is the library formerly known as github.com/codegangsta/cli -- Github will automatically redirect requests to this repository, but we recommend updating your references for clarity.

https://github.com/urfave/cli/blob/master/README.md
This patch will update reference to codegangsta/cli to urfave/cli.

Thanks!


unexpected client error status code: 302

Hello, when trying to bootstrap argocd to a cluster on GCP I get the following error:
rpc error: code = Unknown desc = unexpected client error: unexpected requesting "https://git.org.com/mygroup/myrepo.git/info/refs?service=git-upload-pack" status code: 302

I tried to deploy argocd on a local kind cluster to the same repo and it worked. What could be the problem? We are using gitlab.

+1

+1

+1

Running on Mac M2, with Rancher Desktop 1.8.1, Kubernetes 1.26.3, v1.29.0, Darwin 13.2.1 (arm64)

+1


Falling through textures in Havana

After opening the door of the secret stadium in Havana, it is impossible to drive / walk into the underground garage, the car / Tanner simply falls down through the texture of the ground. CD version is NTSC 1.0
Proof 1
Proof 2

Are you sure that it's 1.0 version?

duplicate of #40

Screen from the emulator console


Create: 2405-optimal-partition-of-string.cpp

File(s) Modified: 2405-optimal-partition-of-string.cpp
Language(s) Used: cpp
Submission URL: _https://leetcode.com/submissions/detail/1290513218/_

Important
Please make sure the file name is lowercase and a duplicate file does not already exist before merging.

@realsubodh Thanks for your contribution 👍


http -> https redirect bug

Welcome!

[X] Yes, I've searched similar issues on GitHub and didn't find any.
[X] Yes, I've searched similar issues on the Traefik community forum and didn't find any.

What did you do?
configured traefik on k8s with
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: redirect-http-https
  namespace: infra
spec:
  redirectScheme:
    permanent: true
    scheme: https

expected only http calls to be redirected from http -> https
What did you see instead?
If a client includes a port number in the http header (not sure if this is spec compliant, but some of our are), traefik sends a redirect for calls which are already using https. e.g.
curl -v -H "Host: atom-content-staging.qutics.com:443"  https://atom-content-staging.qutics.com/q
gets sent a  redirect, even though it can in with the correct scheme.
What version of Traefik are you using?
2.4.8 from docker
What is your environment & configuration?
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: redirect-http-https
  namespace: infra
spec:
  redirectScheme:
    permanent: true
    scheme: https

This is added to a regular networking/v1 ingress with an annotation.
If applicable, please paste the log output in DEBUG level
No response

Hello @tcolgate,
Could you please share your complete configuration (with the ingress) and the Traefik log from the beginning ?

Hi,
Our config is rather complicated (two instances of traefik process different ingresses, middlewares are shared). Most of it is irrelevant to the problem at hand though.
This is the main config file.
defaultEntryPoints = []
[entryPoints]
  [entryPoints.http-public]
  address = ":8880"
  [entryPoints.http-public.transport.lifecycle]
  graceTimeOut= "10s"
  requestAcceptGraceTimeOut= "30s"

  [entryPoints.https-public]
  address = ":443"
  [entryPoints.https-public.http.tls]
  [entryPoints.https-public.transport.lifecycle]
  graceTimeOut= "10s"
  requestAcceptGraceTimeOut= "30s"

[tracing.jaeger]
localAgentHostPort = "{{ env "KUBERNETES_HOST_IP" }}:6831"
samplingType = {{ metadata "samplingType" | quote }}
samplingParam = {{ metadata "samplingParam" }}

[web]
address = ":8080"
ReadOnly = true

[web.statistics]
RecentErrors = 10

[respondingTimeouts]
idleTimeout = "10s"

[api]
dashboard = true
insecure = true

[ping]

[metrics]
  [metrics.prometheus]

[accessLog]
format = "json"

[serversTransport]
  rootCAs = [ "/config/ca.crt" ]
  maxIdleConnsPerHost = 20

[providers.kubernetescrd]
allowcrossnamespace = true
ingressclass  = "traefik"
labelselector = "externalAccess in (public,traefikSharedResource)"

[providers.kubernetesingress]
ingressclass  = "traefik"
labelselector = "externalAccess in (public,traefikSharedResource)"

[log]
format = "json"
level = "fatal"

middlware is:
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  labels:
    externalAccess: traefikSharedResource
  name: redirect-http-https
  namespace: infra
spec:
  redirectScheme:
    permanent: true
    scheme: https

(there also a retry middleware, but which is also not relevant)
ingresses are configured using the regular kube ingress plus annotations
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: traefik
    traefik.ingress.kubernetes.io/router.middlewares: infra-retry@kubernetescrd,infra-redirect-http-https@kubernetescrd
  labels:
    externalAccess: public
  name: someapp
  namespace: default
spec:
  rules:
  - host: someapp.example.com
    http:
      paths:
      - backend:
          service:
            name: blocker
            port:
              name: http
        path: /debug
        pathType: Prefix
      - backend:
          service:
            name: someapp
            port:
              name: http
        path: /
        pathType: Prefix

I can't run the impacted instances with debug logging due to traffic volumes, but the actual bug is fairly self evident in the code. The link PR include a test which demonstrates the problem.
Some of our clients are setting Host: appname.example.com:443 in the host header. This gets passed (including the port) into req.Host, The test to determine if the old URL matches the new URL then fails because the new URL evaluates to "https://appname.example.com/", and the where the old name evaluates to "https://appname.example.com:443/".

Thanks.
Sorry, to be clearer, I should only have asked for the complete configuration for a minimal reproducible case. :smile:

It'll take me a while to put together a minimal config, but anything that's using the RedirectScheme middleware be impacted.

We need to be able to enable redirection on a per-ingress basis.

Just to understand better, why do you need per-ingress redirection instead of by Entrypoint ?

Because  we have existing customers that hit some of our older services with clients that wont respect redirects. We want stuff to redirect by default, but for some specific cases, we need to disable it and accept the traffic directly.
I've tested the fix that I've PR'd and it seems to work.

FWIW, the first thing that actually triggered this bug was a grpc client (part of thanos), that failed talking via traefik to another component that sits behind traefik.

@tcolgate the PR ( #8348 ) was reworked because we think the root cause of the problem was different from what you seemed to be describing. Could you please test it and let us know if it works for you?

@mpl the reworked PR looks fine, though It's not immediately obvious that the tests cover the problem cases (unless the contention is that the value of req.Host is irrelevant, but that code does do the host/port split on req.Host, so not setting req.Host somehow in the tests fees incomplete)

Closed by #8348.


Regex replace substitution

Please search existing issues to avoid creating duplicates.
Also please test using the latest insiders build to make sure your issue has not already been fixed: https://code.visualstudio.com/insiders/
I don't have time to check this unfortunately.
I have searched, but nothing specifies the regex substition, they are either lots of files or similar

VSCode Version: 1.33.1
OS Version: Windows 10

Steps to Reproduce:

notice how 4/5 have $1 and one is successful

Does this issue occur when all extensions are disabled?: Yes

looks to be an issue with the anchor $ in the search

This appears to be another CRLF related issue, I can reproduce

Will also probably be fixed by ripgrep's --crlf flag

Should be fixed in the next insiders

This is a pretty bad bug to a core feature, could we get a patch release for this?

This is still happening in latest insiders:
Version: 1.37.0-insider
Commit: 7e03eb1a1fce3beb6ccea5f3b8f828e01940be61
Date: 2019-07-05T18:25:58.865Z
Electron: 4.2.5
Chrome: 69.0.3497.128
Node.js: 10.11.0
V8: 6.9.427.31-electron.0
OS: Darwin x64 18.6.0

Think you need to use a capturing group in your regex.
example\['(.+)'\]

🤦♂


Textual .forward() override bug.

Hi! First of all, thanks a lot for a great example project!
However, I'm pretty sure I have found a bug. onnx_model.encode_text() produces results that are not consistent with regular model output.
There is a statement in .utils:
take features from the eot embedding (eot_token is the highest number in each sequence).
But ruclip uses eos_id = 3, which clearly is not the highest token in the sequence.
This change was clearly made after you released you repo, but there was no version bump.
So, I tried tracing the model with hardcode eos id and original .where line from ruclip's encode_text:
x = x[torch.arange(x.shape[0]), torch.where(text == 3)[1]] @ self.text_projection
and it worked. I will submit a pull request with a fix, if I have free time later, but for now just wanted you to know, that running the default version of your notebook produces incorrect text_encoding results

@Great-Frosty I also found this bug when trying the openclip model (https://github.com/mlfoundations/open_clip). The output from .encode_text() is different from openclip model.

@Great-Frosty were you ever able to take a look at this?
I'm unsure if this affects only ruclip or other clips as well.

Me neither. It got fixed in ruclip though
https://github.com/ai-forever/ru-clip/pull/19


Cant dynamically change the weights of a multi-output model loss functions during training

Please make sure that this is a feature request. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:feature_template
System information

TensorFlow version (you are using):2.4
Are you willing to contribute it (Yes/No):yes

Describe the feature and the current behavior/state.
I have a multi output model using the sub classing API where each output have a different loss function, and i want to change the loss functions weights dynamically in the training  process ( as shown : https://arxiv.org/pdf/1711.02257.pdf  and more papers).
It is reasonable to have the option to alter these weights using a callback
( very much like the learning rate scheduler https://www.tensorflow.org/api_docs/python/tf/keras/callbacks/LearningRateScheduler)
here is the call() method that my sub-class model uses:
NOTE : the ges_out is a classifier output
    def call(self, inputs, training=False, **kwargs):

        # intro block1
        x = self.conv1(inputs)
        x = self.spatial_dropout1(x, training=training)
        x = self.conv2(x)
        x = self.spatial_dropout2(x, training=training)
        # x = self.avgpool1(x)
        x = self.conv3(x)
        intro_out = self.spatial_dropout3(x, training=training)
        # intro_out = self.avgpool2(x)

        # classification block

        g = self.conv_ges1(intro_out)
        g = self.spatial_dropout_ges1(g)
        g = self.maxpool_ges1(g)
        g = self.conv_ges2(g)
        g = self.spatial_dropout_ges2(g)
        g = self.maxpool_ges2(g)
        g = self.flatten_ges(g)
        g = self.fc_ges1(g)
        g = self.dropout_ges1(g, training=training)
        g = self.fc_ges2(g)
        g = self.dropout_ges2(g, training=training)
        ges_out = self.fc_ges_out(g)

        # segmentation block
        s = self.conv_seg1(intro_out)
        s = self.spatial_dropout_seg1(s)
        s = self.avgpool_seg1(s)
        s = self.conv_seg2(s)
        s = self.spatial_dropout_seg2(s)
        s = self.avgpool_seg2(s)
        s = self.flatten_seg(s)
        s = self.fc_seg1(s)
        s = self.dropout_seg1(s, training=training)
        seg_out = self.fc_seg_out(s)

        return {"gesture_out": ges_out, "segmentation_out": seg_out}

and here is how i use model_compile(...):
def model_compile_and_summary():
      metrics = {"gesture_out": ["accuracy"], "segmentation_out": ["mean_squared_error"]}
      loss_functions = {"gesture_out": keras.losses.CategoricalCrossentropy(from_logits=True),
                            "segmentation_out": keras.losses.MeanAbsoluteError()}
      loss_weights = {"gesture_out": 1.0, "segmentation_out": 0.0001}
      optimizer_name = "Adam"
      optimizer = get_optimizer_from_name(optimizer_name)
      MyModel.model.compile(optimizer=optimizer, loss=loss_functions, metrics=metrics, loss_weights=loss_weights)

Will this change the current API? How?
the current API have the option to pass constant loss functions weights in the model.compile(...) method.
i assigned each output to each loss using a dictionary
Who will benefit with this feature?
i think all the researchers/engineers  that currently deal with multi-task learning.

Hi @Ram-WD!
Please post this feature request on keras-team/keras repo too .
To know more see;
https://discuss.tensorflow.org/t/keras-project-moved-to-new-repository-in-https-github-com-keras-team-keras/1999 . Thanks!


[Chore]: Create e2e tests that validate user role management for ServiceAccounts

Background
#2279 added the ability to add and remove roles from service accounts and to log in as those service accounts. However, we only have unit tests for this functionality and problems were discovered during acceptance that the unit tests didn't catch.
Action to take
Add tests to our current e2e suite that do the following

As an admin user, make a service account user into an org manager for an org
As an admin user, make the service account user into a space developer for a space
As the service account user, create an app in the space
As an admin user, remove the org and space roles from the service account user
Verify that the roles were removed (e.g. attempt to create or update an app in the space as the user, find a way to list the org and space users if it exists and verify the user is missing)

Impact
These new e2e tests will ensure that we don't have regressions in our support for e2e users.
Dev Notes
You may need to change the scripts that set up test users and service accounts to run the e2e tests. This may also affect korifi-ci

Consider adding a test that it's possible to add a role for 2 different service accounts with the same name, but different namespaces. See https://github.com/cloudfoundry/korifi/issues/2385 for details

To support these new tests, we'll likely need to change CI in the same way we did to support the crd tests. Here are some places to look:

Storing the user info for EKS: https://github.com/cloudfoundry/korifi-ci/blob/2b2d03b17ed9302f7df8b7e2b6e6461040c73385/pipelines/scripts/set-account-env-vars.sh#L29-L32
Creating the user and exporting the info for non-EKS: https://github.com/cloudfoundry/korifi/blob/b7de3ee4fe54ee7d617bdbbe3727bb381795888b/scripts/account-creation.sh#L67-L73

I dug around but couldn't find where the terraform variables we use for EKS are coming from. You may need to ask the EMEA folks

EKS setup is in the cf-k8s-secrets repo, in particular:

https://github.com/cloudfoundry/cf-k8s-secrets/blob/main/ci-deployment/eks-e2e/cf-users.tf
https://github.com/cloudfoundry/cf-k8s-secrets/blob/main/ci-deployment/eks-e2e/outputs.tf


Loading Google Maps JavaScript API needs a callback

Is there a way to pass a parameter for the URL generated for Google Maps to include a callback method? I am getting this in console not:
Loading the Google Maps JavaScript API without a callback is not supported: https://developers.google.com/maps/documentation/javascript/url-params#required_parameters

Hey mate, are you just trying to fix the error or do you need an additional callback appended to the url?
If it's to fix the error, I cannot replicate it and it appears that it may be an issue with your API key you have provided:

Your Google Maps may not be functioning for a number of reasons, but the most frequent one is that your Google Map API key may not be set or is set wrongly. You must register your app project on the Google Cloud Platform Console in order to access the Google Maps JavaScript API. Once you have your Google API key, you can add it to your app.
Both errors and warnings are returned by the Maps JavaScript API. An error means that there was a serious problem loading the Maps JavaScript API. The API won't function on your page since it can't load correctly. An additional notification regarding the loading of the Maps JavaScript API is known as a caution. The warning identifies probable problems in your code that load the Maps JavaScript API or explains the causes of an error. The API will function properly on that page if you just get warnings and no errors. However, we advise you to address these potential problems as well.

Source
Make sure you have included your Google Key as described in the docs, and that your Google Key is authorised for the IP address you're calling it from. This is what my key page looks like in the Credentials page.

Hi there, since I'm using your tool, I would hope you understand that I am familiar with Google API integrations and setup.
Yes, I have a Google Key with the set appropriate APIs enabled, and the IPs setup, however those are all 100% completely irrelevant to my statement.
If you look here at Googles URL Docs, you'll see there are 2 required parameters, one for the API key, and one for the callback.
In my initial issue message, the console message was in reference to the second required parameter, the function callback.
If you inspect this page, you'll see generally in the URL, you'll have ?key=XYZ&callback=initMap where the initMap can be whatever method instantiates your Google Map. I'm asking if there is a way with your plugin to provide this callback function.

Ah ok. So you'd like your own callback to be run alongside initMap?
It's not in there now but shouldn't be too hard to add in. I'll try find some time to explore that.

Yeah, I'm just looking for a way to pass in a callback function name to it.
A nice to have, would be able to pass in other parameters, like language, so that the maps language can be changed if using a site that it utilizing localization

Updated the way the config is loaded to the way I do in the Vue 3 version and added a callback property in the options that can be used.
Check out Configuration and let me know if it makes sense (and meets your need). Thanks for the suggestion.

Ayyy, this is perfect! This did the trick for me. Thank you!


Update user types for Screener

It is annoying that the types are wrong on Screener, I find myself looking at the source code to determine the types of the Screener class so this is a fix

what do you mean the "types are wrong"?
What does it mean to "determine the types of the Screener class"?
How can we improve the documentation for Screener to avoid "looking at the source code"?

1. what do you mean the "types are wrong"?

The return types on the Screener are wrong (for example dict not dict_keys on predefined_bodies which seems trivial but the methods are different on them) etc

2. What does it mean to "determine the types of the Screener class"?

I go into the source code to work out what types different properties are as when I run code (which would be correct with the types) it fails (due to incorrect types provided)
I also have to go looking for what string literals are allowed in set_predefined_body when they are hard coded in and most type checkers can prompt you to which ones to use.

3. How can we improve the documentation for Screener to avoid "looking at the source code"?

Mostly fixed it with this PR. Take a look if you want.
I also made the the types run faster and be more runtime safe: by putting the type in a string it only has to construct a string and not the full class and it also means it's easier to fix circular import errors or the class not actually existing for some reason as some of them could be moved under t.TYPE_CHECKING

@ValueRaider What do you think?

How can we improve the documentation for Screener

Improve how user sets up a screener query with a simpler intermediate form:
from dataclasses import dataclass
from typing import List, Union, Any

@dataclass
class Query:
    operator: str
    operands: List[Union['Query', Any]]

    def to_dict(self) -> dict:
        return {
            "operator": self.operator,
            "operands": [op.to_dict() if isinstance(op, Query) else op
                         for op in self.operands]}

def value_in(key: str, *values):
   return Query("or", [Query("eq", [key, v]) for v in values])

query = Query(
    "and",
    [
        value_in("exchange", "NMS", "NYQ"),
        Query("lt", ["epsgrowth.lasttwelvemonths", 15])
    ]
)

Should shorten the screener html page by a lot.

Should I also add #154 into this one?

ok

@ValueRaider any other predefined keys you know off? Still don't know where you found that 52 week gain

Also why is Query and EquityQuery different? EquityQuery doesn't even have region etc.

@ValueRaider any other predefined keys you know off? Still don't know where you found that 52 week gain

I just traversed the gainers webpage with Firefox network inspector open.

@ValueRaider Still need to add proper logging and docs etc, but what do you think of this.
I am getting the error {"finance":{"result":null,"error":{"code":"internal-error","description":"sort field is not a field: fundnetassets"}}} returned by yahoo. I don't understand why as this part is the same as the original and that works fine


Expand Interface FormFactor / Types

Might be easiest to make this an admin-configured lookup table.
Rack tables supports dozens of interface types, to the point at which it gets excessively dumb.  But being able to define single or multimode for SFPs, and adding Fiber Channel (in 2/4/8/16/32Gb flavors) would probably be a decent start...

Makes sense.  When you say FormFactor do you also mean XFP, SFP+, QSFP?

I mean FormFactor as Netbox labels it :)

+1
We have dozens of FibreChannel and Infiniband switches, hosts, and storage systems. Being able to track these interfaces would be extremely helpful. I'm less concerned with the interface speed as much as I am that it's not an ethernet interface.  If this can't be configurable by the admin, the form factors could be "FibreChannel (SFP)", "Infiniband (CX4)", and "Infiniband (QSFP)". This would keep the drop-down from being too unwieldy.

another +1, this would be great. configurable types/form factors would be great as well.

Interface form factors are hard-coded because only a limited set exist in the world. For instance, there is a well-defined set of Ethernet interfaces as standardized by IEEE. Allowing users to create their own pseudo-types is inviting chaos.
Of course, I'm happy to create additional form factors, but let's compile a list of exactly what they should be called.

Would it be possible to add a Form Factor for Serial, or some other descriptor for low speed interfaces?
Could just be generic, like "Serial".  Or specific like, "T1, DS3"  As these low speed interfaces are becoming less and less common probably just "Serial" would suffice.

I think you'll find that form factors are almost infinitely variable depending on how far back in the past you go and what way you stretch the definition of form factor.
In some cases you could argue that some of these share the same form factor (ATM, FDDI, POS) because they share the same interface (SC fiber usually) and the primary difference in speeds and even interface type is the transceiver used.  I've seen Cisco cards converted from ATM to POS by desoldering the transceiver and replacing them with different ones.
But, if you make that argument then everything that rides over an SFP, or an RJ45 is the same form factor.   I can't think of any others right now.

There's no need to include obsolete form factors, just the ones that are likely to be present in modern networks. #326 and #361 have some additional examples.

Looking around my server room the network "sockets" (something I plug into) on the equipment are:
RJ45
SFP/SFP+
QSFP
XFP
CX4
GBIC
Add a few other ports (i.e. CFP, CPAK, XENPAK, and perhaps just "Fiber" for non-transceiver ports) and you've got a solid top-10 of form factors.
I've combined SFP & SFP+ here as the same form factor to reduce bloat. Otherwise we'd also have to consider things like SFP28, QSFP+, QSFP28, CFP2, etc... as their adoption grows.

I've added some additional form factors in 76b9a1c. If anyone has any others they'd like to add, please comment here with the group and specific name you'd use for each.

@jeremystretch Looks like a good approach compared to how we were originally discussing to handle form factors.

I do not even use them, but the only modular that I can see people using are the X2s & XENPAKs (both 10G) .

Yeah, I was considering XENPAK and X2, but are those around any more? I suppose the same could be asked of GBICs.

Yes ha, unfortunately, they are.

Would it be possible to add "Unknown" or "Unspecified"?  Those might cover the rare cases where someone is doing something weird.  It's better than having to enter incorrect data if what you have is not close to any of the choices.

Don't forget to add SFP28 if you are adding QSFP28.

Like @peelman said, please consider adding SFP Fibre Channel interfaces (2,4,8,16 and 32G). Thanks

Consider adding CXP-100G

While the "form factors" have been updated, they are still almost exclusively in support of Ethernet standards and not other technologies. Eliminating the terminology "1GE", "10GE", and so on from the "modular" section would correct this and allow FibreChannel, Infiniband, and others to use existing "form factors" without a discrepancy in the stated terminology. As others have suggested, an "other" category would be helpful where an option is not available (i.e. CX4 or X2)

Eliminating the terminology "1GE", "10GE", and so on from the "modular" section would correct this and allow FibreChannel, Infiniband, and others to use existing "form factors" without a discrepancy in the stated terminology.

Infiniband doesn't come in 1G though, it comes in 2.5, 10 and 30.  FC also has different classes of throughput.  If anything, it would be better to add "SFP (2.5IB)" "SFP (2FC)" to differentiate them, however that means just adding a whole bunch of extra classes.
I am sure there is a better way.  Also, remember not everyone will be happen no matter what end result we get here.

Coax and RJ11 would be nice to have. Agree with original comment, this list ought to be user-configurable. That seems the only possible way to make everyone happy judging by this thread and the related issues closed in favor of this one. :)

Some Dell switches use HDMI for stacking, and there are are least two other types of Dell switches with completely sifferent stack ports I know of with a vendor specific connector. Amazing. So how about we just have an "Other stack port" as well for eventualities.
We need External SAS connector types for tracking SAN cable plant.
Also, eSATA, USB1.1/2 and USB3 for things like external hard drives, console cables.
I can envision uses for HDMI, phono and 3.5mm, and telephony jacks (RJ11 and british standard) as we manage our office cable plant in

How does this look as far as Ethernet and FibreChannel are concerned?
IFACE_FF_CHOICES = [
    [
        'Virtual interfaces',
        [
            [IFACE_FF_VIRTUAL, 'Virtual'],
        ]
    ],
    [
        'Ethernet (fixed)',
        [
            [IFACE_FF_100ME_FIXED, '100BASE-TX (10/100M)'],
            [IFACE_FF_1GE_FIXED, '1000BASE-T (1GE)'],
            [IFACE_FF_10GE_FIXED, '10GBASE-T (10GE)'],
        ]
    ],
    [
        'Ethernet (modular)',
        [
            [IFACE_FF_1GE_GBIC, '1GE (GBIC)'],
            [IFACE_FF_1GE_SFP, '1GE (SFP)'],
            [IFACE_FF_10GE_SFP_PLUS, '10GE (SFP+)'],
            [IFACE_FF_10GE_XFP, '10GE (XFP)'],
            [IFACE_FF_10GE_XENPAK, '10GE (XENPAK)'],
            [IFACE_FF_10GE_X2, '10GE (X2)'],
            [IFACE_FF_25GE_SFP28, '25GE (SFP28)'],
            [IFACE_FF_40GE_QSFP_PLUS, '40GE (QSFP+)'],
            [IFACE_FF_100GE_QSFP28, '100GE (QSFP28)'],
            [IFACE_FF_100GE_CFP, '100GE (CFP)'],
        ]
    ],
    [
        'Fibrechannel',
        [
            [IFACE_FF_1GFC_SFP, '1G (SFP)'],
            [IFACE_FF_2GFC_SFP, '2G (SFP)'],
            [IFACE_FF_4GFC_SFP, '4G (SFP)'],
            [IFACE_FF_8GFC_SFP_PLUS, '8G (SFP+)'],
            [IFACE_FF_8GFC_XFP, '8G (XFP)'],
            [IFACE_FF_10GFC_SFP_PLUS, '10G (SFP+)'],
            [IFACE_FF_10GFC_XFP, '10G (XFP)'],
            [IFACE_FF_16GFC_SFP_PLUS, '16G (SFP+)'],
            [IFACE_FF_16GFC_XFP, '16G (XFP)'],
        ]
    ],
    [
        'Serial',
        [
            [IFACE_FF_T1, 'T1 (1.544 Mbps)'],
            [IFACE_FF_E1, 'E1 (2.048 Mbps)'],
            [IFACE_FF_T3, 'T3 (45 Mbps)'],
            [IFACE_FF_E3, 'E3 (34 Mbps)'],
        ]
    ],
    [
        'Stacking',
        [
            [IFACE_FF_STACKWISE, 'Cisco StackWise'],
            [IFACE_FF_STACKWISE_PLUS, 'Cisco StackWise Plus'],
        ]
    ],
    [
        'Other',
        [
            [IFACE_FF_OTHER, 'Other'],
        ]
    ],
]

[
'Fibrechannel',
[
[IFACE_FF_1GFC_SFP, '1G (SFP)'],
[IFACE_FF_2GFC_SFP, '2G (SFP)'],
[IFACE_FF_4GFC_SFP, '4G (SFP)'],
[IFACE_FF_8GFC_SFP_PLUS, '8G (SFP+)'],
[IFACE_FF_8GFC_XFP, '8G (XFP)'],
[IFACE_FF_10GFC_SFP_PLUS, '10G (SFP+)'],
[IFACE_FF_10GFC_XFP, '10G (XFP)'],
[IFACE_FF_16GFC_SFP_PLUS, '16G (SFP+)'],
[IFACE_FF_16GFC_XFP, '16G (XFP)'],
]
],

FibreChannel is all SFP/SFP+ and in 2/4/6/8/16 speeds. The XFP and 10G items are not relevant to FibreChannel.

@jeremystretch have you considered locking IFACE_FF_CHOICES to a static list,  then allowing a user defined list read in from a config file (TOML?).
You could then do a merge of the two lists to present the form fields.
As others have mentioned there are other "modern" interface types not being considered.  HDMI, SDI (my use case for video networks),  USB, eSATA,  etc.
Hard coding all of these options, isn't desirable.   But not letting users add them, is limiting as well.  Unless you're drawing a hard line, and saying "network" interfaces only?

@jamesboswell It makes no sense to allow users to define their own form factors, since a very finite set exist in the real world. And yes, NetBox is intentionally limited to only including network interface types.

This is what I'm including with v1.6:
IFACE_FF_CHOICES = [
    [
        'Virtual interfaces',
        [
            [IFACE_FF_VIRTUAL, 'Virtual'],
        ]
    ],
    [
        'Ethernet (fixed)',
        [
            [IFACE_FF_100ME_FIXED, '100BASE-TX (10/100ME)'],
            [IFACE_FF_1GE_FIXED, '1000BASE-T (1GE)'],
            [IFACE_FF_10GE_FIXED, '10GBASE-T (10GE)'],
        ]
    ],
    [
        'Ethernet (modular)',
        [
            [IFACE_FF_1GE_GBIC, 'GBIC (1GE)'],
            [IFACE_FF_1GE_SFP, 'SFP (1GE)'],
            [IFACE_FF_10GE_SFP_PLUS, 'SFP+ (10GE)'],
            [IFACE_FF_10GE_XFP, 'XFP (10GE)'],
            [IFACE_FF_10GE_XENPAK, 'XENPAK (10GE)'],
            [IFACE_FF_10GE_X2, 'X2 (10GE)'],
            [IFACE_FF_25GE_SFP28, 'SFP28 (25GE)'],
            [IFACE_FF_40GE_QSFP_PLUS, 'QSFP+ (40GE)'],
            [IFACE_FF_100GE_CFP, 'CFP (100GE)'],
            [IFACE_FF_100GE_QSFP28, 'QSFP28 (100GE)'],
        ]
    ],
    [
        'FibreChannel',
        [
            [IFACE_FF_1GFC_SFP, 'SFP (1GFC)'],
            [IFACE_FF_2GFC_SFP, 'SFP (2GFC)'],
            [IFACE_FF_4GFC_SFP, 'SFP (4GFC)'],
            [IFACE_FF_8GFC_SFP_PLUS, 'SFP+ (8GFC)'],
            [IFACE_FF_16GFC_SFP_PLUS, 'SFP+ (16GFC)'],
        ]
    ],
    [
        'Serial',
        [
            [IFACE_FF_T1, 'T1 (1.544 Mbps)'],
            [IFACE_FF_E1, 'E1 (2.048 Mbps)'],
            [IFACE_FF_T3, 'T3 (45 Mbps)'],
            [IFACE_FF_E3, 'E3 (34 Mbps)'],
        ]
    ],
    [
        'Stacking',
        [
            [IFACE_FF_STACKWISE, 'Cisco StackWise'],
            [IFACE_FF_STACKWISE_PLUS, 'Cisco StackWise Plus'],
        ]
    ],
    [
        'Other',
        [
            [IFACE_FF_OTHER, 'Other'],
        ]
    ],
]

I'm going to close out this issue now as it's been dragging on for months. If anyone would like to request additional interface types, please do so by opening a new issue and enumerating the specific interface form factors you would like to see added.

The way this is currently done seems like a recipe for it to just balloon out of control but still not support how users actually need to use the field.
I would be strongly in favour of turning this into a configurable table. I am mostly behind NetBox's strict adherence to its data model, but this is not a data model question, it's about the data itself. Other interface types fit the data model just fine, and yes, they do exist. Is there any reasoning behind why NetBox should only be used for IP network (and console, and power, and storage, and stacking) but not things like KVM or POTS or video distribution or sensor cabling or RF or timecode or RS485 or $proprietary_whatever or whatever other point-to-point interconnect you might find in a datacentre? All these use cases fit the model just fine. Many of them are present in most DCs, which means they need fragmented DCIM or to use $something_else, just because of this. All this seems to achieve is a list that will continually grow but will never be quite right, a built-in expiry date for the code, artificially making NetBox unsuitable (or users abusing what form factors do exist to mean something other than what they say). This is not a good source of truth, nor is it good for usability to have a bunch of locally-unused options there (serial? WTF?).
Users should be able to control it, and configure NetBox to handle their 'weird' DCIM needs, or simply adjust the list to better suit their usage. I don't even see SONET in the list, 2.5GBASE-T and 5GBASE-T are coming RSN, it'd be nice to remove clutter I will never need, and it would be REALLY nice to differentiate between 10GBASE-SR and 10GBASE-LR. It doesn't break the model, it just lets the user define how granular they want to be and what interfaces are actually present in their system.
Form factor also seems an inappropriate title. It's currently sort of a blend of form factor and a short list of wire protocols. If there's no flexibility, I'd prefer sticking to a strict definition of form factor and get rid of the duplicates (10/100/1000base-T is all the same form factor (usually also T/E carrier), likewise SFP(+) for Ethernet and FC). Perhaps the issue here is conflating the two, and NetBox should consider the RackTables model of having separate physical form factor and media/protocol types.
/2c

Is there any reasoning behind why NetBox should only be used for IP network (and console, and power, and storage, and stacking) but not things like KVM or POTS or video distribution or sensor cabling or RF or timecode or RS485 or $proprietary_whatever or whatever other point-to-point interconnect you might find in a datacentre?

These concepts are simply out of scope for NetBox. Sorry, but we have to draw the line somewhere. NetBox is necessarily limited to network, console, and power in the interest of maintainability. If these are hard requirements, NetBox simply isn't the right choice.

This is not a good source of truth, nor is it good for usability to have a bunch of locally-unused options there

Here's my counterpoint: Allowing users to create arbitrary physical interface types, of which only a limited set exist in the real world, will inevitably lead to chaos. All it takes is for one person to create a type named FastEthernet (100 Mbps) and someone else to create a type named 100Mbps Ethernet (because they were in a hurry and didn't see the first one) and now you've got partitioned data. Using a fixed set of choices, while potentially limiting, avoids this problem.

I don't even see SONET in the list, 2.5GBASE-T and 5GBASE-T are coming

Happy to add these if someone submits a feature request. We put out new releases of NetBox far more frequently than someone releases a new interface type.

it would be REALLY nice to differentiate between 10GBASE-SR and 10GBASE-LR

This isn't dependent on the device, but rather the type of transceiver installed within the device. It's inevitably going to vary from one device to another, so it doesn't make sense to specify as part of the device type. In the future we may be able to map transceiver details (gleaned from inventory data) to interfaces but that's still a ways off.

These concepts are simply out of scope for NetBox. Sorry, but we have to draw the line somewhere. NetBox is necessarily limited to network, console, and power in the interest of maintainability. If these are hard requirements, NetBox simply isn't the right choice.

Yeah, I understand that, but artificially limiting it doesn't seem useful. Out of scope doesn't mean it needs to be excluded; tools should be as flexible as possible as long as it's not compromising their intended purpose. Especially when most of us probably have some oddball things that fit this description that don't really have a good tracking home of their own.

Here's my counterpoint: Allowing users to create arbitrary physical interface types, of which only a limited set exist in the real world, will inevitably lead to chaos. All it takes is for one person to create a type named FastEthernet (100 Mbps) and someone else to create a type named 100Mbps Ethernet (because they were in a hurry and didn't see the first one) and now you've got partitioned data. Using a fixed set of choices, while potentially limiting, avoids this problem.

This is what permissions and operational guidelines are for. This is not a problem for which the solution needs to be baked into the software. Your concern can easily be addressed by not allowing normal users to edit device types (I assume this would live in the Django admin panel, so this would be the default situation), placing it in the configuration file, or simply with administrative 'sanctions' for screwing it up. You can make the exact same argument about Device/Circuit Types. It's even worse for Roles, as they are pretty much completely up to the organization to define how they are used.

Happy to add these if someone submits a feature request. We put out new releases of NetBox far more frequently than someone releases a new interface type.

The point was more about bloating of the list with a bunch of stuff most users aren't going to use but some will need and happen to be interface types you deem legitimate. Obviously nothing added can ever be removed in this model...the list already contains some 'garbage' IMO.

This isn't dependent on the device, but rather the type of transceiver installed within the device.

Fair enough. I don't really care about modelling the pluggables themselves, but I do want to know what media type a connection is using. Setting this manually on the port, either with or separate from the form factor is probably preferable to me than having to model a module, but either way works, I just want the functionality.

Out of scope doesn't mean it needs to be excluded; tools should be as flexible as possible as long as it's not compromising their intended purpose.

The concepts you mentioned would need their own set of models, for the same reasons console, power, and network are each discrete sets. Thus, in this context, out of scope equates to not supported.

This is what permissions and operational guidelines are for.

I have no control over how users assign permissions in NetBox. (I don't have any data to back this up, but I suspect that many people only use the superuser account.) This is why it needs to be baked into the code. Ensuring that users cannot create illegitimate form factors is part of the value NetBox provides.

You can make the exact same argument about Device/Circuit Types. It's even worse for Roles, as they are pretty much completely up to the organization to define how they are used.

Totally different concepts. Prefix/VLAN roles are entirely arbitrary organizational constructs, whereas interface form factors are discrete and formally defined.

The point was more about bloating of the list with a bunch of stuff most users aren't going to use but some will need and happen to be interface types you deem legitimate.

Nine months into the public release, there are a total of 29 interface types currently supported. I don't feel that this is at all excessive.

the list already contains some 'garbage' IMO

I suspect the many people who use those form factors in their network would disagree.

Also you do for now. Making this user configurable gives this project runway if you move on to a new gig that isn't so supportive, or lose interest and the project doesn't get picked up by someone else.

Such is the nature of any open source project. If this is a risk you're not comfortable with, I'd suggest sticking with commercial products (and hoping the selected company doesn't fold).

I don't really care about modelling the pluggables themselves, but I do want to know what media type a connection is using.

I suspect we'll see more work on this front once #20 gets under way.
Bottom line, the current scheme works well for the functions NetBox aims to fill. While I'm happy to add additional form factors if there's demand for them, it will not be transitioning to a dynamic model, for the reasons I've explained in previous posts. If that does not fit your specific needs, there are plenty of other products to choose from.

Thus, in this context, out of scope equates to not supported.

No, apparently it means 'actively blocked', which is quite a different thing. I don't think this will stop me from using NetBox, but it certainly grinds my gears. 'My way or the highway' is a really user-hostile attitude when it comes to things like this, especially when your primary argument can be easily addressed with non-programmatic solutions, if the end-user decides it is in fact a problem. If someone is actively seeking out the 'interface types' table in the Django admin UI, they are probably doing so for a reason. It's not like I'm advocating for this to be a free-form text field with not referential checks.

I suspect the many people who use those form factors in their network would disagree.

This is pretty much my point. There is no list that perfectly suits every user, there will either be useless clutter or missing items in almost every deployment, even if I give you that this field should only ever contain things formally defined as some vague idea of a valid network interface (even if it's not an IP interface, despite NetBox allowing that binding).

it will not be transitioning to a dynamic model, for the reasons I've explained in previous posts. If that does not fit your specific needs, there are plenty of other products to choose from.

Understood, I don't think your argument is sound, and I doubt I'm the only one that likes flexibility where it's free, but I'm not going to push further. If the chosen solution for #20 includes media type on the interfaces, we'll still disagree on this but I'll have what I need 😃 .

'My way or the highway' is a really user-hostile attitude when it comes to things like this, especially when your primary argument can be easily addressed with non-programmatic solutions, if the end-user decides it is in fact a problem.

Since you seem to think I'm just being stubborn, let's walk through the data model (which you can reference in netbox/dcim/models.py). The Interface model represents a network interface. Connections between Interfaces are represented by an InterfaceConnection. (An interim model is necessary because unlike console and power connections, interface connections are bidirectional.) Interfaces can be designated as "management only" and they can be grouped by a parent LAG interface. Additionally, multiple IPAddresses can be assigned to an Interface. All of this is and will be true for all form factors.
Please provide an example of a form factor you would add if given the ability that would make sense in this model.

I will throw my two cents in here (though it is probably worth less).
@jeremystretch, I find the argument about partitioned data limited at best, if not invalid because in almost every other context in netbox, this issue exists in the same way you described. I could very easily create two device types for the same device model (say ex-4300 and Juniper-EX4300) because I am not paying attention.
I tend to agree that you seem to be artificially limiting the scope of netbox in the name of overly strict constraints. Obviously that is a subjective statement, and don't get me wrong I absolutely respect what you have done with netbox and thank you for your work and dedication; it truly is an amazing product!
But I think you are overly worried about scope creep in certain areas. As the developer you don't have to account for every single use case (think network vendor here) but you can facilitate a greater number of user's wants by opening up the model a bit. There will certainly always be supported use cases and scenarios and for good reason but a little extensibility never hurt anyone. And let's face it, the vast majority of netbox users are somewhat technically inclined, and are more likely to understand the implications of a given use case (no data to really back that claim up though).
Again, just you thoughts, take them for what they are worth.

@lampwins Incidentally, my intention early on was to publish a library of device types in JSON format so that people could simply import the ones they wanted, but it never got off the ground (see #451). Still, not being able to prevent that problem doesn't mean we shouldn't prevent this one.

I tend to agree that you seem to be artificially limiting the scope of netbox in the name of overly strict constraints.

All I'm saying is that the network interface model should not be used to represent things that are not network interfaces. It's the same reason we have different models for console and power connections. Is that unreasonable? The suggestion here is that users should be able to repurpose the network interface model to represent whatever they want. Can you see why that's a bad idea?

@jeremystretch so I agree it should be for physical interfaces, but I think the argument being made here is that there exist more interface form factors than one person could possibly imagine by themselves.

The suggestion here is that users should be able to repurpose the network interface model to represent whatever they want.

Not whatever they want, rather any type of physical interface they want. Just to give another arbitrary example, what if I wanted to model the physical GPS connections on my router? That is the intent, and yes a user could potentially represent a "non-physical" interface with this but i argue this is there prerogative and should not be real concern as no actual technical constraints have been violated.
By your method, I would need to open an issue, defend my stance and that this type of interface really exists, and wait for an update (granted you are doing great job of getting the releases out often). If that is something you want to do in the long run, more power to you but wouldn't it be better if I could just do it myself and save everyone else the trouble?
I understand your argument is that this will open the door to "chaos" but my counter is why does it matter and how much chaos is there really going to be by allowing users to do this?
I don't mean to exacerbate this argument as I think everyone has already done so above so I will leave this here. Besides, I think there are more pressings issues that everyone involved could get more value out of anyway. :)

So let's say we create a GPS interface on a device. Here are the problems we run into:

We have no way of signifying whether this interface produces or consumes GPS signal.
This interface can be assigned an IP address in NetBox, but not in the real world.
This interface can be designated as a management interface to the device.

Do you see how a GPS interface differs in function from a network interface? This is why we're not going to use the same model for both. It's the same reason we use different models for console and power: There's other data specific to each class of connection that we might want to track. (Granted, we don't today, but will in the near future.) In fact, the console model would be a much closer fit for a GPS connection. The only reason people fixate on the interface model is because it tracks form factor and they see an opportunity for a quick hack to get what they want.

@jeremystretch  If the network interface model isn't the right fit, then that's fine.   The ask from me, and others above is there is a need for other types of "network" connections.  How that is data modeled is up for debate.
From a user perspective,  there is a large desire to be able to track other types of interfaces.   GPS as mentioned above is one example.
In my use case it's a lot of Serial Digital Interface connections on devices that also have true IP addresses GigE connections as well.  Ingesting baseband video for example and outputting a IP multicast  stream.    These aren't your standard routers and servers,  but they are mission critical data connections
It would be really nice to be able to at least create a SDI  port, assign it a port number and map it's relationship to another device/entity.    In my case,  I don't need IP assignment, etc.   So maybe console data model is correct.   If I could add a custom field, that would be icing on the cake.

@jamesboswell I understand, and it's probably a perfectly valid use case. But the fact is that NetBox simply does not support that type of connection today. It probably will at some point in the future, but it doesn't right now. What some users fail to recognize is that the absence of a feature does not justify trying to shoehorn data into a different feature to try and make it work. If this is a hard requirement for someone, as I've said before, NetBox just isn't the right choice for them yet. If it's merely a nice-to-have, one can certainly use NetBox for everything it does today and simply abstain from modeling unsupported connections until the data model has been extended appropriately.
We've beat the matter to death at this point, so I'm going to ask that people refrain from continuing to comment in this long-closed issue.

My specific use case for this is that we have some DWDM MUX/DEMUX's that are completely passive, and may have many types of traffic going through them. I'd love it if we had generic physical connector options (Such as SC, LC, ST, maybe add the simplex/duplex options) since on my mux that's all that really exists. It isn't Ethernet, or Fiberchannel until you plug something in that is one of those.
Right now they're all just "Virtual" but it would be very nice to see the physical connectors properly labeled.

Adding all of these would be great to have support for 100BASE-SX and -FX fiberglas connections.
https://en.wikipedia.org/wiki/Fast_Ethernet


Allow users to configure behavior of ReadSourceData when missing an RR

This PR introduces a REQUIRE_RR environment variable to ReadSourceData. When REQUIRE_RR is True, the default value, and an RR cannot be found for an incoming eICR ReadSourceData will not trigger the ingestion pipeline, but instead log an error and raise an exception. When REQUIRE_RR is False and an RR cannot be found ReadSourceData will simply log a warning explaining that an RR wasn't found and then trigger the pipeline with only data from the eICR.

Uploaded an eICR without an RR and observed the expected behavior.


Parquet source format doesn't work in cluster environment.

parquet format never copies to local. Causes null pointer exception.


Adds a test to check similarity between audit logs generated from REST layer and TRANSPORT layer

Issues Resolved

Resolves #3099

Testing
Integration testing
Check List

[x] New functionality includes testing
[x] New functionality has been documented
[x] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Backport is currently blocked by https://github.com/opensearch-project/security/issues/3230

@DarshitChanpura Can you attach this to the pull request that will incorporate all of the AuthZ in the Rest layer changes into the 2.x branch?


⚠ Jenkins Plugins Artifactory has degraded performance

In 8bc44e5, Jenkins Plugins Artifactory (https://get.jenkins.io/plugins/artifactory/) experienced degraded performance:

HTTP code: 200
Response time: 1728 ms

Resolved: Jenkins Plugins Artifactory performance has improved in 40907f3 after 24 minutes.