Re-consider Qt

After doing some UI work and looking over the existing work that is left considering Qt again might actually be the correct way forward.
One of the reasons of switch from Qt was slow compile times and large library. Those things won't go away but as the project is growing it may become less of an issue.
The other reason for switching was poor performance. I have been pointed out that the current implementation we had of the HexView could have been done much faster.
So, while annoying (and spending quite a bit of the on the imgui UI) perhaps doing some prototyping and investigating, looking into Rust bindings for Q.

I have been pointed out that the current implementation we had of the HexView could have been done much faster.

By "faster" you mean that it could be implemented faster or it could work faster (consuming less CPU)?

Yes. The problem was that it was quite slow (CPU wise in Qt) I'm very convinced it can be implemented with less code in Qt.

Wrote a long post about this and UI's in general over here http://prodbg.com/rethinking-ui/

You could also have a look at https://github.com/libnui/nui3/ . I used it to write a GUI for LLDB as well (more a proof of concept than anything useful): https://github.com/meeloo/xspray
NUI works fine on macOS/iOS and used to work on linux and windows (I haven't had the time to support those ports for some years unfortunately, but reviving is doable).

@meeloo Thanks I will take a look. I have to make sure to pick something that is support on Mac/Linux and Windows though. Even if Qt is very big it's also widely supported and has a team working on it.

Have you looked at CEF? We use it and it's wonderful, you control the chrome version so no HTML/JS cross compatibility to worry about, it's super flexible, you benefit from all the millions of JS frameworks and it's easy for anyone to jump into and work with the UI and sending messages back and forth is just a matter of JSON between the layers.

@olafurw I haven't but I heard horror stories about building large applications in JS. I will investigate non the less, thanks!

We have a pretty big application where the frontend is all in JS, I would recommend decoupling data and views and setup some sort of data bus where all data is requested through so views never have to worry, the UI should be stupid and fed data from the application end. Works pretty well.

@olafurw alright. Thanks for the info!

Where do you perceive the pain related to plugins including and linking to QT? If Windows compilation - Microsoft is going to release a C/C++ package manager very soon (vcpkg last time I saw it's name), which might remove the last big platform that doesn't have a painless "-dev" package installation system. If a plugin author simply needs to go "apt-get install qt-dev", or "vcpkg install qt", etc, then maybe it's really not so bad.

Slower link times (main app already links with Qt)
Main app and plugins must have matching Qt version.
As related to two if I upgrade main app to Qt 6.0 (or something) all plugins needs to be re-compiled to match.

With my current approach C plugins doesn't need to link with a single library which is very nice.

3 Have plugins being totally separate executables and have them communicate with the host app.

3bis: have plugins being coded in 2 parts:

a "core side" plugin code
separated gui code.

Solve speed/fast problematic, allow clean gui separated code and prevent core side being "slowed down" by gui code (events/etc.).
Just to be sure to include: record/replay RPC communication between gui/core so that unit-testing core side of plugins are robust. (much easier to debug)
May seems complicated, but a big win in the end imho, as gui code tends to pollute codebase very very easily.

@Kuranes That sounds interesting. Care to elaborate more in detail on that?

Did that once for a tool with plugins, that had to be run in cli and gui, but in the end the core/gui plugin separated felt very much like a win.

plugins added dll (C++ based code) to the core, code that did the real work, each "work" unit of the plugin cleanly organised in "commands" that were registered with name, etc to the main core. (using c++ delegates)

plugins gui was optional (plugins could be activated with just cli args ). Plugins gui registered observers on "events" from the core to get updates/visu, and sent RPC "commands"  to the core (pluginname/id/parameters)

Unfortunately I cannot seem to find opensource example of that.
( only I can is http://vorlonjs.com/ which does that in js with code in "core" is in the debugged page, and code "gui" is in the dashboard thing)

I'd mostly make sure that you can have non-UI plugins and UI plugins. Might turn out that the number of UI plugins in the end is rather small as you can provide some hooks for the plugins to get some UI done (message box, ask for user input, that kind of thing -- similar to what Powershell for example provides.)

Yeah so the way I do it now is that I have backend and view plugins. Backends is for implementing a new debugger backend while views are for displaying some data from a backend.
Right now there is fewer backend than views. Example of backends:

LLDB
Amiga UAE
WinDBG Engine

Views:

Disassembly
Registers
SourceCode
Callstack
Threads
Memory
...

So in my case there will be more view plugins than backends (I would guess at least)
The idea is that each view can do fairly complicated UI stuff so I need to expose a fair amount of the UI API to them in one way in another.

Performance could probably be solved by only updating the UI when there is a state change (though things like mouseover, etc might be tricky and require special code paths).
I think a functional view on UI (data → view) is most useful. Even in frameworks that are state based (DOM) there is a tendency to move towards functional APIs and then do a "DOM diff" to figure out what state changes to apply.
Getting good performance with CEF/HTML is certainly possible, but as with any top-down solution it can be tricky, because there are a lot of layers. I find it easier to control performance in a bottom-up scenario (i.e. IMGUI or similar).
For plugins, I would try to define an independent API interface so that they don't have to link against and become dependent on Qt (or another UI library). But it is tricky to find a balance between limiting the plugins (by not exposing enough) and exposing so much that the API becomes tightly bound to the backend (whether that is IMGUI /Qt/HTML). Maybe best to provide two levels of API. A high level that just talks about predefined controls (buttons, treeviews, etc) -- and a low level that allows custom drawing and control implementation. That way, "most" plugins can just use the high-level API and only those that need special capabilities need to deal with the low-level API.
What issues with text rendering did you run into? Is this something that just appears when rendering large amounts of text (i.e. documents), or even for small interface strings?
My current take on this is:

HTML/Cef is doable, but requires lots of effort that you don't accidentally create performance issues. Nice things are that it can be scripted, tested in browser, reloaded, and you know it will be around and worked on. (Because the web will be around.)
Imgui looks attractive, especially for performance/simplicity, but I haven't tried building anything big in it, so I'm not sure what issues you might run into.
Qt looks scary to me -- it is a huge DLL, I don't like moc, I don't like class-based designs. But I haven't made anything big with it, so I don't know. Lots of successful applications use it.
I wouldn't use anything other than one of these three (wxWidgets, GTK, Fox, etc).
How to build UIs in a good way is a hard and still "unsolved" problems. Just look at how many different JavaScript frameworks that keep appearing all the time. Once you've picked a path it is really hard to change, because you will have a lot of UI code.

Thanks for a detailed reply!

What issues with text rendering did you run into? Is this something that just appears when rendering large amounts of text (i.e. documents), or even for small interface strings?

Two things:

Supporting various languages (Arabic, Chinese, Thai, etc)
Quality of text rendering

Both solvable but time consuming.
I think you are totally right about making an Independent API around the UI and I like the idea about two levels of the API so I will likely try to head in that direction.
I think I will try Qt first. I agree that it's huge and I'm no fan of class-based and the moc stuff either but perhaps all of that can be wrapped away with the Independent API. I wouldn't want to expose that directly because it gets very tied to Qt then. I also ran into this thing https://blog.qt.io/blog/2016/08/18/introducing-the-qt-lite-project-qt-for-any-platform-any-thing-any-size/ So maybe there is something that can be used later on.
wxWidgets is a good candidate also but I would like to look to be same across platforms if possible (which Qt does) but if there is a good abstraction around maybe it would be possible do do anyway.
And yes, UIs are really hard and for sure still "unsolved"

I'm the main developer of x64dbg that makes heavy use of Qt. While there are drawbacks (complex threading model and complex parent/child model/hard to profile performance issues) the biggest upside is speedy development, even with custom-drawn widgets. Signals and slots are a great way of working.
I don't quite understand the 'plugin' model of ProDBG but if you are going to make everything a plugin you will be stuck in development hell forever (also if you try to 'wrap' Qt in a C interface). All I provide to plugins is AddQWidgetTab that adds a QWidget* to the main interface and lots of custom APIs that are also used to communicate between the GUI and the backend. My suggestion would be to write various components that each expose a custom interface for plugins and not let plugins manipulate the GUI unless they fully provide it by themselves.
In x64dbg the main challenge has been to optimize custom text drawing (syntax highlighting for disassembly). Qt and fonts are very complex and I would suggest basing your views on QTextEdit and not make them fully custom-drawn.

Thanks for these insights with some good info to consider. I'm a bit torn on the way forward for sure. My approach will likely be that I experiment a bit and see what level I should put the UI API on or if I should expose Qt directly.

I bet plenty of stuff doesn't need a real UI beyond a few buttons and checkboxes. These type of things you can support abstractly with an API that allows the plugin to say: "Here's my (typed) set of things and their current values. Let me know if they are changed" - you can map that to Qt or any other widget set on the app side and the plugin ignores the rest. You can get as fancy as you want here, as long as all you need is settings, trees of things, sliders, arrays etc.
But if you want plugins to actually take part in the screen redraw and be first class citizens in the app, then I don't see how you can avoid linking them to the same UI technology the app is using. Whatever level of abstraction you would add in between is probably just going to slow you down.
Recompiling plugins when there's an new Qt version doesn't seem particularly hard to do or even much of a pain. It sucks for commercial software because you typically don't have source, but it's not a big deal for open source stuff. And if someone has a hypothetical plugin that needs Qt 5.7 and you upgrade to 5.8 they can always chose not to get the latest version of ProDBG while they wait for an upgrade of this mystical plugin. This seems like a lot of work to solve a non-problem.

Having used Qt professionally for a while I can definitely recommend it. In my opinion it's the only cross platform toolkit out there that isn't full-on amateur hour. Stuff actually works across different platforms, and it fits in pretty well with other native apps.
Pros

Extremely comprehensive. Basically all the things included.
Model/view separation is reasonably well done and enables you to focus on (and unit test) data sources - then hook them up to stock views that are pretty solid in terms of features and performance.
Documentation (docs, tutorials, examples) are best in class, especially if you consider other open source projects
It's reasonably performant if you follow best practices. It certainly smokes anything we ever had running in a browser by a huge margin.
Implicit sharing and read only data structures make concurrency an attractive and easy-to-use option for many tasks. We leverage this heavily to kick off tasks to the thread pool and block the UI thread as rarely as possible.
Signals and slots are a clean way to connect random stuff together without crazy coupling.
Layouts basically solve the "scalable UI" problem
Hi-DPI support is basic but workable in Qt 5. Stuff looks all right for the most part, but you need to provide @2x.png-style icons (or use vector icons)
There's a designer that produces .ui files that automate the most tedious part of UI dev, creating widgets and hooking crap up. You can still do it all by hand (if you want to), there's zero magic here. It's just convenient. This is another codegen step (uic) that generates C++ code.
You have full source on all platforms, so crashes are rarely mysteries

Cons

QString is UTF-16. It's a legacy choice, and one that I'm sure they regret. So interacting with a lot of UTF-8 sources can be allocate-y. Best practice is to go to QString as soon as possible and then stay there. Use QStringLiteral to avoid dynamic memory allocations for constants (bakes them into .rodata).
It does need the MOC to generate bindings for signals and slots. Not a huge deal breaker in practice, but it does take some build system love.
There's a ton of crap no one needs for desktop work like Qt Quick, web browser libraries (two of them!), bluetooth and all kinds of other useless junk. Just stick to basic QtWidgets, QtGui, QtConcurrent and QtCore and those are the only thing you need to ship with the binaries. Perhaps QtXML if you're unfortunate enough to have to use XML. It's not a huge dependency at that point IMO compared to what you're getting. There's a deployment tool that makes a custom little side-install of Qt based on just what your app uses which is perfect for shipping of releases.
Some of the advanced APIs are complicated to learn and use correctly. Especially implementing QAbstractItemModel from scratch is a rite of passage. (Tip: Use the modeltest code and unit tests.)
It's very OO heavy. I'm not a huge fan of OO in general, but it's not too awful for what you get.  I just accept it as the cost of doing business in the space.

Best practices to consider right off the bat IMO:

Use UI files and layouts.
Use QRC files for resources.
Use QSettings
Disable implicit const char* -> QString conversions (compile-time macro)
Disable QT legacy keywords (compile-time macro)
Use member function connect syntax exclusively and never the SIGNAL(), SLOT() stuff from pre Qt-5. Compile errors are better than runtime errors.

Thanks for the write up. Very appreciated.
I guess my biggest issue with Qt is that it's C++ and binding C++ to other languages is lots of work esp given how Qt is working. I honestly really dislike to write code in C++ unless I really have to (at home there is a choice) but given the options here I don't see any other way around it until there are some binding for it.
I really don't think the overhead for an abstraction would be that much. Yes it would be a bunch more indirect calls to functions but that isn't likely a big issue here.
With all that being said I will go for Qt with C++ for now. This will allow me to learn to know the API and implement plugins directly (without having them as proper plugins to begin with) and see how it works out.
Still backends can be written in any language as they are independent from the UI and talk with each other over a message passing like API.

I really don't think the overhead for an abstraction would be that much. Yes it would be a bunch more indirect calls to functions but that isn't likely a big issue here.

I think the problem is writing and maintaining that wrapper. Have a look at QPainter, and all the surrounding classes you'd need to expose a C API to (QPen, QFont, QBrush, QFontMetrics just to name a few essential ones to get anything done). It's a lot of cost to carry forward for very little value (depending on how much value you place on using Rust of course). Personally I would just write all of this in C++, because at the end of the day you want a usable debugger with a solid UI. I get that Rust is your preference, but it seems to be putting imaginary roadblocks on the path to completing the actual goal: a debugger.

Point taken. If one really wanted to wrap Qt one would have to auto-generate the wrappings from the C++ headers. Some efforts has been done in this direction like https://github.com/kitech/qt.rs/blob/master/src/gui/qpainter.rs (this isn't actually working or that great generated but shows that it's possible)
So as you suggest I will go with C++. If there (at some point) would be a wrapper that auto-generates the bindings (i.e no manual work) then I will re-consider but now I think C++ is the best option.

Code refactoring over to Qt is now being done over here https://github.com/emoon/ProDBG/tree/qt
Thanks everyone for the replies in this thread as it has been really useful.


support zstd

Closes #102.
A couple of notes:

I never was able to install :ezstd with optional: true, but since it seemed to be isolated to me I installed without that setting, then updated mix.exs afterward. We'll see if it It apparently works in CI.
Whenever the NIF is loaded :ezstd prints to stdout; I could not figure out how to capture/suppress that, even when wrapping the calls to it in ExUnit.CaptureIO.capture_io/1. It doesn't hurt anything, just makes test output unnecessarily messy.

I added a terrible hack to suppress the output: https://github.com/wojtekmach/req/commit/7f8032a95574de25ee34b49b037f9b949c17a49a. Don't try this at home :D


Tier0 WMAgent deployment model for Alma9/RHEL9

Impact of the new feature
Tier0 WMAgent
This issue may be closed by: https://github.com/dmwm/CMSKubernetes/pull/1466
Is your feature request related to a problem? Please describe.
This can potentially become another meta-issue, but it's important to start tracking the Tier0 requirements such that we can properly plan for an impending migration of the Tier0 WMAgent stack, together with condor schedd migration to Alma9 OS.
This goes along the plans we have for central production agent, tracked in this meta-issue: https://github.com/dmwm/WMCore/issues/11314
but we need to clarify whether the Tier0 agrees with the same deployment model and/or what changes are required for the T0 environment.
Describe the solution you'd like
For now, central production WMAgent deployment will be:

package and upload wmagent package to PyPi
build WMAgent docker images based on the https://github.com/dmwm/CMSKubernetes/tree/master/docker/pypi/wmagent and PyPi WMAgent package.
assume condor_schedd will be deployed on the host
For MariaDB (Fermilab agents), run it from a Docker container
For Oracle, there is no need to run any service, however we do need to have an up-to-date tnsnames.ora file
(docker) compose all these images together

Describe alternatives you've considered
None
Additional context
See containerization meta-issue: https://github.com/dmwm/WMCore/issues/11314

Thanks for creating this issue @amaltaro
Some updates here on that.
There was non 0 effort between me and @LinaresToine  last year for bringing the T0 deployment tests up to the level where the WMAgent containers currently are.  We have reached to the point where the Oracle functionalities for the manage scripts and the CouchDB container are needed. Once we resolve: https://github.com/dmwm/WMCore/issues/11720 and https://github.com/dmwm/WMCore/issues/11312, T0 Team will be automatically able to fully run T0 agents either from a WMAgent container, or from the OS directly (provided the underlying OS is alma9 or rhel9 or equivalent one supporting python 3.8.16 or higher) by using the scripts we provide here: https://github.com/dmwm/WMCore/blob/master/deploy/deploy-wmagent-venv.sh. Last year we tracked down all the needed packages equivalences between distributions and OS dependencies during our meetings - Hopefully Antonio steel keeps record on the list :)
FYI @germanfgv @LinaresToine

logging the current state of this issue:
Yesterday, We took the chance that the two mandatory PRs (https://github.com/dmwm/CMSKubernetes/pull/1409 && https://github.com/dmwm/CMSKubernetes/pull/1451) needed for T0 to start their tests are already in a semifinal stage, and we had a meeting yesterday between me, Andrea, German and Antonio to discuss the new initialization mechanisms of the agents with a small hands on demonstration from my side on how things work currently. During this process I used mostly Docker containers for both CouchDB and WMAgent, because the initialization process for the agent is immutable and independent of the deployment methods, being it Docker or virtual env.
We agreed on another meeting next week when we would have everything merged from our side, such that next time they would be able to perform the hands on activities with guidance on my side.

p.s. on the comment from above.
During the meeting, we also agreed on starting the proper communication with CERN IT in order to asses how safe it is to add the OS UIDs and GIDs in the Docker repositories, as explained in this comment: https://github.com/dmwm/CMSKubernetes/pull/1412#issuecomment-1967363719
I am about to start this communication today, and will include the proper set of people involved from our side.
FYI: @amaltaro @vkuznet  @khurtado @anpicci

Again for logging the activity on this topic:
Today we had yet another meeting during which T0 Team was doing the hands on activities. We found several issues like:

Additional account management needed to happen directly inthe scripts ... will be fixed once we add the T0 account to the CMSKubernetes repository, but for that we'll have to wait for reply from CERN IT
We failed with CouchDB connection between the agent and the the CouchDB instance from the Docker container.

Once the later is debugged we plan to have another meeting for tomorrow.
FYI: @amaltaro  @germanfgv @LinaresToine @anpicci

Logging the activities again.
Today we were having yet another long hands on meeting between me, @LinaresToine and  @anpicci. This time we we had  to resolve few more bugs such as a

Mismatch between T0 and Prodction agents configurations - T0 are not configuring ACDC server,
Typo in fetching Team name from WMAgent.secrets file
Missing Tier0.* account handling  in init.sh script
Skipping agent config upload step for T0 agents

A PR is coming for resolving all of these.
At the very end we managed to fully deploy and initialize  the agent to the very end. But we failed to start the services because of the old licurl - to - pycurl backend mismatch (nss vs. openssl). @LinaresToine is about to try later to resolve this issue, so they can start testing the deployment of the T0 related packages and eventual injections.
FYI: @amaltaro

A PR in the CMSKubernetes repository was created to fix isues pointed out in the previous comment. See https://github.com/dmwm/CMSKubernetes/pull/1457.
I continue to look into the libcurl - to - pycurl issue pointed out by @todor-ivanov

Hi @LinaresToine  I've left one comment in the Pr. Please take look

hi @LinaresToine  while you are removing that line I asked in the review, here is your solution about the backend ssl library mismatch in pycurl. You need to follow both of those steps:

At the Host:

[root@vocms0290 data]# yum install curl-openssl libcurl-devel libcurl-openssl-devel

Inside the Virtual Environment:

(WMAgent.venv3) cmst1@vocms0290:WMAgent.venv3 $ pip uninstall pycurl
(WMAgent.venv3) cmst1@vocms0290:WMAgent.venv3 $ export PYCURL_SSL_LIBRARY=openssl
(WMAgent.venv3) cmst1@vocms0290:WMAgent.venv3 $ pip install --no-cache-dir --global-option=build_ext --global-option="-L/usr/local/opt/openssl/lib" --global-option="-I/usr/local/opt/openssl/include"  pycurl

Thank you very much @todor-ivanov. Regarding the PR, I will modify it right now. I will point out an observation in the PR itself.
Regarding the backend solution, I did try the steps you mention inside the virtual environment. I was missing out on the step in the host. Thank you for your insight!!

Hello @todor-ivanov. I created a script to keep track of all the steps taken until the manage start-agent step. This script can be seen in https://github.com/dmwm/T0/pull/4931. Worth clarifying that I am sourcing the script, not executing it.
From our meetings I remember the couchdb-docker-build.sh step is not always necessary, however I include it for now.
The script includes the debugging of the pycurl issue with the environment activated. I may be missing something, but I still get the same error message upon running manage start-agent. I continue to look into it.

hi @LinaresToine

I continue to look into it.

Lets quickly meet again next week and we will try to resolve this together.

@todor-ivanov @LinaresToine Can you give an update on this issue?  Has there been any more progress since the meeting referenced above?

Hello @klannon
Tier 0 recently got an Alma 9 machine, allowing us to successfully deploy the new agent. That is an update regarding the issue with libcurl and pycurl backend misconfigurations. Regarding the fixes in the init and manage scripts, perhaps @todor-ivanov can give an update on that? Such fixes are documented in https://github.com/dmwm/CMSKubernetes/pull/1457

hi @klannon  we plan to close this issue this week. There are few more lines that need to get into those fixes, which would be faster if I make them myself. And then we meet with @LinaresToine  to complete the process on another hands on meeting.

So, after our long meeting yesterday with @LinaresToine  we ended  up with with another possible fix addressing more than one problems: https://github.com/dmwm/CMSKubernetes/pull/1466
But there is still few minor details before we close it. I'll push the more changes later tonight.

hi @amaltaro  @anpicci @vkuznet  with our latest commits to https://github.com/dmwm/CMSKubernetes/pull/1466 we ( me @LinaresToine  @germanfgv ) to initialize a T0 agent alma9 machine properly and test this deployment process during our meeting today. There were some new library issues  that have popped up the last minute when German decided to actually start a replay (he or Antonio are going to update the issue with the new error). But the biggest success here was that we managed to finish the deployment properly on top of all the rest of the changes  we  did lately on the WMAgent containers. Please feel free to start your review on the PR in the CMSKuberenetes  repository.

Hello all. As Todor mentions in the comment above, we were able to initialize and start the T0 agent successfully. After starting, the Tier0Feeder component of the agent displays the following error message:

ERROR:StdBase:About to raise exception <@========== WMException Start ==========@>
Exception Class: WMSpecFactoryException
Message: 'xml.etree.ElementTree.Element' object has no attribute 'getchildren'
ClassName : None
ModuleName : WMCore.WMSpec.WMWorkloadTools
MethodName : _validateArgFunction
ClassInstance : None
FileName : /data/tier0/WMAgent.venv3/srv/WMCore/src/python/WMCore/WMSpec/WMWorkloadTools.py
LineNumber : 139
ErrorNr : 0
Traceback:
File "/data/tier0/WMAgent.venv3/srv/WMCore/src/python/WMCore/WMSpec/WMWorkloadTools.py", line 130, in _validateArgFunction
if not valFunction(value):
File "/data/tier0/WMAgent.venv3/srv/WMCore/src/python/WMCore/WMSpec/StdSpecs/StdBase.py", line 1034, in
"CMSSWVersion": {"validate": lambda x: x in releases(),
File "/data/tier0/WMAgent.venv3/srv/WMCore/src/python/WMCore/ReqMgr/Tools/cms.py", line 219, in releases
return TC.releases(arch)
File "/data/tier0/WMAgent.venv3/srv/WMCore/src/python/WMCore/Services/TagCollector/TagCollector.py", line 89, in releases
for row in self.data():
File "/data/tier0/WMAgent.venv3/srv/WMCore/src/python/WMCore/Services/TagCollector/TagCollector.py", line 83, in data
for row in xml_parser(data, pkey):
File "/data/tier0/WMAgent.venv3/srv/WMCore/src/python/WMCore/Services/TagCollector/XMLUtils.py", line 62, in xml_parser
get_children(elem, event, row, key)
File "/data/tier0/WMAgent.venv3/srv/WMCore/src/python/WMCore/Services/TagCollector/XMLUtils.py", line 74, in get_children
for child in elem.getchildren():

@amaltaro @vkuznet @todor-ivanov @anpicci @germanfgv

Hi @LinaresToine @todor-ivanov , it seems an error with the XML file itself, rather than a bug, right? In addition, the stacktrace looks like incomplete.

The WMException ended there after the attribute getchildren was not found. However, I can include this portion also:

2024-05-02 19:02:18,253:139701761603136:ERROR:Tier0FeederPoller:Can't configure for run 359688 and stream Calibration
Traceback (most recent call last):
File "/data/tier0/WMAgent.venv3/srv/WMCore/src/python/WMCore/WMSpec/WMWorkloadTools.py", line 130, in _validateArgFunction
if not valFunction(value):
File "/data/tier0/WMAgent.venv3/srv/WMCore/src/python/WMCore/WMSpec/StdSpecs/StdBase.py", line 1034, in
"CMSSWVersion": {"validate": lambda x: x in releases(),
File "/data/tier0/WMAgent.venv3/srv/WMCore/src/python/WMCore/ReqMgr/Tools/cms.py", line 219, in releases
return TC.releases(arch)
File "/data/tier0/WMAgent.venv3/srv/WMCore/src/python/WMCore/Services/TagCollector/TagCollector.py", line 89, in releases
for row in self.data():
File "/data/tier0/WMAgent.venv3/srv/WMCore/src/python/WMCore/Services/TagCollector/TagCollector.py", line 83, in data
for row in xml_parser(data, pkey):
File "/data/tier0/WMAgent.venv3/srv/WMCore/src/python/WMCore/Services/TagCollector/XMLUtils.py", line 62, in xml_parser
get_children(elem, event, row, key)
File "/data/tier0/WMAgent.venv3/srv/WMCore/src/python/WMCore/Services/TagCollector/XMLUtils.py", line 74, in get_children
for child in elem.getchildren():
AttributeError: 'xml.etree.ElementTree.Element' object has no attribute 'getchildren'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/data/tier0/WMAgent.venv3/srv/WMCore/src/python/WMCore/WMSpec/StdSpecs/StdBase.py", line 944, in masterValidation
validateArgumentsCreate(schema, argumentDefinition)
File "/data/tier0/WMAgent.venv3/srv/WMCore/src/python/WMCore/WMSpec/WMWorkloadTools.py", line 276, in validateArgumentsCreate
_validateArgumentOptions(arguments, argumentDefinition, "optional")
File "/data/tier0/WMAgent.venv3/srv/WMCore/src/python/WMCore/WMSpec/WMWorkloadTools.py", line 160, in _validateArgumentOptions
arguments[arg] = _validateArgument(arg, arguments[arg], argDef)
File "/data/tier0/WMAgent.venv3/srv/WMCore/src/python/WMCore/WMSpec/WMWorkloadTools.py", line 101, in _validateArgument
_validateArgFunction(argument, value, argumentDefinition["validate"])
File "/data/tier0/WMAgent.venv3/srv/WMCore/src/python/WMCore/WMSpec/WMWorkloadTools.py", line 139, in _validateArgFunction
raise WMSpecFactoryException(str(ex))

We are  in the process of final tests here. More details I gave in my comment to the PR with which I called @amaltaro and @anpicci  for final review: https://github.com/dmwm/CMSKubernetes/pull/1466#issuecomment-2107090214

The xml library have removed the getchildren attribute after version 3.9: https://docs.python.org/3.8/library/xml.etree.elementtree.html#xml.etree.ElementTree.Element.getchildren


POST request JSON must be added to req after the attachment of cookies--why?

The following test passes:
var request = require('supertest');
var superagent = require('superagent');

describe('Example:', function() {
  var agent;
  beforeEach(function(done) {
    agent = superagent.agent();
    var req = request('http://example.com').get('/getsomecookies');
    req.end(function(err, result) {
      if (!err) {
        agent.saveCookies(result.res);
        done();
      } else {
        done(err);
      }
    });
  });

  it('should successfully post some JSON', function(done) {
    var req = request('http://example.com').post('/postsomejson')
    agent.attachCookies(req);
    req.send({my: 'JSON'});
    var csrfToken = (/XSRF-TOKEN=(.*?);/.exec(req.cookies)[1]);
    req.set('X-XSRF-TOKEN', unescape(csrfToken));
    req.end(function(err, result){
      expect(res.status).to.eql(200);
      done();
    });
  });
});

However, simply swapping the attachment of cookies with the setting of the JSON causes the test to fail--the server responds with 403. That is, the following fails:
  it('should successfully post some JSON', function(done) {
    var req = request('http://example.com').post('/postsomejson').send({my: 'JSON'});
    agent.attachCookies(req);
    ...
  });

When I look on the server (express proxied by nginx, using the express.csrf middleware) to see what is going on, in the unsuccessful case the secret loaded from req.session.csrfSecret, which is used to match the X-XSRF-TOKEN header, is undefined. In the successful case, the secret loaded from req.session.csrfSecret is exactly what it should be, namely the secret created by the GET request to /getsomecookies.
Interestingly, if I make the POST request without trying to send any JSON, I at least don't get a 403 response (though the test would fail with a 400 because I didn't provide any JSON for the server to do something with). That is, this works too:
  it('should successfully post some JSON', function(done) {
    var req = request('http://example.com').post('/postsomejson');
    agent.attachCookies(req);
    ...
  });

So, what is it about attachCookies that is interfering with the setting of the JSON sent in the request? Clearly the server is seeing something different in each case. Is this problem documented anywhere?

:+1:

+1

Where is any of this saveCookies/attachCookies stuff documented? I don't see any of it in https://github.com/visionmedia/superagent/blob/master/test/node/agency.js

Also, this means that agent.get(url).end(err,res) doesn't get its cookies set (as an agent), which is completely bonkers.


When I search for resource path, I want to only list paths that ends with my search

When I search for resource path, I want to only list paths that ends with my search.
Today, this is not possible, and if I search for foo.jar, I may get all the foo.jar-extract and more
It would be nice to have a way to force and "ends with" only search as an option.

@pombredanne a new advanced search was implemented. You can refer to the search syntax in the documentation at https://scancodeio.readthedocs.io/en/latest/user-interface.html#search-syntax or from the inline help next to the search box.

It would be nice to have a way to force and "ends with" only search as an option.

You can use the following search syntax: name:foo.jar


char* to int conversion

The STAT and RECV section has a problem with checking the correct value. There is a error free char* to int conversion needed.

ede6d4b143ca6176784ad56c497909abb0f7c82b fixed


Rails4

These are the commits to get orchestrated on rails 4. The only thing that might be an issue afaik is we removed the attr_accessible call in order to use strong_parameters in rails 4. Not tested this on rails 3.
Renamed the Orchestration_state.png to make windows file systems happy (there was an error).

Aaand…merged. Finally. Again, thank you for your PR!


Eureka endpoint - online/offline service instances

Provide Blue/Green deployments through endpoint. Use the endpoint to set registered instances to offline

This functionality is provided in Spring via the service-registry endpoint: https://github.com/spring-cloud/spring-cloud-commons/blob/master/spring-cloud-commons/src/main/java/org/springframework/cloud/client/serviceregistry/endpoint/ServiceRegistryEndpoint.java


Old Bugs not quashed (unit movement, right click, arrow keys)

Loading a new game, units won't move. Multiple instances of CQUI listed in "additional content," reinstalling Civ6 did not solve the issue, nor did resubscribing to the CQUI mod. On a Mac.

Are you subbed to multiple CQUI versions on the workshop? And have you installed the mod manually at any point? Because that will cause multiple listings (and issues with the mod itself).
If you could post your logs and your savegame for us, that'd make it a lot easier.
You can find those in ..\Documents\My Games\Sid Meier's Civilization VI\

I think I resolved it, the initial mod was hiding in a different folder from usual that I missed but someone else pointed out. I do stop getting reminders to change cultural policies at some point though, not sure why that happens.

Good to hear you had your issue resolved. If you want to report your other issue it's best if you make a new issue, and follow the bug report format. That way we can actually look into it.


I see this message when I create azure web app in vscode. (I try to deploy loopback app to azure)

Repro steps:

Action: appService.Deploy
Error type: BadRequest
Error Message: Exceeded the limit of 1 free tier linux server farm per subscription.
Version: 0.15.0
OS: win32
Product: Visual Studio Code
Product Version: 1.35.1
Language: en

Call Stack
module.exports.l.pipeline extension.bundle.js:262:3299936
s extension.bundle.js:262:112563
s extension.bundle.js:157:26036
extension.bundle.js:157:27276extension.bundle.js:157:27276
o extension.bundle.js:208:7054
extension.bundle.js:157:24957extension.bundle.js:157:24957
$.module.exports.t.requestLibrarySink.t.headersOnly.t.streamedResponse.t.body.function.t.body.pipe.a [as _callback] extension.bundle.js:1:534414
$.module.exports.$.init.t._callback.t.callback.t.callback extension.bundle.js:238:5461
$.emit events.js:182:13
$.<anonymous> extension.bundle.js:238:18274
$.emit events.js:187:15
IncomingMessage.<anonymous> extension.bundle.js:238:17116
Object.onceWrapper events.js:273:13
IncomingMessage.emit events.js:187:15
endReadableNT _stream_readable.js:1090:12
process._tickCallback next_tick.js:63:19

Hi @onwebcity
Unfortunately Azure only offers one free Linux app service plan per subscription.  You can either create your web app in the existing plan or you can create a free Windows app service plan.
Please let me know if that resolves your issue!

Hi @nturinski
In my case, @kms-binhtran do that on DevOps and it is resolved, Thank you for helps

Hi @nturinski , Can you help me to change the config of virtual paths in the devops? What is the key for that and directory i wanted to change is /home/site/wwwroot/frontend/dist/frontend. How can i make it to that path?

Hi @onwebcity,
What kind of project are you deploying?  I'm not sure if this is handled by Dev Ops as much as your project's settings.  For instance, in a node.js project, I believe the package.json's "main" property is used as the start-up script.


How to read audio attachment from bot framework message extension?

I'm trying to download an audio attachment from a message payload coming from Teams arriving to a message extension bot. The attachment part of the JSON payload has the following form:
"attachments": [
   {
     "id": "xxx",
     "contentType": "application/vnd.microsoft.card.audio",
     "contentUrl": null,
     "name": null,
     "thumbnailUrl": null,
     "content": "{\"duration\":\"PT13S\",\"media\":[{\"url\":\"https://eu-api.asm.skype.com/v1/objects/0-neu-d6-xxxx/views/audio\"}]}"
  }
]

I wasn't able to find any proper documentation about getting an access token from skype to download it. Is it possible at all?
Posted the issue here as suggested on this question in SO
Thanks in advance

hi @Subhasish-msft, the audio message is recorded from the MS Teams iOS app

Hi @eduardocampano , Sorry for the delayed response, Currently file download is not supported through message extension. We are discussion with platform team to consider this in future release.


Add an option to skip hash as identification method

More frequently than it should be, hash doesn't grants that the subtitle being downloaded is the correct one. Probably because of typing error or confusion/copy-paste when the uploaders post it.
In those cases... we should be able to put a parameter in command line to ask Subliminal to NOT use hash as a method of search, but searching for filename / release instead.

I second this. I frequently see the wrong subtitle downloaded because the provider has the wrong hash for the subtitle.

Challenging the hash with other matches is already done in opensubtitles here and there. Please try with develop branch and report back.

Hey Diaoul, I've read the codes you quoted. But there are lot of times where even inside the same Title there are errors in hashes.
Eventually I find myself relying much more on full-names (title, year, quality, codec, release group) than hashes.
If you could skip hash check or give it a 0 (zero) score when a specific parameter is given... we would appreciate.
** -- AND MORE: --**
Recently I've began to create a manual (automatic, soon) procedure of adjusting several metadata of my video files. Removing embeeded subtitles, removing unused audio track, tagging it...etc...
When that happens, the adjusted video file comes with a different hash. So I'm forced to remember using subliminal right after download, because later this won't be possible, forever for that file!.
So... using exclusively the filename and/or metadata instead of hash has to be a option.
Thanks.

Please provide examples.
Researching by hash doesn't dismiss other searches so searching with a wrong hash will yield no results but the other full text searches will.

I'm not in my home computer right now... so testing will only be possible at night.
I'll respond as soon as possible, then.

Any updates on this?

Sorry. had a lot in my mind and I forgot to come back here.
Debugging some more revealed me an ugly truth... hashes aren`t perfect and they NEED to have an option to be ignored completely.
An episode called The.Big.Bang.Theory.S01E08.The.Grasshopper.Experiment.HDTV-720p.x264.AC3-MRSK.mkv (**MRSK **is also know as **Max **in some files) in debug mode of subliminal uses the hash c48b1267359eacbf . When I went manually inside the site to search using the hash, it returned me a generic subtitle that  "apparently" could serve more than 20 different releases, including ReEnc-Max release.
But thats a lie! the times in release uploaded are totally wrong, Legendas.TV although is perfect for the same release / subtitle name.... go figure.

Do you have a link to that subtitle? Or the video hash? Please provide more information. I doubt I will be able to do something about it given the nature of the issue. Full text search might just as well return the same result from opensubtitles if the uploaded did say this subtitle matched all those releases.
The hash search result is already challenged with other properties (series, season and episode for an Episode and title for a Movie).

Sorry to hijack this issue, but I have the same problem (running on master branch)
Example:

2016-04-19 21:10:02,262 INFO     [CP Server Thread-2 :: subliminal.video] Scanning video u'Southpaw.2015.1080p.BluRay.x264-SPARKS.mkv' in u'\\\\SERVER\\couchpotato-autosubliminal\\Southpaw.2015.1080p.BluRay.x264-SPARKS[rarbg]'
2016-04-19 21:10:02,371 INFO     [CP Server Thread-2 :: enzyme.mkv] Reading Segment element
2016-04-19 21:10:02,371 INFO     [CP Server Thread-2 :: enzyme.parsers.ebml.core] MasterElement EBML ignored
2016-04-19 21:10:02,387 INFO     [CP Server Thread-2 :: enzyme.parsers.ebml.core] Maximum level 0 reached for children of MasterElement Segment
2016-04-19 21:10:02,387 INFO     [CP Server Thread-2 :: enzyme.mkv] Reading SeekHead element
2016-04-19 21:10:02,387 INFO     [CP Server Thread-2 :: enzyme.mkv] Processing element Info from SeekHead at position 4151
2016-04-19 21:10:02,387 INFO     [CP Server Thread-2 :: enzyme.mkv] Processing element Tracks from SeekHead at position 4284
2016-04-19 21:10:02,387 INFO     [CP Server Thread-2 :: enzyme.mkv] Processing element Tags from SeekHead at position<PHONE_NUMBER>
2016-04-19 21:10:02,387 INFO     [CP Server Thread-2 :: enzyme.mkv] Processing element Chapters from SeekHead at position 5555
2016-04-19 21:10:02,387 INFO     [CP Server Thread-2 :: subliminal.api] Listing subtitles for
2016-04-19 21:10:02,387 INFO     [CP Server Thread-2 :: subliminal.api] Listing subtitles with provider u'opensubtitles' and languages set([])
2016-04-19 21:10:02,387 INFO     [CP Server Thread-2 :: subliminal.api] Initializing provider opensubtitles
2016-04-19 21:10:02,387 INFO     [CP Server Thread-2 :: subliminal.providers.opensubtitles] Logging in
2016-04-19 21:10:02,605 INFO     [CP Server Thread-2 :: subliminal.providers.opensubtitles] Searching subtitles [{'moviebytesize': '9385937011', 'sublanguageid': u'dut', 'moviehash': 'fc2eede6b9d200ec'}]
2016-04-19 21:10:02,667 INFO     [CP Server Thread-2 :: subliminal.api] Skipping provider u'addic7ed': not a valid video
2016-04-19 21:10:02,667 INFO     [CP Server Thread-2 :: subliminal.api] Found 0 subtitle(s)
2016-04-19 21:10:02,667 INFO     [CP Server Thread-2 :: subliminal.api] Terminating provider opensubtitles
2016-04-19 21:10:02,667 INFO     [CP Server Thread-2 :: subliminal.providers.opensubtitles] Logging out

When a hash is present, it only seems to search for the hash.
When I, just to test, use an empty file with the same name, hash calculation is skipped, query is used and a subtitle is found on OpenSubtitles

2016-04-19 22:06:10,769 INFO     [CP Server Thread-9 :: subliminal.video] Scanning video u'Southpaw.2015.1080p.BluRay.x264-SPARKS.mkv' in u'c:\\temp\\autosubliminal\\completed'
2016-04-19 22:06:11,015 WARNING  [CP Server Thread-9 :: subliminal.video] Size is lower than 10MB: hashes not computed
2016-04-19 22:06:11,016 INFO     [CP Server Thread-9 :: enzyme.mkv] Reading Segment element
2016-04-19 22:06:11,115 ERROR    [CP Server Thread-9 :: subliminal.video] Parsing video metadata with enzyme failed
Traceback (most recent call last):
  File "C:\Tools\GitHub\Auto-Subliminal\lib\subliminal\video.py", line 322, in scan_video
    mkv = MKV(f)
  File "C:\Tools\GitHub\Auto-Subliminal\lib\enzyme\mkv.py", line 42, in __init__
    raise MalformedMKVError('No Segment found')
MalformedMKVError: No Segment found
2016-04-19 22:06:12,394 INFO     [CP Server Thread-9 :: subliminal.api] Listing subtitles for
2016-04-19 22:06:15,760 INFO     [CP Server Thread-9 :: subliminal.api] Listing subtitles with provider u'opensubtitles' and languages set([])
2016-04-19 22:06:15,762 INFO     [CP Server Thread-9 :: subliminal.api] Initializing provider opensubtitles
2016-04-19 22:06:30,119 INFO     [CP Server Thread-9 :: subliminal.providers.opensubtitles] Logging in
2016-04-19 22:06:48,733 INFO     [CP Server Thread-9 :: subliminal.providers.opensubtitles] Searching subtitles [{'query': u'Southpaw.2015.1080p.BluRay.x264-SPARKS.mkv', 'sublanguageid': u'dut'}]
2016-04-19 22:06:59,391 INFO     [CP Server Thread-9 :: subliminal.api] Skipping provider u'addic7ed': not a valid video
2016-04-19 22:06:59,392 INFO     [CP Server Thread-9 :: subliminal.api] Found 5 subtitle(s)
2016-04-19 22:06:59,394 INFO     [CP Server Thread-9 :: subliminal.api] Terminating provider opensubtitles
2016-04-19 22:06:59,397 INFO     [CP Server Thread-9 :: subliminal.providers.opensubtitles] Logging out
2016-04-19 22:06:59,720 INFO     [CP Server Thread-9 :: subliminal.api] Downloading subtitle
2016-04-19 22:06:59,723 INFO     [CP Server Thread-9 :: subliminal.api] Downloading subtitle
2016-04-19 22:06:59,726 INFO     [CP Server Thread-9 :: subliminal.api] Initializing provider opensubtitles
2016-04-19 22:07:00,562 INFO     [CP Server Thread-9 :: subliminal.providers.opensubtitles] Logging in
2016-04-19 22:07:00,809 INFO     [CP Server Thread-9 :: subliminal.providers.opensubtitles] Downloading subtitle
2016-04-19 22:07:01,759 INFO     [CP Server Thread-9 :: subliminal.api] Downloading subtitle
2016-04-19 22:07:01,760 INFO     [CP Server Thread-9 :: subliminal.api] Downloading subtitle
2016-04-19 22:07:01,762 INFO     [CP Server Thread-9 :: subliminal.providers.opensubtitles] Downloading subtitle
2016-04-19 22:07:03,158 INFO     [CP Server Thread-9 :: subliminal.api] Downloading subtitle
2016-04-19 22:07:03,160 INFO     [CP Server Thread-9 :: subliminal.api] Downloading subtitle
2016-04-19 22:07:03,161 INFO     [CP Server Thread-9 :: subliminal.providers.opensubtitles] Downloading subtitle
2016-04-19 22:07:03,885 INFO     [CP Server Thread-9 :: subliminal.api] Downloading subtitle
2016-04-19 22:07:03,887 INFO     [CP Server Thread-9 :: subliminal.api] Downloading subtitle
2016-04-19 22:07:03,887 INFO     [CP Server Thread-9 :: subliminal.providers.opensubtitles] Downloading subtitle
2016-04-19 22:07:04,677 INFO     [CP Server Thread-9 :: subliminal.api] Downloading subtitle
2016-04-19 22:07:04,678 INFO     [CP Server Thread-9 :: subliminal.api] Downloading subtitle
2016-04-19 22:07:04,680 INFO     [CP Server Thread-9 :: subliminal.providers.opensubtitles] Downloading subtitle
2016-04-19 22:07:05,282 INFO     [CP Server Thread-9 :: subliminal.api] Terminating provider opensubtitles
2016-04-19 22:07:05,282 INFO     [CP Server Thread-9 :: subliminal.providers.opensubtitles] Logging out
2016-04-19 22:07:07,332 INFO     [CP Server Thread-9 :: subliminal.subtitle] Computing score for matches set(['year', 'video_codec', 'title', 'resolution']) and
2016-04-19 22:07:07,334 INFO     [CP Server Thread-9 :: subliminal.subtitle] Computed score 43
2016-04-19 22:07:07,880 INFO     [CP Server Thread-9 :: subliminal.subtitle] Computing score for matches set(['year', 'title']) and
2016-04-19 22:07:07,881 INFO     [CP Server Thread-9 :: subliminal.subtitle] Computed score 35
2016-04-19 22:07:08,371 INFO     [CP Server Thread-9 :: subliminal.subtitle] Computing score for matches set(['video_codec', 'year', 'title']) and
2016-04-19 22:07:08,372 INFO     [CP Server Thread-9 :: subliminal.subtitle] Computed score 39
2016-04-19 22:07:08,578 INFO     [CP Server Thread-9 :: subliminal.subtitle] Computing score for matches set(['video_codec', 'year', 'title']) and
2016-04-19 22:07:08,578 INFO     [CP Server Thread-9 :: subliminal.subtitle] Computed score 39
2016-04-19 22:07:08,910 INFO     [CP Server Thread-9 :: subliminal.subtitle] Computing score for matches set(['year', 'video_codec', 'format', 'title', 'release_group']) and
2016-04-19 22:07:08,911 INFO     [CP Server Thread-9 :: subliminal.subtitle] Computed score 56

@Diaoul can you have a look at this? Is this sufficient info for you?


ci: test commit to echo gitLab variable

:tada: This PR is included in version 1.1.11 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


add hill climb search

#695 Solve Caeser Cipher Problem Using Hill Climb Search.

I am not that familiar with the hill-climbing/caesar cipher combo, so I can't be of much help. Maybe you can receive as input the possible combinations and go from there?
I can see that in some problems that input would be large, but remember we are not building real-world applications, so don't worry about size. We assume the students will have small examples.

hill-climbing-search should be  a separate function, like uniform-cost-search. It should take as inputs an initial state, a function to generate neighboring states, and an evaluation function to say how good each neighbor is. Neighbors could be formed by swapping two letters, so there are 26*25, not 26! options. Or you could choose a random subset of those. See page 232 of http://norvig.com/ngrams/ch14.pdf

Thanks for the help, I will implement it as soon as possible
On Feb 7, 2018 2:10 AM, "Peter Norvig"<EMAIL_ADDRESS>wrote:

hill-climbing-search should be a separate function, like
uniform-cost-search. It should take as inputs an initial state, a
function to generate neighboring states, and an evaluation function to say
how good each neighbor is. Neighbors could be formed by swapping two
letters, so there are 26*25, not 26! options. Or you could choose a random
subset of those. See page 232 of http://norvig.com/ngrams/ch14.pdf
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/aimacode/aima-python/pull/697#issuecomment-363558008,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AXEpPtRq-QIwIpbLMxFsUy_mYWdK8zHTks5tSLi2gaJpZM4RrrLP
.

I read the link, what you mentioned above is true for substitution cipher, but for the case of caeser cipher won't the sample space will be 26 ( 1 for each of the 26 keys).
Please correct me if I have misunderstood.

That's right -- for Caeser (shift) cipher, there are just 26 possibilities (25 without the identity shift), and you can try them all. No hillclimbing required.


Reduce number of operators known

Simplify such that we have less tOp defined:

opNotEqual can be eliminated in favor of tUnop(opNot, tBinop(opEqual, left, right))
opGreaterThanOrEqual can be eliminated in favor of tBinop(opOr, tBinop(opEqual, left, right), tBinop(opGreaterThan, left, right))
opLessThan can be eliminated in favor of tBinop(opGreaterThan, right, left) (note inversion of left and right)
opLessThanOrEqual can be eliminated in favor of tBinop(opOr, tBinop(opEqual, right, left), tBinop(opGreaterThan, right, left))

For an API standpoint on Number, it's nice to have convenience methods such as GreaterThanOrEqual etc. but we shouldn't need to dispatch to them within tBinop's Compute method.

@workmajj Figured you'd enjoy tackling this cleanup.

@pascallouisperez 👍


[ownership] Refactor out def-use compatibility checking out of gatherUsers() into a helper method.

This together with Operand::isConsumingUse() allows me to completely hide
OperandOwnershipKindMap from the main flow of the algorithm which makes it
easier to understand.

@swift-ci smoke test and merge

I did this using Xcode's refactoring engine!


struggling to understand llama parse node based parser's benefits

I’m struggling to understand Node based parser helps with the retrieval process. Even though there are PREVIOUS AND NEXT relationship attributes in each node, they are not included (encoded) when we do embeddings. Each node is parsed independently and doesn’t include explicit information about relationships like PREVIOUS or NEXT nodes when creating embeddings.
So my question is:
How does a node-based parser like LlamaParse improve retrieval if it doesn’t pass any relationship context (like PREVIOUS or NEXT) along with the node's content?
What’s the advantage of using a node-based structure for retrieval compared to using regular chunks of text without splitting it into nodes?
Is there an inherent benefit to node-based parsing in the retrieval pipeline, even if the relationships between nodes aren’t explicitly encoded in the embeddings?
I’d appreciate any insights into how node-based parsers can still be useful and improve retrieval effectiveness.

Its just a way to maintain references to where that node was from when doing retrieval. Sometimes you might have an algorithm to grab neighbouring nodes, or maybe you need to re-assemble the entire document. This makes it possible.


[rcore] GetMouse... Delta X and Y don't work correctly on web

WARNING: Please, read this note carefully before submitting a new issue:
It is important to realise that this is NOT A SUPPORT FORUM, this is for reproducible BUGS with raylib ONLY.
There are lots of generous and helpful people ready to help you out on raylib Discord forum or raylib reddit.
Remember that asking for support questions here actively takes developer time away from improving raylib.

Please, before submitting a new issue verify and check:

[x] I tested it on latest raylib version from master branch
[x] I checked there is no similar issue already reported
[x] My code has no errors or misuse of raylib

Issue description
Looking at the source code, it is clear that platform web is not supported by the functions mentioned above. I noticed this while working on my gamejam project. I might create a pr tomorrow. Note that the problem is not the mouse, but when using touch. I get there are the GetTouchX and Y functions, but touch doesn't give an option for when touch is released. If this would be used as a counter argument, then there is no reason why GetMousePosition does work.
Environment
Building for the web
Issue Screenshot
If possible, provide a screenshot that illustrates the issue. Usually an image is better than a thousand words.
Code Example
Can't currently provide one

@mandar1jn Are you sending a PR to solve this issue?


Color Picker shortcut doesn't work when minimized

Microsoft PowerToys version
"0.57.2"
Running as admin

[X] Yes

Area(s) with issue?
ColorPicker
Steps to reproduce
Press the shortcut buttons for color picker
✔ Expected Behavior
When the application is minimized in the system tray, pressing shortcut for color picker will trigger color picker option.
❌ Actual Behavior
Nothing happens pressing the shortcut button. But it works when the application is opened.
Other Software
No response

What application do you mean?

What application do you mean?

Microsoft PowerToys

/bugreport

PowerToysReport_2022-05-02-11-18-55.zip

I'm not sure what you mean by the "But it works when the application is opened.". Could you be more specific? Perhaps some screenshots will help.

/needinfo


Ensure spritefonts adders are calling library initialization function appropriately

https://discord.com/channels/724320164371497020/724320324375806123/1007080091844612106

https://github.com/JujuAdams/Scribble/commit/78937efb70806b24e0858c7e9e489147c755c96b#diff-12b7fbf851b4eb4fcd9400ac99726dba8845a93261b3242e22bcacd33b30daa3


Otel allocation profiling POC

What does this PR do?
Make libdd_profiling.so compatible for use with otel profiler by creating a new mode that does not start ddprof and adding a USDT probe inside sampling path.
(TODO) better description

[ RUN      ] allocation_tracker.start_stop
/go/src/github.com/DataDog/apm-reliability/ddprof-build/ddprof/test/allocation_tracker-ut.cc:141: Failure
Expected: (sample->dyn_size_stack) < (hdr->size - sizeof_allocation_event(0)), actual: 18088 vs 18088
[  FAILED  ] allocation_tracker.start_stop (323 ms)

[ RUN      ] allocation_tracker.start_stop
/go/src/github.com/DataDog/apm-reliability/ddprof-build/ddprof/test/allocation_tracker-ut.cc:141: Failure
Expected: (sample->dyn_size_stack) < (hdr->size - sizeof_allocation_event(0)), actual: 18088 vs 18088
[  FAILED  ] allocation_tracker.start_stop (323 ms)

Just saw that, USDT probe seems to have a big impact on stack size in sanitized mode.

I'm not sure if the crash is reproducible and a product of the PR

I agree on the TODO about a better description :smile:


Fixed support for python 3.8 can't use 'is' for comparing equality

Hello, thank you for creating this project it's very useful. I ran into an error when using python 3.8: "SyntaxWarning: "is" here is a fix.

Yay!  Thank you @alkrav !


updated postgresql command in v20.1 and v20.2

updated gssapi_authentication.md in v20.1 and v20.2 to have an updated postgresql command
@jseldess  Mentioning you here because the contribution doc specified that all pull requests be assigned to you.

This change is

Online preview: http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/cf7f8d58b80c7853d6cd98cf27c55ff6af08e6a5/
Edited pages:

v20.1/gssapi_authentication.md
v20.2/gssapi_authentication.md

Thanks, @zuliatowoade! I'll pass this to @Amruta-Ranade for review.


Update ORT training examples & add summarization example

What does this PR do?

[x] Replace metrics in the example from datasets to evaluate.
[x] Add summarization example for ORTSeq2SeqTrainer.
[x] Enable inference_with_ort in training examples.
[ ] Update Readme to introduce inference with ORT.

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

P.S. This PR is based on the implementation of ORT inference for ORTSeq2SeqTrainer,  it is built on the top of PR #189 .


enhancement: don't show all motions in the agenda

Currently all motions appear in the agenda, either as official agenda entry or as internal entry.
This is a big problem, because for larger assemblies (tens of motions) this pollutes the agenda. But an agenda is there to keep the overview. Therefore normally "motions" is the agenda point and the assembly managers will handle the different motions within this point. Maybe they add some main motions as subitem to this point, but never all motions because the motions menu is already the electronic version of the printed book of motions.
So it would therefore be very helpful if motions would not appear in the agenda, except

the option to appear in the agenda is set
an option to set it as internal entry

@donovaly The last year we done some improvements there. You can now collapse childrens of agenda items and use better (persistend) filter options to hide items. Is this issue solved in your opinion?

Yes, this is fixed with OS 2.3.


Add password when opening RarReader

When using the following code the program will crash because there is no password specified
var archive = RarArchive.Open(***);
var entries = archive.ExtractAllEntries();
while(entries.MoveToNextEntry()) { // cryptographic exception here
   // do something with entries
}

This pull requests passes the password to the RarReader

Oh I see.  If the Archive is SOLID and Password protected, it threw an exception?


Cannot debug installed package

Issue Type: Bug
It used to be possible to debug into the standard library or pip-installed 3rd party packages with the python extension for vscode.
At some point over the last months this has stopped working however. python.dataScience.debugJustMyCode: false is already set.
Example code:
import logging
logger = logging.getLogger(__name__)

Put a breakpoint on the last line and try to step into it with F11: The debugger does not step into logging.getLogger() but steps over it.

If not, "python.dataScience.debugJustMyCode" does not apply here. You need to specify "justMyCode":true in your launch.json instead.

Thanks for reminding me. - No I was trying to debug a python file.
I fell into this trap already twice before and keep forgetting it. The setting "python.dataScience.debugJustMyCode" REALLY need a warning that it applies only to nodebooks. Ideally with a not on how to enable justMyCode. I cannot be the only one that keeps stumbling with this.

@luabud, something to consider

Yes, that is a very good point. https://github.com/microsoft/vscode-python/issues/7296 for reference.

Since we have a separate issue to track the option, I'm going to close this one.


Enhancement: Duplicate host option

It would save a lot of time and effort to be able to configure a host, then duplicate the entry with a new name. Many hosts are setup with common options, and this would allow a host to be used as a template for new hosts, minimizing the configuration time.

Duplicate of https://github.com/vx/connectbot/issues/57


[2.0] Merge up to 1.9

This is a continuation of the work of #526, continuosly porting all the new stuff from the master branch into the 2.0. This is the list of merged PRs on master that I've ported here (by merge order on master):

[x] #543: docs fix
[x] #546: adding application/json input handling
[x] #547: docs fix bis
[x] #548: fix Client PHPDoc
[x] #551: fix syntax highlighting in the README.md
[x] #556: docs fix ter
[x] #557: use badge/poser in the README.md
[x] #555: Cleanup the 'site' value if it's empty site is a deprecated field, no longer present
[x] #564: add runtime info already covered by #562
[x] #570: fix official docs definition
[x] #572: handle non-default host port
[x] #573: prepare 1.8.4 release
[x] #578: Fix x-forwarded-proto detection will be handled with #594
[x] #583: exclude_exceptions with child types already handled in #487
[x] #554: allow serialization 5 levels deep
[x] #559: make string limit customizable
[x] #587: avoid looping when exception throws exception
[x] #586: Fix monolog handler not accepting Throwable
[x] #553: fix serializer to account for non-UTF-8 chars
[x] #588: fix for undefined variable not applicable, CurlHandler no longer present
[x] #590: update for 1.9.0 release
[x] #591: prepare 1.10.x release not applicable

I will handle those PHPT regression tests separately:

[ ] #571: add regression tests for fatal handling
[ ] #574: avoid double reporting
[ ] #575: force send with curl_method async
[ ] #576: more regression PHPT tests and fixes
[ ] #589: fix regression test

@ste93cry while trying to port #555 I discovered that the site field got lost during the development of the 2.x. Is it a custom field that we are ditching since we already have the standard server_name?

It was removed because it's a deprecated field

@ste93cry looking at #578 I discovered that the trust_x_forwarded_proto option is handled but never used... what should we do?

According to the security issues that could arise by using the X_FORWARDED_* headers I would remove the configuration option entirely and not take any action regarding support of this particular feature in the PHP SDK

This is no longer WIP, I will handle the PHPT tests separately, please review!

I know that this argument has already been discussed many times and widely, but I strongly disagree with gathering the POST data and deserialize it if it's JSON. Now that we have a well organized way to collect all information we and users need through middlewares everyone can easily add whatever they want to the event data, and the SDK specs also says:

This data should not be provided by default as it can get quite large

This is the main reason I did not collect POST data by default in the RequestInterfaceMiddleware middleware. If we really want to keep going this way, we should at least use the PSR-7 methods to get the data instead of using file_get_contents directly.

This data should not be provided by default as it can get quite large

Oh well, then I think we should create a separated middleware to do that!

This is the main reason I did not collect POST data by default in the RequestInterfaceMiddleware middleware. If we really want to keep going this way, we should at least use the PSR-7 methods to get the data instead of using file_get_contents directly.

What's the correct method to do it?

I think you should use the getBody() method of the ServerRequestInterface interface

Ok I've reverted the commit about #546, I will handle it in a separate PR where I will create a separated middleware to handle POST body, with JSON handling too.

#587 should not apply to 2.x as we don't set anymore the event_id property

The event_id was set in the code of 1.x. The core of the issue is an exception that throws an other exception on __set, which would be triggered in a number of ways; in 1.x was the event_id.

What I meant is that I don't see anything in the test that tries to set the event_id property on the exception object (which in turn should throw another exception), so how can the test work? By the way, are you sure that the same case is not covered by my error handler PR?

It's possible that is already covered by your PR, yes.


update version

@lkillgore,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot


Ipyvolume cannot be imported in Jupyterlite: KeyError: 'emscripten'

Reproduce

Open https://jupyter.org/try-jupyter/lab/
Open a new notebook
run a cell containing

import piplite
await piplite.install('ipyvolume')

run a cell containing

import ipyvolume

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
Cell In[2], line 1
----> 1 import ipyvolume

File /lib/python3.10/site-packages/ipyvolume/__init__.py:6
      4 from ipyvolume import styles  # noqa: F401
      5 from ipyvolume import examples  # noqa: F401
----> 6 from ipyvolume import datasets  # noqa: F401
      7 from ipyvolume import embed  # noqa: F401
      8 from ipyvolume.widgets import *  # noqa: F401, F403

File /lib/python3.10/site-packages/ipyvolume/datasets.py:13
      9     from urllib.request import urlretrieve  # py3
     11 import numpy as np
---> 13 osname = dict(darwin="osx", linux="linux", windows="windows")[platform.system().lower()]
     14 data_dir = os.path.expanduser("~/.ipyvolume/datasets")
     15 if not os.path.exists(data_dir):

KeyError: 'emscripten'

Context
I am trying to improve the user experience of running Panel in its custom build of Jupyterlite called Panelite in Panel #4388.
Now I am working with the notebook reference/panes/IPyWidget.ipynb which needs to import ipyvolume. But it fails.

@MarcSkovMadsen
don't know if it helps. But this is what I am doing in JupyterLite:
# cell 1
%pip install -q ipywidgets plotly ipyvolume ipython_genutils

# cell 2
def PatchIPyVolume(filename="/lib/python3.10/site-packages/ipyvolume/datasets.py"):
    # osname = dict(darwin="osx", linux="linux", windows="windows")[platform.system().lower()]
    with open(filename,"r") as f: lines=f.readlines()
    lines=["osname = platform.system().lower()\n" if line.startswith("osname =") else line for line in lines]
    with open(filename,"w") as f: f.writelines(lines)
PatchIPyVolume()

# cell3
import ipyvolume as ipv

This should be fixed in 0.6.1, thanks for reporting Marc!
Nice patch @scrgiorgio

Thx all.


hamming: update generator

For issue #632
Use .Header in template.
The generated cases_test.go has a new test case from canonical-data.json,
so update the test program and example solution to bump
testVersion and targetTestVersion to 6.

So an interesting question out of curiosity: Would it be helpful to take advantage of https://help.github.com/articles/closing-issues-via-commit-messages/ ? That way you would not have to manually go and close #632 after this gets merged. Well, all good if you prefer to do it manually, but I'm obligated to make sure the option has been explicitly known and rejected if it is to be rejected.


No Mention of Windows 11

This (and Associated Pages) don't make any mention of Windows 11 nor include the DDF files for Windows 11, does this mean that Windows 11 is not supported via CSP management?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 162c8434-b3ab-e235-43ce-ed435d6cd06e
Version Independent ID: 18b29b82-f1ad-81b8-2ea4-f7bebc506487
Content: Configuration service provider reference - Windows Client Management
Content Source: windows/client-management/mdm/configuration-service-provider-reference.md
Product: w10
Technology: windows
GitHub Login: @Dansimp
Microsoft Alias: dansimp

Hello @ld0614 ,
Windows 11 is supported via CSP management, but the documentation is not fully updated yet.
As you can see, CSP articles already have Windows 11 in the applicability table, but it will take some time until all the articles in the MDM section of documentation is updated.
Thank you


Activity 17.1 solution for Glances v3.1.3 with psutil v5.5.1

When I try the solution glances --disable-memswap --theme-white as seen in the pdf I encountered the error unrecognized arguments. The version I have is the following,
pax@bookr:~$ glances -V
Glances v3.1.3 with psutil v5.5.1

I also have the latest Ubuntu live server installed: 20.04.3 LTS
My solution was glances --theme-white --disable-plugin memswap and I think it's correct given the similarity in the screenshot. To figure out the name of the module, I used the argument --modules-list.

Sharing it here in case someone else encounters this, or to notify authors of the change in case PDF can be updated.

Thanks for finding this out. As I understand any fixes like this will be in the next edition, or some errata might be published.


Sidenav not working well with align end

The behavior for the sidenav is janky when opened from the right (align="end")
See demo site https://material2-dev.firebaseapp.com/sidenav for example

Has someone a workaround for this?
Thank you!

It seems this was fixed by #1990

@mmalerba  Is this fix on master branch?

@mmalerba  This commit you reference to is older than the reported issue

It does appear to have been fixed in master by #1990. The jank is no longer there on the demo site. BTW this issue was reported in the morning and the commit was pushed to master in the afternoon.

Thank you for clarification @andrewseguin, really appreciate.


Add mem_gb to SUSAN and MELODIC nodes to avoid memory errors

I have been having some issues with BrokenProcessPool errors (i.e., out-of-memory errors), and these usually occur around the melodic and susan commands. Would it be possible to add a mem_gb of a few gigabytes to both of these, so that the nipype scheduler knows not to start too many at the same time?
https://github.com/nipreps/fmriprep/blob/a45ea486a8aa58541add3370a60ae87a70af8efb/fmriprep/workflows/bold/confounds.py#L668-L673

Absolutely. Do you have a sense of how much they use, as a function of input file size?

I believe these nodes will only ever be called with resampled data in MNI152NLin6Asym space, so the only parameter would be the number of volumes. I do not have a sense of that though, I would need to set up a few test runs.
I think it may also be a challenge to estimate memory usage for melodic, because it infers the number of components it estimates automatically, and that number may also impact memory usage.

At a guess, 2x the uncompressed 4D array should be the dominant term in memory complexity. For SUSAN, the pre-smoothed and smoothed arrays. For MELODIC, the maximum number of components would be number of original volumes.
Might be another significant term I'm not thinking of, but we could try that to start.

I just checked, one of our uncompressed inputs to susan was 1.6 GB in size, but susan used more than 10 GB of memory. I'm actually not sure what kind of memory complexity it has.

Can some of it be explained by a type conversion? If you have int16 data, that'll be promoted to float or double. 2 x prod(shape) x sizeof(double) might get pretty close.

Yes, that seems to be exactly what is happening. I tested two different images, and the calculation you suggested seems to be accurate for the susan part, and even has a few gigabytes of safety margin. For melodic, it seems that 1 x prod(shape) x sizeof(double) would be sufficient.

I would be happy to contribute a pull request, but I'm not sure how deal with the type conversion. As far as I can tell, the workflow only has the mem_gb argument, and that contains the input file size. I don't think the workflow function will be able to tell if the file size refers to int16 data that is going to be converted or a file that is already float64.

I wouldn't worry about the type conversion and just assume it's double. So I think we just need to know the xyz size of the MNI volume to come up with a scale factor that we then multiply by the number of volumes in the series.

I think this can be transferred to fmripost-aroma.


Created a new tab for private and public projects

Closes #1842

@Cheshiriks thank you so much! May be you will also try to release it to saveourtool.com then?


Lay out a foundation for integration tests

This adds pytest-docker as a pytest plugin that allows to add a docker service as a fixture for test cases.
Parameterizes the Superset version, and allows to not load example data which takes a very long time.
Adds a very simeple test case, just to confirm that the integration test setup works.
Refs #36

@opus-42 If you create a free account on https://coveralls.io and configure this repo there we can add coverage reporting to the test suite as well.

@opus-42 If you create a free account on https://coveralls.io and configure this repo there we can add coverage reporting to the test suite as well.

Hello, this is a great idea !


Add is_system_group field to UserGroup.

First commit just adds the field to model.
Second commit restricts editing of system groups from API.
Third commit adds the new field to obejcts sent to clients.
Fourth commit hides the system user groups from settings UI and typeaheads.

Related Issue - #19525.

@timabbott updated the PR with the changes asked above.

@timabbott updated the PR with the changes asked above.

Resolved the conflicts

@timabbott this is ready for review.

Merged, after tiny tweaks to API documentation, thanks @sahil839!
I have a few follow-up requests:

We should make the is_system_group: true examples have a name appropriate for a system group (E.g. "moderators").
I do want the for_mention parameter (it'll be useful down the line when we make a permission for who can mention a given user group), but probably we don't want to allow mentioning system groups at this time, since effectively allowing that would let you circumvent existing wildcard mention notification restrictions.


Exception raise when specifying suffix to Vix.Vips.Image.new_from_enum/2

Image streaming working well. Tested with File.stream! and also ExAws.S3 with minio.  However when specifying a suffix the following error occurs:
iex(1)> stream = File.stream! "test/images/puppies.jpg", [], 2048
%File.Stream{
  line_or_bytes: 2048,
  modes: [:raw, :read_ahead, :binary],
  path: "test/images/puppies.jpg",
  raw: true
}
iex(2)> Vix.Vips.Image.new_from_enum(stream, ".jpg")
{:error, "Failed to create image from VipsSource"}
** (EXIT from #PID<0.377.0>) shell process exited with reason: an exception was raised:
    ** (MatchError) no match of right hand side value: {:error, "Broken pipe"}
        (vix 0.9.0) lib/vix/vips/image.ex:178: anonymous fn/2 in Vix.Vips.Image.new_from_enum/2
        (elixir 1.13.2) lib/enum.ex:942: anonymous fn/3 in Enum.each/2
        (elixir 1.13.2) lib/enum.ex:4144: anonymous fn/3 in Enum.each/2
        (elixir 1.13.2) lib/stream.ex:1559: Stream.do_element_resource/6
        (elixir 1.13.2) lib/enum.ex:4144: Enum.each/2
        (vix 0.9.0) lib/vix/vips/image.ex:177: anonymous fn/2 in Vix.Vips.Image.new_from_enum/2

This also results in an iex restart.
If specifying the suffix only as "jpg" the same error is returned but without an exception being raised:
iex(1)> stream = File.stream! "test/images/puppies.jpg", [], 2048
%File.Stream{
  line_or_bytes: 2048,
  modes: [:raw, :read_ahead, :binary],
  path: "test/images/puppies.jpg",
  raw: true
}
iex(2)> Vix.Vips.Image.new_from_enum(stream, "jpg")
{:error, "Failed to create image from VipsSource"}

This is mostly incorrect param name and documentation. The second parameter is not suffix, but loader options.

Load options may be given in option_string as "[name=value,...]"

Loader options are similar to what we specify for new_from_file ex: [shrink=2] for jpegload

Ahhhh, ok, got it.  In face in Image.open/2 I used a keyword list as options and then format them as loader options so I can reuse that for new_from_enum/2. I should have checked the libvips documentation, sorry for the noise.

In face in Image.open/2 I used a keyword list as options and then format them as loader options so I can reuse that for new_from_enum/2

Nice! we should do something similar here as well.
Btw this is a valid issue which needs to be fixed. Thanks for raising it :)


brought back the week locking

adds:

weeks can now be properly locked again (or just atleast lock overhaul)
on default: all weeks are unlocked
score reset option
settings reset option
(not at bottom of each page but resolves #572)

Doesn't this already work? Or did actually unlocking the weeks not work

well either way I find it much more comfortable this way

how about I change it that instead of it being a txt file this just puts all of the week data into one array ?


Don't import TypeGuard if TYPE_CHECKING is false

Description
Small patch to prevent issues when using older version of typing_extensions.
I don't think we should force people to upgrade as we only use TypeGuards for
type checking.
Types of Changes

[x] Core
[x] Bugfix
[ ] New feature
[ ] Enhancement/optimization
[ ] Documentation

Thanks for adding the RELEASE.md file!

Here's a preview of the changelog:

This release removes a TypeGuard import to prevent errors
when using older versions of typing_extensions.


go-playground/validator module is not imported with recommended URL

go-playground/validator module is imported with its github.com URL (github.com/go-playground/validator) as opposed to its gopkg.in (gopkg.in/go-playground/validator.v9) URL. The go-playground/validator README suggests importing it with its gopkg.in URL (gopkg.in/go-playground/validator.v9), and other projects follow this notation (e.g. gin-gonic/gin). This can be fixed by changing the URL where the module is imported.

I think this has been fixed, thanks!


Added missing tests for locationresolver

Wrote new tests to cover the unhandled statements in the NewFixPathLocationResolver function.
Overview

@matthyx Do you mind if I ask why is the code coverage action not working for the last couple of days? Did the API token expire?

@yonatanamz is this relates to the token issue?

Putting the code coverage action aside, I guess this PR is ready for a code review.


[Experiment] Re-render the schedule when the timezone changes

Mostly sending this PR so I get a deploy preview URL which I can use as a demo :)
If you want to actually land something like this, I'd want to make some changes first.

Screencast demo: https://www.youtube.com/watch?v=zKv7bbCravs
I can’t wait for the timezonechange event to become a thing so web developers don’t need the polling anymore.

You are such a troll for using that URL in the video 😂

If you want, I could create a separate PR that only adds the timezonechange listeners, without the polyfill. That would be the minimal change required to let us use the main CDS web app as a demo (because we’d then just run the polyfill in DevTools first).

Love it. I'm closing this since it was only used to trigger a deploy, which is now at https://deploy-preview-335--cds2019.netlify.com/devsummit/.


adding nodeSelector to the chart

As mentioned in this issue it would be handy to have nodeSelector added to the chart, especially because this is using hostvol and scheduling to another node would create a new world.

Tested out the templating but didn't deploy to a live cluster. Running helm template with a nodeSelector in the values files returned:
...
spec:
  nodeSelector:
    kubernetes.io/hostname: test
  containers:
  - image: lloesche/valheim-server:latest
    name: vailheim-server
...


[FEATURE] getWIFIIPV4Address

Get the device's WiFi interface IP address, if it is connected.

Thanks for this.  I believe it's nice to have-- I would just like for it to be consistent on the iOS side.  Do you know if iOS has the same problem this is resolving?  If yes, maybe we make this the default and create another method for getting all the IPs of any interfaces?

I don't know if iOS has the same problem. However, looking through the code, I saw that the method follows almost the same structure as the Android version (I could be wrong, I don't know much about Objective-C).
In ios/RNNetworkInfo.m in line 172: NSArray *searchArray = @[ IOS_WIFI @"/" IP_ADDR_IPv4, IOS_CELLULAR @"/" IP_ADDR_IPv4 ];, looks like it iterates through the interfaces WiFi first and Cellular second, so it will probably have the same problem described in issue #72.
For the getWIFIIPV4Address() in iOS, I just copied getIPV4Address() from ios/RNNetworkInfo.m and changed the following line:
+ Line 197: NSArray *searchArray = @[ IOS_WIFI @"/" IP_ADDR_IPv4 ];
So in getWIFIIPV4Address() it should only look for the device's WiFi interface.

Oh, sorry, I was looking at the PR on my phone and overlooked you had implemented it in iOS as well.  Great!
Let me think about the API a little.  I don't love having both getWIFIIPV4Address and getIPV4Address but you're right, this is a nice method to have.

A possibility is to have different methods for WiFi, Cellular and Ethernet (?). However, it might be a better idea to have a getIPV4Address with a parameter to select which interface you want to look for (defaulting on WiFi, because I think that would be the most common use).

A possibility is to have different methods for WiFi, Cellular and Ethernet (?). However, it might be a better idea to have a getIPV4Address with a parameter to select which interface you want to look for (maybe defaulting on WiFi, because I think that would be the most common use).

I am totally agree with this approach, as we might need to get the ip address for wifi or cellular.

A possibility is to have different methods for WiFi, Cellular and Ethernet (?). However, it might be a better idea to have a getIPV4Address with a parameter to select which interface you want to look for (maybe defaulting on WiFi, because I think that would be the most common use case).

What about, for now, if we change it so the getIPV4Address method will return the wifi IP first, if it exists.  Otherwise, it will return the cellular IP?  I agree that getting the local wifi address is the most common use case.
I think a bigger, but nicer, future change we could just have one method getIPAddress and it would return a hash of all IPs with details about each (wifi, cellular, loopback, etc).

What about, for now, if we change it so the getIPV4Address method will return the wifi IP first, if it exists. Otherwise, it will return the cellular IP? I agree that getting the local wifi address is the most common use case.

I adapted the code. Checking if the WiFi address exists is done in JavaScript. However, I don't know TypeScript, so I don't know if the file NetworkInfo.d.ts needs to be changed too.

I adapted the code. Checking if the WiFi address exists is done in JavaScript. However, I don't know TypeScript, so I don't know if the file NetworkInfo.d.ts needs to be changed too.

No, that should still be fine.  Thanks for making those changes!  I made one quick code formatting change.  I'm traveling so I don't have access to my Android device to test on.  Can you give it one last test using my latest changes?  If it's okay, I think we are good to merge this one.
Thanks for your help!

I have tested it on my Android device, Google Pixel 3. This is what I am getting:
WiFi & Cellular enabled: '192.168.1.xx'
WiFi enabled: '192.168.1.xx'
Cellular enabled: '10.xx.xx.xx.xx'
None: '<IP_ADDRESS>'


Learn Django - First Steps

Learn the Django Framework enough to be able to accomplish the below tasks :
Create project and app
Create Models and migrate them to the database
Create Controllers (C of MVC)  which is corresponding to the Views in Django
Create Views (V of MVC) which is corresponding to the Templates in Django
Create and use URL patterns to the apps from project
Create and use URL patterns to the views from app
Create a Masterpage ( Template inheritance ) and blocks ( content blocks )
Use a Masterpage in a content page
Create "Static" folder to hold css,js,images etc folder and files.
Create and use links and references to static folder files.

Done.


Finding uncaught checked exception

https://discuss.kotlinlang.org/t/in-future-could-kotlin-have-checked-exception/1579/8

I would very much like a static analysis option to show all the possible lines where checked exceptions could be generated and escape (uncaught).

Even though checked exceptions are very bad for functional programming, it's still pretty useful sometimes to have this check to know what exception we forgot to catch. :)

I am guessing this would need type resolution as we would have to check the signature of each function that we call to see if it would throw?

Yes. And then we probably need to handle dependency issues (exception declared in dependencies)... If only JetBrains could do this in their compiler or IDE...

Furthermore, one would also have to check java sources when interop is used.

Just curious is there was any progress on it?
@3flex do you have some idea how to implement it?

No progress, but I was planning to look into it soon.
It should be easy enough to determine if the callee throws an exception that is a subclass of Throwable (but not RuntimeException or Error) but will have to see how that works with Java interop as mentioned above.
If it's challenging I might abandon it for a while as I'm focusing on false positives, bugs and low hanging fruit at the moment.

@3flex Thanks for details!
I have interest in providing such functionality so would like to try to contribute if you will not have time.

That would be great! Please go ahead, the more external contributes we have the better :)

@arhont375 do you have updates on this rule for us? We gladly assist if you have any questions or need help.

Sorry for the delay, so many things happened since then and I got a bit busy so after initial investigation I didn't do much :cry:
If someone else would be able to take it please go ahead


limit amount of repositories after language filter

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@b8dc50d). Click here to learn what that means.
The diff coverage is 100%.

@@            Coverage Diff            @@
##             master      #66   +/-   ##
=========================================
  Coverage          ?   75.66%
  Complexity        ?       43
=========================================
  Files             ?       14
  Lines             ?      226
  Branches          ?       14
=========================================
  Hits              ?      171
  Misses            ?       51
  Partials          ?        4

Impacted Files
Coverage Δ
Complexity Δ

...ikov/mightywatcher/external/RestGithubApiClient.kt
81.63% <100%> (ø)
5 <0> (?)

...igorperikov/mightywatcher/service/GithubService.kt
84.61% <100%> (ø)
5 <0> (?)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b8dc50d...31a3a23. Read the comment docs.


Have you thought about a 'fork me on github' ribbon?

https://github.com/blog/273-github-ribbons - I'll add and PR if others think it's a good idea.

I DO think this is a good idea.  Please make sure it resizes nicely with the theme.
I'll add you to the repo.

Take a look at #2


Fix #950 - Streamline C2D README

Fixes #950
Changes proposed in this PR:

Reduce to one line creating nft, datatoken and asset for dataset
Reduce to one line creating nft, datatoken and asset for algorithm

Note: this PR's title and description were pointing to a github issue that was specified confusingly (title was different than description; and the issue itself was duplicate). So: I closed that github issue, and have the changed the title and description of this PR to point to the correct github issue.

Overall, great to see this. I'm glad you were able to take advantage of create_url_asset() to chop complexity down. As well as metadata updates.
Before the next request for review, please make sure that the end-to-end flow has been tested to work. The current one doesn't for sure, since there would have been an issue due to the missing Service (as I pointed out in a comment). And there will likely be an issue due to referencing services[1] not services[0] in two places (as I also pointed out).

The code that is already in the repo runs well, I did this several times a couple of days ago.
I did the same with the new one.
There are issue running this in mumbai because of the time that it takes to get things from Aquarius.
The comment about the services was addressed in a previous comment, there are two services.

The comment about the services was addressed in a previous comment, there are two services.

Sure, there are two services now. And there were two before, right? So why is a different service being pointed to? (services[1] vs services[0]). Perhaps the order was reversed?

There are issue running this in mumbai because of the time that it takes to get things from Aquarius.

To be clear: you can run with your local repo, and with a remote network (Mumbai). And that's what you should be doing.
C2D README explicitly runs with remote components, not local, because local causes many issues, and it isn't a practical flow anyway.

Details: up until about 6 weeks ago, we did have C2D README on local. But we had issue after issue because via barge, we end up running docker-in-docker which is error-prone and tedious to debug. Then we realized that it wasn't a practical flow anyway, and so the obvious fix was to go remote. And we did, via ocean.py#909.
From that work, we also found that we didn't have any other simpler remote READMEs, so we did that too, via ocean.py#933.
From both those issues, we also realized that we didn't have any unit / CI testing for remote networks. So we did that too, via ocean.py#923.
In short: there's history on this! And the punchline is that we C2D README needs to be remote. And therefore working well for remote

There are issue running this in mumbai because of the time that it takes to get things from Aquarius.

To be clear: you can run with your local repo, and with a remote network (Mumbai). And that's what you should be doing.
C2D README explicitly runs with remote components, not local, because local causes many issues, and it isn't a practical flow anyway.

Details: up until about 6 weeks ago, we did have C2D README on local. But we had issue after issue because via barge, we end up running docker-in-docker which is error-prone and tedious to debug. Then we realized that it wasn't a practical flow anyway, and so the obvious fix was to go remote. And we did, via ocean.py#909.
From that work, we also found that we didn't have any other simpler remote READMEs, so we did that too, via ocean.py#933.
From both those issues, we also realized that we didn't have any unit / CI testing for remote networks. So we did that too, via ocean.py#923.
In short: there's history on this! And the punchline is that we C2D README needs to be remote. And therefore working well for remote

Yes, I am running this with my local repo and in mumbai as I stated in my last comment. It seems, for your comment, that you have missunderstood or that I was not able to explain myself correctly.
The comment about mumbai is just to make everyone aware (I also shared this yesterday in slack) that you should await for at least 5 minutes to get things from aquiarious.
Let me know if there is something else that you think should be done. Otherwise as soon as you approve the PR I will merge it.