Questions about the linux patch

I have installed the 4.19 kernel version as instructed on a physical machine. And I need to reinstall the NIC driver. However, I get an error about missing /lib/modules/linux-4.19-xx/build .
Therefore, I would like to ask you whether you installed the linux patch on a virtual machine or a physical machine and whether you have encountered problems that require reinstalling drivers such as the NIC driver.

I would also like to know if there are any other effects on the physical machine as a result of changing a custom kernel version

Hi @intothesea,
Some notes regarding your questions.

In our setting, we had multiple servers dedicated to different tasks, some using our own patched kernel, some using vanilla versions. For example, the whole training process was conducted on servers with vanilla kernels. If you read the SIGCOMM'23 paper, you'll realize that it was part of our key motivations to be able to simply use the existing general-purpose ML clusters (without any change) for the training part.

We didn't use any VMs in our setting.

For the patched servers, tbh I don't remember any issues with NIC drivers. When generating the Debian kernel files, I tried to enable different general drivers by default. That said, if time permits (which unfortunately is not the case in a near future for me), I'll try to clean-up the source code of the patched kernel and enable interested users to build their own patches (and enable/disable diff drivers).

re the effects on the physical machine: no, there wouldn't be any impact on "physical" machine.

Hope these help.

Hi @intothesea, Some notes regarding your questions.

In our setting, we had multiple servers dedicated to different tasks, some using our own patched kernel, some using vanilla versions. For example, the whole training process was conducted on servers with vanilla kernels. If you read the SIGCOMM'23 paper, you'll realize that it was part of our key motivations to be able to simply use the existing general-purpose ML clusters (without any change) for the training part.
We didn't use any VMs in our setting.
For the patched servers, tbh I don't remember any issues with NIC drivers. When generating the Debian kernel files, I tried to enable different general drivers by default. That said, if time permits (which unfortunately is not the case in a near future for me), I'll try to clean-up the source code of the patched kernel and enable interested users to build their own patches (and enable/disable diff drivers).
re the effects on the physical machine: no, there wouldn't be any impact on "physical" machine.

Hope these help.

First of all, thank you very much for your response. I will continue to work on resolving the issues with the patched Linux kernel. I installed Ubuntu 16.04 alongside Windows 11, and I suspect that the problems I'm experiencing might be related to a faulty kernel installation, causing issues with network cards, graphics drivers, and other components.
Secondly, I would like to confirm the role of this kernel. Is it the TCP kernel module mentioned in the paper 'Computers Can Learn from the Heuristic Designs and Master Internet Congestion Control'? It is used for user-space and kernel-space interaction, providing observation values and executing actions based on model outputs (such as the 'getsock( sock_for_cnt[i], SOL_TCP, TCP_INFO, (void *)&sage_info, &sage_info_length)' and 'setsockopt(sock_for_cnt[i], IPPROTO_TCP,TCP_CWND, &target_ratio, sizeof(target_ratio))' codes for TCP congestion control).

I am now relatively certain that the issue is related to the code '#include <sys/socket.h>' line28 in the '/usr/src/linux-headers-4.19.112-0062/include/linux/if.h' file, which references a user-space library. This issue seems similar to the one described in this GitHub issue: https://github.com/openzfs/zfs/issues/10601.
In fact, there may be other problems with the kernel module that are preventing the installation of network card drivers and NVIDIA GPU drivers (basically, anything that relies on kernel header files). I would like to know if you can confirm this issue. When using Sage for inference, did you use only the CPU, or were there any GPU-related dependencies?

Hi, the main point of having a patched kernel is to enable TCP PURE that you need to run Sage during the inference.
re the inference, for the SIGCOMM'23 paper, we only used CPU.
Hope that helps

I have encountered the same question. I cannot install NVIDIA drivers with this version of kernel, It seems I can only use CPU to run sage..

I have encountered the same question. I cannot install NVIDIA drivers with this version of kernel, It seems I can only use CPU to run sage..

Train the model using GPU and then test it with CPU ⁺◟('꒳')


[Bug] Destroy object by itself cause crash

self.scene.removeObject(self.owner)

Fixed with build#106


Hibernate region_prefix configuration doesn't apply to redis key redisson-hibernate-timestamp

Redis version
Any
Redisson version
3.38.1
Redisson configuration
I don't believe the Redisson config makes a difference but this is the Spring Boot/Hibernate config to set a Hibernate L2 cache region prefix.
spring:
  jpa:
    properties:
      hibernate:
        cache:
          region_prefix: test-prefix

What is the Expected behavior?
When the region_prefix Hibernate cache configuration is set then all redis keys should be prefixed with that prefix, including the key redisson-hibernate-timestamp.
What is the Actual behavior?
The redisson-hibernate-timestamp is not prefixed.
Other keys are prefixed correctly.
Additional information
No response

Can you share a sample application to reproduce?

Sure, here is an example application https://github.com/rockinrimmer/redisson-region-prefix-example
If you run this Spring Boot app with redis running locally, you should see a redisson-hibernate-timestamp key with no prefix.
Something like this:


Ensure cross lane functions are device qualified only

Cross-lane ops use GPU backends. Should only qualify them for device usage.

@dlangbe  or @mkarunan please review


serving from production build locally

So I am using this in my app and love it. I do have a problem. When I run the dist of the example in redux-oidc-example the application can't seems to parse the hash/url fragment returned by google because the webserver reports a 404 on /callback and basically the callback component never gets mounted.
(i am using npms http-server to serve the spa using your) I am getting a similar problem against another oidc provider.
It seems like I need to somehow tell my webserver that when the provider calls /callback it should pass that to the app. Do you know how this would be done in express?
basically how do i expose my apps callback? i see your prod server doesn't have anything that might give me a clue about this. TIA

Figured it out


Ratelimits

Hi!
I loooove your Module, and very very thanks for making it. I wanna use this module in an app of mine, that is in development right now. That app would involve searching and downloading files. So I wanted to know if there are any ratelimits involved. If there are any ratelimits, then I will have to think of any other way, but good work! Would be greatful  if u either tell about the ratelimits or link me to the docs or both...
Thank you and good work...

Hey, @Hunter2807 I just had the same question, seems to me by this point you would have enough experience dealing with mega.py is there a ratelimit or how much is it?
Thanks :)

Hi there @OttomanZ!
Sorry but i didn't use this module in the app. More of, the app development has been ceased. Sorry.

I am proceeding to go ahead and run mega.py on my web application for easier management, but I guess I will then surely have a fall back to Dropbox in case mega.py goes ahead and gives it up. I guess that would be much safer bet, or I could by a proxy list and with anonymous_login off to the races.


scala-runners: init at unstable-2020-02-02

Resubmission of  #84068  (apparently you can't push changes and then reopen a PR on github?)

Motivation for this change
It's a good script and useful for testing with pre-release and non-current versions of Scala.
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[x] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
21.11 Release Notes (or backporting 21.05 Relase notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

the title of the pr and commit should be scala-runners: init at unstable-2020-02-02

As it now is. Thanks.


net: shell: Ping command needs target host set

The ping command was not checking if the user gave target
host as a parameter. This would lead to NULL pointer access.
Fixes #4827
Signed-off-by: Jukka Rissanen<EMAIL_ADDRESS>

recheck


Compile desktop app on linux and mac

Unfortunately, our current desktop app is a monolith...
Are you suggesting to rewrite it to use the web-service, or to just use Microsoft EWS API on linux and mac?
Native mac client for the existing server looks like a low-hanging fruit - we can use the same backend code as in iPhone and Apple Watch applications.

there is RESTful API for Exchange, there is probability to use it instead of EWS API


exclude multh-files to a extern repo

My multh_listworker.hpp  should be external library.
Its generic, so it can have its own repo.

Created a repo. now I must figure out how to import an external repo...

git submodule with a bunch of links should do it


Create LICENSE

Adding an Apache 2.0 License

I'll do another pass. This is going to be true of everything in the org.


notebook() fails in new installation of IJulia in Julia 0.7 on Ubuntu 16.04

I have just installed IJulia under Conda in Julia 0.7, but...
(v0.7) pkg> add IJulia
 Resolving package versions...
  Updating `~/.julia/environments/v0.7/Project.toml`
  [7073ff75] + IJulia v1.11.1
  Updating `~/.julia/environments/v0.7/Manifest.toml`
  [7073ff75] + IJulia v1.11.1
  [739be429] + MbedTLS v0.6.0
  [b85f4697] + SoftGlobalScope v1.0.4
  [c2297ded] + ZMQ v1.0.0

(v0.7) pkg> build IJulia
  Building Conda ──→ `~/.julia/packages/Conda/m7vem/deps/build.log`
  Building ZMQ ────→ `~/.julia/packages/ZMQ/ABGOx/deps/build.log`
  Building MbedTLS → `~/.julia/packages/MbedTLS/Qo8TN/deps/build.log`
  Building IJulia ─→ `~/.julia/packages/IJulia/k5o7j/deps/build.log`

julia> using IJulia
[ Info: Recompiling stale cache file /home/troels/.julia/compiled/v0.7/IJulia/nfu7T.ji for IJulia [7073ff75-c697-5162-941a-fcdaad2a7d2a]

julia> notebook()
Process(setenv(`/home/troels/.julia/packages/Conda/m7vem/deps/usr/bin/jupyter notebook --NotebookApp.iopub_data_rate_limit=2147483647`; dir="/home/troels"), ProcessExited(1))

...and nothing else happens, i.e. the notebook does not open. For what it is worth, IJulia worked flawlessly under Julia 0.6.4 on the same machine.
Is this a bug or something I've messed up?

Does running /home/troels/.julia/packages/Conda/m7vem/deps/usr/bin/jupyter notebook manually in the terminal work?

No, unfortunately not:
troels@troels-700Z7C:~$ /home/troels/.julia/packages/Conda/m7vem/deps/usr/bin/jupyter notebook
[I 08:51:15.143 NotebookApp] Writing notebook server cookie secret to /run/user/1000/jupyter/notebook_cookie_secret
[I 08:51:15.663 NotebookApp] Serving notebooks from local directory: /home/troels
[I 08:51:15.663 NotebookApp] The Jupyter Notebook is running at:
[I 08:51:15.663 NotebookApp] http://localhost:8888/?token=cad702ce02da537a672d7e102f618a6aebc1c1f2e6f64f84
[I 08:51:15.663 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
Traceback (most recent call last):
  File "/home/troels/.julia/packages/Conda/m7vem/deps/usr/lib/python3.7/webbrowser.py", line 58, in get
    command = _browsers[browser.lower()]
KeyError: '/usr/bin/google-chrome'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/troels/.julia/packages/Conda/m7vem/deps/usr/bin/jupyter-notebook", line 11, in <module>
    sys.exit(main())
  File "/home/troels/.julia/packages/Conda/m7vem/deps/usr/lib/python3.7/site-packages/jupyter_core/application.py", line 266, in launch_instance
    return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
  File "/home/troels/.julia/packages/Conda/m7vem/deps/usr/lib/python3.7/site-packages/traitlets/config/application.py", line 658, in launch_instance
    app.start()
  File "/home/troels/.julia/packages/Conda/m7vem/deps/usr/lib/python3.7/site-packages/notebook/notebookapp.py", line 1697, in start
    browser = webbrowser.get(self.browser or None)
  File "/home/troels/.julia/packages/Conda/m7vem/deps/usr/lib/python3.7/webbrowser.py", line 60, in get
    command = _synthesize(browser)
  File "/home/troels/.julia/packages/Conda/m7vem/deps/usr/lib/python3.7/webbrowser.py", line 116, in _synthesize
    register(browser, None, controller, update_tryorder)
TypeError: register() takes from 2 to 3 positional arguments but 4 were given

Since the error occurs when you run jupyter notebook manually, it is unrelated to IJulia and should be filed with the Jupyter project if it is still occurring.


Obsoletion notice: GO:0000188 | inactivation of MAPK activity & related

Dear all,
The proposal has been made to obsolete the following terms:
GO:0000188 | inactivation of MAPK activity'
GO:0051389 | inactivation of MAPKK activity'
GO:0051390 | inactivation of MAPKKK activity'
The reason for obsoletion is that those represent molecular functions. There are 72 annotations to those terms:
Contributor   (Assigned by)
ARUK-UCL | 1
CGD | 2
FlyBase | 1
GOC | 1
HGNC-UCL | 1
MGI | 8
PomBase | 5
SGD | 36
UniProt | 10
WB | 6
ZFIN | 1
Annotation review ticket is here: https://github.com/geneontology/go-annotation/issues/3736
There are no mappings to these terms; these terms are not present in any subsets.
You can comment on the ticket here: https://github.com/geneontology/go-ontology/issues/21308
We are opening a comment period for this proposed obsoletion. We’d like to proceed and obsolete this term on May 15, to leave time for groups to make corrections. Unless objections are received by May 15, 2021, we will assume that you agree to this change.

Hi @suzialeksander @marekskrzypek @sabrinatoro
Your annotations are not fixed yet, according to the spreadsheet. Can you please have a look?
Thanks, Pascale


Fix dependency mismatch in Playground Java runner

fixes #24058
We are fixing the base image for java runner container base beam java SDK image version and bumping the grpc vendor version

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Choose reviewer(s) and mention them in a comment (R: @username).
[ ] Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

R: @eantyshev

R: @damccorm


Examples for installing SMA using SC2016

Hey,
Can you post some examples of DSC configuration scripts for installing SMA using System Center Orchestrator 2016?

Can you describe the scenario a bit more, what you try to accomplish? What is the reasoning for wanting to do this with System Center Orchestrator 2016?

I'm not working on this anymore. I wanted to install only SMA using the software package of SC Orchestrator 2016. This was a long time ago, and as I didn't get any answers at the time I put this on the side.

@bgelens Aha, I see!
@ricardogaspar2 There are examples here; https://github.com/PowerShell/xSCSMA/blob/dev/Examples. Although not sure if they are tested with SMA that comes with SCO2016. There are also more resources here that should be moved here (issue #13); https://github.com/robwi/xSCSMA/tree/master/DSCResources


Animals table

Project milestone: Create animals table
Highlights:

Create a database named vet_clinic.
Insert data
Screenshots:
Screenshot of the ff queries:
Find all animals whose name ends in "mon".
List the name of all animals born between 2016 and 2019.
List the name of all animals that are neutered and have less than 3 escape attempts.
List the date of birth of all animals named either "Agumon" or "Pikachu".

Screenshots of the remaining queries:

Thank you 👍


Make header translucent on scroll down

Checklist

[x] I have read the Contribution & Best practices Guide.
[x] My branch is up-to-date with the Upstream development branch.
[x] I have added necessary documentation (if appropriate)

Short description of what this resolves:
The PR makes the header translucent when the user scrolls down. The original motivation it to make it look similar to the FOSSASIA website, as mentioned in #2108.
Changes proposed in this pull request:

The PR adds an additional class to the navbar, that is specifically used to get the DOM node of navbar inside JS.
Adds JS that executes on the main event page (homepage), can be expanded by moving JS to footer.hbs, need suggestion on this!

Closes #2108

@notlmn Put the js code inside the navbar.js(https://github.com/fossasia/open-event-webapp/blob/development/src/backend/assets/js/navbar.js) file.
Also it will be great if you could share a link of sample event(gh-pages of the generated event) here. It will help move things quickly. For event samples please check https://github.com/fossasia/open-event/tree/master/sample

@notlmn Put the js code inside the navbar.js(/src/backend/assets/js/navbar.js@development) file.

Done!

Also it will be great if you could share a link of sample event

Published to https://notlmn-dump.github.io/opev-gen-googio using the GoogleIO17 sample!

(PS: There maybe a problem with the header logo which is not caused by this PR, that may be another issue for #2106 to handle)

Great work. LGTM :+1:
Please squash the commits.

Implement disappear on scroll up as on chat.susi.ai/overview as mentioned in #2085.

@shubham3211 I didn't want this PR to close @2085 because the last point in that issue is a bit unclear, I thought it better be another PR that implements hide-on-scroll and the other unclear point!

Please squash the commits.

This could be done in GH UI too right? How would this be helful if I locally squash commits and force push to GH? I thing it would be too hard to follow the progress if squashed!

:tada: Yay!

@notlmn I asked you to squash the commits because the third commit is not relevant, there is no need for anyone to know that you moved the code from event.hbs to navbar.js .
Great Work keep contributing :+1: .


refactor(ingest/unity): Use databricks_sdk for all requests

Checklist

[ ] The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
[ ] Links to related issues (if applicable)
[ ] Tests for the changes have been added/updated (if applicable)
[ ] Docs related to the changes have been added/updated (if applicable). If a new feature has been added a Usage Guide has been added for the same.
[ ] For any breaking change/potential downtime/deprecation/big changes an entry has been made in Updating DataHub

Note to self: remove databricks_cli dependency

Closing as changed merged by https://github.com/datahub-project/datahub/pull/8238


Quoted arguments are mishandled

A quoted search expression, e.g. :Ack "Error code", fails as follows:
|| [Search results for '"Error']
zsh|1| unmatched "

Notice it is searching for "Error instead of "Error code".
Maybe use <arg>? I believe that handles shell quoting in the normal manner.

May want to check how ack.vim is able to handle quoted arguments correctly.
https://github.com/mileszs/ack.vim/blob/36e40f9ec91bdbf6f1adf408522a73a6925c3042/autoload/ack.vim#L50-L52

Opened PR #10 to fix this issue.

FYI: Same problem, using ag/(f)grep to find
'foobar = foo(bar)'
yields:
1 || [Search results for ''\''foobar']                                                                                2 || /bin/bash: -c: line 0: syntax error near unexpected token('
3 || /bin/bash: -c: line 0: ag --vimgrep  -riF ''\''foobar' = foo(bar)' *' 4 || [Search command exited with status 1]


Experiment with Bolt12

See https://github.com/getAlby/hub/issues/47

I hear it more often from users, signaling demand! :)
https://feedback.getalby.com/-alby-hub-request-a-feature/posts/bolt-12-support-

There's an open PR here: https://github.com/getAlby/hub/pull/307
It needs a bit more work and planning. We will do a BOLT 12 hackday in the future and can maybe get some minimal BOLT-12 features in

@redimongo note that this is not limited to Bolt12 - you can already do this with lightning addresses or NWC. Bolt12 simply allows the invoice to be requested over the lightning network, rather than using an external protocol.

@redimongo as long as you have the connection secret of the user's wallet, you can create an invoice from your own wallet, and pay it with the user's wallet.
We have an example app that does scheduled payments based on this idea:
https://zapplanner.albylabs.com/
Most of the complexity is setting up the scheduling to actually make payments on time. ZapPlanner also has some support for setting up subscriptions with some data attached to them. e.g. this link
Source: https://github.com/getAlby/ZapPlanner

@redimongo NWC should work for all your requirements. As long as you keep track of paid invoices for users in your system, you can tell how much each user has paid.

It would be great to generate a static payment identifier Bolt12 offer that I could share. Even better if my friends & family sub accounts could get their own static payment identifier.


Share message as an env variable

The default tweet text should be config sensible for each deployment.

@slifszyc you may check the DEMOS fork for additional stuff on these things. We really customized that one.

Yeaaaaaaaaaahhh... no, we shouldn't.
https://github.com/Legislatura/demos/blob/demos-phase2/lib/proposal-article/article.jade#L92

:+1:


Color performance adjusted curve for coded_quantizer map

This change is relevant to issue #8. Commit https://github.com/barrbrain/daala/commit/6066bf0f30ee62e69f46f758f35d05293f39b485 - Use 3/4 of the existing quantizers and distribute along a quadratic curve. Subjective improvements in chroma error observed at 0.1, 0.04 and 0.02bpp. This gives an idea of how improved chroma quantization matrices should be distributed.
I made a visualization of the effect of this curve on stills in subset1:
https://youtu.be/sO0whnADzFk

I made a visualization of the effect of this curve on stills in subset1: https://youtu.be/sO0whnADzFk

For commit https://github.com/barrbrain/daala/commit/ca50754369054bbacba3deab7aaf162f5c803fb0 I measured rate distortion with an objective color metric that has been calibrated against subjective scores ("Color Image Quality Assessment Based on CIEDE2000" Yang Yang, Jun Ming and Nenghai Yu, 2012). I controlled against flattening OD_DC_RES to all 17 as that change has a significant subjective impact at low rates - skies tend to go grey without it. For the "subset1" still image suite I measured a 7.15% gain. Previous tests demonstrate this is at a penalty of around 2.2% luma-only PSNR.

For commit https://github.com/barrbrain/daala/commit/941557f6c80f975ce49da03098b7dbefc368f8a6 we see a total gain of 13.5% in the previously mentioned metric.

(PSNR label should read log2-mean-CIEDE2000)

Refs (and possibly fixes) #8 #55 #94

Luma only BD rate (ntt-short-1):
           RATE (%)  DSNR (dB)
    PSNR 4.67451  -0.13996
 PSNRHVS 4.37615  -0.20431
    SSIM 4.51729  -0.10501
FASTSSIM 4.19643  -0.11438

BD rate, all planes, vs just flat OD_DC_RES (subset1):
              RATE (%)  DSNR (dB)
CIEDE2000  -9.66866   0.0692133
  PSNRHVS  -2.20841   0.156146
     SSIM  -2.79758   0.115485
 FASTSSIM -13.2972    0.348243

BD rate for all planes and complete patch (subset1):
              RATE (%)  DSNR (dB)
CIEDE2000 -13.4864    0.103007
  PSNRHVS  -3.84045   0.256033
     SSIM  -3.44664   0.14143
 FASTSSIM -16.1921    0.429639

An rational quadratic approximation of the curve with 0.062 MSE: 202/250-(x-1)*(x-251)/240

For commit https://github.com/barrbrain/daala/commit/8b487c3f2da8fbd38afd5404e75442ca1eaf12c0 we see a total gain of 14.7% in the previously mentioned metric.

Luma only BD rate (ntt-short-1):
           RATE (%)  DSNR (dB)
    PSNR 5.63782  -0.16973
 PSNRHVS 5.35094  -0.25057
    SSIM 5.47958  -0.12749
FASTSSIM 5.16922  -0.14113

BD rate, all planes, just flat OD_DC_RES (subset1):
              RATE (%)  DSNR (dB)
CIEDE2000  -4.41287   0.0337937

BD rate, all planes, complete patch (subset1):
              RATE (%)  DSNR (dB)
CIEDE2000  -14.6804   0.112439

I've attempted to find a simple continuous function with similar shape to the best curve so far.
As a function of coded_quantizer, producing an offset: a*pow(sin((x/64)*pi/2)^2, b)
Best parameters found: a = 11.304068080811136, b = 1.4619592172471414
The following shows the optimal curve, the smooth curve and the pow-2.25 curve:

So, there's still a gradient flipping issue with the visual results. I observe that it goes away when I encode with --no-activity-masking. It also only occurs in the Y plane but that is not surprising if it is related to activity masking.

Commit https://github.com/barrbrain/daala/commit/d06b4ddf464efa197b73d61311cc5b5695cb7b01 is https://github.com/barrbrain/daala/commit/ec33da59220dba8896e803cee7fc13624d08075c rebased after the unexpected skip bug was fixed in https://github.com/barrbrain/daala/commit/fa67c327a51710f3165ddcfe82e68c0d533a79bc
This was blocking the final visualization which is now available: https://youtu.be/pjWQRvi3VD8

Commit https://github.com/barrbrain/daala/commit/ad49582ddc9b795b6ea80d7f6969ca9b24784560 translates simplified versions of these curves into scaling prior to quantization. Overall color performance is comparable to commit https://github.com/barrbrain/daala/commit/941557f6c80f975ce49da03098b7dbefc368f8a6 but has a lower impact on luma-only metrics. It updates existing scaling and interpolation tables rather than changing the meaning of coded_quantizer.
Luma only BD rate (ntt-short-1):

           RATE (%)  DSNR (dB)
    PSNR 4.14342  -0.12667
 PSNRHVS 4.08045  -0.19292
    SSIM 4.08956  -0.09648
FASTSSIM 3.85425  -0.10587

BD rate for all planes (subset1):

            RATE (%) DSNR (dB)
CIEDE2000 -13.3345 0.100967

Can you also submit your chroma metric in a separate PR?

@tdaede Sure, I'm happy to submit the metric. Would it be best to create a C version first? I got part way through hacking one together before giving up and using python.

Commit https://github.com/barrbrain/daala/commit/cc6687b245056f2979e10b8372793836add2f3ec fully specifies the values of OD_DC_RES within the Q4 range (5 through to 17). Overall color performance is equivalent to commit https://github.com/barrbrain/daala/commit/941557f6c80f975ce49da03098b7dbefc368f8a6 to 4 significant figures.
Luma only BD rate (ntt-short-1):

           RATE (%)  DSNR (dB)
    PSNR 4.26355  -0.13027
 PSNRHVS 4.18239  -0.19767
    SSIM 4.17944  -0.09857
FASTSSIM 3.92156  -0.10771

BD rate for all planes (subset1):

            RATE (%) DSNR (dB)
CIEDE2000 -13.4849 0.102379

So I think we can get extremely close to the AC curve in commit https://github.com/barrbrain/daala/commit/ec33da59220dba8896e803cee7fc13624d08075c with just 8 sample points. (MSE 0.0001 in log of values) We round the scale to 1.0 for the first 14 quantizers to avoid the rounding anomaly at the start of the original curve.

For commit https://github.com/barrbrain/daala/commit/e574897d0e8527a7fc675f208148885e9e1011c8 we see a gain of 14.7991% in the color metric. This is the best gain witnessed so far and is achieved with 2 simple curves in AC and DC quantizer scaling. The ratio of average regression in luma-only metrics to gain in color metric is 6.1% better than with commit https://github.com/barrbrain/daala/commit/8b487c3f2da8fbd38afd5404e75442ca1eaf12c0 which was the tightest curve in the coded quantizer coordinates.
Luma only BD rate (ntt-short-1):

           RATE (%)  DSNR (dB)
    PSNR 5.20237  -0.15816
 PSNRHVS 5.19815  -0.24441
    SSIM 5.17940  -0.12145
FASTSSIM 4.98343  -0.13608

BD rate for all planes (subset1):

            RATE (%) DSNR (dB)
CIEDE2000 -14.7991 0.667919

I hand-tweaked the od_dc_res() curve to get this nice-looking result:

There are not many points in the space that curve inhabits, so it was easy to spot and fix the off-trend regions. This gave a rather tiny boost from -14.7982 to -14.7991% but the smoothness of the curve gives me some confidence.
No graph for AC quantizer scaling as it is a single linear segment (in log-log coordinates) that spans the entire quantizer range.

@jmvalin collaborated on unifying the DC scaling for I-frames and P-frames. He suggested replacing oc_dc_res() with a lookup into pvq_qm_q4[]. I ran with this and tuned the interpolation points. Waiting on AWCY for luma stats and visual results but the RD curve is promising:

BD rate for all planes (subset1):

            RATE (%) DSNR (dB)
CIEDE2000 -15.9887 0.741882

The Python version is fine, it can always be converted to C later.

Commit https://github.com/barrbrain/daala/commit/10677de6dbc3acd83b02c1ee8ac597156c1b7d2c winds back the strength of the chroma effect. The ratio of average regression in luma-only metrics to gain in color metric is 4.6% better than with commit https://github.com/barrbrain/daala/commit/e574897d0e8527a7fc675f208148885e9e1011c8 as a result. It is a direct translation of the curve in commit https://github.com/barrbrain/daala/commit/ec33da59220dba8896e803cee7fc13624d08075c into the three-segment shaped defined by 2 points in OD_DEFAULT_QMS[] and accounting for rounding in Q4.
Luma only BD rate (ntt-short-1):

         RATE (%) DSNR (dB)
    PSNR 3.65608  -0.11200
 PSNRHVS 3.61704  -0.17133
    SSIM 3.58143  -0.08467
FASTSSIM 3.41183  -0.09387

BD rate for all planes (subset1):

          RATE (%)   DSNR (dB)
CIEDE2000 -10.7459   0.483104
    *PSNR  -0.14829 -0.000423
 *PSNRHVS  -2.30856  0.156448
    *SSIM  -2.91     0.114373
*FASTSSIM -11.9042   0.306553

*Take with ample salt.

Do you want to tune more or is this ready for review/landable?

Merged in https://github.com/xiph/daala/commit/27e3704d919b8b987ecfe513f083cbaf6eaa2d4f


CU-59q6ye Fix vault file size is 0.0; CU-59q0hp Change vault api for …

…vault usage unit to MB

Task linked: CU-59q6ye Fix vault file size is 0.0

Task linked: CU-59q0hp Change node api for vault usage unit to MB


Add missing state-write for ShaderEXTWrapper

fixes trimmed captures for VK_EXT_shader_object

CI gfxreconstruct build queued with queue ID 181117.

CI gfxreconstruct build # 4117 running.

CI gfxreconstruct build # 4117 passed.


Keyboard support of "Ctrl/Command + Mouse scroll wheel" in the workflow editor

Reviewed via Slack.


Add awatiable_builder decorator

This PR adds a awatiable_builder decorator, the decorated function will return AiiDA ProcessNode. The WorkGraph will wait for the returned nodes.
The main purpose is to implement the submission_controller using WorkGraph.
Example
This is the WorkGraph version of the the submission_controller, use the the example from the aiida-submission-controller
from aiida_workgraph import WorkGraph, task
from aiida import load_profile, orm

load_profile()

@task()
def prepare_input(n, m):
    """I want to compute a 12x12 table.
    I will return therefore the following list of tuples: [(1, 1), (1, 2), ..., (12, 12)].
    """
    inputs = []
    for left_operand in range(0, n):
        for right_operand in range(0, m):
            inputs.append([left_operand, right_operand])
    print("inputs", inputs)
    return inputs

@task()
def compare(index, inputs):
    return index < len(inputs)

@task()
def update_index(index):
    return index + 1

@task.awaitable_builder()
def submit_job(index, inputs, code_label):
    """
    This function is responsible for submitting a task to the scheduler.
    """
    from aiida.calculations.arithmetic.add import ArithmeticAddCalculation
    from aiida_workgraph.engine.utils import submit
    from random import randint

    builder = ArithmeticAddCalculation.get_builder()
    builder.code = orm.load_code(code_label)
    builder.x = orm.Int(inputs[index][0])
    builder.y = orm.Int(inputs[index][1])
    builder.metadata.options.sleep = randint(1, 10)
    node = submit(builder)
    return {f"index_{index}": node}

wg = WorkGraph("test_submission_controller")
wg.context = {"index": 0,
              "should_run": True}
prepare_input1 = wg.add_task(prepare_input, name="prepare_input", n=10, m=1)
while1 = wg.add_task("While", name="While", conditions=["should_run"])
submit_job1 = wg.add_task(submit_job, name="submit_job", code_label="add@localhost",
                        inputs=prepare_input1.outputs["result"],
                        index="{{index}}")
update_index1 = wg.add_task(update_index, name="update_index", index="{{index}}")
update_index1.set_context({"result": "index"})
update_index1.waiting_on.add(submit_job1)
compare1 = wg.add_task(compare, name="compare", inputs=prepare_input1.outputs["result"],
                       index = update_index1.outputs["result"])
compare1.set_context({"result": "should_run"})
while1.children.add([submit_job1, update_index1, compare1])
wg.max_number_jobs = 3

# wg.run()
wg.submit()

WorkGraph

Timeline
We can see that the maximum number of running processes is 3 for all the time.

Codecov Report
Attention: Patch coverage is 34.32836% with 44 lines in your changes missing coverage. Please review.

Project coverage is 78.62%. Comparing base (5937b88) to head (2774f56).
Report is 34 commits behind head on main.

Files
Patch %
Lines

aiida_workgraph/engine/utils.py
39.39%
20 Missing :warning:

aiida_workgraph/engine/workgraph.py
29.16%
17 Missing :warning:

aiida_workgraph/decorator.py
30.00%
7 Missing :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #239      +/-   ##
==========================================
+ Coverage   75.75%   78.62%   +2.87%
==========================================
  Files          70       62       -8
  Lines        4615     4623       +8
==========================================
+ Hits         3496     3635     +139
+ Misses       1119      988     -131

Flag
Coverage Δ

python-3.11
78.62% <34.32%> (+2.96%)
:arrow_up:

python-3.9
?

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


JS Iterate Through the Keys of an Object , challenge bug

Describe your problem and how to reproduce it:
so the challenge ask the user to  Iterate Through the Keys of an Object with a for...in Statement,
and return the number of  users where online = true;
although i could pass the test without using a for in statement and just writing return 5; which is used in the assert test which is definitely not what we are expecting .
Add a Link to the page with the problem:
https://learn.freecodecamp.org/javascript-algorithms-and-data-structures/basic-data-structures/-iterate-through-the-keys-of-an-object-with-a-for---in-statement
Tell us about your browser and operating system:

Browser Name: Mozilla
Browser Version: quantum
Operating System: windows 10

If possible, add a screenshot here (you can drag and drop, png, jpg, gif, etc. in this box):

i think we should add a test on the user code and test if he used for in statement and the returned value must be equal to 2 , i can work on this if you agree

This particular challenge has more than one issue.  Behind the scenes, 3 additional users are added to the users object with an online property set to true.  I think instead of adding extra users behind the scenes, the existing users object should either have one new user with online property set to true or change one of the existing users which has an online property of false to true.  This way, the actual number of true values will be different than the number of false values for online in case someone accidentally writes the logic to get the online values of false.  Then, the tests would need to look for the use of for x in users  in the code using regex.  Also, we would want to validate the user does not hard code the number of users with online = true in some way.  This could be done, but would make the tests more complicated than necessary.
Honestly, I think the best way to approach this challenge is not rewrite it so that the challenge seed starts with the following:
function countOnline(obj) {
  // change code below this line

  // change code above this line
}

console.log(countOnline(users));

and the Instructions section shows an example of a users object that could be passed to the function.  That way, we can generate the objects in the tests with various outcomes for the count of users with online property = true and actually show the users object being tests in the text of the tests.  I am thinking there would only be 3-4 tests.  The first would validate the campers's code uses the for x in z syntax.  It will have to be flexible enough to handle a valid value for x and z, because the user could can use any valid variable name for x and the user could change the obj parameter to something else which is fine and can be handled via regex.  The other tests should have objects with different number of users with online property set to true, so we don't have to worry about the user trying to hard-code the return values as much.
Before creating a PR, lets get some feedback from some other mods (@thecodingaviator, @Manish-Giri, @ojeytonwilliams).

@RandellDawson Your suggested changes sound good to me. Just to be sure though, I think you were proposing a challenge re-write here -

I think the best way to approach this challenge is not(?) rewrite it so that the challenge seed starts with the following

I also agree that we shouldn't be adding objects behind the scenes with online: true. Best to have a ready-to-use example in the challenge page itself that can be used directly.
Speaking of the tests, while we're re-writing the challenge anyway, how about checking to make sure something like .filter() isn't used in the code -
return Object.values(obj).filter(e => e.online).length;

Although if we're using regex to check for for..in, this might not be necessary.

@Manish-Giri I made some changes locally which could be used for a PR.  Basically, the challenge instructions could look something like below:

The tests would look something like below:

I used an After Test div section to create the three userObjs used in the tests.

.

/

.

.

/

I think what should be done is just change the challenge seed to have the user just write the code and in the background, add the object containing users

@RandellDawson  about the After Test Div did you add it in the same file i mean iterate-through-the-keys-of-an-object-with-a-for---in-statement.md ? if yeah can i go and create a PR with this solution ?

@sil3nthill I am not 100% sure if we are all in agreement yet.

@thecodingaviator  i didnt understand what do you mean by " in the background, add the object containing users"

@sil3nthill If the other mods agree with what I have proposed above, then I already have a branch ready to create the PR.

@RandellDawson alright happy that i helped to spot the issue ! thank you @RandellDawson !

@thecodingaviator i didnt understand what do you mean by " in the background, add the object containing users"

@sil3nthill We'll add on object in the background i.e. the user would not see it being added. The object would contain the contents of the users object from the screenshot on the first post in this issue

@RandellDawson I like the proposed changes, too. So, please go ahead and submit the PR - it sounds like it's basically ready.

@ojeytonwilliams Done (PR #35899)


Win32 cwd

Here's my first attempt at implementing cwd for window, mostly based on the code of PSUtil.
Fixes #105
It works by parsing the PEB data structure found in each process. This data structure, along with the APIs used to acquire it, are unstable and, according to microsoft, the APIs may change. In practice, a significant amount of software is built on top of this API, and as such it is unlikely to be broken. If it does break, I expect microsoft to provide a stable API to acquire this information, like they did for the CommandLine.
I tested this with various combination on intel (x86 and x64) CPUs:

Host: 32-bit, Heim: 32-bit, Target: 32-bit
Host: 64-bit, Heim: 64-bit, Target: 64-bit
Host: 64-bit, Heim: 64-bit, Target: 32-bit
Host: 64-bit, Heim: 32-bit, Target: 32-bit

I did not implement this under ARM, as it requires using a fairly new API. We'd need to do a runtime check to see if the API is present, I'm not sure what the best way to do this is.
Furthermore, getting a 64-bit process' CWD from a 32-bit Heim is not implemented, as it requires relying on even more unstable APIs, and I don't personally need it.

Coverage increased (+0.01%) to 37.087% when pulling eb5f4c396c5ad66de3ad9bd1b6c774bfb805624f on roblabla:win32-cwd into e22e23570c8fcc66a922ca9252f3f6643032ba79 on heim-rs:master.

Heads up: I know this PR was kinda abandoned, my apologies about that. I'll try to check as much as possible in a next neek.

Alright, it is really unfair from me to keep this thing in the backlog for that long; I thought that it would be better to merge it as is and mark Process::cwd for Windows as an "unstable" function somehow (probably just in the documentation, since we don't really have any better mechanisms?).
Further bug reports (if there will be any) will help us reiterate on current functionality.
Thank you, @roblabla !


[WIP]AO-8448 log overriden

Still work in progress, more thorough tests are needed to make sure it works properly .

Print all non-default configuration values in default level
Mask the service token while printing it
Refactored the code of configuration initialization and printing to avoid boilerplate. Now all the configurations are managed in a standalone package. Other packages can call the API provided by this package to get the current value of a specific configuration.
Changed the default print level from ERROR to WARN -- This is TBD. I think ERROR is too high as a default level, and it looks a bit weird to print the non-default configurations in ERROR level. However, it needs further discussion as the change breaks the backward-compatibility.

Codecov Report

Merging #33 into master will increase coverage by 2.97%.
The diff coverage is 45.67%.

@@            Coverage Diff             @@
##           master      #33      +/-   ##
==========================================
+ Coverage   79.65%   82.63%   +2.97%
==========================================
  Files          21       21
  Lines        2379     2355      -24
==========================================
+ Hits         1895     1946      +51
+ Misses        430      354      -76
- Partials       54       55       +1

Impacted Files
Coverage Δ

v1/ao/internal/reporter/utils.go
59.49% <ø> (-6.17%)
:arrow_down:

v1/ao/internal/reporter/reporter_udp.go
0% <0%> (ø)
:arrow_up:

v1/ao/internal/reporter/metrics_linux.go
89.47% <0%> (ø)
:arrow_up:

v1/ao/internal/reporter/reporter.go
92.85% <100%> (-2.39%)
:arrow_down:

v1/ao/internal/reporter/event.go
99.25% <100%> (ø)
:arrow_up:

v1/ao/internal/reporter/metrics.go
74.1% <16.66%> (-0.21%)
:arrow_down:

v1/ao/internal/reporter/reporter_grpc.go
68.81% <40.74%> (+15.07%)
:arrow_up:

v1/ao/internal/reporter/oboe.go
88.82% <66.66%> (-0.54%)
:arrow_down:

v1/ao/internal/reporter/context.go
93.6% <75%> (-0.03%)
:arrow_down:

... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6b575a5...852f322. Read the comment docs.


Add old release IP to callbacks to embed-yourself (fixes #108)

Thanks for clarifying all the comments.
Merged the pull request.


Include pt-BR locale

I would like to ask you include the "pt-BR" locale on isAlphaNumeric. The pt-PT can be used as a basis, as far as I know there are no differences.

Happy to accept a PR. Have a look at https://github.com/chriso/validator.js/blob/master/lib/alpha.js to see how you can setup a locale alias:
alpha['pt-BR'] = alpha['pt-PT'];
alphanumeric['pt-BR'] = alphanumeric['pt-PT'];

Added in 7d0bee566d17a075d4158e9713ce8e4a0bc03bf5.


add .strict method which only returns GA spec'd utm params

utm.strict only returns utm params that are spec'd by GA.
(these ones:)

FYI @wcjohnson11 =D
@ndhoule plz review.

@ndhoule thanks! good catch on that exponential operation. all comments accounted for and rebased.


Wasm: Generics test fails if run in a loop

Wrap the simple Generics test in a for (var i = 0; i < 100; i++) and it will fail.  It may not be consistent, but this stack is common.
exception thrown: RuntimeError: memory access out of bounds,RuntimeError: memory access out of bounds
    at S_P_CoreLib_System_Runtime_TypeLoaderExports__LookupInCache (wasm-function[4572]:210)
    at S_P_CoreLib_System_Runtime_TypeLoaderExports__CacheMiss_0 (wasm-function[4574]:248)
    at S_P_CoreLib_System_Runtime_TypeLoaderExports__GVMLookupForSlot (wasm-function[4584]:507)
    at Generics_Program_TestSimpleGVMScenarios__TestWithStruct (wasm-function[12618]:347)
    at Generics_Program_TestSimpleGVMScenarios__Run (wasm-function[12548]:1938)
    at Generics_Program__Main (wasm-function[12531]:144)
    at Generics__Module___MainMethodWrapper (wasm-function[12518]:3)
    at StartupCodeMain (wasm-function[12509]:108)
    at __managed__Main (wasm-function[22612]:67)
    at main (wasm-function[2072]:126)

This might be solved by #8171 - it is in a test around structs.  Wanted to create a specific task for this failure so it doesn't get forgotten (by me at least).  I think this is the same error that occurs in CI sometimes.

Fingers crossed it's the same issue!


NIFI-4545: Improve Hive processors provenance transit URL

Thank you for submitting a contribution to Apache NiFi.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
For all changes:

[x] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?

[x] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.

[x] Has your PR been rebased against the latest commit within the target branch (typically master)?

[x] Is your initial contribution a single, squashed commit?

For code changes:

[x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder?
[ ] Have you written or updated unit tests to verify your changes?
[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?
[ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?
[ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?

For documentation related changes:

[ ] Have you ensured that format looks appropriate for the output in which it is rendered?

Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.

Reviewing...

+1 LGTM, ran build with contrib-check and unit tests, also tried various HiveQL commands, verified the input and output tables attributes are being correctly populated. Thanks for this improvement! Merging to master


#18: Use full file name when requiring JSON assets

PR to address #18

Thank you for allowing this. If you also get a chance to release a new NPM package after this merges, that would be most appreciated.

Released as 1.0.6!


Install ign command

Signed-off-by: Louise Poubel<EMAIL_ADDRESS>
🦟 Bug fix
Fixes https://github.com/ignitionrobotics/ign-gazebo/issues/886
Summary

It was removed by mistake on #55
Checklist

[x] Signed all commits for DCO
[ ] Added tests
[ ] Updated documentation (as needed)
[ ] Updated migration guide (as needed)
[ ] codecheck passed (See contributing)
[ ] All tests passed (See test coverage)
[x] While waiting for a review on your PR, please help review another open pull request to support the maintainers

Note to maintainers: Remember to use Squash-Merge
🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸

ouch, thanks @chapulina !


Set-up project's basic

What?
This pull-request should set up the project's basics, like:

package.json
build
CI

Why?
We'd like to establish a clear separation between the libraries' source-codes and hosted versions (minified).

Also, one suggestion, you can add checks for the PR, i.e. block merging the PR until all comments are resolved, all jobs from the workflow passed, etc.

@mp-dthr thanks! I've applied some of your suggestions, and we could iterate in the following PRs for another one. For now merging this as it is :+1:


Initial implementation of Cucumber messages for Perl

Types of changes

[x] New feature (non-breaking change which adds functionality).

Checklist:

[x] I've added tests for my code.
[x] I have updated the CHANGELOG accordingly.

Caution: there is already a default.mk for perl in the templates.
At some point we will have to move your code from the default.mk you have created into the one in templates/perl, and rsync the files to other modules which have perl implementations.

Caution: there is already a default.mk for perl in the templates.
At some point we will have to move your code from the default.mk you have created into the one in templates/perl, and rsync the files to other modules which have perl implementations.
Edit: I noticed that there may have been an attempt in #1673?

Yes,  that was an attempt, based on misunderstanding the situation. I've since merged a better version and rebased this branch. By setting up a '.rsync' file to ensure the shared infrastructure remains current, this new library should now be integrated in the regular build infrastructure.


Stop logging IP addresses

Right now, we are logging IP addresses of users in several locations:

journalctl -f -u dovecot
journalctl -u<EMAIL_ADDRESS>-f
journalctl --facility=2 in general (showing all mail related log messages in journald)
/var/log/nginx/access.log
/var/log/nginx/error.log
...other places? don't think so.

postfix (and dovecot as well I think) doesn't allow to specify a log format. Three possible solutions:

it allows to specify a log file though. Maybe we could write a log daemon which receives postifx logs and rewrites them before saving them to disk/journald.
we could use nginx in stream mode as a reverse TCP proxy, at least on the port(s) Delta Chat connects to. Then all IPs postfix sees would be "localhost". Could this cause problems with validating MTA-STS or reverse DNS lookups though?
we could pass the log lines to rsyslog, use an rsyslog config template to scrub the IP addresses from the log line... and maybe even pass them to journald again in the end?

Config templates seem to be able to rewrite log messages, if I'm understanding this correctly: https://stackoverflow.com/questions/52636918/is-there-a-way-parse-log-messages-using-rsyslog-config-and-transform-them-to-str

related: such a feature was proposed for journald, but is "out-of-scope". :<
https://github.com/systemd/systemd/issues/2447

autistici is forwarding the logs from journald to rsyslog: https://www.autistici.org/orangebook2/orangebook2-en.html#logging
We can probably look up in https://git.autistici.org/ai3/ how they do it exactly

okay, so we set journald to memory and define rsyslog as the output. Before being saved into rsyslog, the logs are filtered / anonymized via rsyslog-filters. Do we stop here, or do we want to collect our logs in a more ergonomic logging service?
For our own deployment it could be nice to put them into a central grafana-compatible DB, but what do we do for chatmail?

In any case, I'll take c2.testrun.org for today and try to implement the clear stuff :) If rsyslog isn't comfortable enough, we still can forward the logs further

...or we just store all logs only in memory. Then we don't need to sanitize them at all. Should be fine, it's just one line of journald config.
Oh, and we need to convince nginx to log to journald instead of /var/log/nginx.


Run task concurrency but key is same

I want to run eslint and prettier concurrency, but their file glob is same:
"lint-staged": {
        "*.{js,ts,json,md}": "eslint --fix",
        "*.{js,ts,json,md}": "prettier --write",
}

If I changed to:
"lint-staged": {
        "*.{js,ts,json,md}": ["eslint --fix", "prettier --write"]
}

They run sequentially

I would suggest not to run tasks that modify files concurrently since this can lead to race conditions.
If you still want this, I believe it’s doable with an advanced configuration format. Check out our docs.

You could also setup eslint-plugin-prettier to integrate the two together, and only run ESLint for those files!


mwc-wallet need to handle Coinbase transactions as Coinbase Outputs

We don't want to see Coinbase transactions that are not confirmed.
Such transactions need to be deleted similar to what we do for the orphans CoinBase outputs.
In case of Reorg and reversion back, new transactions will be created every time.
Reason: Nobody likes current behavior, it is confusing and don't have much benefits.

Reopen because we have potential problem with different branch jittering.  Or returning orphan case. As a result coinbase will get duplicate transactions.
Chances for that is pretty low, but fix still needed.

For testing need to simulate the reorg scenario for mining wallet. switch with main branch to attacker and back need to be done at least twice (total 4 switched form branch to branch).
I did such testing by tweaking the code. Tweak just skip outputs, so it looks like a switch to attacker branch for the wallet.
Note, that issue and fix are for mining wallet only. Regular wallet is not affected

Testing will take more time, but should not block release.


Add HeckelDiff

Project URL
https://github.com/mcudich/HeckelDiff
Description
A fast Swift diffing library.
Why it should be included to awesome-ios (optional)
Checklist

[x] Only one project/change is in this pull request
[x] Addition in chronological order (bottom of category)
[x] Supports iOS 9 or later
[x] Supports Swift 3
[x] Has a commit from less than 2 years ago
[x] Has a clear README in English

          1 Error

:no_entry_sign:
Found 1 link issue

Link issue by awesome_bot

Line
Status
Link

2257
301
http://www.ioscreator.com/ redirects tohttps://www.ioscreator.com/

  Generated by :no_entry_sign: danger

Hey @mcudich, nice lib! 👍
Thanks for contributing


feat: Workflow files added (#24)

https://werkbotstudios.teamwork.com/#/tasks/29567211
Summary
Release PR for git flow workflow files
Testing Steps

[ ] confirm release draft action completed https://github.com/werkbot/silverstripe-module-seeder/actions
[ ] confirm draft release looks good https://github.com/werkbot/silverstripe-module-seeder/releases

Issues/Concerns

I will monitor the actions behavior after merging

Git Flow

DO NOT delete "release/*" or "hotfix/*" branches after merging a PR. These are used to publish the next release, and they are deleted automatically.
"Squash and merge" is good on "feature/*" into "develop"
"Create a merge commit" is good on "release/*" or "hotfix/*" into "main"

@aletail The same issue that occurred in the search module occurred here https://github.com/werkbot/silverstripe-module-seeder/runs/8136098691?check_suite_focus=true#step:9:38
"main" has a branch protection rule that prevented the changelog updater from making a commit without a review.


allow crusher suicide

About the PR
title
Why / Balance

good for when you want to borg yourself
antag funnies, when i was rev i mindswapped a seccie and ran into crusher but couldnt turn it on. i asked a bozo to do it but he threw cfl3 at me instead :trollface:

Technical details
remove check
Media
squeeeeeeze

[X] I have added screenshots/videos to this PR showcasing its changes ingame, or this PR does not require an ingame showcase

Breaking changes
no
Changelog
no cl no fun

M-MICHAEL DON'T LEAVE ME HERE!!!!!

Do you think you can make it so if you put someone/something in there but they don't have hands (like being cuffed) they can't just run out?

nitrous oxide :trollface:


Ensure bokeh/panel extension is loaded only once

If the panel extension is already loaded do not load it a second time, instead simply initialize the selected HoloViews backend.

Coverage increased (+2.6%) to 73.493% when pulling be6afd4469e393e37b27509d556e438377d8ae2a on load_ext_once into e0767f2533daf0ba8ed5b6ea2f28000803d99b91 on master.


Add controls for boost and pause

The reason this hasn't been added, is that those should be added as services. I had trouble developing/debugging this because of the connection issues, but that should be fixed in this PR: #19.

Very good, well done!
I don't know how services is implemented right now but can help to test when there is some code.

After looking into this, I think we need to implement a fan.py to do this 🤔
https://developers.home-assistant.io/docs/dev_101_services/#entity-services
If so, we need to restructure the project and move things over to fan.py, like speed, is_on etc.

Is this something you have looked into more since last post?

My "hack" for boost is to use the constant speed feature, but then I had to add some logic to deactivate it.


[KFP] kubeflow stand alone deployment with vpc

Would it be of interest to add instruction on how to set up a stand alone deployment with a vpc cluster? Let me know and if yes I can added had a hard time getting it right my self the first time. I guess it would be in this instruction file here

/cc @joeliedtke


Fixed a compiler error, I didn't verify if it worked but it seemed safe enough.

Definitely safe. My bad, I missed that one.


Preparation for point release

[x] Pull request represents a single change (i.e. not fixing disparate/unrelated things in a single PR)
[x] Title summarizes what is changing
[ ] Has a news entry file (remember to thank yourself!)
[ ] Appropriate comments and documentation strings in the code
[ ] Has sufficient logging.
[ ] Has telemetry for enhancements.
[ ] Unit tests & system/integration tests are added/updated
[ ] Test plan is updated as appropriate
[ ] package-lock.json has been regenerated by running npm install (if dependencies have changed)
[ ] The wiki is updated with any design decisions/details.

Codecov Report

Merging #7268 into release will decrease coverage by 0.42%.
The diff coverage is n/a.

@@             Coverage Diff             @@
##           release    #7268      +/-   ##
===========================================
- Coverage    58.46%   58.03%   -0.43%
===========================================
  Files          485      485
  Lines        21437    21437
  Branches      3462     3457       -5
===========================================
- Hits         12533    12441      -92
- Misses        8123     8211      +88
- Partials       781      785       +4

Impacted Files
Coverage Δ

src/client/testing/serviceRegistry.ts
48.64% <0%> (-46.85%)
:arrow_down:

src/client/testing/codeLenses/main.ts
44.44% <0%> (-33.34%)
:arrow_down:

...t/datascience/jupyter/jupyterDataRateLimitError.ts
66.66% <0%> (-33.34%)
:arrow_down:

src/client/formatters/serviceRegistry.ts
75% <0%> (-25%)
:arrow_down:

src/client/api.ts
62.5% <0%> (-25%)
:arrow_down:

src/client/common/utils/platform.ts
64.7% <0%> (-23.53%)
:arrow_down:

src/client/common/utils/icons.ts
83.33% <0%> (-16.67%)
:arrow_down:

src/client/datascience/cellFactory.ts
75.86% <0%> (-13.8%)
:arrow_down:

src/client/providers/providerUtilities.ts
83.33% <0%> (-5.56%)
:arrow_down:

src/client/common/platform/pathUtils.ts
66.66% <0%> (-5.56%)
:arrow_down:

... and 9 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 799344b...8cfed63. Read the comment docs.


Fixed non-Maya builds

I changed how setup_precompiled_headers worked in 4fd6f80 but forgot to update the other CMakeLists.txt files.

Thanks for the fix.


Search by title fails

For the CZO we have Titles that are automatically created as a concatenation of several attributes and look like this
BCCZO - Well Water Levels - (GGU_GW_1,2,6_Pducer_Tran) - Gordon Gulch: Upper (2011-2017)
![1]
if i search for this in discover I get this error:
Field delimiter 'Gulch' is not recognized. Field delimiters include title, contributor, subject, etc. Please try again.

looks like the colon creates a problem

I'm sure @alvacouch can elaborate here, but this is related to the fact that HydroShare enables search terms to be entered that search specific metadata "Fields" for specific terms where the field and terms are separated by colons. For example the following search string looks for all resources with an author of "Horsburgh" and a subject keyword of "Temperature":
author:Horsburgh subject:Temperature
So, if you include colons in resource titles and search using a big long string that has colons in it, it will cause problems.

I will ignore bad search terms and treat them as plaintext. This is easy. Other changes are much harder.

@alvacouch - that seems like a reasonable approach. For what it's worth - I really like the ability to do the search using the {field}:{value} syntax and so would be sad to see it go.

Fixed in pr #3309 .

@alvacouch hi we are discussing this right now just marking

I remain convinced that I should put keywords in ALL CAPS, e.g., TITLE: AUTHOR: CONTRIBUTOR: ....
in order to avoid collision. Other than this, PR #3309 addresses this issue.

@martinseul I successfully searched for "BCCZO - Well Water Levels - (GGU_GW_1,2,6_Pducer_Tran) - Gordon Gulch: Upper (2011-2017)" as the custom behavior with colons is no longer a part of Discover.


dnsmasq/DHCP Issues - Unable to get IP from vtrust network

I've used the docker version of this library on my laptop with no issues in the past, however with my latest attempts I'm running into issues getting my phone (or any device for that matter) to connect to the vtrust-flash wifi network.
I've run through all of the recommended fixes for the problem however it seems as though dnsmasq just isn't running/responding to any of the requests from my devices.
As such the flash process isn't working at all.
Has anyone else experienced this issue and if so, how did you resolve it?
I noticed that the dnsmasq service was running in the foreground thus preventing the AP from starting, so I added an & to the end of this line: https://github.com/ct-Open-Source/tuya-convert/blob/master/scripts/setup_ap.sh#L47
That at least started the AP & dnsmasq services but dnsmasq doesn't seem to be receiving any requests for IPs, it doesn't output anything at all.
Meanwhile the AP service is happily logging connects/disconnects each time my phone attempts to connect to the AP.

For anyone encountering the same problem as me, the solution is simple - you just have to wait (or at least, that's what I had to do).
I noticed that DNSMasq was consuming 100% CPU when it was starting up, for nearly as much as 10 minutes. After about 10 minutes the CPU usage dropped and the screen session for smarthack-wifi outputted some of the DNSMasq start-up lines. After that everything worked perfectly. (See notes below on additional mods/changes I made to make it work from other issues).
I have absolutely no idea what's causing DNSMasq to take so long to start, but it was just a case of being patient and then everything worked fine.
Other changes I made/ran before running tuya start in docker:

sudo systemctl stop NetworkManager
sudo killall wpa_supplicant
sudo systemctl stop firewalld

In the scripts/setup_ap.sh file I added -d & to the end of the dnsmasq command (line 47), this allowed it to run in debug mode in the background so that I could see the output in the screen session.
I hope this helps some other people that run into the same problem. I'm purposefully not marking this as closed as ideally these changes will be worked into the documentation/scripts/process for the docker method, also, people tend to search through the open issues rather than the closed ones when they have issues, so hopefully it'll help someone in the same position as me.


Android Virtual Assistant Sample App

Android is a popular client of Virtual Assistants especially in the Automotive space. Most customer conversations end up with a demo running on an example Android device/head-unit and require a custom app to be built that interfaces with STT, Direct-Line and TTS, Renders Adaptive Cards, etc. which requires a lot of up-front time delaying initial traction. In comparison a VA solution can be deployed in around 15 minutes.
Describe the solution you'd like
A sample that once compiled enables a customer to build, drop an APK onto a device, provide the appropriate configuration (keys, voice names, endpoints) and demonstrate VA right away. This will work against a standard VA deployed straight from our repo codebase and not require any changes.
This then provides the basis for a customer/partner to then iterate for any demo. We'll keep this up to date with planned Speech changes including Edge.
Key features:

Branding should be "Virtual Assistant"
No Wake-Words or constant recognition. Interaction is started by pressing a button in the UX. We have a DDK example that shows keyword spotting
Unified Speech APIs to be used (not CRIS)
All Speech configuration should be configurable including keys, endpoint, voice names and streaming should be used to improve performance (as per the current codebase)
DirectLine key should be configurable along with UserId (GUID) ensuring Linked Accounts scenarios work as expected.
Adaptive Cards are rendered using the Adaptive Cards Rendering SDK for Android
App respects InputHints returned from the Bot to control the Microphone
Documentation detailing the Android SDK/Tools required to build, deploy and configure
Added to the assistant-android folder within the testharnesses folder folder.
Custom Events sent by the old app are removed and the client should work against a normal Bot as well as VA.
Location event is sent on startup with a latitude/longitude value sent enabling PoI scenarios to work. Other bots can just ignore

Do I have commit rights on the repo?


Usage inside CSS

When using vue, svelte, ... or other frameworks, often images are part of styles:
<style>
.rice-drum {
    background-image: url(../components/static/Rice_Drum.jpg?format=webp);
}
</style>

vite resolves and bundles these images properly, but it seems imagetools doesn't see them - they stay in the source format and are not transformed.
Is this expected?

+1

Has anyone found a solution to this problem?

@Abilov599
As a workaround;

Set parent container to display: relative;
Create the picture element inside parent container, and set display: absolute; left: 0; top: 0; height: 100%; width: 100%

This would need support in Vite: https://github.com/vitejs/vite/issues/14686

I just learned about image-set in CSS which may be useful here as well

https://developer.mozilla.org/en-US/docs/Web/CSS/image/image-set
It seems that it's pretty well adopted now https://caniuse.com/?search=image-set


added marmara university student mail domain

Marmara University uses both marun.edu.tr and marmara.edu.tr as seen:
https://bidb.marmara.edu.tr/en/e-mail-service

@mustafahmetoglu I am sorry, the domain marun.edu.tr is removed from our trusted domains list because it was abused in the past. If you are the student of this school, please enroll to GitHub Student Developer Program. Then apply for JetBrains educational subscription with your GitHub account (GitHub tab at https://www.jetbrains.com/shop/eform/students).


Add test for TD and BQ export and preview filters

Jira: BIG-99 BIG-98 TER-78
KBC: https://github.com/keboola/connection/pull/4135
Before asking for review make sure that:
Checklist

[x] New client method(s) has tests
[x] Apiary file is updated

Release

[x] I gave the PR a proper label:

major (BC break)
minor (new feature)
patch (backwards compatible fix)
no release (just test changes)

🔗 jeste prihodim navazujici PR s testama pro TD: https://github.com/keboola/storage-api-php-client/pull/952

Ještě sem přidal další testy


Scaffold-DbContext not correctly configuring auto incrementing primary key in from Sqlite database

I'm using EF.Core scaffold to create a datamodel for a DB with an auto increment primary key but the context is being generated with the key annotated with ValueGeneratedNever() instead of ValueGeneratedOnAdd().  As a result EF framework requires each record to have a primary key set instead of allowing the DB to set it.
Exception message:  N/A
Stack trace:  N/A

Steps to reproduce
Create an SQLite DB with this table:
CREATE TABLE "LastSyncTime"
(
ID INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT UNIQUE,
SyncTime TEXT NOT NULL,
TableUpdated TEXT NOT NULL
)
In the package manager console run:
Scaffold-DbContext "DataSource=MyDatabase.sqlite;" Microsoft.EntityFrameworkCore.Sqlite -OutputDir Models -force
Examine the context class, it generates with:
        modelBuilder.Entity<LastSyncTime>(entity =>
        {
            entity.HasIndex(e => e.Id)
                .IsUnique();

            entity.Property(e => e.Id)
                .HasColumnName("ID")
                .ValueGeneratedNever();

            entity.Property(e => e.SyncTime).IsRequired();

            entity.Property(e => e.TableUpdated).IsRequired();
        });

ValueGeneratedNever() should be ValueGeneratedOnAdd()
Further technical details
EF Core version: (found in project.csproj or packages.config) 2.1.0-rc1-final  (also happens in 2.0.2)
Database Provider: (e.g. Microsoft.EntityFrameworkCore.Sqlite)
Operating system:  Win 10 1709
IDE: (e.g. Visual Studio 2017 15.6.3)

Possibly related, also in SQLite. I found out that with my entity:
public class Account
{
    [Key, DatabaseGenerated(DatabaseGeneratedOption.Identity)]
    public int Id { get; set; }

    public int Balance { get; set; }
}

giving Balance a default value in OnModelCreating:
modelBuilder.Entity<Account>()
    .Property(ac => ac.Balance)
    .HasDefaultValue(1500);

scaffolds the following in a migration:
migrationBuilder.CreateTable(
    name: "Accounts",
    columns: table => new
    {
        Id = table.Column<int>(nullable: false)
            .Annotation("Sqlite:Autoincrement", true),
        Balance = table.Column<int>(nullable: false, defaultValue: 1500)
            .Annotation("Sqlite:Autoincrement", true) //WTF?
    });

Why does the scaffolding think that the second column needs auto-increment?
Commenting out the bit in OnModelCreating doesn't emit the annotation for the Balance column to the migration scaffold.

I have the same problem with GUID.
[Id]        UNIQUEIDENTIFIER NOT NULL

I'm seeing this as well but with the error AUTOINCREMENT is only allowed on an INTEGER PRIMARY KEY. My database looks right. It looks like whatever is generating the SQL is trying to put auto increment on the wrong field.
Microsoft.EntityFrameworkCore.Database.Transaction:Debug: Beginning transaction with isolation level 'Serializable'.
Microsoft.EntityFrameworkCore.Database.Command:Debug: Executing DbCommand [Parameters=[], CommandType='Text', CommandTimeout='30']
CREATE TABLE "Device" (
    "id" INTEGER NOT NULL CONSTRAINT "PK_Device" PRIMARY KEY AUTOINCREMENT,
    "name" varchar(90) NULL,
    "publicId" uniqueidentifier NOT NULL
);
Microsoft.EntityFrameworkCore.Database.Command:Information: Executed DbCommand (2ms) [Parameters=[], CommandType='Text', CommandTimeout='30']
CREATE TABLE "Device" (
    "id" INTEGER NOT NULL CONSTRAINT "PK_Device" PRIMARY KEY AUTOINCREMENT,
    "name" varchar(90) NULL,
    "publicId" uniqueidentifier NOT NULL
);
Microsoft.EntityFrameworkCore.Database.Command:Debug: Executing DbCommand [Parameters=[], CommandType='Text', CommandTimeout='30']
CREATE TABLE "DeviceAction" (
    "deviceId" INTEGER NOT NULL,
    "id" int NOT NULL CONSTRAINT "PK_DeviceAction" PRIMARY KEY AUTOINCREMENT,
    "name" varchar(90) NULL,
    "value" varchar(4000) NULL,
    CONSTRAINT "FK_DeviceAction_Device_deviceId" FOREIGN KEY ("deviceId") REFERENCES "Device" ("id") ON DELETE CASCADE
);
'dotnet.exe' (CoreCLR: clrhost): Loaded 'C:\Program Files\dotnet\shared\Microsoft.NETCore.App\2.1.0\System.Diagnostics.StackTrace.dll'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
'dotnet.exe' (CoreCLR: clrhost): Loaded 'C:\Program Files\dotnet\shared\Microsoft.NETCore.App\2.1.0\System.Reflection.Metadata.dll'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
Microsoft.EntityFrameworkCore.Database.Command:Error: Failed executing DbCommand (28ms) [Parameters=[], CommandType='Text', CommandTimeout='30']
CREATE TABLE "DeviceAction" (
    "deviceId" INTEGER NOT NULL,
    "id" int NOT NULL CONSTRAINT "PK_DeviceAction" PRIMARY KEY AUTOINCREMENT,
    "name" varchar(90) NULL,
    "value" varchar(4000) NULL,
    CONSTRAINT "FK_DeviceAction_Device_deviceId" FOREIGN KEY ("deviceId") REFERENCES "Device" ("id") ON DELETE CASCADE
);

Microsoft.Data.Sqlite.SqliteException (0x80004005): SQLite Error 1: 'AUTOINCREMENT is only allowed on an INTEGER PRIMARY KEY'.
   at Microsoft.Data.Sqlite.SqliteException.ThrowExceptionForRC(Int32 rc, sqlite3 db)
   at Microsoft.Data.Sqlite.SqliteCommand.PrepareAndEnumerateStatements(Stopwatch timer)+MoveNext()
   at Microsoft.Data.Sqlite.SqliteCommand.ExecuteReader(CommandBehavior behavior)
   at Microsoft.Data.Sqlite.SqliteCommand.ExecuteNonQuery()
   at Microsoft.EntityFrameworkCore.Storage.Internal.RelationalCommand.Execute(IRelationalConnection connection, DbCommandMethod executeMethod, IReadOnlyDictionary`2 parameterValues)
Exception thrown: 'Microsoft.Data.Sqlite.SqliteException' in Microsoft.EntityFrameworkCore.Relational.dll
SQLite Error 1: 'AUTOINCREMENT is only allowed on an INTEGER PRIMARY KEY'.

The thread 0x59d4 has exited with code 0 (0x0).
An exception of type 'Microsoft.Data.Sqlite.SqliteException' occurred in Microsoft.EntityFrameworkCore.Relational.dll but was not handled in user code
SQLite Error 1: 'AUTOINCREMENT is only allowed on an INTEGER PRIMARY KEY'.

Ran into the same issue with EFCore 2.1.2
dotnet ef dbcontext scaffold ... -f -d -v

renders a PRIMARY KEY AUTOINCREMENT field as a ValueGeneratedNever in the generated DbContext, and the reverse operation
dotnet ef migrations add ...

generates PRIMARY KEY AUTOINCREMENT columns in the SQLite db where the property either has no ValueGenerated... at all, or is explicitly ValueGeneratedNever().
Looking forward to 2.2.0 I guess.

Still no fixes for this ValueGeneratedNever() problem ?

I have a problem with Auto Increment when a table has more than one primary key column.
Model:
public class Product
{
    public int Id { get; set; }
    public int Tr {get; set; }
}

Configuration:
modelBuilder.Entity<Product>()
                .HasKey(model => new { model.Id, model.Tr });

modelBuilder.Entity<Product>()
                .Property(model => model.Id)
                .ValueGeneratedOnAdd();

which creates this:
CREATE TABLE "Products" (
    "Id" INTEGER NOT NULL,
    "Tr" INTEGER NOT NULL,
    CONSTRAINT "PK_Products" PRIMARY KEY ("Id", "Tr")
)

As you see, the Id column is not AutoIncrement. But if I change the configuration to have only one primary key, then it works!

Problem also exists while using SQL Server, not just Sqlite.

Any updates on this?

Willing to take a PR for this?

@ErikEJ Of course! It might depend on #10228


Fix stringified JSON response (#22450)

Description
TL;DR
This commit fixes unexpected stringified JSON in the following plugins

REST API Plugin
GraphQL plugin
SaaS plugins (Airtable)

The detailed analysis can be found in the comment section of the issue itself (#22450)
Slack Conversations

1
2

Add a TL;DR when description is extra long (helps content team)

Fixes #22450
Media
Type of change

Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

Manual

Test Plan

Add Testsmith test cases links that relate to this PR

Issues raised during DP testing

Link issues raised during DP testing for better visiblity and tracking (copy link from comments dropped on this PR)

Checklist:
Dev activity

[ ] My code follows the style guidelines of this project
[ ] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes
[ ] PR is being merged under a feature flag

QA activity:

[ ] Test plan has been approved by relevant developers
[ ] Test plan has been peer reviewed by QA
[ ] Cypress test cases have been added and approved by either SDET or manual QA
[ ] Organized project review call with relevant stakeholders after Round 1/2 of QA
[ ] Added Test Plan Approved label after reveiwing all Cypress test

/ok-to-test sha=8486d59

/build-deploy-preview skip-tests=true

/ok-to-test sha=6e4e050

/ok-to-test sha=77c448d

/ci-merge-check

@Rhitottam Can you please take a look at the perf-tests failure?

/perf-test


Upgrade to Ubuntu 16.04 LTS

Follow up to #106 for easier certbot installation in #159.

Guide available - https://www.digitalocean.com/community/tutorials/how-to-upgrade-to-ubuntu-16-04-lts

No more actual after #173.


[WIP] Write-Ahead log and crash recovery

[x] Write-Ahead log implementation in RWFileSafe
[x] Env implementation for POSIX
[ ] File locks in RWFileSafe implementation
[x] Tests for RWFileSafe
[ ] Tests for file locks
[ ] Tests for ENV
[ ] Tests for recovery mechanism
[x] Successful build (fails because of FixedTable)

:rage:

PR is too big. I'll merge it with master to allow others pull changes.


Does not copy compile_commands.json if invoked from VSCode task

Brief Issue Summary
In my .vscode/settings.json I have following lines:
...
    "cmake.copyCompileCommands": "${workspaceFolder}/compile_commands.json",
    "cmake.buildDirectory": "${workspaceFolder}/build/${buildKit}/${variant:Config}",
    "cmake.buildTask": true,
    "cmake.configureSettings": {
        "CMAKE_EXPORT_COMPILE_COMMANDS": "ON",
    },
    "cmake.generator": "Unix Makefiles",
...

And I have following task in tasks.json:
...
        {
            "label": "CMake Configure",
            "type": "cmake",
            "command": "configure",
            "presentation": {
                "reveal": "never"
            }
        },
...

I expect that both >CMake: Configure and task CMake Configure, will copy my compile_commands.json  to the specified path, however it happens only with >CMake: Configure and not with custom task.
In debug logs it can be seen that task CMake Configure does not get finished with [extension] [8877] cmake.configure finished (returned 0), but >CMake: Configure does.
CMake Tools Diagnostics
{
  "os": "linux",
  "vscodeVersion": "1.72.2",
  "cmtVersion": "1.12.27",
  "configurations": [
    {
      "folder": "/code",
      "cmakeVersion": "3.22.1",
      "configured": true,
      "generator": "Unix Makefiles",
      "usesPresets": false,
      "compilers": {}
    }
  ],
  "cpptoolsIntegration": {
    "isReady": true,
    "hasCodeModel": false,
    "activeBuildType": "",
    "buildTypesSeen": [
      ""
    ],
    "requests": [],
    "responses": [],
    "partialMatches": [],
    "targetCount": 0,
    "executablesCount": 0,
    "librariesCount": 0,
    "targets": []
  },
  "settings": [
    {
      "communicationMode": "automatic",
      "useCMakePresets": "auto",
      "configureOnOpen": null
    }
  ]
}

Debug Log
task CMake Configure (trimmed output)

[driver] Start configure
[driver] Running pre-configure checks and steps
[driver] Using CMake toolchain clang/x86 for configuring
[cmakefileapi-driver] Configuring using kit
[cmakefileapi-driver] Invoking CMake /usr/bin/cmake with arguments ["--no-warn-unused-cli","-DCMAKE_EXPORT_COMPILE_COMMANDS:BOOL=TRUE"...
[proc] Executing command: /usr/bin/cmake --no-warn-unused-cli -DCMAKE_EXPORT_COMPILE_COMMANDS:BOOL=TRUE...
[cmakefileapi-parser] Read reply folder: /code/build/.../.cmake/api/v1/reply
[cmakefileapi-parser] Found index files: ["..."]
[extension] Not updating the configuration provider because "C_Cpp.intelliSenseEngine" is set to "Disabled"

> CMake: Configure (trimmed output)

[main] Configuring folder: code
[main] Saving open files before configure/build
[driver] Start configure
[driver] Running pre-configure checks and steps
[driver] Using CMake toolchain clang/x86 for configuring
[cmakefileapi-driver] Configuring using kit
[cmakefileapi-driver] Invoking CMake /usr/bin/cmake with arguments ["--no-warn-unused-cli","-DCMAKE_EXPORT_COMPILE_COMMANDS:BOOL=TRUE"...
[proc] Executing command: /usr/bin/cmake --no-warn-unused-cli -DCMAKE_EXPORT_COMPILE_COMMANDS:BOOL=TRUE...
[cmake] Not searching for unused variables given on the command line.
...
[cmake] -- Configuring done
[cmake] -- Generating done
[cmake] -- Build files have been written to: /code/build/...
[cmakefileapi-parser] Read reply folder: /code/build/.../.cmake/api/v1/reply
[cmakefileapi-parser] Found index files: ["..."]
[extension] Not updating the configuration provider because "C_Cpp.intelliSenseEngine" is set to "Disabled"
[main] Copying compile_commands.json from /code/build/.../compile_commands.json to /code/compile_commands.json
[extension] [8877] cmake.configure finished (returned 0)

I thought the same.
But currently it is said that option "cmake.buildTask": true, is applicable only to "build" commands.
CMake: Configure (same as CMake: Clean) is not using "task" backend but normal processing.
See discussion in #2768 and related #2770


roots-contentful doesn' play well with roots-yaml

I'm trying to use roots-contentful in combination with roots-yaml. Basically I need to access the data object created by roots-yaml from inside the single entry views compiled by roots-contentful. Unfortunately, the data object is undefined when I try to reference it from my views.
The problem seems to be that the views are compiled during the set-up of roots-contentful (inside the setup hook), but the data object is created by roots-yaml much later, in the before_pass compile hook.
The ordered flag has no effect, of course, since the extension is not dealing with files like roots-yaml is. If roots extensions had some sort of cleanup hook, the compilation could be moved in there, but there isn't.
The only solution I could find is to make the compile_entries function available on the Roots object, and to call it from the after hook in my project's app.coffee.
root-contentful/lib/index.coffee
# ...

    setup: ->
      types = opts.content_types
      types = if _.isPlainObject(types) then reconfigure_alt_type_config(types)
      @roots.compileContentfulViews = compile_entries.bind(@, types)

      configure_content(types).with(@)
        .then(get_all_content)
        .tap(set_urls)
        .tap(set_locals)

# ...

app.coffee
# ...

    after: (roots) ->
      roots.compileContentfulViews()

# ...

+1 for issue.


Integration tests enhancement + sample upgrade

Addresses #5495
Fixes #8713
I don't agree with the OP on #8713. A dB insert test using the SUT's dB context feels like a unit test to me. An integration test checks the response of the app. However, this can be performed easily enough (and thus indicate that the dB must have an inserted object) with AngleSharp applied to a test response. I thought this would be a nice little add to one of the tests to show how easy it is to do.
@javiercn There's a lot of 2.2 updating going on here; so to facilitate your review, I'll paste the critical code bits in here with the added language to the topic. We can handle the 2.2 upgrade review on our side.
Content ...

You can use AngleSharp to parse SUT responses and make assertions about content returned to the client.
The Post_AddMessageHandler_ReturnsRedirectToRootAndAddsMessage test in the sample app:

Loads the Index page of the SUT.
Adds a test message with SendAsync.
Makes second request for the Index page in order to assert that the test message is present in the returned content. If the test message appears in the returned content, the test message must be present in the database. A separate request is made because the client for these tests is configured not to follow redirects, which is the response of SendAsync. See the highlighted lines in the following code from the sample app.
Asserts the following:

The first request for the Index page loads successfully (HttpStatusCode.OK).
The response status code for adding the test message is a redirect (HttpStatusCode.Redirect).
The response location for the redirect is the Index page ("/");
The response from the second Index page request has a list of messages (list items in an unordered list) that includes the test message added by the test. See the highlighted lines in the following code from the sample app.

@javiercn 	please review or suggest another reviewer.

@javiercn please review or suggest another reviewer.


Small typo in Embedding Chakra Wiki: "CharkaCore.h"

Hi,
There's a small typo in the C++ code example at: https://github.com/Microsoft/ChakraCore/wiki/Embedding-ChakraCore
"CharkaCore.h" should be "ChakraCore.h".
Was just wondering why my VS2015 couldn't find any classes, functions etc. :)

Regards and many thanks for the Engine :+1:

Fixed. Thanks a ton @brakmic


Explain HTTPS ports (443, 6443) and differences for API server

It's really ambiguous from the documentation which port the API server resides on:

"the API serves on port 443, protected by TLS. "

"default is port 6443"

Both refer to a secure, TLS, API Port.
It would be good to explain the distinction (if any) and how to change these ports. It would also help to explain if these are ports on the host or in the cluster.
Further confusion is the fact that "metrics-service" runs on the ClusterIP port 443 (internally).

/retitle Explain HTTPS ports (443, 6443) and differences for API server
The API server listens on port 6443 (by default). When you set up a non-toy cluster, you should expose the API server on port 443; for example, using a load balancer that isn't part of Kubernetes.
My suggestion: add extra notes to https://k8s.io/docs/reference/ports-and-protocols/ about that, and then edit https://k8s.io/docs/concepts/security/controlling-access/ to link to that updated page.

/assign


Include @doc annotations for main methods in usage

@doc annotations for main methods are described in documentation.  Everything is already in place to process them, but they're not displayed in the usage.
With this change, those docs are included on the line following the method name.

Looks good to me, thanks!


No autocomplete or debugging on targets using custom rules

Description of the issue. Please be specific.
I'm trying to run and debug bazel targets in my workspace which use custom rules also defined in my workspace. I can execute these targets manually from the command line and can hardcode them when setting up a run configuration, but I don't get autocomplete when trying to find the target during configuration editing.

Note in this screenshot how I've hardcoded //helm/api:staging and that it doesn't show up in the autocomplete list when adding a second target. I can run this target, but the UI doesn't let me debug it which I guess is part of the same problem.

This target uses a custom rule defined in a helm.bzl file located in the same workspace. Looking through the docs, it seems like the bazel plugin ignores some rules which might contribute to this.
What's the simplest set of steps to reproduce this issue? Please provide an example project, if possible.
I'm still working on a simple repro and will update this question once I have it figured out.
Version information
GoLand: 2020.1.4
Platform: Mac OS X 10.15.5
Bazel plugin: 2<IP_ADDRESS>.3
Bazel: 2.2.0

Hi there! We're doing a clean up of old issues and will be closing this one. Please reopen if you’d like to discuss anything further. We’ll respond as soon as we have the bandwidth/resources to do so.


Container Instance and Registry - false positive for deletion deploying with Bicep

Describe the noise
Resource type (i.e. Microsoft.Storage/storageAccounts)

'Microsoft.ContainerInstance/containerGroups@2023-05-01'
'Microsoft.ContainerRegistry/registries@2023-01-01-preview'

apiVersion (i.e. 2019-04-01)
Client (PowerShell, Azure CLI, or API)
Bicep CLI version 0.17.1 (d423d61882)
Relevant ARM Template code (we only need the resource object for the above resourceType and apiVersion, but if it's easier you can include the entire template
Expected response (i.e. "I expected no noise since the template has not been modified since the resources were deployed)
Container instances and registry were not changed, but always show up as deleted in what-if.
Current (noisy) response (either include a screenshot of the what-if output, or copy/paste the text)
Resource and property changes are indicated with these symbols:
  - Delete
  ~ Modify
  = Nochange

The deployment will update the following scope:

Scope: /subscriptions/b3639acc-d43d-4bdf-ab9d-21b13afbffbd/resourceGroups/rg-datainfrastructure-dev

  - Microsoft.ContainerInstance/containerGroups/dev-dynamics-wwesel4juuqom

      id:       "/subscriptions/b3639acc-d43d-4bdf-ab9d-21b13afbffbd/resourceGroups/rg-datainfrastructure-dev/providers/Microsoft.ContainerInstance/containerGroups/dev-dynamics-wwesel4juuqom"
      location: "uksouth"
      name:     "dev-dynamics-wwesel4juuqom"
      type:     "Microsoft.ContainerInstance/containerGroups"

  - Microsoft.ContainerInstance/containerGroups/dev-freshdesk-wwesel4juuqom

      id:       "/subscriptions/b3639acc-d43d-4bdf-ab9d-21b13afbffbd/resourceGroups/rg-datainfrastructure-dev/providers/Microsoft.ContainerInstance/containerGroups/dev-freshdesk-wwesel4juuqom"
      location: "uksouth"
      name:     "dev-freshdesk-wwesel4juuqom"
      type:     "Microsoft.ContainerInstance/containerGroups"

  - Microsoft.ContainerInstance/containerGroups/dev-instagram-wwesel4juuqom

      id:       "/subscriptions/b3639acc-d43d-4bdf-ab9d-21b13afbffbd/resourceGroups/rg-datainfrastructure-dev/providers/Microsoft.ContainerInstance/containerGroups/dev-instagram-wwesel4juuqom"
      location: "uksouth"
      name:     "dev-instagram-wwesel4juuqom"
      type:     "Microsoft.ContainerInstance/containerGroups"

  - Microsoft.ContainerInstance/containerGroups/dev-mailchimp-wwesel4juuqom

      id:       "/subscriptions/b3639acc-d43d-4bdf-ab9d-21b13afbffbd/resourceGroups/rg-datainfrastructure-dev/providers/Microsoft.ContainerInstance/containerGroups/dev-mailchimp-wwesel4juuqom"
      location: "uksouth"
      name:     "dev-mailchimp-wwesel4juuqom"
      type:     "Microsoft.ContainerInstance/containerGroups"

  - Microsoft.ContainerRegistry/registries/devcontainerregistrywwesel4juuqom

      id:       "/subscriptions/b3639acc-d43d-4bdf-ab9d-21b13afbffbd/resourceGroups/rg-datainfrastructure-dev/providers/Microsoft.ContainerRegistry/registries/devcontainerregistrywwesel4juuqom"
      location: "uksouth"
      name:     "devcontainerregistrywwesel4juuqom"
      sku.name: "Basic"
      sku.tier: "Basic"
      type:     "Microsoft.ContainerRegistry/registries"

Additional context
Add any other context about the problem here.

I use concat to generate the environmental variables for my containers, combining some global and container specific vars. I suspect this is causing the false positive according to issue #317 , therefore marking this as closed to avoid duplication


Brainstorm: Mimir syncing mechanism

This issue is a record of a conversation that we had around multitenancy in Mimir and our company's specific use-case and how OSKO could support that use-case without it being intrusive for other people using the project
Brainstorm

We sync PrometheusRule resources to Mimir based annotation at the SLO resource level - if no annotation, we don't do this and just create the PrometheusRule (osko.com/mimir-sync: true)
Mimir sync that's either tenant based or not from PrometheusRule based on if osko.com/target-tenant is set or not, default no tenant
TenantID based on annotation or default (namespace name) (osko.com/target-tenant: "infra")
Source tenants (osko.com/source-tenants: ["infra", "logging"])

source_tenants, if multi-tenant, have to include target
osko.com/source-tenants should be Datasource-based (therefore, SLI-based)
osko.com/target-tenant defaults to namespace name, unless it's set at the META Datasource or SLO level
Datasource META resource - osko.com/meta-ds: true

includes connectionDetails
will include target-tenant for our use-case
may include source-tenants, appended to all SLIs

RESULT (GOLDEN PATH)
1 datasource with:

connectionDetails
targetTenant
sourceTenants

Example:
apiVersion: openslo.com/v1
kind: Datasource
metadata:
  labels:
    app.kubernetes.io/name: datasource
    app.kubernetes.io/instance: datasource-sample
    app.kubernetes.io/part-of: slo-kubernetes-operator
    app.kubernetes.io/managed-by: kustomize
    app.kubernetes.io/created-by: slo-kubernetes-operator
  name: datasource-main # should it have some special name/annotation?
spec:
  description: Mimir Example Datasource
  type: mimir
  connectionDetails:
    address: https://mimir.monitoring.dev.heu.group/
    syncPrometheusRules: true
    targetTenant: billing
    sourceTenants:
      - billing
      - infra

If neither targetTenant or sourceTenants are present, treat the type: mimir datasource as not multitenant. If either is present, the other has to be as well.
Process
We will create PrometheusRule resources and sync them to the Mimir cluster using another controller as part of osko.
With that same controller, sync other PrometheusRule resources from this namespace to the cluster as well, as defined by the optional targetTenant (if syncPrometheusRules is true)

I removed mimir: object from the connectionDetails as we agreed on it during the meting.

I will convert this issue into something on the osko.dev page and then close it

@Hy3n4 wait, annotations can't be a list? 😮

That is true. This is what the Kubernetes docs says about annotations: Note: The keys and the values in the map must be strings. In other words, you cannot use numeric, boolean, list or other types for either the keys or the values.

Interesting - maybe labels can handle this? ... I should probably look more into this myself.

I think the lifetime of this issue is kind of over...