kernel-devel and kernel-headers matching kernel-core not available

Describe the bug
When running either rpm-ostree update or ujust update, I get the report that there's nothing to update.
However I need to build a custom out of tree kernel driver for the lighting on my laptop keyboard (Asus Predator device), so I need the kernel-devel installed.  I tried this and then discovered it didn't match the current kernel-core that was installed by default.
After installing kernel-core and kernel-devel:
$ rpm -q kernel-core
kernel-core-6.9.12-205.fsync.fc40.x86_64

$ rpm -q kernel-devel
kernel-devel-6.10.12-200.fc40.x86_64

$ rpm -q kernel-headers
kernel-headers-6.10.3-200.fc40.x86_64

And when I try to install the matching version for kernel-core explicitly:
$ rpm-ostree install kernel-devel-$(uname -r) kernel-headers-$(uname -r)
Checking out tree 5b620b6... done
Enabled rpm-md repositories: copr:copr.fedorainfracloud.org:matte-schwartz:sunshine copr:copr.fedorainfracloud.org:rodoma92:kde-cdemu-manager copr:copr.fedorainfracloud.org:rodoma92:rmlint copr:copr.fedorainfracloud.org:rok:cdemu updates fedora rpmfusion-free-updates-testing rpmfusion-free-updates rpmfusion-free code updates-archive
Importing rpm-md... done
rpm-md repo 'copr:copr.fedorainfracloud.org:matte-schwartz:sunshine' (cached); generated: 2024-09-13T09:52:44Z solvables: 2
rpm-md repo 'copr:copr.fedorainfracloud.org:rodoma92:kde-cdemu-manager' (cached); generated: 2024-04-20T12:39:00Z solvables: 12
rpm-md repo 'copr:copr.fedorainfracloud.org:rodoma92:rmlint' (cached); generated: 2024-05-10T10:16:28Z solvables: 4
rpm-md repo 'copr:copr.fedorainfracloud.org:rok:cdemu' (cached); generated: 2024-09-18T12:52:44Z solvables: 23
rpm-md repo 'updates' (cached); generated: 2024-10-11T03:33:46Z solvables: 28845
rpm-md repo 'fedora' (cached); generated: 2024-04-14T18:51:11Z solvables: 74881
rpm-md repo 'rpmfusion-free-updates-testing' (cached); generated: 2024-10-09T14:33:29Z solvables: 2
rpm-md repo 'rpmfusion-free-updates' (cached); generated: 2024-10-09T14:33:22Z solvables: 164
rpm-md repo 'rpmfusion-free' (cached); generated: 2024-04-20T12:11:51Z solvables: 422
rpm-md repo 'code' (cached); generated: 2024-10-11T21:14:29Z solvables: 459
rpm-md repo 'updates-archive' (cached); generated: 2024-10-10T02:51:28Z solvables: 47039
error: Packages not found: kernel-devel-6.9.12-205.fsync.fc40.x86_64, kernel-headers-6.9.12-205.fsync.fc40.x86_64

This is a brand new install of the latest Bazzite, with the only change to the repo being the addition of the VSCode RPM repo.  There is no matching kernel-devel or kernel-headers for the fsync kernel-core available in any repos.
What did you expect to happen?
I expected there to be a kernel-devel and kernel-headers matching the default shipped kernel-core.  On Bazzite this is a custom UBlue fsync version (apparently) and doesn't keep up with the latest from upstream Fedora, so UBlue would be responsible for providing the associated kernel-devel and kernel-headers to match their custom kernel-core.
Output of rpm-ostree status
State: idle
Deployments:
● ostree-image-signed:docker://ghcr.io/ublue-os/bazzite-nvidia-open:stable
Digest: sha256:f0a715ec44064ebfa14504db0c72b3790542ce3bcd3d7f3b9fb068e6a04197e7
Version: 40.20240930.0 (2024-09-30T15:25:37Z)
LayeredPackages: code grubby kernel-devel kernel-devel-uname-r sunshine zsh
ostree-image-signed:docker://ghcr.io/ublue-os/bazzite-nvidia-open:stable
Digest: sha256:f0a715ec44064ebfa14504db0c72b3790542ce3bcd3d7f3b9fb068e6a04197e7
Version: 40.20240930.0 (2024-09-30T15:25:37Z)
LayeredPackages: code grubby kernel-devel sunshine zsh
Hardware
$ inxi
CPU: 14-core (6-mt/8-st) 12th Gen Intel Core i9-12900H (-MST AMCP-)
speed/min/max: 771/400/4900:5000:3800 MHz
Kernel: 6.9.12-205.fsync.fc40.x86_64 x86_64 Up: 27m
Mem: 8.37/31.04 GiB (27.0%) Storage: 92.53 GiB/-4784 Procs: 460 Shell: Zsh
inxi: 3.3.34
$ cat /sys/devices/virtual/dmi/id/product_name
Predator PH315-55s
Extra information or context
The particulars of the device don't really matter, the problem is that there seems to be no repos to provide updates or support for the custom fsync kernel-core that ships with Bazzite.
It's reasonable that the kernel-header and kernel-devel matching the kernel-core aren't installed by default in the Bazzite ostree refs, and it's certainly easier to distribute the custom kernel-core purely by including it directly there rather than providing it via an RPM repo on copr or something, but some method of getting the kernel-devel and kernel-headers matching it are a minimum necessity of having a custom kernel in the first place.

Incidentally, this brought up basically the same topic and was closed almost immediately without any resolution whatso ever: https://github.com/ublue-os/bazzite/issues/899

I'm not at all sure where the fsync kernel-core being used is coming from, but at a rough guess it looks like maybe this copr could add what's needed, presuming it keeps old enough builds that it would still have the 6.9 kernel (the oldest kernel version Fedora itself still supports). https://copr.fedorainfracloud.org/coprs/sentry/kernel-fsync/

Yeah ok. This is a clear initial comfig bug in UBlue. Any ostrees shipping with the fsync kernel require the copr repo to be included in the /etc/yum.repos.d/ files, and the default kernels from the fefora-updates.repo to be disabled. Exactly as the https://copr.fedorainfracloud.org/coprs/sentry/kernel-fsync/ explains. If you want to add a ujust command to toggle the Fedora kernels back on that could make sense so people could then install newer kernels without fsync patches, but the default has to be to use the matching kernel packages from the copr.

I see why the copr repo isn't installed and setup as the default.  The copr only keeps the most recent build, none of the past ones, which means it has only the 6.11 kernel right now.  The current latest UBlue/Bazzite-provided kernel-core-6.9.12-205.fsync.fc40.x86_64 is relatively old, and is not supported by the copr anymore (probably hasn't been for >1 year).
I guess if UBlue is going to ship with the fsync kernel and not keep to the bleeding edge like the upstream copr supports, it's going to have to figure out how to support a cached version of all the associated kernel packages so it has the older fsync versions available. Maybe UBlue needs to setup it's own copr just to host cached copies of things?
In the meantime, it's not possible to build kernel drivers for any fsync UBlue images as far as I can tell.  The minimum necessary kernel-devel doesn't exist anymore and (apparently) wasn't cached before it was removed from the upstream copr.  From all appearances, it looks like even the UBlue akmods layer isn't functional for fsync kernels (it doesn't actually appear to be functional for anything to be honest, relying on GitHub releases of container images to keep cached copies of the kernel RPMs but the most recent release being from April 2024 and therefore way out of date).

Looking more at the UBlue akmods, it appears there's actually nightly updates to the akmods container images that just aren't published as releases. So doing this would get the set of RPMs from the latest for the fsync kernel:
$ podman pull ghcr.io/ublue-os/fsync-kernel:40
...
$ podman container create --name=foo ghcr.io/ublue-os/fsync-kernel:40
...
$ podman container cp foo:/tmp/rpms $(pwd)
...
$ ls rpms/
kernel-6.11.2-201.fsync.fc40.x86_64.rpm        kernel-devel-matched-6.11.2-201.fsync.fc40.x86_64.rpm  kernel-modules-core-6.11.2-201.fsync.fc40.x86_64.rpm
kernel-core-6.11.2-201.fsync.fc40.x86_64.rpm   kernel-headers-6.11.2-201.fsync.fc40.x86_64.rpm        kernel-modules-extra-6.11.2-201.fsync.fc40.x86_64.rpm
kernel-devel-6.11.2-201.fsync.fc40.x86_64.rpm  kernel-modules-6.11.2-201.fsync.fc40.x86_64.rpm        kernel-uki-virt-6.11.2-201.fsync.fc40.x86_64.rpm

This is running into a similar issue as the copr though, it only has the latest available kernel version.

You are currently using the fsync-ba kernel.

You are currently using the fsync-ba kernel.

Does the -ba part of this mean something?
And if so, how can you tell when the reported kernel version from both inxi and kernel-core exactly match with what the copr fsync packages/kernel report?

-ba zzite
We are currently in the process of moving building the kernel to github. This means that for you, you'll have access to all the versions for a long time.
https://github.com/hhd-dev/kernel-bazzite
With a changelog as well.

Looking more at the UBlue akmods, it appears there's actually nightly updates to the akmods container images that just aren't published as releases. So doing this would get the set of RPMs from the latest for the fsync kernel:
$ podman pull ghcr.io/ublue-os/fsync-kernel:40
...
$ podman container create --name=foo ghcr.io/ublue-os/fsync-kernel:40
...
$ podman container cp foo:/tmp/rpms $(pwd)
...
$ ls rpms/
kernel-6.11.2-201.fsync.fc40.x86_64.rpm        kernel-devel-matched-6.11.2-201.fsync.fc40.x86_64.rpm  kernel-modules-core-6.11.2-201.fsync.fc40.x86_64.rpm
kernel-core-6.11.2-201.fsync.fc40.x86_64.rpm   kernel-headers-6.11.2-201.fsync.fc40.x86_64.rpm        kernel-modules-extra-6.11.2-201.fsync.fc40.x86_64.rpm
kernel-devel-6.11.2-201.fsync.fc40.x86_64.rpm  kernel-modules-6.11.2-201.fsync.fc40.x86_64.rpm        kernel-uki-virt-6.11.2-201.fsync.fc40.x86_64.rpm
This is running into a similar issue as the copr though, it only has the latest available kernel version.

There exist these things called tags. You can get all previous versions. See https://github.com/ublue-os/kernel-cache/pkgs/container/fsync-kernel

Looking more at the UBlue akmods, it appears there's actually nightly updates to the akmods container images that just aren't published as releases. So doing this would get the set of RPMs from the latest for the fsync kernel:
$ podman pull ghcr.io/ublue-os/fsync-kernel:40
...
$ podman container create --name=foo ghcr.io/ublue-os/fsync-kernel:40
...
$ podman container cp foo:/tmp/rpms $(pwd)
...
$ ls rpms/
kernel-6.11.2-201.fsync.fc40.x86_64.rpm        kernel-devel-matched-6.11.2-201.fsync.fc40.x86_64.rpm  kernel-modules-core-6.11.2-201.fsync.fc40.x86_64.rpm
kernel-core-6.11.2-201.fsync.fc40.x86_64.rpm   kernel-headers-6.11.2-201.fsync.fc40.x86_64.rpm        kernel-modules-extra-6.11.2-201.fsync.fc40.x86_64.rpm
kernel-devel-6.11.2-201.fsync.fc40.x86_64.rpm  kernel-modules-6.11.2-201.fsync.fc40.x86_64.rpm        kernel-uki-virt-6.11.2-201.fsync.fc40.x86_64.rpm
This is running into a similar issue as the copr though, it only has the latest available kernel version.

There exist these things called tags. You can get all previous versions. See https://github.com/ublue-os/kernel-cache/pkgs/container/fsync-kernel

In Github, you can only see container tags if it was listed as a release, or if it's still present and someone gives you a link. If you don't have permissions to run builds, you also can't see a list of them. That's what releases are intended to show. So currently there are no public lists of any container builds at all in that repo.
By reading the Dockerfile code and README I was able to manually reconstruct what tag was pulled during the build for the kernel-archive image, and where files in it lived. Which is where I found the build (GitHub will show you the info about the digest matching the image tag if you go there in a browser instead of using docker pull). The tag matching the current latest non-testing bazzite ostree release (which has the 6.9 fsync kernel still) doesn't match with the akmods build for the same configuration description (fsync-kernel:40), which is why I assumed it was a nightly.
If you're talking about git tags instead, then sure if there were some way to relate that to an image digest for the kernel-archive, then I would. But all akmod builds use a moving tag (e.g. 40) and don't track what the digest was at the moment it resolved the tag. And remember, builds lists can only be seen if you personally have permission to run them.

In Github, you can only see container tags if it was listed as a release, or if it's still present and someone gives you a link. If you don't have permissions to run builds, you also can't see a list of them. That's what releases are intended to show. So currently there are no public lists of any container builds at all in that repo. By reading the Dockerfile code and README I was able to manually reconstruct what tag was pulled during the build for the kernel-archive image, and where files in it lived. Which is where I found the build (GitHub will show you the info about the digest matching the image tag if you go there in a browser instead of using docker pull). The content matching the current latest non-testing bazzite ostree release (which has the 6.9 fsync kernel still) doesn't match with the akmods build for the same configuration description (fsync-kernel:40), which is why I assumed it was a nightly.
If you're talking about git tags instead, then sure if there were some way to relate that to an image digest for the kernel-archive, then I would. But all akmod builds use a moving tag (e.g. 40) and don't track what the digest was at the moment it resolved the tag. And remember, builds lists can only be seen if you personally have permission to run them.

Uh yes it does. The last kernel in stable was fsync-ba, and you can see that the last tag in fsync-ba-kernel is the one currently in bazzite.
As for what the digest was, the images are tagged using the kernel version. So, seeing that in bazzite stable you could derive that the current image is 6.9.12-210.fsync.fc40.x86_64.
Below is an image of the kcache in incognito mode. I can see all of the tagged versions.

I am not going to say this is ideal and it is something that we are moving away from.
Here is the repo:
https://github.com/ublue-os/kernel-cache/pkgs/container/fsync-ba-kernel
We moved to fsync-ba in August due to 6.10 being completely broken. We are ready to move on to 6.11 now with a cleaner build system.

In Github, you can only see container tags if it was listed as a release, or if it's still present and someone gives you a link. If you don't have permissions to run builds, you also can't see a list of them. That's what releases are intended to show. So currently there are no public lists of any container builds at all in that repo. By reading the Dockerfile code and README I was able to manually reconstruct what tag was pulled during the build for the kernel-archive image, and where files in it lived. Which is where I found the build (GitHub will show you the info about the digest matching the image tag if you go there in a browser instead of using docker pull). The content matching the current latest non-testing bazzite ostree release (which has the 6.9 fsync kernel still) doesn't match with the akmods build for the same configuration description (fsync-kernel:40), which is why I assumed it was a nightly.
If you're talking about git tags instead, then sure if there were some way to relate that to an image digest for the kernel-archive, then I would. But all akmod builds use a moving tag (e.g. 40) and don't track what the digest was at the moment it resolved the tag. And remember, builds lists can only be seen if you personally have permission to run them.

Uh yes it does. The last kernel in stable was fsync-ba, and you can see that the last tag in fsync-ba-kernel is the one currently in bazzite.
As for what the digest was, the images are tagged using the kernel version. So, seeing that in bazzite stable you could derive that the current image is 6.9.12-210.fsync.fc40.x86_64.
Below is an image of the kcache in incognito mode. I can see all of the tagged versions.

I am not going to say this is ideal and it is something that we are moving away from.
Here is the repo: https://github.com/ublue-os/kernel-cache/pkgs/container/fsync-ba-kernel
We moved to fsync-ba in August due to 6.10 being completely broken. We are ready to move on to 6.11 now with a cleaner build system.

Oh you meant the kernel image tag is fsync-ba-kernel, I didn't understand that part.
Oh, I see what you mean, if I open the image name without a version tag in a browser, it redirects to the image itself to see the version tags. I've only ever followed to a specific version tag. I guess I'm just used to the pretty layout from registries like Quay.io or hub.Docker.com.
OK, that helps get me over the hump then. It might be worth adding something to grab these packages automatically as a ujust command, then all the digging on these particulars don't matter as much.

skopeo inspect is also good for tag sleuthing: https://github.com/containers/skopeo/blob/main/docs/skopeo-inspect.1.md

skopeo inspect is also good for tag sleuthing: https://github.com/containers/skopeo/blob/main/docs/skopeo-inspect.1.md

I did just find that yesterday  (unrelated actually), but apparently it wouldn't have helped in this case because I didn't even have the right basename.


How to use antv g6 with vue cli

G6 Version:
Platform:
Mini Showcase(like screenshots):
CodePen Link:

g6 works perfectly with Vue. Create a new project and install g6:
vue create testapp
cd testapp
npm install @antv/g6

In your vue component:
<template>

</template>
[...]


Revert "Shortlist downloaded state (for labs)"

Reverts alphagov/digitalmarketplace-buyer-frontend#599

:fire: :fire: :fire:
:+1:
:fire: :fire: :fire:


Publish 7.6-20221117002350+0000

Publish 7.6-20221117002350+0000

@bot-gradle test and merge

Your PR is queued. See the queue page for details.


Compact cver option

When checking what needs updating with cver, the output is very bloated and requires a lot of scrolling to view, e.g.

I'd like to see nice compact output like the following:
Name            Installed           Available           Upgradable
=============== =================== =================== ==========
7zip            <IP_ADDRESS>30618    <IP_ADDRESS>30618
7zip.install    <IP_ADDRESS>30618    <IP_ADDRESS>30618
Console2        2.0.148             2.0.148
fiddler4        <IP_ADDRESS>             <IP_ADDRESS>0130128     Y
git             1.8.4               <IP_ADDRESS>             Y
git.install     1.8.4               <IP_ADDRESS>             Y
...

Perhaps the current output can stay as the default with the compact output as an option. Or perhaps the compact should be the default with the verbose being an option.

How does this look?
This uses limitoutput option (-r) to keep the information relevant. false at the end tells you whether it is pinned or not:
c:> choco upgrade all --noop -r
linqpad|2.46.09|4.51.03|false
grepwin|1.6.1|1.6.6|false
vim|7.4.27|7.4.560|false
sumatrapdf.commandline|2.5.2|3.0|false
notepadplusplus.install|6.7.3|6.7.4|false
mingw|<IP_ADDRESS>40107|<IP_ADDRESS>41208|false
1password|<IP_ADDRESS>1|<IP_ADDRESS>0|false
notepadplusplus.commandline|6.7.2|6.7.4|false
PowerShell|4.0.20140915|4.0.20141001|false
autohotkey|<IP_ADDRESS>|<IP_ADDRESS>|false
webpicommandline|7.1.40719.0|7.1.50430.0|false
chocolatey.server|0.1.0|0.1.1|false
wincommandpaste|1.0.6|1.0.8|false
git|1.9.4|<IP_ADDRESS>50114|false
javaruntime|7.0.51|7.0.75|false
gitextensions|2.47.3|2.48.03|false
fiddler|<IP_ADDRESS>|<IP_ADDRESS>|false
VirtualCloneDrive|<IP_ADDRESS>30920|<IP_ADDRESS>|false
ChocolateyPackageUpdater|<IP_ADDRESS>|<IP_ADDRESS>|false
git.commandline|1.9.4|1.9.5|false
TestDriven.Net|3.3.2779|3.5.2830|false
DotNet3.5|3.5|3.5.20140403|false
resharper|8.0.2000.2660|9.0.1|false
yumi|<IP_ADDRESS>|<IP_ADDRESS>|false
notepadplusplus|6.7.3|6.7.4|false
autohotkey.install|<IP_ADDRESS>|<IP_ADDRESS>|false
papercut|2<IP_ADDRESS>|2014.11.5|false
puppet|3.7.0|3.7.4|false
ccleaner|4.17.4808|5.02.5101|false
keepass.portable|2.27|2.28|false
sumatrapdf|2.5.2|3.0|false
pip|1.1.1|1.2.0|false
ruby.devkit|<IP_ADDRESS>11231|<IP_ADDRESS>20101|false
boxstarter.chocolatey|1.0.33|2.4.159|false
autohotkey_l|<IP_ADDRESS>|<IP_ADDRESS>40119|false
DotNet4.0|4.0.30319.20130222|4.0.30319.20141222|false
ketarin|<IP_ADDRESS>4|1.7.1|false
nodejs|0.10.30|0.10.36|false
autohotkey.portable|<IP_ADDRESS>|<IP_ADDRESS>|false
Pester|2.0.2|3.3.5|false
WindowsLiveWriter|2011.0.0|2012.0.0|false
dropbox|2.8.4|3.0.5|false
webpicmd|7.1.50430.0|7.1.50430.20141001|false

A table would be better but I can understand the difficulty in determining column widths. Perhaps a space either side of the pipe symbol would be a bit more readable?
What do you mean by 'pinned'?

That did just point out a possible bug in new choco install logic. NOTE: It believes chocolatey <IP_ADDRESS> is installed. :/

Pinned, means you "pin" to a particular version. It's a way to suppress upgrades...

Dang it, not a bug... :/

Attempting to upgrade shows the proper version:

It's there in the package directory:

Anyone want to guess what I forgot to pass? Yes, -pre. I think localonly should always prefer the prerelease. So it's a bug.

https://github.com/chocolatey/choco/issues/46 (and fixed)

FYI, the -r option is meant for machines to read, not humans.


web related files in umbrella

I don't think lib folder is the place for roda_plugins or page.rb. Maybe I am wrong.

Fixed in https://github.com/dry-rb/dry-web-roda/commit/b050575e3304da873d3a249e8820037d42824d57


SAK-29973 avoid an extra hibernate db hit when adding privacy records

The setCacheable is nice for performance, but it doesn't invalidate when updated..... It only invalidates when stale. This can result in constraint errors.

This makes sense as need to be careful about using Query.setCacheable(true) when not used in conjunction with Hibernate's second level cache, since state is maintained by the second level cache and the query cache does not.


Added get_puppet, helpers and tests

Closes #868.

Overall looks okay. I'll take a deeper look and merge it later on.
@MegaIng  @evilnose
Slightly related: What do you think of renaming the "puppet" to "interactive parser" ?
Then instead of starting it in "puppet mode", you'll be starting the parser in "interactive mode".
An alternative name is "single step parser". Another suggestion that's been raised is "parser cursor".
Let me know your thoughts.


Ensure Documentation of Own Components and Features are Updated

@CS2103JUN2017-T2/developers
Hi all, I created a branch called documentation-update, we can all use this branch for documentation update.


ListObservable doesn't support .push()

Can you provide an example of how you would use angularfire2offline to write to a firebase location? Or would you just use angularfire to write to locations and only use angularfire2offline for the reads?

Right now AngularFire2Offline only works with read-only methods.
You can use both AngularFire2 for writing to Firebase and AngularFire2Offline when you would like to be able to access data offline.
If you would like to have write methods added to this library, feel free to request that feature.
Here's an example of using both libraries together:
import { Component } from '@angular/core';
import { AngularFire, FirebaseListObservable } from 'angularfire2';
import {
  AngularFireOffline,
  ListObservable,
  ObjectObservable } from 'angularfire2-offline';

@Component({
  selector: 'project-name-app',
  template: `

{{ (info | async)?.name }}

      {{ item?.name }}

  <button (click)="addItem('itemNameHere')">Add item</button>
  `
})
export class MyApp {
  info: ObjectObservable<any>;
  items: ListObservable<any[]>;
  afItems: FirebaseListObservable<any>;
  constructor(
    af: AngularFire,
    afo: AngularFireOffline) {
    this.info = afo.database.object('/info');
    this.items = afo.database.list('/items');
    this.afItems = af.database.list('/items');
  }
  addItem(itemName) {
    this.afItems.push({ name: itemName });
  }
}

+1 i would like that feature - to have pushes stored in localstorage that are synced on connect. unfortunately my ts skills are not yet good enough to implement it myself

@niklasp Saving to Firebase while offline sounds like a useful feature!

Currently you can save to Firebase while offline if you don't do a complete refresh after saving data. This is actually a supported feature of Firebase.
After implementing the suggested feature of saving data offline (using local storage) the data should persist through a page refresh and write back to Firebase when an internet connection is available.

I've just find this library and I think that is very great!
Saving data to Firebase while offline would be great, how do you plan to manage conflicts?

Hi @giorgiopiatti
AngularFire2 Offline currently supports saving data to Firebase while offline for the related AngularFire2 write operations: Read object, Read list, Write object, Write list.
Write conflicts can happen with any Firebase app, but the issue is magnified with offline based writes.
 This is a key piece of the angularfire tech for mobile phones where you can't step in when the user is about to force quit the app.

@cweidinger  Awesome, glad it's helpful!

@adriancarriger Thankyou very much for the examples! I think that with timestamp, all can be done fine.


Make capitalization more consistent, clean up code formatting

Electrosmith -> Electro-Smith
Usb -> USB
Firmware -> firmware

Also ran Prettier and deleted some stray blank lines

Nice! Much appreciated!


theater users can't speak over moderation

#40 removes this non-standard feature so let's close this


Having trouble with prefixes.

I may have set up my groups wrong, but I am having trouble adding multiple prefixes. If I set a format to be the highest on track, it'll set the player's rank to the highest group inside of the parented group. How would I set it so it shows all of the prefixes they have?

Give them all of the groups that you want them to have a prefix for, From my experience, Giving someone the highest rank in a track does not give them multiple prefixes, Because the priority is always gonna be the top rank prefix, Unless you give them multiple ranks. From another track

Also
https://github.com/lucko/LuckPerms/wiki/Prefix-&-Suffix-Stacking


Doesnt work at all

C:\Users\gregc2\Desktop\ionicsignup - Copy\cordova>cordova-icon
Checking Project & Icon
√  platforms found: android
?  icon.png does not exist in the root folder
C:\Users\gregc2\Desktop\ionicsignup - Copy\cordova>cordova-ico
Checking Project & Icon
√  platforms found: android
√  icon.png exists
√  config.xml exists
Generating Icons for android
{ [Error: Command failed: Invalid Parameter - -set
] timedOut: false, killed: false, code: 4, signal: null }

Hey @ioudas210, do you have any update on this?

Nah gave up. Thanks tho.

didn't work on ImageMagick-6.9.3-8-Q16-x64-dll.exe, dese this work on specific version of ImageMagick?

When installing ImageMagick on Windows, you need to check the box that says something about "legacy". That is what includes convert.exe (of which this is reliant on). You also need to change line 255 of npm\node_modules\cordova-icon\node_modules\imagemagick\imagemagick.js to be:
//exports.convert.path = 'convert';
exports.convert.path = "C:\Program Files\ImageMagick-7.0.2-Q16\convert";
Chat it to whatever your path is.

When installing ImageMagick on Windows, you need to check the box that says something about "legacy". That is what includes convert.exe (of which this is reliant on). You also need to change line 255 of npm\node_modules\cordova-icon\node_modules\imagemagick\imagemagick.js to be:
//exports.convert.path = 'convert';
exports.convert.path = "C:\Program Files\ImageMagick-7.0.2-Q16\convert";
Chat it to whatever your path is.

its not working for me

@ganeSH-Ganz just FYI, that comment was given over 3 years ago.

but still there no proper solution for it

Maybe not, who knows, it's from years ago, which is ancient in terms of software. I'm sure there's plenty of other solutions for sizing icons, even if it's not this tool.


Add separate endpoint to fetch jobs for a scan

The jobs and there output can get little large ... so GET /scans endpoint is starting to have fetch lot of data ...

Add pagination of scans will resolve this with out need to change the jobs endpoints ... Fixes by: #94

We currently only see 100 scans in Distiller. The Edit field has important information. Will we be able to see older scans and data with this update? i.e. separate pages of 100 scans.

Yes, they will just be paginated. So you will be able to load them in batches.

Now deployed


[SPARK-34262][SQL][3.1] Refresh cached data of v1 table in ALTER TABLE .. SET LOCATION

What changes were proposed in this pull request?
Invoke CatalogImpl.refreshTable() in v1 implementation of the ALTER TABLE .. SET LOCATION command to refresh cached table data.
Why are the changes needed?
The example below portraits the issue:

Create a source table:

spark-sql> CREATE TABLE src_tbl (c0 int, part int) USING hive PARTITIONED BY (part);
spark-sql> INSERT INTO src_tbl PARTITION (part=0) SELECT 0;
spark-sql> SHOW TABLE EXTENDED LIKE 'src_tbl' PARTITION (part=0);
default	src_tbl	false	Partition Values: [part=0]
Location: file:/Users/maximgekk/proj/refresh-cache-set-location/spark-warehouse/src_tbl/part=0
...

Set new location for the empty partition (part=0):

spark-sql> CREATE TABLE dst_tbl (c0 int, part int) USING hive PARTITIONED BY (part);
spark-sql> ALTER TABLE dst_tbl ADD PARTITION (part=0);
spark-sql> INSERT INTO dst_tbl PARTITION (part=1) SELECT 1;
spark-sql> CACHE TABLE dst_tbl;
spark-sql> SELECT * FROM dst_tbl;
1	1
spark-sql> ALTER TABLE dst_tbl PARTITION (part=0) SET LOCATION '/Users/maximgekk/proj/refresh-cache-set-location/spark-warehouse/src_tbl/part=0';
spark-sql> SELECT * FROM dst_tbl;
1	1

The last query does not return new loaded data.
Does this PR introduce any user-facing change?
Yes. After the changes, the example above works correctly:
spark-sql> ALTER TABLE dst_tbl PARTITION (part=0) SET LOCATION '/Users/maximgekk/proj/refresh-cache-set-location/spark-warehouse/src_tbl/part=0';
spark-sql> SELECT * FROM dst_tbl;
0	0
1	1

How was this patch tested?
Added new test to org.apache.spark.sql.hive.CachedTableSuite:
$ build/sbt -Phive -Phive-thriftserver "test:testOnly *CachedTableSuite"

Authored-by: Max Gekk<EMAIL_ADDRESS>Signed-off-by: HyukjinKwon<EMAIL_ADDRESS>(cherry picked from commit d242166b8fd741fdd46d9048f847b2fd6e1d07b1)
Signed-off-by: Max Gekk<EMAIL_ADDRESS>

Test build #134599 has started for PR 31379 at commit fb1d319.

Test build #134599 has started for PR 31379 at commit fb1d319.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39186/

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39186/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39186/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39186/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39186/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39186/

Merged to branch-3.1.

Merged to branch-3.1.

Test build #134599 has finished for PR 31379 at commit fb1d319.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test build #134599 has finished for PR 31379 at commit fb1d319.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134599/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134599/


Update USAGE.md

After splitting the original README, both ways --binary loaded or locally built-- should be mentioned. Also move prescipt how to call the binary from INSTALL.md here.

Danke!


Update deps

Change code where needed to remove warnings and make tests pass.

Pull Request Test Coverage Report for Build 0d6b2888773d58e8c15a1b0116ebd91c20277885-PR-589
Details

0 of 0   changed or added relevant lines in 0 files are covered.
10 unchanged lines in 1 file lost coverage.
Overall coverage increased (+0.04%) to 80.486%

Files with Coverage Reduction
New Missed Lines
%

lib/edgehog/repo.ex
10
6.25%

Totals

Change from base Build 18048171c49591875b5a74be13b952133162e1e6:
0.04%

Covered Lines:
1225

Relevant Lines:
1522

💛  - Coveralls


feature/auth/public-api

feature/auth/public-api

@TalhaAli00 I think we merged it into wrong branch


Post audit tweaks

We've had a few really nice audits done on this code and things have generally been positive. There are a few pieces of feedback that I think are worth addressing before we settle on the final code though. Here is a list, ordered by priority:

Verify timestamp is non-zero -- currently the implementation allows an input of zero to return a zero root. This is because the current mainnet timestamp is not perfectly divisible by our modulus. Although no proof could be authenticated against the zero root, it's still an ugly correctness edge case that I think should be addressed.
Update buflen to 93600 -- this value will share 500-1000^1 more remainders if the slot time were to change versus the current modulus, decreasing the overall storage use of the contract (h/t @adietrichs).
Load calldata once -- minor nit, but in case the cost of retrieving calldata changes in the future, it would make more sense to only access it via calldatacopy a single time, instead of paying for that cost at each load. This require a bit of dup'ing and swap'ing, but still extremely manageable. As a part of this I was also able to delete the get_input macro (h/t @adietrichs).

Don't forget to update the readme (and any other reference) with the new code, so the repo is consistent.

I have a few more nits. I don't like that revert is a label, revert() is a macro, and, well, revert is also an opcode. I tried to change this earlier, but I think it has crept back. What I would want is

lbl_revert for the label (and lbl_-prefix for all labels)
macro_revert for the macro

I'll make a PR

This change is fine, IMO, but if one were to complain, it would be that the flow is a bit awkward.

Check-length, if ok: goto loadtime,
fail revert
loadtime: check input == zero, if bad, goto revert
verify stored==load, if ok, goto loadroot
fail revert
loadroot: ...

The goto revert is used exactly once, and the inlined macro revert  is used twice (three times if we count the usage inside the goto revert). A more consistent control-flow would be

Check-length, if bad: goto revert,
Check input == zero, if bad, goto revert
loads root..
Check stored==load, if bad, goto revert
loadroot: ...
7.revert: reverts..

Then we would have one label revert, but no macro revert. This would be a somewhat larger change, since we need to reverse some of the checks.


Update skip lock option help info since it is used in several operations

The issue
Currently --skip-lock option is used for several commands, such as install, uninstall. The Ignore locking mechanisms when **installing** is not appropriately anymore.
The checklist

[ ] Associated issue
[ ] A news fragment in the news/ directory to describe this fix with the extension .bugfix, .feature, .behavior, .doc. .vendor. or .trivial (this will appear in the release changelog). Use semantic line breaks and name the file after the issue number or the PR #.

Does this mean that the pipenv install with the --skip-lock flag will install things in the same greedy fashion that pip would? If so, I still think the help is confusing.
Maybe something like, "Skip dependency resolution provided by locking and fall back to a pip-style greedy installation of the requirements specified in the Pipfile"

This means nothing because nothing is different than it was, and yes, if you skip locking pipenv just passes your dependencies to pip, that isn't greedy, it just doens't use a dependency resolver.

As a user who was unfamiliar with the lingo of pipenv, it wasn't clear to me that flag would cause pipenv to fall back to pip-style installation, but I get it now. Thanks

that isn't greedy, it just doens't use a dependency resolver

Perhaps greedy was the wrong term. I was referring to this, from the Pip User Guide, "pip doesn’t have true dependency resolution, but instead simply uses the first specification it finds for a project", which I thought of as a greedy (sub-optimal) algorithm, as opposed to an algorithm which can find the globally optimal solution (a. la dependency resolution).


KAFKA-4982: add protocol and listener tags to socket-server-metrics

Added tags

Do you think you'll be able to update the PR today? If so, we can probably get it merged today.

Thanks for the detailed feedback. I've updated the PR


Abort previous ajax calls in jsTree

How to cancel previous ajax call while loading child nodes.. am using below format to load nodes using ajax. And trying to abort previous ajax call in the beforesend function which unfortunately called multiple times and causes throbe icon to display multiple times(switching between previous and current nodes).
.jstree({
'core': {
'data': {
'async': true,
'data': function (node) {....},
'url':function(node){....},
'beforesend':function(jqXhr){
if(prevRequest!=undefined && prevRequest!=''){
prevRequest.abort();
}
prevRequest=jqXhr;
},
'success';function(data){.....}
.......});

Switch to using core.data as a function - that way you have full control over the AJAX requests (since you will be making the manually).

Thanks a lot, can you provide any sample which uses core.data as a function ?

Try using instance.get_node(prevNode).state.loaded = false;

'$tree.jstree(true).get_node(prevNode).state.loaded = false;'
it did help but also brought back the issue i was facing previously. it just triggers multiple ajax request which makes the flickering(switching between current and previous nodes).


[5.2] DeserializeSIL: Fix two places where we want the SILType inside of the current function according to the TypeExpansionContext

rdar://58095210

@swift-ci Please test

Original PR: https://github.com/apple/swift/pull/28901


liberation-circuit: 1.3 -> unstable-2022-01-02

Motivation for this change
While the project hasn't seen a new official release in a few years, development is still semi-active and has seen some important fixes and refactorings. Fixes #135942
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
22.05 Release Notes (or backporting 21.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

@PoustouFlan


Ignoring Keep-Alive settings of Global Agent

Hello,
AWS approach for configuring is more flexible.
This line prevent to use http(s).globalAgent.options.keepAlive

We might consider changing this default behavior in future, but for now, users have to manage their connection pool by passing us an agent explicitly.
Relevant spec: https://fetch.spec.whatwg.org/#connections

I agree that overriding global agent options causes unexpected behavior for NodeJS devs so getNodeRequestOptions should onor them. This is expecially true where you have to use node-fetch aside other http clients (ie. some cloud sdk) that you want to configure once.


refactor(api-client): create usePipetteSettingsQuery for pipette settings slideout

closes #10506
Overview
This fixes the bug in pipette settings. Rather than using the getAttachedPipetteSettings selector, this creates a hook in api-client called usePipetteSettingsQuery that returns the pipette settings. Also there are some small refactors in Configure Pipette
Changelog

modify ConfigurePipette
create usePipetteSettingsQuery and test

Review requests

does it work as expected? is the logic correct?
open up the pipette settings slideout, you should now see the form fields to input the values to change the pipette settings.

Risk assessment
low

Figma space showing annotations.
Looking so good! Just a few minor things:

[ ] (1) Update colour to our dark-black
[ ] (2) Update to our new checkbox component (new icon, various states and label is 13px vs 11px)
[ ] (3) Can we remove the box shadow on the button and opt to put it on the container behind it? Loom video explaining.


Add blog post "Equality"

Adding a new blog post from the Steering Committee titled "Equality".

/hold


Update ml5 version number to @latest

We recently released the 0.5.0 release version of ml5. This PR updating examples here to specify the new version.
I wonder if we should specify @latest here instead though cc @joeyklee.
EDIT
Switched to @latest instead of a specific version number.

@bomanimc - thank you for opening this PR!
I think it is ok now to go for @latest because it is transparent which version number of ml5 is running (in the console) and we can inform the Teachable Machine team if any of our changes are breaking w/ PRs, rather than needing to make PRs here if things are being fixed. Does that sound reasonable?

@joeyklee sounds good! Updated this PR

@bomanimc - Thanks for updating the PR!
@HalfdanJ - Many thanks for merging! 🙏


I am getting the below error after connecting to wandb and using ./run.sh [path/to/config]

GPUs in usage: 0
Running Centralized experiment with 828 epochs
Setting up distributed...
Let's use 1 GPUs!
Done
Initializing wandb...
wandb: Currently logged in as: swapnikvarala. Use wandb login --relogin to force relogin
Problem at: /usr/local/lib/python3.11/site-packages/wandb/sdk/wandb_init.py 837 getcaller
Traceback (most recent call last):
File "/content/FedDrive/src/run.py", line 31, in
run_experiment()
File "/content/FedDrive/src/run.py", line 15, in run_experiment
main(args)
File "/content/FedDrive/src/centr_setting/main.py", line 17, in main
writer, device, rank, world_size, logger, label2color, denorm = setup_env(args)
^^^^^^^^^^^^^^^
File "/content/FedDrive/src/utils/utils.py", line 36, in setup_env
logger = CustomWandbLogger(name=get_job_name(args), project=get_project_name(args.framework, args.dataset),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/content/FedDrive/src/utils/logger.py", line 7, in init
super(CustomWandbLogger, self).init(name=name, project=project, group=group, entity=entity, offline=offline,
File "/usr/local/lib/python3.11/site-packages/pytorch_lightning/loggers/wandb.py", line 358, in init
_ = self.experiment
^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/lightning_fabric/loggers/logger.py", line 114, in experiment
return fn(self)
^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/pytorch_lightning/loggers/wandb.py", line 406, in experiment
self._experiment = wandb.init(**self._wandb_init)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/wandb/sdk/wandb_init.py", line 1173, in init
raise e
File "/usr/local/lib/python3.11/site-packages/wandb/sdk/wandb_init.py", line 1154, in init
run = wi.init()
^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/wandb/sdk/wandb_init.py", line 770, in init
raise error
wandb.errors.CommError: Run initialization has timed out after 60.0 sec.
Please refer to the documentation for additional information: https://docs.wandb.ai/guides/track/tracking-faq#initstarterror-error-communicating-with-wandb-process-
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 24118) of binary: /usr/local/bin/python3
Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in _run_code
File "/usr/local/lib/python3.11/site-packages/torch/distributed/run.py", line 798, in
main()
File "/usr/local/lib/python3.11/site-packages/torch/distributed/elastic/multiprocessing/errors/init.py", line 346, in wrapper
return f(*args, **kwargs)
^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/torch/distributed/run.py", line 794, in main
run(args)
File "/usr/local/lib/python3.11/site-packages/torch/distributed/run.py", line 785, in run
elastic_launch(
File "/usr/local/lib/python3.11/site-packages/torch/distributed/launcher/api.py", line 134, in call
return launch_agent(self._config, self._entrypoint, list(args))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/torch/distributed/launcher/api.py", line 250, in launch_agent
raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:
run.py FAILED
Failures:
<NO_OTHER_FAILURES>
Root Cause (first observed failure):
[0]:
time      : 2023-07-18_17:50:12
host      : f905eb7441a0
rank      : 0 (local_rank: 0)
exitcode  : 1 (pid: 24118)
error_file: <N/A>
traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html

I have same problem.
How to solve Wandb problem?

I disabled wandb mode as you suggested above, still I am facing issues
RuntimeError: CUDA error: invalid device function
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 581611) of binary: /home/cvblhcs22/anaconda3/envs/feddrive/bin/python3
Traceback (most recent call last):
File "/home/cvblhcs22/anaconda3/envs/feddrive/lib/python3.9/runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/home/cvblhcs22/anaconda3/envs/feddrive/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/home/cvblhcs22/anaconda3/envs/feddrive/lib/python3.9/site-packages/torch/distributed/launch.py", line 193, in
main()
File "/home/cvblhcs22/anaconda3/envs/feddrive/lib/python3.9/site-packages/torch/distributed/launch.py", line 189, in main
launch(args)
File "/home/cvblhcs22/anaconda3/envs/feddrive/lib/python3.9/site-packages/torch/distributed/launch.py", line 174, in launch
run(args)
File "/home/cvblhcs22/anaconda3/envs/feddrive/lib/python3.9/site-packages/torch/distributed/run.py", line 710, in run
elastic_launch(
File "/home/cvblhcs22/anaconda3/envs/feddrive/lib/python3.9/site-packages/torch/distributed/launcher/api.py", line 131, in call
return launch_agent(self._config, self._entrypoint, list(args))
File "/home/cvblhcs22/anaconda3/envs/feddrive/lib/python3.9/site-packages/torch/distributed/launcher/api.py", line 259, in launch_agent
raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:
run.py FAILED
Failures:
<NO_OTHER_FAILURES>
Root Cause (first observed failure):
[0]:
time      : 2023-11-23_19:21:01
host      : ubuntu-22-0-4
rank      : 0 (local_rank: 0)
exitcode  : 1 (pid: 581611)
error_file: <N/A>
traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html

I also have the same issue as @kagHarsh above despite WandB working. How do I fix the issue of the local rank? I did all the instructions given in the README.


Assignment 1 Pim Vree

Introduction
Please briefly introduce yourself here, for example: Hi all, my name is Pim Vree and I'm a PhD student in the van der Sar lab. I also have two cats!
Reflections on the importance of RDM videos
Reflect about what you heard on the video and briefly write your thoughts and your horror stories in less than 5 sentences.
Nice and insightful of why data management is important. My horror stories mostly include experiments where either important measurement settings where not stored properly in the data files or not at all.
What would you like to learn during this course?
Are there any things in particular that you would like to get out of this course? Do you have any goals that you would like to work on?
I think a good way to structure, store research data and a structured why to have an overview of my data analysis.
Checklist assignments

[x] Assignment 1: creating a GitHub issue (before Class 1)
[x] Respond to the GitHub issue on 'data challenge'
[x] Assignment 2: Data Flow Map 1 (link](https://surfdrive.surf.nl/files/index.php/s/a4sNyvcNpq0Dbg2))
[x] Respond to the GitHub discussion on 'licenses'
[x] Respond to the GitHub discussion on 'folder structure'
[x] Respond to this GitHub issue with your readme file
[x] Assignment 3: Data Flow Map 2 (share a link in this issue before 14 March 13:00)
[ ] Assignment 4: Data Management Plan (before Class 2)
[ ] Respond to the GitHub discussion on 'Data Management Plans' if you have any questions (optional)
[ ] Assignment 5: Data Flow Map 3: submit your slide (before Class 2)

Hi Pim (@phvree), I am Paula and I am the instructor from the library supporting Esther with this run of the course. I normally teach in the RDM101 provided to all faculties.
Thank you very much for your assignment! I had a look and I have some questions/suggestions;
About the listing of data types and code. It good good to think separately about data, design data/information and code. But, I think for the purpose of the assignment and also thinking about the assignment 2, it would be good to maybe split a bit more or create subcategories of data and code. For example, for the experimental data you will be also collecting different types of data within that category.  The instruments used for collecting images will be different as those for collecting sensor data, right? It means that the file formats, the software use for collecting them and the documentation you will need for them will also be different, right?
The same goes for the experimental data from TNO. If you do not have detailed information about the data you will have access to and in which format, etc., it is fine to group that data for now. But, it will be useful to create a more detail listing and descriptions, especially when thinking about data organization.
It got my attention that you describe the experimental data as .mat files. Is that because you will process the raw images and the raw sensor data in MatLab? If that is the case, then you should probably distinguish between raw data and the description, file formats and size of those, from the analyzed data, which might be stored in .mat format. I can be saying something totally wrong, so it would be great if you could comment on that, so I also learn from your workflows :-)
Maybe you could have a look at @sercan's assignments to compare? He seems to have similar data types and code as you, maybe it can help to look at your data and code from another point of view?
A couple of your peers have mentioned this NAS server, which is fantastically synchronized with the computer of instruments and laptops. If the computers of the instruments are synchronized with the NAS, why are you also using an external drive? Is that your personal external drive? Or, is that an external drive which is always connected to the PCs?
I hope that TNO is willing to use one of the secure alternatives for sharing data that TU Delft recommends? Either Project Data (U:) drive or SURFdrive, maybe (if the data is not highly confidential)?
@EstherPlomp, is this NAS server something provided by ICT for applied sciences? Or, is this NAS server administrated by the faculty?

Here is my template for the readme
ReadmePhD_Pim_vree.txt

@MpaulaL thanks for the extensive feedback.
In our measurement setup almost all sensors and systems are controlled and read out through Matlab, so that we have a central system that coordinate and control everything during a measurement. Typically, the raw data is collected through the drivers also in Matlab where it is stored in a .mat file together with relevant experimental data ( which is most of the time also controlled through matlab) such as the temperature of the sytem, laser power, position of magnet etc. These parameters are thus directly set through matlab but important to store for further analysis.  Typicaly, we have thus .mat files that store the data and .m files for matlab scripts and functions to analyze and capture the data, you are right that I could further divide those files in a subcategory.
The external drive is connected to the lab PC and it is mainly used to store the data during the measurements ( with the main benefit that it will also work without an internet connection and is a lot faster)
For TNO the exact data is still and work method is not yet properly defined but I will try to be a bit more specific, but I guess it will be mainly a carbon copy of my main project datasets.

RDM101_Assignment2_Week2_DataFlowMap_Template.pptx
Here is the link to the ppt containing my updated datasets based on your feedback and the information containing to the FAIR principles.
I did have a hard time answering some of these questions, cause I really did not know what is standard in the field and how to di all of these things in an efficient manner, but keeping the current consistent structure of the project. As already mentioned it is not yet clear how confidential the TNO data is ( or at all), so I could not really draw any different conclusions for these data sets.

@MpaulaL thanks for the extensive feedback.
In our measurement setup almost all sensors and systems are controlled and read out through Matlab, so that we have a central system that coordinate and control everything during a measurement. Typically, the raw data is collected through the drivers also in Matlab where it is stored in a .mat file together with relevant experimental data ( which is most of the time also controlled through matlab) such as the temperature of the sytem, laser power, position of magnet etc. These parameters are thus directly set through matlab but important to store for further analysis. Typicaly, we have thus .mat files that store the data and .m files for matlab scripts and functions to analyze and capture the data, you are right that I could further divide those files in a subcategory.
The external drive is connected to the lab PC and it is mainly used to store the data during the measurements ( with the main benefit that it will also work without an internet connection and is a lot faster) For TNO the exact data is still and work method is not yet properly defined but I will try to be a bit more specific, but I guess it will be mainly a carbon copy of my main project datasets.

Thanks for the explanation @phvree! Although I am not a fan of MatLab for being a proprietary format, I think t is a great automated process that you have their at your lab! :-D

Hi @phvree, thank you very much for your assignment! And thanks a lot for considering my feedback and extending the data types and code list! I think it gives a much better overview.
There are several of you using the group server for collecting the data from instruments. This is very nice to make sure that all the data is securely save, but it can be a challenge to make the data easily findable and linked to other data. Maybe in the next class you could discuss a bit about this with your peers.
But, I think you have done a very nice reflection on how to better organise the data and code. You will need to get started and then see if the plan fits with your workflow. If not, you could do some adjustments.
I've seen also your assignment of folder structure and it looks a bit different to the one in the PowerPoint. Do you have an inclination on which one to use?
I think it will be very important for you to discuss this with your supervisor and maybe team members. If the improvements in the folder structure will be only used by you in the common server, it might challenge others to find the data. If there is no wish to improve the folder structure from your supervisor and team members, maybe you could keep the main folder structure:
──./NAS/Measurement_data/Setup/
│└── type of sample
│ │ └── sample
│ ││ └── analysis <- scripts for intermediate and final analysis
│ ││ └── Measurement data <- raw and intermediate data
And do little changes, like adding some ReadMe files for documentation on how to navigate the folder structure, describing the file naming conventions and adding the paths to linked data or code. You could also maybe create a good subfolder structure within the analysis folder, separating the scripts for intermediate analysis from the final analysis. And in the measurement data, separate the raw from the intermediate data?
It is good and great actually that you now know better ways to organise the data. If at any point you can decide yourself how to do it, you have already a pretty good idea :-) Maybe there is an opportunity to organise the TNO data in the way you envisioned?
About documentation. Great that you thought through what type of information you should collect. About the question using a ReadMe file or include it in the .m file. I think there are pro-and cons- to both. A . txt file can be opened with any text editor, open file format yay!. But, if there is a way to automate the collection of documentation, it is a nice things to do thinking about how efficiently document well the data.
Since you are also using OneNote as kind of lab book, make sure that documentation you have in other places is linked to your experiments. It could be just indicating the path to where the right documentation is. If possible, try to be consistant on how you organise your experiments in OneNote. Maybe you can create kind of templates? Here you can remember some of the basic tips that we provide for paper notebooks, they can also help here when using tools like OneNote as a lab book:
•	Write in a commonly agreed language, e.g. English
•	Write down the date for each record
•	Make sure that it is possible to separate the different experiments – e.g., by using meaningful names
•	Note where the raw experimental data will be stored and the name of the corresponding data file
About your idea of writing a ReadMe file for some of the whole datasets, I think is great. Sometimes, you don't need a ReadMe file per file, but you can create a ReadMe for the folder where to tell a bit about what type of data is compile in that folder, how it was collected (which instruments, model of it), description of file naming conventions, description of file format and how to open the files, etc. That type of information would apply to all datasets.
About metadata. I totally agree with your decision. If metadata is collected automatically during the measurement and is attached to the data, that is great. A good file naming convention, providing meaningful extra metadata, is a great choice!
File formats. Just remember to add the information in the ReadMe file about what is .m and which software to use to open them. The same for Autocad and Kicad. If in your group it is a standard to work with MatLab, ti will require enormous effort to transform the code to .py. So, just make sure you provide the relevant information about file formats in the documentation.
About Access. I see that there are uncertainties about making data openly available. I guess that is why there is also not a section for Data Publication? It is important that you discuss this with your supervisor or ask your team mates if they have any repository of choice. 4TU.ResearchData is always a good choice. It provides a DOI per dataset and/or code, you can provide a license, you can add more standardized metadata and as TU Delft researcher you can deposit up to 1TB per year.
It would be great if besides the raw data, you also publish the analysis scripts to demonstrate how you generated figures and graphs in your publications. The same for the scripts of the model.
About the license, CC-BY-SA. It is an intermediate open license. If possible, I would go for an even more open option, CC-BY. Sometimes we have good intentions restricting the licenses, to ensure that the data is also shared openly. But, if somebody builds upon the data of your project, maybe adding more data to the dataset and would like to publish it as open as possible, they could not use a CC-BY or a CC0 license if they have reuse the data of your project.
I hope you found the exercise useful Pim. Do not get discourage if you can't implement all best practices immediately or at once. If needed do it on incremental steps!

Thanks for sharing your READme @phvree, well done!
Just a couple of pointers:

Will you have enough space in the paper to fully elaborate on all the method/code details?
It can be helpful to at least have the contributor information listed in the READme file so that people know who to contact.
I have been leaving some information about ORCIDs in other people's feedback, so I'll also share here just in case: An ORCID an identifier for researchers. Similar like the identifier to the publication, it provides a way to easily identify you as a person and associate your research outputs with you. You can set up your own ORCID very easily (and use it for some scholarly platforms such as Zenodo/publishing platforms). You can have a look at Toeno's ORCID to see what it looks like when there is information in the profile.

Hope that helps!


bower.json: Package this lesson for Bower

Docs for bower.json are here and here.  Once this gets
merged, you'll want to tag a 0.1.0 release:
$ git tag -s -m 'version 0.1.0' v0.1.0
$ git push --tags

You can tweak the tag options as you see fit, but that's how I like to
sign my tags.  Then register the package with:
$ bower register awk-lesson git://github.com/BillMills/awk-lesson.git

I'm starting with version 0.1.0 as [recommended for semantic
versioning] 4.
Let me know if you think the content needs any tweaks.  Personally, I
think ‘lesson.md’ should be moved to ‘README.md’, which would require
a matching update to the ‘main’ entry.

Thanks for all the detail, that's super useful as I've never made a bower package before. I'm not familiar with 'signing tags' - can I just make a tag on GitHub without installing any more deps?

On Tue, Mar 24, 2015 at 09:36:10PM -0700, Bill Mills wrote:

I'm not familiar with 'signing tags'…

It's just using your GPG key to sign the tag so folks pulling from
your repository can verify that the tagged content has not been
tampered with (baring SHA-1 collisions, which are hard to do if you're
limited to textual content).  Docs in [1,2,3].

… can I just make a tag on GitHub without installing any more deps?

Sure.  It gives a wider window for malicious parties to inject their
own content, but for an awk lesson that probably doesn't matter.  It's
more problematic if you're tagging a release in the Linux kernel or
something else that's going to get compiled and installed without a
careful read through (I certainly don't read the whole source before
building a kernel ;).
You can also make a lightweight (no metadata), unsigned tag from the
command line with:
$ git tag v0.1.0
Or you can make an annotated (metadata and tag message), unsigned tag
with:
$ git tag -m 'Some message' v0.1.0
See ‘-a’ and ‘-F’ for other ways to fill in the tag message.  Then
push just like you'd push a signed tag:
$ git push --tags origin

Cool:
bower install study-group-awk-lesson

seems to work - thanks again for walking me through this!
So, I think this workflow is really good for anything that has dependencies - getting them all with a bower install would be really convenient (it does handle deps, right? I'm assuming it works the same as an npm install). But for the awk lesson, I'm not sure it's much better than a git clone (if we break the lessons out into their own repos exactly as you suggested).
But on the other hand, it does teach about the culture of using packages and package managers, which is a really really good example to set.
Thanks for this - this is a serious contender for how we distribute lesson material, and enlightening for me personally :)

Grr, it looks like Bower doesn't actually use the keyword entries
yet (see bower/registry#44 and 1).  It looks like fixing that is
reasonably major work (but probably doable).  Until then, we'll have
to stuff keywords into the project name 2.
It looks like work on the new registry is still in progress 3, and
they're using a backing database now 4, so keyword-based searches
shouldn't be hard to put in once they deploy bower/registry.

On Thu, Mar 26, 2015 at 01:21:16PM -0700, Bill Mills wrote:

So, I think this workflow is really good for anything that has
dependencies - getting them all with a bower install would be
really convenient (it does handle deps, right? I'm assuming it works
the same as an npm install).

Not quite.  npm is designed for server-side code, so installing
something will recursively pull all dependencies into a tree.  That
allows you to install multiple versions of the same library if one of
your deps needs <2.0 and another needs >2.0.  Bower, on the other
hand, is designed for client-side libraries, so it installs all
dependencies in a single, flat space (you don't want clients running
two versions of jQuery simultaneously).  Bower will warn you if
there's an incompatible dependency though, which is more than you'll
get out of pip (pypa/pip#988 ;).

But for the awk lesson, I'm not sure it's much better than a git clone (if we break the lessons out into their own repos exactly as
you suggested).

In terms of delivering the content to your local box, it's probably
worse (since we have better penetration with the ‘git’ binary).  I was
suggesting it as a way to offload discoverability:
$ bower search awk-lesson
Search results:
  study-group-awk-lesson git://github.com/BillMills/awk-lesson.git

$ git search awk-lesson
git: 'search' is not a git command. See 'git --help'.
;).  The keyword issue is unfortunate, but hopefully a temporary
problem.


Allow user to send error stack trace when an unexpected error happens

When we get an unexpected error we should have a button to 'Report to Hathor' that sends a report to Sentry.

Duplicate #74


Add support for Homebrew-installed software

Fixes #220. For software installed by and managed by Homebrew, site_data_dir and site_config_dir should point to the Homebrew-managed directories. (I added site_cache_dir as well just because it was easy to add.) This also means that the multipath option is now relevant for software running on macOS, as well as Linux/Unix.
Note that Homebrew actually has three default prefixes: one for Intel macOS computers, one for Apple Silicon macOS computers, and one for Linux computers. This pull request only addresses Apple Silicon macOS computers, where the default prefix is /opt/homebrew. I did this for several reasons:

On macOS, /opt/homebrew is the clearest indication that the software is indeed running under Homebrew; /usr/local could mean user-installed software.
It will probably be awhile before this change affects end-users; by that time, Intel Macs may no longer be supported.
Very, very few people use Homebrew on Linux; it doesn't seem worth it to support that use-case (yet).

Thoughts?

All CI runners are now passing on my pull request to my own fork. Could you please run the workflow again?

I made the changes requested. Could I get another review, please?


Optimize server startup log

Purpose of the pull request

Brief change log

Verify this pull request

This pull request is code cleanup without any test coverage.
(or)
This pull request is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:

(or)
If your pull request contain incompatible change, you should also add it to docs/docs/en/guide/upgrede/incompatible.md

Codecov Report
Attention: 60 lines in your changes are missing coverage. Please review.

Comparison is base (4e06999) 37.80% compared to head (14d1a92) 37.78%.

:exclamation: Current head 14d1a92 differs from pull request most recent head 401b9d0. Consider uploading reports for the commit 401b9d0 to get more accurate results

Files
Patch %
Lines

...inscheduler/server/master/config/MasterConfig.java
0.00%
21 Missing :warning:

...inscheduler/server/worker/config/WorkerConfig.java
0.00%
17 Missing :warning:

...hinscheduler/extract/base/NettyRemotingServer.java
42.85%
8 Missing :warning:

...apache/dolphinscheduler/common/utils/NetUtils.java
50.00%
2 Missing and 1 partial :warning:

...che/dolphinscheduler/alert/rpc/AlertRpcServer.java
0.00%
2 Missing :warning:

...inscheduler/server/master/rpc/MasterRpcServer.java
0.00%
2 Missing :warning:

.../server/master/runner/StateWheelExecuteThread.java
0.00%
2 Missing :warning:

...inscheduler/server/worker/rpc/WorkerRpcServer.java
0.00%
2 Missing :warning:

...ache/dolphinscheduler/dao/entity/TaskInstance.java
0.00%
1 Missing :warning:

...er/master/builder/TaskExecutionContextBuilder.java
0.00%
1 Missing :warning:

... and 1 more

Additional details and impacted files
@@             Coverage Diff              @@
##                dev   #15362      +/-   ##
============================================
- Coverage     37.80%   37.78%   -0.02%
+ Complexity     4681     4679       -2
============================================
  Files          1304     1303       -1
  Lines         44937    44905      -32
  Branches       4812     4789      -23
============================================
- Hits          16990    16969      -21
+ Misses        26098    26087      -11
  Partials       1849     1849

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

Please add some pr description.

Please add some pr description.

Done


[FR] Allow hiding sidebar items (map, logbook, etc)

Is your feature request related to a problem? Please describe.
It would be great to allow hiding certain sidebar buttons from non-admin users, or even altogether.
Describe the solution you'd like
Ability to hide the following buttons: Map, Logbook, History, Media, Calendar.
Describe alternatives you've considered
Using custom-sidebar lovelace plugin but it misses a general "non-admin" approach, which kiosk-mode does.
Thank you.

Hi @clau-bucur,
Kiosk-mode is intended to hide elements only in Lovelace dashboards, and the sidebar is present in every dashboard even if it is not a Lovelace one. On top of that elements in the sidebar are not fixed, every user has their own, and kiosk-mode only works with fixed elements.
As you mentioned, there is a specific custom plugin to do that, if that plugin doesn't have a certain feature, it makes more sense to request that feature there instead of requesting it in a plugin that is not intended to perform those tasks.
Regards


Can I change RIP address by this?

I want to change RIP address using this debugger.
However I can't see there are external method to do exception handler.
Is it possible to do this?

This is currently not possible because the FPBREAKPOINT_CALLBACK prototype does not allow breakpoint callbacks to modify the guest instruction pointer. This is an oversight, and I'm working on a fix now.

2700fb109dc9ccc19e06233f45c0ca2e00fd2020 allows breakpoint callbacks to modify the guest instruction pointer. If a callback modifies the guest ip parameter then the guest will be redirected to that modified address on VM entry.
You can achieve code hooking by implementing a new breakpoint callback and its driver interface. e.g., Define a breakpoint callback which uses the context parameter to store the hook destination address. The body of the callback writes the context parameter to the guest instruction pointer so that execution is redirected to your hook on VM entry.
I like this idea so I will implement it in the future. I plan to include documentation which explains the process of adding new callbacks using this idea as a working example.


Add device condition support for media_player

Description:
Adds basic device condition support for media_player integration - on, off, idle, paused, playing.
Related issue (if applicable): fixes #27004
Checklist:

[x] The code change is tested and works locally.
[x] Local tests pass with tox. Your PR cannot be merged unless tests pass
[x] There is no commented out code in this PR.
[x] I have followed the development checklist

If user exposed functionality or configuration variables are added/changed:

[ ] Documentation added/updated in home-assistant.io

If the code does not interact with devices:

[x] Tests have been added to verify that the new code works.

Is there anything I need to do in home-assistant-polymer? When I went to see what it looked like in the UI, the condition dropdown items were all blank for me.

Also, what kind of documentation should I add, if any?

Great first contribution 👍  Run script/translations_develop to get UI strings to work.

This can be merged when you format your code with Black so the tests pass 👍
You can run black path/to/file.py

The formatting error was caused by #28171, I've rebased this branch so the Black issue is fixed now 😊

Oh snap, didn't realize. Wish Black would give some more output so we know where to look.
Great work ! 🐬


Misc enhancements

Distribute RDY across producers
Catch errors connecting to lookupd
Add a timeout that I don't remember the reason for

cc @franklinhu

@danielhfrank thanks, what do you think about backwards compatibility here?
I'm inclined to just push this RDY change since it fixes the behavior according to what one would expect from the official libraries.
Alternatively, we could make this mode configurable/optional?

Yeah, I think we can go forward with this for the reasons you outlined. It's just as much work to understand and set a config flag as it is to adjust the max-in-flight for your number of producers, IMO.


fix: do not use noAssert

Support for the noAssert argument was removed for Node.js 10.x
and all input will be validated from now on nevertheless.
This removes all noAssert entries and fixes the possible overflow.
Refs: https://github.com/nodejs/node/pull/18395

Ping @theturtle32

Ping @theturtle32 again

I pushed a new change. The reason why I limited the mask to 24-bits is that the value was actually capped at 2 ** 31 - 1 due to the | 0 coercion. If a number was bigger than that, it would have been a negative number even though a unsigned integer function is used and no negative values are valid in that case. 0xFFFFFFFF is actually 2 ** 32 - 1. So in 50% of the times, a wrong value was passed through so far and this fixes a real bug in your code as it seems.
The fix is to change | 0 to >>> 0 as this will actually coerce the value in the proper range.

@BridgeAR - That's a fantastic description, and thank you very much for the update!


v0.17.0

#71  👈 (View in Graphite)
main

This stack of pull requests is managed by Graphite. Learn more about stacking.


AsyncHttpClient returns empty body on slow stream

When I request a slow stream with AsyncHttpClient I get an empty body. The request times out but it does not throw an error. In the test below there are two cases: one checks that exception is thrown, another that we get the full result. Neither of the tests succeed.
There are two questions:

How increase the timeout? Neither setRequestTimeout nor setReadTimeout seem to work
How to make sure it does not fail silently in case of timeout?

import java.util.Calendar

import cats.effect._
import cats.implicits._
import org.http4s.{HttpRoutes, Method, Request, Status, Uri}
import org.http4s.syntax._
import org.http4s.dsl.io._
import org.http4s.implicits._
import org.http4s.server.blaze._
import fs2.Stream
import fs2.text.utf8DecodeC
import org.http4s.client.Client
import org.http4s.client.asynchttpclient.AsyncHttpClient
import org.http4s.server.Server

import scala.concurrent.duration._
import org.scalatest.FunSuite

import scala.concurrent.ExecutionContext

class ExampleSuite extends FunSuite {

  val singleThreadExecutor = ExecutionContext.fromExecutor((task: Runnable) => task.run())
  implicit val contextShift = IO.contextShift(singleThreadExecutor)
  implicit val timer = IO.timer(singleThreadExecutor)

  val host = "localhost"
  val port = 4002
  val uri = Uri.unsafeFromString(s"http://$host:$port/test")

  val hang = Stream.awakeEvery[IO](120.seconds).take(5)

  val hangService = HttpRoutes.of[IO] {
    case GET -> Root / "test" => Ok(hang.map(_.toString))
    case POST -> Root / "test" => Ok(hang.map(_.toString))
  }.orNotFound

  def runServer(f: (Server[IO]) => IO[List[String]]): IO[List[String]] = {
    BlazeServerBuilder[IO]
      .bindHttp(port, host)
      .withHttpApp(hangService)
      .resource.use(f)
  }

  def runClient(f: (Client[IO]) => IO[List[String]]): IO[List[String]] = {
    AsyncHttpClient.resource[IO]().use(f)
  }

  def runRequest(client: Client[IO]): IO[List[String]] = {
    client.stream(Request[IO](Method.GET, uri)).flatMap { response =>
      Console.println(Calendar.getInstance().getTime, response)
      response.status match {
        case Status.Ok => response.body.chunks.through(utf8DecodeC)
        case _ => Stream.raiseError[IO](new Exception(response.status.reason))
      }
    }.compile.toList
  }

  test("should timeout") {
    val result = runServer { _ =>
      runClient { client =>
        runRequest(client)
      }
    }

    assertThrows[Exception] {
      result.unsafeRunSync()
    }
    // Expected exception java.lang.Exception to be thrown, but no exception was thrown
  }

  test("should succeed") {
    val result = runServer { _ =>
      runClient { client =>
        runRequest(client)
      }
    }

    assert(result.unsafeRunSync().size == 5)
    // List() had size 0 instead of expected size 5
  }

}

// build.sbt

name := "http4s-test"

version := "0.1"

scalaVersion := "2.12.6"

//val http4sVersion = "0.20.1"
val http4sVersion = "0.21.4"

libraryDependencies ++= Seq(
  "org.http4s" %% "http4s-dsl" % http4sVersion,
  "org.http4s" %% "http4s-blaze-server" % http4sVersion,
  "org.http4s" %% "http4s-blaze-client" % http4sVersion,
  "org.http4s" %% "http4s-async-http-client" % http4sVersion,
  "org.scalatest" % "scalatest_2.12" % "3.1.2" % "test"
)

scalacOptions ++= Seq("-Ypartial-unification")

It's late here, so I'll need to take a closer look tomorrow, but I appreciate the detailed repro.  That will make it much easier to get to the bottom of.
To be clear, you tried to set those timeout options in the runClient call?

So I did some more testing
Server timeout
I saw in the BlazeServer logs that it closes connection after 60 seconds.
11:20:45.804 [scala-execution-context-global-32] DEBUG org.http4s.blazecore.IdleTimeoutStage - Idle timeout after 60000 ms.
11:20:45.805 [scala-execution-context-global-32] DEBUG org.http4s.server.blaze.Http1ServerStage$$anon$1 - Shutting down due to idle timeout

Not sure if it is correct behavior for the server to close connection while streaming result. Anyway, I used this server just for testing so I fixed it by setting servers idle timeout to infinity.
def runServer(f: (Server[IO]) => IO[List[String]]): IO[List[String]] = {
    BlazeServerBuilder[IO]
      .withIdleTimeout(Duration.Inf)
      .bindHttp(port, host)
      .withHttpApp(hangService)
      .resource.use(f)
  }

Client timeout
This however did not fix the problem. I see timeout exception logged now and then it is swallowed somewhere apparently. Request still does not fail and returns result, this time one item.
java.util.concurrent.TimeoutException: Read timeout to localhost/<IP_ADDRESS>:4002 after 60000 ms
	at org.asynchttpclient.netty.timeout.TimeoutTimerTask.expire(TimeoutTimerTask.java:43)
	at org.asynchttpclient.netty.timeout.ReadTimeoutTimerTask.run(ReadTimeoutTimerTask.java:56)
	at io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:672)
	at io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:747)
	at io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:472)
	at java.lang.Thread.run(Thread.java:748)

List("120014630836 nanoseconds") had size 1 instead of expected size 5

So I increased the read timeout now for the client.
def runClient(f: (Client[IO]) => IO[List[String]]): IO[List[String]] = {
  val defaultConfig = AsyncHttpClient.defaultConfig
  val config = new DefaultAsyncHttpClientConfig.Builder()
    .setMaxConnectionsPerHost(defaultConfig.getMaxConnectionsPerHost)
    .setMaxConnections(defaultConfig.getMaxConnections)
    .setRequestTimeout(Integer.MAX_VALUE)
    .setReadTimeout(240*1000) // 4 minutes
    .setThreadFactory(defaultConfig.getThreadFactory)
    .setCookieStore(defaultConfig.getCookieStore)
    .build()
  AsyncHttpClient.resource[IO](config).use(f)
}

This time it actually succeeded, so increasing read timeout actually seems to work.
Conclusion
There is still a problem that when read timeout happens the exception is swallowed and result is returned.
There is a workaround: to set read timeout high enough so that it is not reached. Still, if it is reached, the client will silently return the incomplete result.

The cb is already completed with a response.  For streaming purposes, the response headers are available before the timeout happens, so I think the best we can do is to raise an error in the Stream body.
The reason it's not is because the error is going to onThrowable instead of being raised by the Publisher.  We need some way of getting an error into that body we create in onStream.


Add F-Droid build flavor

Fixes #82.

@dimqua Please fix appveyor's and circleci's config to run tests from playDebug or fdroidDebug. I've fixed circleci config and it should run after next commit.


Icon put in wrong place in Install Script

Fixes #135

Updated the desktop to show a terminal so downloading of docker updates/ install can be seen.


Fixed flyway 4.0 upgrade for Redshift (it does not support alter column)

@axelfontaine
I fixed the flyway 4 metadata upgrade for Redshift. Redshift does not support ALTER COLUMN, so we have to create new columns and copy the data instead.

Thanks!


Update get_started.md

Correcting output.

Can one of the admins verify this patch?

✔ Please visit https://cla.developers.google.com/ to sign.

@tensorflow-jenkins test this please

@rohan100jain thank you for merging. Can you tell me how the process of building and deploying the documentation is beeing handle by you guys? I see many differences between the source documentation and the one published.

The root of the website will always be from the latest stable version, which is currently 1.0.
Your changes in master will be deployed under https://www.tensorflow.org/versions/master/get_started/get_started.
Your changes will also be in the next release cut which will be 1.2 (1.1 is already cut, and nearing the final release.)

Thanks for the info. 👍


Migrating this into JC

The tool JC https://github.com/kellyjonbrazil/jc has been for me an epicenter for converting things into json.
To the point that ansible integrated it into their pipeline.
Would you want to connect to that maintainer and pull this code into his project?
I just want a general tool to rule them all

Hey Ron,
thanks for reaching out but jc is written in Python and I do not feel like porting my Rust code to Python tbh.
Thanks and best
moertel


[SPARK-21911][ML][PySpark] Parallel Model Evaluation for ML Tuning in PySpark

What changes were proposed in this pull request?
Add parallelism support for ML tuning in pyspark.
How was this patch tested?
Test updated.

Test build #81411 has started for PR 19122 at commit 596ce35.

Test build #81411 has finished for PR 19122 at commit 596ce35.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81411/
Test PASSed.

Test build #81457 has started for PR 19122 at commit 0a94344.

Test build #81457 has finished for PR 19122 at commit 0a94344.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81457/
Test PASSed.

@BryanCutler code updated. thanks!

Test build #81705 has started for PR 19122 at commit d6cf103.

Test build #81705 has finished for PR 19122 at commit d6cf103.

This patch fails Python style tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81705/
Test FAILed.

Test build #81707 has started for PR 19122 at commit 7122884.

Test build #81707 has finished for PR 19122 at commit 7122884.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81707/
Test PASSed.

@BryanCutler Do you have more comments?  I can check it out now but don't want to review at the same time.

Test build #82058 has started for PR 19122 at commit 3464dfe.

Test build #82058 has finished for PR 19122 at commit 3464dfe.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82058/
Test PASSed.

ping @jkbradley

Test build #82275 has started for PR 19122 at commit 93ab39a.

Test build #82275 has finished for PR 19122 at commit 93ab39a.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82275/
Test PASSed.

Discussed elsewhere: We'll delay the multi-model fitting optimization in favor of getting this in for now.  Taking a look now...

Test build #3961 has started for PR 19122 at commit 93ab39a.

Test build #3961 has finished for PR 19122 at commit 93ab39a.

This patch passes all tests.
This patch does not merge cleanly.
This patch adds no public classes.

Test build #83075 has started for PR 19122 at commit 8b3ef97.

Test build #83075 has finished for PR 19122 at commit 8b3ef97.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83075/
Test PASSed.

LGTM
Merging with master
Thanks @WeichenXu123 , @BryanCutler and @viirya !

Whoops, could you please send a follow-up PR to do 1 doc update?

Update https://github.com/apache/spark/blob/master/docs/ml-tuning.md to say this is supported in Python.  (Search for "NOTE: this is not yet supported in Python")

@jkbradley Sure I will!


Vendor<EMAIL_ADDRESS>and adapt changes for workerless Shoots

How to categorize this PR?

/area networking
/kind enhancement
What this PR does / why we need it:

Vendor gardener v1.70.2
Skip mutation and validation for shoots if it's workerless.

Which issue(s) this PR fixes:
Fixes #
Special notes for your reviewer:
Release note:

The following dependency has been updated:
  - github.com/gardener/gardener 1.67.1 -> 1.70.2

/invite @ScheererJ


[WIP] Add function for creating base scale

Description

Closes issue #32

Added function generateBaseScale()

Generates color scale using colorKeys and colorspace parameters
Converts all new colors to HSLuv
Filters values to remove duplicates for each whole value of HSLuv lightness
Returns filtered array in Hex format

Added tests for createBaseScale
Updated README with details on how to use the function
Updated Demo app to use createBaseScale as method of integrating Leonardo for brightness controls.

Motivation

To make brightness controls easier within Leonardo
Ensure colors used in brightness controls are uniform in display colorspace (HSLuv) brightness values
Ensure output is easily consumable and easy to reason about (0-100 scale)

Screenshots

To-do list

[x] I have read the CONTRIBUTING document.

[ ] This pull request is ready to merge.

After consideration, this may be better suited as a separate package, ie @adobe/leonardo-base-scale.
See issue #34 for possible addition to packages, and how this could come into play.

Work for creating base-scale as separate package moved to separate branch. Given the contrast-colors package is stated as housing all functions necessary for creating colors based on contrast ratio, and since generateBaseScale is intended for brightness control and use within future generateContrastTheme functions, it may be better suited within the same package after all.

@lazd I am curious which direction makes the most sense to you:

Add this (and all other contrast-based color generation functions) to the leonardo-contrast-colors
Or
Create a new package as leonardo-base-scale

I feel that the package name leonardo-contrast-colors is very specific to the generateContrastColors function. As we add functions for generating an entire palette (if part of the same package), the package name seems a bit ambiguous. At which point a more generic name would have been more appropriate.
I am leaning more towards separate packages but want your input.

@NateBaldwinDesign hmm, I'm conflicted. If we consider the use cases, will people often be using just either generateBaseScale or generateContrastColors, or will they often be using both together?
On one hand, we don't want folks to have to search to find the package they need to provide the right function.
On the other hand, we don't want a single package full of so many methods that it's confusing as to which should be used for a specific use case.
And on the, uh, third hand, perhaps we need to consider unifying these under a single package with a more generic name. It really depends on how people are using these packages.

@lazd true. Once we implement the generateAdaptiveTheme function, it may be less likely that the individual generateContrastColors function is used, but it would be almost certain that the generateBaseScale is less used directly. The base scale function is an aspect of implementing the theme in its entirety but may be useful outside of that, hence the separation.
This may help to show how the functions are used.

If we separate these all into unique packages, they will all need to import createScale, and the generateAdaptiveTheme function will need to import both generateBaseScale and generateContrastColors.

My thoughts on this are

If we separate into different packages, we will need a "bundle" for the generateAdaptiveTheme that depends on all of them.
If we keep them in the same package, contrast-colors is not generic enough.
If we generalize the name, we will need to consider I want to add (in the future) helper functions for creating sequential and diverging scales for datavis, which would need the createScale helper function but nothing else. Since they're not "contrast-based", they'd likely be separate packages.

One thought, off the top of my head, would be:

leonardo-contrast: Contrast based color and theme generation (all current functions)
leonardo-sequential: Sequential color scale
leonardo-diverging: Diverging color scale

However, since the sequential/diverging have not yet been implemented, there's also no guarantee that I'll need a package for those. For instance, the web tool may simply provide arguments necessary to output a custom scale using D3's existing API (no wrapper or unique functionality).

Like we discussed today, let's reach out to some potential developers that could use Leonardo, give them a demo, and ask them how they imagine Leonardo integrating with their app. Would it be in the build process? CSS-in-JS? CSS custom properties (variables) manipulated at run time? I think this will inform how these functions are separated into different packages and give us a much better idea of how we should recommend the package(s) be used.


fix deprecation warning

Fixes the following warning: DeprecationWarning: The 'warn' function is deprecated, use 'warning' instead
(This seems to be the only location in the code where the deprecated method is used.)

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

unfortunately, signing the CLA is mandatory. I added the fix to #316, i.e., you should no longer get that warning about warnings ;)


Simplified subcategories to build automation

Categorised in a list the three subcategories to Build automation.

[x] I have read freeCodeCamp's contribution guidelines.
[x] My pull request has a descriptive title (not a vague title like Update index.md)
[x] My pull request targets the master branch of freeCodeCamp.
[x] None of my changes are plagiarized from another source without proper attribution.
[x] My article does not contain shortened URLs or affiliate links.

If your pull request closes a GitHub issue, replace the XXXXX below with the issue number.
Closes #XXXXX

Thank you for your contribution and congratulations on your first PR merge to this repo! 🎉
Hope to see more contributions from you in the future.


Software BOM (package-url)

In issue #113 a cve-search is proposed. Maybe as a first step create a software BOM and use https://github.com/package-url/packageurl-python instead of CPE following the arguments given in the Veracode article in that issue.
The generated software BOM van be used for other tools such as https://plugins.jenkins.io/dependency-track/
See https://medium.com/@steve_springett/using-software-bill-of-materials-to-drive-change-and-reduce-risk-5901b7a339e3

See also the requirements for sbom in the new executive order about cyber security as analysed by David Wheeler: https://www.linuxfoundation.org/en/blog/how-lf-communities-enable-security-measures-required-by-the-us-executive-order-on-cybersecurity/?s=09

A tool did something similar for west+cmake: https://GitHub.com/swinslow/cmake-spdx

Some more background info here: https://www.zephyrproject.org/how-to-generate-a-software-bill-of-materials-sbom-with-open-source-standards-and-tooling/

Here is small example of a python package generating an CycloneDX SBoM from a conan recipe: https://github.com/CycloneDX/cyclonedx-conan/blob/main/src/command.py

Created a Proof-of-concept, this generates the following SBoM:
{
    "bomFormat": "CycloneDX",
    "specVersion": "1.3",
    "serialNumber": "urn:uuid:a187961b-118b-440d-8ca7-ca9fee4333a6",
    "version": 1,
    "components": [
        {
            "type": "library",
            "name": "cpputest",
            "version": "v3.4",
            "purl"<EMAIL_ADDRESS>            "group": "cpputest",
            "licenses": [
                {
                    "license": {
                        "name": "BSD 3-Clause \"New\" or \"Revised\" License"
                    }
                }
            ]
        },
        {
            "type": "library",
            "name": "jsmn",
            "version": "",
            "purl": "pkg:github/zserge/jsmn#zserge/jsmn.git",
            "group": "zserge",
            "licenses": [
                {
                    "license": {
                        "name": "MIT License"
                    }
                }
            ]
        }
    ],
    "metadata": {
        "timestamp": "2021-11-26T22:42:12.090050+00:00",
        "tools": [
            {
                "vendor": "CycloneDX",
                "name": "cyclonedx-python-lib",
                "version": "0.9.1"
            }
        ]
    }
}


Updated PywalZen for 1.0.1-a.14 + Preferences

I fixed the theme for the latest update and added some preferences, I hope they work. I couldn't find a way to test them with the browser.

Thanks!


early_stopper wrong documentation

Hi,
I just was working on a Project with keras and added talos for hyperparameter opimization.
Probably its just my mistake, but I noticed that the documentation (example) of the early_stopper implementation ist wrong.
out = model.fit(x_train, y_train, batch_size=params['batch_size'], epochs=params['epochs'], verbose=0, validation_data=[x_val, y_val], callbacks=early_stopper(params['epochs'], mode='strict')) ```
https://autonomio.github.io/docs_talos/#probabilistic-reduction
correct is to put the early_stopper call in brackets.
out = model.fit(x_train, y_train, batch_size=params['batch_size'], epochs=params['epochs'], verbose=0, validation_data=[x_val, y_val], callbacks=**[**early_stopper(params['epochs'], mode='strict') ])```
It took me a bit of time to find the problem, that's why I just wanna save someone else some time ;-)

Thanks. This is related with #307.
As we make the move to the next LTS version, which starts from v.0.6 (now available in daily-dev branch), part of it involves ensuring that the documentation is 100% in accord with the software. It might take a little bit of time though. To that end, all kinds of contributions will be most welcome.

This is done now, will be in the new docs once it makes to master. Closing here.